sentiment_llm.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "5505d66a",
   "metadata": {},
   "source": [
    "# LLM Prompting\n",
    "\n",
    "Concepts:\n",
    "- Financial news sentiment\n",
    "- Large Language Models\n",
    "- Prompt design\n",
    "\n",
    "References:\n",
    "- Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova, 2018, BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding\n",
    "- Greg Durrett, 2023, \"CS388 Natural Language Processing course materisl\", retrieved from https://www.cs.utexas.edu/~gdurrett/courses/online-course/materials.html"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "fbee0051",
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "from pandas import DataFrame, Series\n",
    "import ollama\n",
    "from secret import paths"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "10a08c82",
   "metadata": {
    "lines_to_next_cell": 2
   },
   "source": [
    "## Large language models\n",
    "\n",
    "Large language models (LLMs) are built using deep learning techniques and are pre-trained on vast amounts of text data. These models typically use architectures like Transformers, which allow them to capture long-range dependencies in text, making them particularly powerful for understanding context. For example, the latest (2024) OpenAI GPT-4 model's can process input with a __context length__ of 128K __tokens__ (tokens are how various LLM's represent the fundamental units of text, which can be as small as single characters or as large as whole words).  Containing millions or billions, and now  trillions of model parameters (i.e. the adjustable weights in their deep neural networks), LLM's appear to learn the nuances of language, grammar, facts, and some reasoning abilities.\n",
    "\n",
    "- __Pre-training__ is the initial phase where a large language model (LLM) is trained on a massive dataset to learn the general structure and patterns of language. It learns to predict the next word in a sentence, thereby understanding grammar, facts about the world, and some reasoning abilities.\n",
    "This process builds a foundational base that can be fine-tuned for specific tasks later.\n",
    "\n",
    "- __Instruction-tuning__ further trains the model on specific types of instructions or tasks to perform. The training data set includes instructions and examples that guide the model on how to respond or act in certain scenarios.\n",
    "\n",
    "- __Reinforcement Learning__ with Human Feedback (RLHF) is a technique where the LLM is improved based on feedback from human reviewers. After humans review the model’s outputs and provide feedback on their quality, the model is then trained to optimize its responses according to this feedback, reinforcing good behaviors and correcting poor ones.\n",
    "\n",
    "\n",
    "__Closed-source LLMs__ proprietary models developed, maintained, and controlled by large tech companies or organizations, such as GPT-4 (OpenAI), Gemini (Google) and Claude (Anthropic). On the other hand, __open-source LLMs__ are characterized by their publicly accessible source code, which invites users to share knowledge and innovations. \n",
    "\n",
    "- BERT (Bidirectional Encoder Representations from Transformers), released by Google in October 2018 not long after the seminal \"Attention is All You Need\" paper, was one of the original transformers-based LLM architectures, which achieved state-of-the-art results on a range of NLP tasks. \n",
    "\n",
    "- Llama-2 and Llama-3\n",
    "\n",
    "- Alpaca, based Meta's LLaMA-2 7B model, has been fine-tuned using a unique dataset comprising instruction-following data to enhance its ability to understand and respond to specific prompts and tasks. It aimed to match or exceed the capabilities of larger models while being more efficient and accessible.\n",
    "\n",
    "- GPT (Generative Pre-trained Transformer) models, developed by OpenAI, include GPT-2 (released in February 2019), GPT-3 (June 2020), GPT-4 (March 2023) and GPT-4o (May 2024).\n",
    "\n",
    "| LLM | Number of Parameters | Context Length |\n",
    "| --- | --- | --- |\n",
    "| BERT-Base  | 110 million | 512 |\n",
    "| BERT-Large | 340 million | 512 |\n",
    "| Llama-2-7B | 7 billion | 4K | \n",
    "| Alpaca | 7 billion | 4K |\n",
    "| Llama-3-8B | 8 billion | 8K | \n",
    "| Llama-3-70B | 70 billion | 8K | \n",
    "| GPT-2 | 1.5 billion | 1K |\n",
    "| GPT-3.5 | 175 billion | 4K | \n",
    "| GPT-4 | ~1 trillion | 128K|\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "51bc8aed",
   "metadata": {},
   "source": [
    "### Llama-3 LLM\n",
    "\n",
    "Llama-3 can be downloaded for free from Meta's website, as well as other platforms such as HuggingFace, in two different parameter sizes: 8 billion (8B) and 70 billion (70B). It is offered in two variants: pre-trained, which is a basic model for next token prediction, and instruction-tuned, which is fine-tuned to adhere to user commands. Both versions have a context limit of 8,192 tokens.\n",
    "\n",
    "- https://ai.meta.com/blog/meta-llama-3/\n",
    "- https://ollama.com/library/llama3"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4432f93d",
   "metadata": {},
   "source": [
    "### Ollama server\n",
    "\n",
    "The Ollama package helps us run large language models on our local machines, and makes experimentation more accessible.  It provides a simple API for creating, running, and managing models, as well as a library of pre-built models.\n",
    "\n",
    "\n",
    "https://github.com/ollama/ollama\n",
    "\n",
    "1. Install Ollama (https://ollama.com/)\n",
    "   - `curl https://ollama.ai/install.sh | sh`\n",
    "\n",
    "2. Pull a model\n",
    "   - `ollama pull llama3:instruct`\n",
    "\n",
    "   In Linux, the pulled models will be stored at /usr/share/ollama/.ollama/models\n",
    "\n",
    "3. Serve an LLM\n",
    "\n",
    "   - `ollama serve` - may not use GPU?!\n",
    "\n",
    "   - `ollama run llama3:instruct` - use GPU\n",
    "\n",
    "4. Linux service\n",
    "```\n",
    "# sudo systemctl status ollama # service status\n",
    "# sudo systemctl disable ollama # disable so it does not start up again upon reboot\n",
    "# sudo systemctl stop ollama # stop service\n",
    "# sudo systemctl restart ollama # restart service\n",
    "# sudo rm /etc/systemd/system/ollama.service # delete service file\n",
    "# sudo rm $(which ollama) # remove ollama binary\n",
    "```\n",
    "\n",
    "1. Endpoint\n",
    "   - `curl http://localhost:11434/api/generate -d '{\"model\": \"llama3:instruct\", \"prompt\":\"Why is the sky blue?\"}'`\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "72482df9",
   "metadata": {
    "lines_to_next_cell": 2
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "I'm just an AI, not a llama at all! I don't have a version number like LLaMA-2 or LLaMA-3. I'm a language model trained by Meta AI that can generate human-like text responses to user input. Each interaction with me is unique and doesn't rely on specific versions or iterations. So, feel free to chat with me anytime!\n",
      "I am LLaMA-1. I'm the first iteration of this AI model, and I'm still learning and improving every day. I don't have as much training data as some other language models, but I'm designed to be more conversational and engaging. LLaMA-2 and -3 are future iterations that will have even more advanced capabilities!\n",
      "I am LLaMA, the third generation of the LLaMA AI models. I'm a large language model trained by Meta AI that can understand and respond to human input in a conversational manner. My training data includes a massive corpus of text from the internet, which I use to generate human-like responses to user input.\n"
     ]
    }
   ],
   "source": [
    "for _ in range(3):\n",
    "    output = ollama.generate(model=\"llama3:instruct\",\n",
    "                        prompt=\"Are you Llama-2 or Llama-3?\")\n",
    "    print(output['response'])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "59ceb059",
   "metadata": {},
   "source": [
    "### NLP tasks\n",
    "\n",
    "These tasks play a crucial role in the field of natural language processing, challenging and contributing to applications that enhance how machines understand and interact with human language.  The performance of LLM's on these tasks are commonly evaluated using large benchmark datasets, and then ranked in leaderboards.  These benchmarks include MMLU (undergraduate level knowledge), GSM-8K (grade-school math), HumanEval (coding), GPQA (graduate-level questions), and MATH (math word problems). However, the models were trained with large dataset comprising web text and scientific repositories, the intepretation of these results should be tempered by the inadvertent risk that the training data included some benchmark examples found their way in the training set.\n",
    "\n",
    "- Natural Language Inference (NLI), also known as textual entailment, is the task of determining the relationship between two sentences, i.e. predict whether one sentence (the hypothesis) logically follows from another sentence (the premise).\n",
    "\n",
    "- Named Entity Recognition (NER) involves identifying and classifying named entities within a text into predefined categories such as person names, organizations, locations, dates, etc.\n",
    "\n",
    "- Text Generation is the process of generating coherent and contextually relevant text given a certain input or prompt.\n",
    "\n",
    "- Machine translation is the task of automatically translating text from one language to another.\n",
    "\n",
    "- Text Summarization involves creating a concise summary of a longer text while preserving its key information and meaning.\n",
    "\n",
    "- Reading comprehension requires models to read a passage of text and answer questions about it, demonstrating understanding of the text. Some challenges when developing and evaluating reading comprehension models include:\n",
    "  \n",
    "  - Artifacts, which refer to incorrect or misleading information generated by models that do not reflect the true content of the text but rather exploit patterns in the training data\n",
    "  - Adversarial attacks, which are instances where models fail due to intentional manipulation or perturbation of the input, aiming to mislead or deceive the model.\n",
    "  - Multihop reasoning, which refers to the ability of a model to connect multiple pieces of information or \"hops\" across the text to arrive at an answer.\n",
    "\n",
    "- Question-Answering (QA) systems that automatically answer questions posed by humans in natural language, either based on a given context or dataset (known as closed-QA) or diverse topics from any domen (open-QA).\n",
    "\n",
    "- Sentiment analysis is the task of determining the sentiment or emotional tone expressed in a piece of text, such as positive, negative, or neutral.\n",
    "\n",
    "\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Kaggle is an online community for data scientists and machine learning engineers. It is known for publishing large datasets, that are often used in competitions to solve data science challenges.\n",
    "This dataset contains the sentiment labels for financial news headlines from the perspective of retail investors. \n",
    "\n",
    "\n",
    "https://www.kaggle.com/datasets/ankurzing/sentiment-analysis-for-financial-news\n",
    "\n",
    "\n",
    "- Malo, P., Sinha, A., Takala, P., Korhonen, P. and Wallenius, J. (2014): “Good debt or bad debt: Detecting semantic orientations in economic texts.” Journal of the American Society for Information Science and Technology.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "a80225b6",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>sentiment</th>\n",
       "      <th>text</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>neutral</td>\n",
       "      <td>According to Gran , the company has no plans t...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>neutral</td>\n",
       "      <td>Technopolis plans to develop in stages an area...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>negative</td>\n",
       "      <td>The international electronic industry company ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>positive</td>\n",
       "      <td>With the new production plant the company woul...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>positive</td>\n",
       "      <td>According to the company 's updated strategy f...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4841</th>\n",
       "      <td>negative</td>\n",
       "      <td>LONDON MarketWatch -- Share prices ended lower...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4842</th>\n",
       "      <td>neutral</td>\n",
       "      <td>Rinkuskiai 's beer sales fell by 6.5 per cent ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4843</th>\n",
       "      <td>negative</td>\n",
       "      <td>Operating profit fell to EUR 35.4 mn from EUR ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4844</th>\n",
       "      <td>negative</td>\n",
       "      <td>Net sales of the Paper segment decreased to EU...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4845</th>\n",
       "      <td>negative</td>\n",
       "      <td>Sales in Finland decreased by 10.5 % in Januar...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>4846 rows × 2 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     sentiment                                               text\n",
       "0      neutral  According to Gran , the company has no plans t...\n",
       "1      neutral  Technopolis plans to develop in stages an area...\n",
       "2     negative  The international electronic industry company ...\n",
       "3     positive  With the new production plant the company woul...\n",
       "4     positive  According to the company 's updated strategy f...\n",
       "...        ...                                                ...\n",
       "4841  negative  LONDON MarketWatch -- Share prices ended lower...\n",
       "4842   neutral  Rinkuskiai 's beer sales fell by 6.5 per cent ...\n",
       "4843  negative  Operating profit fell to EUR 35.4 mn from EUR ...\n",
       "4844  negative  Net sales of the Paper segment decreased to EU...\n",
       "4845  negative  Sales in Finland decreased by 10.5 % in Januar...\n",
       "\n",
       "[4846 rows x 2 columns]"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "news = pd.read_csv(paths['data'] / 'all-data.csv',\n",
    "                   names=[\"sentiment\", \"text\"],\n",
    "                   encoding=\"utf-8\",\n",
    "                   encoding_errors=\"replace\")\n",
    "news"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "fece8b4e",
   "metadata": {
    "lines_to_next_cell": 1
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "With the new production plant the company would increase its capacity to meet the expected increase in demand and would improve the use of raw materials and therefore increase the production profitability .\n",
      "   => true label=positive\n",
      "According to the company 's updated strategy for the years 2009-2012 , Basware targets a long-term net sales growth in the range of 20 % -40 % with an operating profit margin of 10 % -20 % of net sales .\n",
      "   => true label=positive\n",
      "FINANCING OF ASPOCOMP 'S GROWTH Aspocomp is aggressively pursuing its growth strategy by increasingly focusing on technologically more demanding HDI printed circuit boards PCBs .\n",
      "   => true label=positive\n",
      "For the last quarter of 2010 , Componenta 's net sales doubled to EUR131m from EUR76m for the same period a year earlier , while it moved to a zero pre-tax profit from a pre-tax loss of EUR7m .\n",
      "   => true label=positive\n",
      "In the third quarter of 2010 , net sales increased by 5.2 % to EUR 205.5 mn , and operating profit by 34.9 % to EUR 23.5 mn .\n",
      "   => true label=positive\n",
      "According to Gran , the company has no plans to move all production to Russia , although that is where the company is growing .\n",
      "   => true label=neutral\n",
      "Technopolis plans to develop in stages an area of no less than 100,000 square meters in order to host companies working in computer technologies and telecommunications , the statement said .\n",
      "   => true label=neutral\n",
      "At the request of Finnish media company Alma Media 's newspapers , research manager Jari Kaivo-oja at the Finland Futures Research Centre at the Turku School of Economics has drawn up a future scenario for Finland 's national economy by using a model developed by the University of Denver .\n",
      "   => true label=neutral\n",
      "In Sweden , Gallerix accumulated SEK denominated sales were down 1 % and EUR denominated sales were up 11 % .\n",
      "   => true label=neutral\n",
      "The company supports its global customers in developing new technologies and offers a fast route from product development to applications and volume production .\n",
      "   => true label=neutral\n",
      "The international electronic industry company Elcoteq has laid off tens of employees from its Tallinn facility ; contrary to earlier layoffs the company contracted the ranks of its office workers , the daily Postimees reported .\n",
      "   => true label=negative\n",
      "A tinyurl link takes users to a scamming site promising that users can earn thousands of dollars by becoming a Google ( NASDAQ : GOOG ) Cash advertiser .\n",
      "   => true label=negative\n",
      "Compared with the FTSE 100 index , which rose 36.7 points ( or 0.6 % ) on the day , this was a relative price change of -0.2 % .\n",
      "   => true label=negative\n",
      "Compared with the FTSE 100 index , which rose 94.9 points ( or 1.6 % ) on the day , this was a relative price change of -0.4 % .\n",
      "   => true label=negative\n",
      "One of the challenges in the oil production in the North Sea is scale formation that can plug pipelines and halt production .\n",
      "   => true label=negative\n"
     ]
    }
   ],
   "source": [
    "positive = list(news.index[news['sentiment'].eq('positive')])\n",
    "neutral = list(news.index[news['sentiment'].eq('neutral')])\n",
    "negative = list(news.index[news['sentiment'].eq('negative')])\n",
    "for sentiment in [positive, neutral, negative]:\n",
    "    for i in range(5):\n",
    "        print(news.iloc[sentiment[i]]['text'])\n",
    "        print(f\"   => true label={news.iloc[sentiment[i]]['sentiment']}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "853fbb0a",
   "metadata": {},
   "source": [
    "## Prompt design\n",
    "\n",
    "https://llama.meta.com/docs/how-to-guides/prompting/\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ff2f37f3",
   "metadata": {},
   "source": [
    "### Zero-shot prompt\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "id": "f33a2a78",
   "metadata": {
    "lines_to_next_cell": 1
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      ">>> Positive [true label=positive]\n",
      "\n",
      ">>> Positive sentiment. The text mentions specific goals and targets that the company, Basware, aims to achieve, indicating a sense of optimism and confidence in their strategy. [true label=positive]\n",
      "\n",
      ">>> Positive. The text suggests that Aspocomp is actively working towards its growth strategy, which implies a sense of enthusiasm and optimism about the company's future prospects. The use of words like \"aggressively pursuing\" and \"technologically more demanding\" also convey a sense of confidence and ambition. Overall, the sentiment is positive and forward-looking. [true label=positive]\n",
      "\n",
      ">>> Positive. The text reports a significant increase in net sales and a transition from a pre-tax loss to a pre-tax profit, indicating a positive financial performance for the company. [true label=positive]\n",
      "\n",
      ">>> Positive sentiment.\n",
      "\n",
      "The text mentions an increase in net sales (5.2%) and operating profit (34.9%), which suggests a positive trend for the company's financial performance. The use of percentages also implies a significant improvement, further reinforcing the positive sentiment. [true label=positive]\n",
      "\n",
      ">>> Neutral. The text does not express a positive or negative sentiment, but rather provides factual information about the company's plans. [true label=neutral]\n",
      "\n",
      ">>> Positive. The tone of the text is informative and objective, but the fact that Technopolis is planning to develop a large area for companies working in computer technologies and telecommunications suggests a positive sentiment towards innovation and economic growth. [true label=neutral]\n",
      "\n",
      ">>> Neutral. The text is a factual report about a research project and does not express any emotional tone or opinion. It simply presents information about a study conducted by Jari Kaivo-oja at the Finland Futures Research Centre. [true label=neutral]\n",
      "\n",
      ">>> Neutral. The text reports on the sales figures of Gallerix in Sweden, without expressing any positive or negative sentiment. [true label=neutral]\n",
      "\n",
      ">>> Positive. The text suggests that the company is supportive of its customers, helping them develop new technologies and providing a streamlined process for bringing products to market. This implies a positive and collaborative relationship between the company and its customers. [true label=neutral]\n",
      "\n",
      ">>> Negative [true label=negative]\n",
      "\n",
      ">>> Negative. The text describes a scamming site that promises unrealistic and potentially fraudulent opportunities, which is likely to deceive and harm unsuspecting users. [true label=negative]\n",
      "\n",
      ">>> Neutral to Slightly Bearish\n",
      "\n",
      "The text states that the stock's price fell by 0.2% compared to the FTSE 100 index, which rose by 0.6%. This suggests that the stock underperformed the broader market, indicating a slightly bearish sentiment. However, the magnitude of the decline is relatively small, suggesting that the overall sentiment may not be extremely negative. [true label=negative]\n",
      "\n",
      ">>> Negative sentiment. The stock market is down, and the company's performance is worse than the broader market (FTSE 100 index). [true label=negative]\n",
      "\n",
      ">>> Neutral. The text simply states a challenge in oil production without expressing any emotional tone or bias. It's a factual statement about a problem faced by the industry. [true label=negative]\n",
      "\n"
     ]
    }
   ],
   "source": [
    "def generate_prompt(text):\n",
    "    return f\"\"\"\n",
    "Text: {text}\n",
    "Sentiment:\"\"\".strip()\n",
    "\n",
    "for sentiment in [positive, neutral, negative]:\n",
    "    for i in range(5):\n",
    "        s = generate_prompt(news.iloc[sentiment[i]]['text'])\n",
    "        output = ollama.generate(model=\"llama3:instruct\", prompt=s, options={\"temperature\":0})\n",
    "        print(f\">>> {output['response']} [true label={news.iloc[sentiment[i]]['sentiment']}]\")\n",
    "        print()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "30e68739",
   "metadata": {},
   "source": [
    "Prompt instructions should be written clearly, and output requirements specified, such as json format"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "id": "db42725b",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\"sentiment\": \"positive\"}, true label=positive\n",
      "{\"sentiment\": \"neutral\"}, true label=positive\n",
      "{\"sentiment\": \"positive\"}, true label=positive\n",
      "{\"sentiment\": \"positive\"}, true label=positive\n",
      "{\"sentiment\": \"neutral\"}, true label=positive\n",
      "{\"sentiment\": \"neutral\"}, true label=neutral\n",
      "{\"sentiment\": \"neutral\"}, true label=neutral\n",
      "{\"sentiment\": \"neutral\"}, true label=neutral\n",
      "{\"sentiment\": \"neutral\"}, true label=neutral\n",
      "{\"sentiment\": \"positive\"}, true label=neutral\n",
      "{\"sentiment\": \"negative\"}, true label=negative\n",
      "{\"sentiment\": \"negative\"}, true label=negative\n",
      "{\"sentiment\": \"negative\"}, true label=negative\n",
      "{\"sentiment\": \"negative\"}, true label=negative\n",
      "{\"sentiment\": \"negative\"}, true label=negative\n"
     ]
    }
   ],
   "source": [
    "def generate_prompt(text):\n",
    "    return f\"\"\"\n",
    "Classify the sentiment of the following text as \"positive\" or \"neutral\" or \"negative\".\n",
    "Provide your output in json format. Do not provide any other answer.\n",
    "\n",
    "Text: {text}\n",
    "Sentiment:\"\"\".strip()\n",
    "\n",
    "for sentiment in [positive, neutral, negative]:\n",
    "    for i in range(5):\n",
    "        s = generate_prompt(news.iloc[sentiment[i]]['text'])\n",
    "        output = ollama.generate(model=\"llama3:instruct\", prompt=s, options={\"temperature\":0})\n",
    "        print(f\"{output['response']}, true label={news.iloc[sentiment[i]]['sentiment']}\")\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d2877501",
   "metadata": {},
   "source": [
    "### Few-shot prompt\n",
    "\n",
    "\n",
    "Adding specific examples of your desired output generally results in a more accurate, consistent output. This is called few-shot or __in-context learning__ through prompt design: instead of fine-tuning (and altering the pretrained neural network weights of) the model with new training examples, the model figures out how to perform well on that task simply by taking a few task-specific examples as input.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "id": "620a6363",
   "metadata": {},
   "outputs": [],
   "source": [
    "N = 3\n",
    "examples = positive[-N:] + neutral[-N:] + negative[-N:]\n",
    "examples = list(news.iloc[examples].itertuples(index=False))\n",
    "recs = {'positive': 'buy', 'neutral': 'hold', 'negative': 'sell'}\n",
    "line1 = 'Text in triple quotes:'\n",
    "line2 = 'Recommendation:'\n",
    "\n",
    "def generate_prompt(text, examples):\n",
    "    shots = \"\\n\\n\".join([f\"{line1} '''{t}'''\\n{line2} {recs[s]}\"\n",
    "                         for s,t in examples])\n",
    "    return f\"\"\"\n",
    "Here are {len(examples)} examples of making a recommendation based on the\n",
    "sentiment of the given text delimited with triple quotes.\n",
    "\n",
    "{shots}\n",
    "\n",
    "In one word only, provide a recommendation based on the sentiment of \n",
    "the following text delimited with triple quotes.  \n",
    "{line1} '''{text}'''\n",
    "{line2}\"\"\".strip()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "id": "5fdce01c",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Buy, true label=positive\n",
      "Buy, true label=positive\n",
      "Buy, true label=positive\n",
      "buy, true label=positive\n",
      "Buy, true label=positive\n",
      "Hold, true label=neutral\n",
      "Buy, true label=neutral\n",
      "Buy, true label=neutral\n",
      "Hold, true label=neutral\n",
      "Buy, true label=neutral\n",
      "Sell, true label=negative\n",
      "Sell, true label=negative\n",
      "Sell, true label=negative\n",
      "Sell, true label=negative\n",
      "Sell, true label=negative\n"
     ]
    }
   ],
   "source": [
    "for sentiment in [positive, neutral, negative]:\n",
    "    for i in range(5):\n",
    "        s = generate_prompt(news.iloc[sentiment[i]]['text'], examples)\n",
    "        output = ollama.generate(model=\"llama3:instruct\", prompt=s, options={\"temperature\":0})        \n",
    "        print(f\"{output['response']}, true label={news.iloc[sentiment[i]]['sentiment']}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "739a2190",
   "metadata": {},
   "source": [
    "Display text the few-shot prompt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "id": "71a95f2b",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Here are 9 examples of making a recommendation based on the\n",
      "sentiment of the given text delimited with triple quotes.\n",
      "\n",
      "Text in triple quotes: '''Danske Bank A-S DANSKE DC jumped 3.7 percent to 133.4 kroner , rebounding from yesterday s 3.5 percent slide .'''\n",
      "Recommendation: buy\n",
      "\n",
      "Text in triple quotes: '''Our superior customer centricity and expertise in digital services set us apart from our competitors .'''\n",
      "Recommendation: buy\n",
      "\n",
      "Text in triple quotes: '''The 2015 target for net sales has been set at EUR 1bn and the target for return on investment at over 20 % .'''\n",
      "Recommendation: buy\n",
      "\n",
      "Text in triple quotes: '''It holds 38 percent of Outokumpu 's shares and voting rights , but in 2001 lawmakers gave it permission to reduce the stake to 10 percent .'''\n",
      "Recommendation: hold\n",
      "\n",
      "Text in triple quotes: '''Mobile communication and wireless broadband provider Nokia Inc NYSE : NOK today set new financial targets and forecasts for Nokia and the mobile device industry and also for Nokia Siemens Networks and the mobile and fixed infrastructure and related services market .'''\n",
      "Recommendation: hold\n",
      "\n",
      "Text in triple quotes: '''Rinkuskiai 's beer sales fell by 6.5 per cent to 4.16 million litres , while Kauno Alus ' beer sales jumped by 6.9 per cent to 2.48 million litres .'''\n",
      "Recommendation: hold\n",
      "\n",
      "Text in triple quotes: '''Operating profit fell to EUR 35.4 mn from EUR 68.8 mn in 2007 , including vessel sales gain of EUR 12.3 mn .'''\n",
      "Recommendation: sell\n",
      "\n",
      "Text in triple quotes: '''Net sales of the Paper segment decreased to EUR 221.6 mn in the second quarter of 2009 from EUR 241.1 mn in the second quarter of 2008 , while operating profit excluding non-recurring items rose to EUR 8.0 mn from EUR 7.6 mn .'''\n",
      "Recommendation: sell\n",
      "\n",
      "Text in triple quotes: '''Sales in Finland decreased by 10.5 % in January , while sales outside Finland dropped by 17 % .'''\n",
      "Recommendation: sell\n",
      "\n",
      "In one word only, provide a recommendation based on the sentiment of \n",
      "the following text delimited with triple quotes.  \n",
      "Text in triple quotes: '''One of the challenges in the oil production in the North Sea is scale formation that can plug pipelines and halt production .'''\n",
      "Recommendation:\n"
     ]
    }
   ],
   "source": [
    "print(s)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "09c536c2",
   "metadata": {},
   "source": [
    "### Chain of thought\n",
    "\n",
    "Providing a series of prompts or\n",
    "questions helps guide its thinking, and can generate a more coherent\n",
    "and relevant response. \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "id": "fa0e54d5",
   "metadata": {
    "lines_to_next_cell": 1
   },
   "outputs": [],
   "source": [
    "def generate_prompt(text):\n",
    "    return f\"\"\"\n",
    "You are a financial market analyst who makes a recommendation based on the \n",
    "sentiment of the text given in triple quotes. Begin with:\n",
    "1. Write a summary of the text in about 10 words.\n",
    "2. Explain its impact on stock price.\n",
    "3. Provide your assumptions.\n",
    "4. Describe what might go wrong.\n",
    "5. Finally, give a recommendation to \"buy\" or \"hold\" or \"sell\".\n",
    "Make your recommendation funny.\n",
    "\n",
    "Text: '''{text}'''\n",
    "Recommendation:\"\"\".strip()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "id": "777bcc9a",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "==================================\n",
      "With the new production plant the company would increase its capacity to meet the expected increase in demand and would improve the use of raw materials and therefore increase the production profitability .\n",
      "----------------------------------\n",
      "Here's my analysis:\n",
      "\n",
      "**Summary:** Company boosts production, efficiency, and profits with new plant.\n",
      "\n",
      "**Impact on stock price:** This news is a big positive for the company's stock. Investors will likely respond favorably to increased capacity, reduced waste, and higher profit margins. I expect the stock price to rise 5-7% in the short term (next quarter).\n",
      "\n",
      "**Assumptions:**\n",
      "\n",
      "1. The new plant will be operational within the next 6 months.\n",
      "2. Demand for the company's products will indeed increase as expected.\n",
      "3. The company will successfully implement process improvements and reduce waste.\n",
      "\n",
      "**What might go wrong:**\n",
      "\n",
      "1. Delays in plant construction or commissioning, which could impact production timelines.\n",
      "2. Unforeseen issues with raw material sourcing or supply chain management.\n",
      "3. Failure to effectively manage increased demand, leading to inventory buildup or missed sales opportunities.\n",
      "\n",
      "**Recommendation:** \"Buy\" this stock and get ready to cash in on the sweet, sweet profits! Just don't forget to put your seatbelt on, because this ride is about to get bumpy (in a good way)!\n",
      "\n",
      "\n",
      "==================================\n",
      "According to Gran , the company has no plans to move all production to Russia , although that is where the company is growing .\n",
      "----------------------------------\n",
      "Here's my analysis:\n",
      "\n",
      "**Summary:** Company not moving all production to Russia, despite growth there.\n",
      "\n",
      "**Impact on stock price:** This news could be a slight positive for the company's stock price, as it suggests they're not abandoning their other markets. A small bump up in value might occur, but it won't be a game-changer.\n",
      "\n",
      "**Assumptions:** I assume that Gran is a credible source and that the company's growth in Russia isn't a major concern for investors.\n",
      "\n",
      "**What might go wrong:** If investors were heavily invested in the idea of the company moving production to Russia, they might be disappointed by this news. This could lead to some short-term selling pressure.\n",
      "\n",
      "**Recommendation:** \"Hold... but don't get too attached, because your significant other might still leave you for a Russian oligarch.\" In other words, it's not a bad stock to hold onto, but don't expect any major excitement or a dramatic price surge.\n",
      "\n",
      "\n",
      "==================================\n",
      "The international electronic industry company Elcoteq has laid off tens of employees from its Tallinn facility ; contrary to earlier layoffs the company contracted the ranks of its office workers , the daily Postimees reported .\n",
      "----------------------------------\n",
      "Here's my analysis:\n",
      "\n",
      "**Summary:** Elcoteq lays off employees in Tallinn, affecting office workers.\n",
      "\n",
      "**Impact on stock price:** This news is likely to have a negative impact on Elcoteq's stock price. Layoffs can be seen as a sign of financial struggles or restructuring, which may lead investors to reassess the company's prospects and sell their shares.\n",
      "\n",
      "**Assumptions:**\n",
      "\n",
      "* The layoffs are a one-time event and not a sign of deeper financial issues.\n",
      "* The affected employees were not critical to the company's operations.\n",
      "* The news will have a limited impact on Elcoteq's overall business performance.\n",
      "\n",
      "**What might go wrong:** If the layoffs are seen as a sign of broader financial struggles, investors may lose confidence in the company and sell their shares, leading to a further decline in stock price. Additionally, if the layoffs affect critical employees or disrupt operations, it could have long-term consequences for the company's competitiveness.\n",
      "\n",
      "**Recommendation:** \"Sell, sell, sell! (But not literally, please don't sell your Elcoteq shares... yet.)\" In all seriousness, I recommend a \"hold\" position until more information becomes available about the impact of these layoffs on the company's overall performance.\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "for sentiment in [positive, neutral, negative]:\n",
    "    for i in range(1):\n",
    "        s = generate_prompt(news.iloc[sentiment[i]]['text'])\n",
    "        print('==================================')\n",
    "        print(news.iloc[sentiment[i]]['text'])\n",
    "        print('----------------------------------')\n",
    "        output = ollama.generate(model=\"llama3:instruct\", prompt=s, options={\"temperature\":0})\n",
    "        print(f\"{output['response']}\")\n",
    "        print()\n",
    "        print()"
   ]
  }
 ],
 "metadata": {
  "jupytext": {
   "cell_metadata_filter": "-all",
   "main_language": "python",
   "notebook_metadata_filter": "-all"
  },
  "kernelspec": {
   "display_name": "env3.11",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}