Practical_6_Parsing.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "orig_nbformat": 2,
    "deepnote_notebook_id": "ec333b79-136f-46c5-bb45-45b1ff889806",
    "deepnote_execution_queue": [],
    "colab": {
      "name": "Practical_6_Parsing.ipynb",
      "provenance": []
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00001-add6b402-508f-463b-8b60-3a331b31485f",
        "deepnote_cell_type": "markdown",
        "id": "RJVMkrDeKg1r"
      },
      "source": [
        "# Syntax (sentence structure) analysis, constiuency and dependency parsing"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00001-ed1aaf98-4180-4d08-a9cd-442a8d1f8c01",
        "deepnote_cell_type": "markdown",
        "id": "s3SMM4SaKg1u"
      },
      "source": [
        "## Some fundamentals"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00001-210c50f8-1ac4-4e69-990c-cbf85b51a793",
        "deepnote_cell_type": "markdown",
        "id": "aIzIVyc2Kg1u"
      },
      "source": [
        "In this notebook, we will move beyond n-grams and talk about **grammar**  - in particular, about **syntax** and **parsing**."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00002-9b302223-e870-4b50-be66-26e5c8405f98",
        "deepnote_cell_type": "markdown",
        "id": "649Fdxc8Kg1v"
      },
      "source": [
        "In linguistics, **grammar** is a set of rules to help us organise meaningful elements into sentences. \n",
        "\n",
        "It includes: \n",
        "- **Morphology** (how do we form words? \"Unforeseeable\" consists of 4 morphemes: \"un\", \"fore\", \"see\", \"able\" - and is constructed using the rules of morphology).\n",
        "- **Syntax** (word order, agreement)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00003-d453e159-e7a3-438b-9ef2-93f938448db8",
        "deepnote_cell_type": "markdown",
        "id": "iOIJ14FNKg1v"
      },
      "source": [
        "In NLP, a **grammar** is a set of **rules** to be used to generate grammarically correct sentences.\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00003-92dcb7fc-19db-4297-849f-3c70c3ee5004",
        "deepnote_cell_type": "markdown",
        "id": "MxqAqziAKg1v"
      },
      "source": [
        "How do we judge if a sentence is grammatically correct or not?\n",
        "Language has unlimited possibilities; we can endlessly embed sentences one into another using existing language constructions.\n",
        "Purpose of grammar: to give an explicit description of a language. "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00006-53dda89e-efb8-4449-a900-45af2a01de81",
        "deepnote_cell_type": "markdown",
        "id": "n2dvU-0PKg1w"
      },
      "source": [
        "## Context-free grammar and constituency "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00006-7356578f-edcf-4267-97bb-029247d1f991",
        "deepnote_cell_type": "markdown",
        "id": "cDbctrjKKg1w"
      },
      "source": [
        "One view on a lingustic structure is constituency/context-free grammar view."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00006-3d6f1b01-bc95-435d-9737-8d232e5ee2a0",
        "deepnote_cell_type": "markdown",
        "id": "h65f_zC5Kg1w"
      },
      "source": [
        "**Context-free grammar (CFG)** is a formal system for modelling constituent structure. It consists of:\n",
        "- **rules** (also known as **productions**) expessing how language symbols can be grouped and ordered together, and \n",
        "- **lexicon** of words and symbols\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "tags": [],
        "cell_id": "00007-eda30276-2266-4d94-adad-d0d35da1cd76",
        "output_cleared": false,
        "source_hash": "4b33b82f",
        "execution_millis": 14906,
        "deepnote_to_be_reexecuted": false,
        "execution_start": 1611663672652,
        "deepnote_cell_type": "code",
        "id": "pfQmUbsCKg1w"
      },
      "source": [
        "!pip install nltk \n",
        "import nltk\n",
        "from nltk import CFG"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00009-2cb333a5-e318-4e0e-af1f-6560c9f9d3dd",
        "deepnote_cell_type": "markdown",
        "id": "68j9A4hQKg1x"
      },
      "source": [
        "Let's define a grammar for our own little language.\n",
        "(From NLTK book):\n",
        "By convention, the left-hand-side of the first rule is the **start-symbol** of the grammar, typically S, and all well-formed trees must have this symbol as their root label. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "tags": [],
        "cell_id": "00011-26d59a04-3b8f-405f-952a-06c59c6b4b26",
        "output_cleared": false,
        "source_hash": "96307baf",
        "execution_start": 1603376286719,
        "execution_millis": 8158,
        "deepnote_cell_type": "code",
        "id": "EMIpirC1Kg1x"
      },
      "source": [
        "!pip install nltk"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "tags": [],
        "cell_id": "00009-e0928f1e-3030-4795-89cd-2b454c8f16f1",
        "output_cleared": false,
        "source_hash": "359f346a",
        "execution_millis": 0,
        "deepnote_to_be_reexecuted": false,
        "execution_start": 1611663734254,
        "deepnote_cell_type": "code",
        "id": "Ne6d8l0nKg1y"
      },
      "source": [
        "import nltk\n",
        "grammar = nltk.CFG.fromstring(\"\"\"\n",
        "S -> NP VP\n",
        "VP -> V NP | V NP PP\n",
        "PP -> P NP\n",
        "V -> \"saw\" | \"ate\" | \"walked\"\n",
        "NP -> \"John\" | \"Mary\" | \"Bob\" | Det N | Det N PP\n",
        "Det -> \"a\" | \"an\" | \"the\" | \"my\"\n",
        "N -> \"man\" | \"dog\" | \"cat\" | \"telescope\" | \"park\"\n",
        "P -> \"in\" | \"on\" | \"by\" | \"with\"\n",
        "\"\"\")"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00011-df506b1d-6975-477f-bf14-6d1fe24e5e31",
        "deepnote_cell_type": "markdown",
        "id": "CQdLQMb9Kg1y"
      },
      "source": [
        "Remember what the codes mean?\n",
        "- S - sentence\n",
        "- NP - noun phrase (a dog)\n",
        "- VP - verb phrase (saw a park)\n",
        "- PP - prepositional phrase (with a telescope)\n",
        "- Det - determiner (the)\n",
        "- N - noun\n",
        "- V - verb\n",
        "- P - preposition (in, on)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00012-94554cd7-7e0e-4968-8a9e-16ee57d87ded",
        "deepnote_cell_type": "markdown",
        "id": "gUjno1gLKg1y"
      },
      "source": [
        "What this grammar means is that the language it defines must have a NP and a VP in any sentence; a VP must consist of a V and a NP (and 0 or more PP); and so on. It also describes the language vocabulary: there are NPs: John, Mary etc; Vs - \"saw\", \"ate\", \"walked\". And so on. "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00013-b1fe64a7-6558-4f79-8374-cc065a89ce96",
        "deepnote_cell_type": "markdown",
        "id": "sZftCvd7Kg1z"
      },
      "source": [
        "Now we can take any sentence from that (imaginary) language a grammar for which we have just defined (making sure a sentence consists of the words available in the defined vocabulary and is built according to the defined rules), and a parser - a special program designed to analyse a phrase structure - will tell us what the sentence consists of. \n",
        "This is referred to as constituency."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00006-d9a20ed8-afce-4670-b85c-15af32552732",
        "deepnote_cell_type": "markdown",
        "id": "tHajxrk1Kg1z"
      },
      "source": [
        "**Constituency**: groups of words behaving as single units (constituents). \n",
        "Example: noun phrase (a sequence of words surrounding a noun) - \"two students from Luxembourg\",  \"we\", \"a nice cafe like Dalmat\"\n",
        "\n",
        "They all can appear before a verb (like a noun): \n",
        "\"two students from Luxembourg arrived\", \"we sat\", \"a nice cafe like Dalmat sells\"\n",
        "\n",
        "NOTE: not all separate words in these noun phrases can be placed before a verb: we usually mark non-grammatical phrases with an asterisk (*).\n",
        "\n",
        "- \"from arrived*\"\n",
        "- \"like sells*\"\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00008-3e61ab8b-f62d-4836-b2fb-81080a416079",
        "deepnote_cell_type": "markdown",
        "id": "J-mPpPIfKg1z"
      },
      "source": [
        "## Parsing"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00006-6fc3ac37-9071-4997-aac1-0527f6cbf91e",
        "deepnote_cell_type": "markdown",
        "id": "HrFrfFiKKg1z"
      },
      "source": [
        "**Parsing (syntax analysis)** is breaking down a text into its component parts with an explanation of the form, function, and syntactic relationship of each part. Two types of parsing we will discuss are:\n",
        "- constituency (phrase-structure) parsing\n",
        "- dependency parsing"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00010-38fae2f3-44bb-40f9-a41c-ccdd23ff9123",
        "deepnote_cell_type": "markdown",
        "id": "_wNFRJZ5Kg10"
      },
      "source": [
        "### Constituency (phrase-structure) parsing"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00013-2f363335-6675-48be-ac8c-338c09aac708",
        "deepnote_cell_type": "markdown",
        "id": "RnymXG6uKg10"
      },
      "source": [
        "Once we have agreed on the grammar we can parse (break down into components and analyse structure of) sentences. "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00028-dee6e7bd-4a9e-4565-8459-151355395c3b",
        "deepnote_cell_type": "markdown",
        "id": "GT7Nga-fKg10"
      },
      "source": [
        "Some commonly used **parsing algorithms**:\n",
        "- Recursive Descent Parsing \n",
        "- Shift-reduce Parsing"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00030-82876b21-669f-4df0-974d-6d4ce1faeb89",
        "deepnote_cell_type": "markdown",
        "id": "OOrNQqAdKg10"
      },
      "source": [
        "### Recusrive Descent Parsing "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00031-9016c2da-d62f-455f-8317-8955bb3b2d10",
        "deepnote_cell_type": "markdown",
        "id": "LJ8VcoNrKg10"
      },
      "source": [
        "Please read a brief description in https://www.nltk.org/book/ch08.html part 4.1 and run the code below to see it working."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "tags": [],
        "cell_id": "00032-f47b5ff9-080f-4be0-8740-c536aead2962",
        "output_cleared": false,
        "source_hash": "1261ea",
        "execution_millis": 0,
        "deepnote_to_be_reexecuted": false,
        "execution_start": 1611663738550,
        "deepnote_cell_type": "code",
        "id": "Uyow-8shKg10"
      },
      "source": [
        "sent = \"Mary saw Bob\".split()\n",
        "rd_parser = nltk.RecursiveDescentParser(grammar)\n",
        "for tree in rd_parser.parse(sent):\n",
        "    print(tree)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00021-ff4126c9-fec2-4f36-b309-6fc2904545d1",
        "deepnote_cell_type": "markdown",
        "id": "zsU1HFVuKg11"
      },
      "source": [
        "The parser tells us that this sentence consists of a NP (noun phrase) \"Mary\" and a VP (verb phrase) (\"saw Bob\"), and the latter consists of a V (verb) (\"saw\") and an NP (noun phrase) \"Bob\"."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00033-5701931e-cd7a-45fd-85a1-9b46053e9f53",
        "deepnote_cell_type": "markdown",
        "id": "tmH74vbnKg11"
      },
      "source": [
        "(1) **CODE IT**: Change the code below to create a little grammar of your own (include 3-5 rules and some vocabulary, like in the example earlier in the notebook) in any language and parse a sentence of your choice."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "tags": [],
        "cell_id": "00026-125d33b2-835f-44a7-9bfc-a727f0ff3be5",
        "output_cleared": false,
        "source_hash": "511ce54f",
        "execution_millis": 284738,
        "deepnote_to_be_reexecuted": false,
        "execution_start": 1611663741650,
        "deepnote_cell_type": "code",
        "id": "821B0PHXKg11"
      },
      "source": [
        "!pip install nltk\n",
        "import nltk\n",
        "grammar = nltk.CFG.fromstring(\"\"\"\n",
        "S -> PS | BS | RS\n",
        "PS -> RS P | P\n",
        "BS -> P RS | RS P RS\n",
        "RS -> N| RS N\n",
        "P -> VP | NP\n",
        "VP -> \"atti\" | \"atar\" | \"gittim\"\n",
        "NP -> \"yesildir\" | \"iyidir\"\n",
        "N -> NL N | \"ali\" | \"topu\" | \"renk\"\n",
        "NL -> \"en\" | \"buyuk\" | \"guzel\"\n",
        "\"\"\")\n",
        "sent = \"ali topu atti\".split()\n",
        "rd_parser = nltk.RecursiveDescentParser(grammar)\n",
        "for tree in rd_parser.parse(sent):\n",
        "    print(tree)\n",
        "\n",
        "print(\"end\")\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "tags": [],
        "cell_id": "00033-40f42fbb-9e78-403d-81a4-538897593a17",
        "output_cleared": false,
        "source_hash": "de0e81ee",
        "execution_millis": 5,
        "execution_start": 1603203213483,
        "deepnote_cell_type": "code",
        "id": "3MLT3xa6Kg12"
      },
      "source": [
        "sent = \"Mary saw Bob\".split()#YOUR SENTENCE (don't forget that the input must be a list of strings)\n",
        "rd_parser = nltk.RecursiveDescentParser(grammar)\n",
        "for tree in rd_parser.parse(sent):\n",
        "    print(tree)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00035-f18dc27c-562d-4429-ab0b-2cd8f5f5201a",
        "deepnote_cell_type": "markdown",
        "id": "OojYxOVfKg12"
      },
      "source": [
        "### Shift Reduce Parser"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00036-29ff8135-fd4d-4cc1-a50f-5209a81bbd90",
        "deepnote_cell_type": "markdown",
        "id": "7tUJfpt5Kg12"
      },
      "source": [
        "Read https://www.nltk.org/book/ch08.html part 4.2. and run the code below to see the difference:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "tags": [],
        "cell_id": "00036-a691d835-fce5-496c-aa5c-0b7ae8fa4622",
        "output_cleared": false,
        "source_hash": "fe747aaf",
        "execution_millis": 6,
        "execution_start": 1603114548556,
        "deepnote_cell_type": "code",
        "id": "_77-atmwKg12"
      },
      "source": [
        "sr_parser = nltk.ShiftReduceParser(grammar1)\n",
        "sent = 'Mary saw a dog'.split()\n",
        "for tree in sr_parser.parse(sent):\n",
        "    print(tree)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00018-2046861a-58da-4364-8884-efca22d57caf",
        "deepnote_cell_type": "markdown",
        "id": "jxHzkCSJKg13"
      },
      "source": [
        "**IF YOU FANCY**: read about some other types of parsers (parts 4.3 and 4.4)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00002-9911ad44-4102-4114-b14e-f54aa8db57b1",
        "deepnote_cell_type": "markdown",
        "id": "dLRswd0rKg13"
      },
      "source": [
        "## Dependency Parsing"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00036-c42e8ec0-1a96-4a00-9bdc-5d6efd59e505",
        "deepnote_cell_type": "markdown",
        "id": "C6L2QagdKg13"
      },
      "source": [
        "While we have talked about **phrase structure parsing** (concerned with how words and sequences combine to form constituents), there is another view on sentence structure - and another type of parsing -  'dependency parsing'. \n",
        "**Dependency parsing** is a distinct and complementary approach focusing on how words relate to other words. \n",
        "**Dependency** - a binary assymetric relation between a **head** and its **dependents**."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00032-b2775474-65a7-4505-b0fa-30fb43cd2b72",
        "deepnote_cell_type": "markdown",
        "id": "mUb9hEWiKg13"
      },
      "source": [
        "**Dependency representation**  -  a labeled directed graph:\n",
        "- nodes: lexical items\n",
        "- labeled arcs: dependency relations from heads to dependents\n",
        "\n",
        "Let's build a dependency representation using SpaCY. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "tags": [],
        "cell_id": "00040-8c774437-a201-4874-89fc-88caf99a37a3",
        "output_cleared": false,
        "source_hash": "a9fe7923",
        "execution_start": 1603101318681,
        "execution_millis": 1904,
        "deepnote_cell_type": "code",
        "id": "Or2njJMvKg13"
      },
      "source": [
        "!pip install spacy\n",
        "import spacy\n",
        "from spacy import displacy"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "tags": [],
        "cell_id": "00039-6349fddf-cee5-4bb5-a90e-4b181b61ad19",
        "output_cleared": false,
        "source_hash": "86c76921",
        "execution_start": 1603101334325,
        "execution_millis": 849,
        "deepnote_cell_type": "code",
        "id": "CoQzpsRiKg14"
      },
      "source": [
        "\n",
        "import en_core_web_sm\n",
        "nlp = en_core_web_sm.load()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "tags": [],
        "cell_id": "00039-1f096d12-9ed9-4233-bcd0-025c85045320",
        "output_cleared": false,
        "source_hash": "e779abc3",
        "execution_start": 1603101347599,
        "execution_millis": 1,
        "deepnote_cell_type": "code",
        "id": "T34C1QZfKg14"
      },
      "source": [
        "from spacy import displacy"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "tags": [],
        "cell_id": "00039-cdd154b4-f932-4db7-9e52-41970815a06d",
        "output_cleared": false,
        "source_hash": "a5196d8c",
        "execution_start": 1603101360241,
        "execution_millis": 14,
        "deepnote_cell_type": "code",
        "id": "hSQWjMZ5Kg14"
      },
      "source": [
        "doc = nlp(\"Mary saw a star with a telescope\")\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "tags": [],
        "cell_id": "00040-78bd428c-72d7-4d19-accd-3e6707c08dd7",
        "output_cleared": false,
        "source_hash": "6e1c9070",
        "execution_start": 1603101369880,
        "execution_millis": 6,
        "deepnote_cell_type": "code",
        "id": "IHqVlbjbKg14"
      },
      "source": [
        "displacy.render(doc, style='dep',jupyter=True)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00037-9267dd96-d742-4f1e-a379-1d748d8c2090",
        "deepnote_cell_type": "markdown",
        "id": "LcDfgMQSKg15"
      },
      "source": [
        "List of the dependency labels used by Spacy: https://github.com/clir/clearnlp-guidelines/blob/master/md/specifications/dependency_labels.md"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00041-0da25e9d-6377-479e-9ccb-2452bb12fa4e",
        "deepnote_cell_type": "markdown",
        "id": "X5d1QNpuKg15"
      },
      "source": [
        "### Ambiguity"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00042-edae97a1-9089-40f2-b9d7-7532fae4e3cb",
        "deepnote_cell_type": "markdown",
        "id": "d6NHhBVVKg15"
      },
      "source": [
        "There are two types of ambiguity:\n",
        "- syntactic\n",
        "- attachement"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00043-d81dfa7a-de32-47a9-8bb6-cad609e98cf4",
        "deepnote_cell_type": "markdown",
        "id": "iU5WxyQ6Kg15"
      },
      "source": [
        "**Syntactic ambigutity** - where the doubt is about the syntactic structure of the sentence (a sentence could be parsed in many syntactical forms). "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00040-35b128df-b3a9-4183-8812-fde4f1dfca05",
        "deepnote_cell_type": "markdown",
        "id": "258UaqNFKg16"
      },
      "source": [
        "(2) **CODE IT:** \n",
        "Write down two paraphrases of \"I like fresh bread and eggs\" where each paraphrase unambiguously expresses one of the meanings."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "tags": [],
        "cell_id": "00040-4a9dfd45-9082-458b-b60c-eb6a6877946d",
        "output_cleared": false,
        "source_hash": "e50c52",
        "execution_millis": 12,
        "execution_start": 1603101931321,
        "deepnote_cell_type": "code",
        "id": "SaoK9_1_Kg16"
      },
      "source": [
        "doc = nlp(\"I like fresh bread and eggs\")\n",
        "displacy.render(doc, style='dep',jupyter=True)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "tags": [],
        "cell_id": "00046-368a6a4d-2c88-457e-aa09-fab6a7796036",
        "deepnote_cell_type": "code",
        "id": "fIdBbjz1Kg16"
      },
      "source": [
        "#you code for paraphrase one"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "tags": [],
        "cell_id": "00047-ad72f6cb-53c2-48a9-812f-9fd79384b6bb",
        "deepnote_cell_type": "code",
        "id": "F4SR9BKnKg16"
      },
      "source": [
        "#you code for paraphrase two"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00049-eb72c2fb-7061-4b0a-8db9-447c15031da8",
        "deepnote_cell_type": "markdown",
        "id": "UCrwMjISKg16"
      },
      "source": [
        "# Ambiguity"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00050-ba4a8192-e210-461b-b2c4-9ea8221ad1e9",
        "deepnote_cell_type": "markdown",
        "id": "lagFv_zVKg17"
      },
      "source": [
        "## Global Ambiguity"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00049-c8f7d409-6dad-4b56-8f59-3e977582d4b8",
        "deepnote_cell_type": "markdown",
        "id": "CuEtg4OpKg17"
      },
      "source": [
        "**Attachment ambiguity**"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00051-285caa2d-ba48-4596-8981-6ded8793abc5",
        "deepnote_cell_type": "markdown",
        "id": "DDAnKbf3Kg17"
      },
      "source": [
        "It is not clear how exactly a phrase or a clause is attached to another part of the utterance (a common situaion: there are more than two PPs (prepositional phrases) is a sentence).\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00053-d87a6e40-d511-44da-b6bb-e04831ff5692",
        "deepnote_cell_type": "markdown",
        "id": "F-ohUJCBKg17"
      },
      "source": [
        "Example: ![picture](https://pbs.twimg.com/media/Db_IWPWUQAAf0Nq?format=jpg&name=small)\n",
        "\n",
        "(one of the comments to that tweet: \"Why didn’t someone just show this poor woman a dog?” )"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00054-e5f03018-4071-4b85-9182-d229c35abcf5",
        "deepnote_cell_type": "markdown",
        "id": "-MQlKVauKg17"
      },
      "source": [
        "https://literalminded.files.wordpress.com/2018/04/neverdidntsmile.png"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00055-2c4b282a-af47-485f-9a90-43bccc26cbed",
        "deepnote_cell_type": "markdown",
        "id": "1acT4NKvKg18"
      },
      "source": [
        "Source: https://literalminded.wordpress.com/category/semantics/ambiguity/attachment-ambiguity/"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00042-4ef517d0-bd66-4dec-a296-ebd878b3bbb0",
        "deepnote_cell_type": "markdown",
        "id": "_FGIj6aVKg18"
      },
      "source": [
        "## Local Ambiguity"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00055-c4b43e4b-7e41-4735-bf2b-0926dda09d1c",
        "deepnote_cell_type": "markdown",
        "id": "nPPhZctmKg18"
      },
      "source": [
        "Try reading the following senrences:\n",
        "\n",
        "1. Fat people eat accumulates.\n",
        "\n",
        "2. The cotton clothing is usually made of grows in Mississippi.\n",
        "\n",
        "3. Until the police arrest the drug dealers control the street.\n",
        "\n",
        "4. The man who hunts ducks out on weekends.\n",
        "\n",
        "5. The girl told the story cried.\n",
        "\n",
        "6. I convinced her children are noisy.\n",
        "\n",
        "7. The horse raced past the barn fell.\n",
        "\n",
        "8. She told me a little white lie will come back to haunt me.\n",
        "\n",
        "9. That Jill is never here hurts.\n",
        "\n",
        "10. The old man the boat.\n",
        "\n",
        "![What?](https://media.giphy.com/media/TigmthadB7D5XlGhhJ/giphy.gif)\n",
        "\n",
        "\n",
        "\n",
        "### Now Read them again\n",
        "\n",
        "1. **(The)** fat **(that)** people eat accumulates **(in their bodies)**.\n",
        "2. The cotton **(that)** clothing is usually made of grows in Mississippi.\n",
        "3. Until the police **(make the)** arrest, the drug dealers control the street.\n",
        "4. The man, who hunts **(animals)**, ducks out on weekends.\n",
        "5. The girl **(who was)** told the story, cried.\n",
        "6. I convinced her **(that)** children are noisy.\n",
        "7. The horse **(which was)** raced past the barn, fell **(down)**.\n",
        "8. She told me **(that)** a little white lie will come back to haunt me.\n",
        "9. **(The fact)** that Jill is never here hurts **(me)**.\n",
        "10. The old **(people)** man the boat.\n",
        "  "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00058-b2fd1f46-cbb5-417b-abbc-075f2a5fc828",
        "deepnote_cell_type": "markdown",
        "id": "vy_o7sLSKg18"
      },
      "source": [
        "## Garden Path Sentences : \n",
        "The sentences you saw above are called garden path sentences, in these sentences your mind (usually) starts to read and parse the sentence, and in the middle of the sentence you encounter a word that your current reading cannot continue anymore and reaches a dead-end.\n",
        "\n",
        "Most statistically trained parsers are unable to **correctly** parse garden path sentences.\n",
        "\n",
        "Consider the example:\n",
        "\n",
        "Fat people eat accumulates\n",
        "\n",
        "1. (NP (NNP Fat) (NNS people))  \n",
        "2. (S (NP (JJ fat) (NNS people)) (VP (VBP eat)))\n",
        "3. DEAD_END when seeing accumulates as the next word\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00058-0511deea-f2c0-49be-87c6-d1018d56159c",
        "deepnote_cell_type": "markdown",
        "id": "MgF8FK0hKg19"
      },
      "source": [
        "![picture] (https://64.media.tumblr.com/6de31f336494b38be8a55359182e121d/tumblr_inline_nnn3faqhIe1rorzyc_1280.png)\n",
        "Source: https://allthingslinguistic.com/post/118396390957/garden-path-sentence-shirts-a-story"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "cell_id": "00058-2ba84258-84be-476d-8327-5f9e544fa93e",
        "deepnote_cell_type": "markdown",
        "id": "aESQfjZgKg19"
      },
      "source": [
        "(3) **OBSERVE AND REFLECT** Use this [Demo](https://parser.kitaev.io/) to construct the constituency parse tree of the two sentences:\n",
        "\n",
        "'The old man the boat' and 'the old people man the boat'\n",
        "\n",
        "    1. Reflect on how the parser interpret the first sentence? \n",
        "    2. Considering the POS tag of the tokens in the sentence, which word(s) is(are) responsible for this local ambiguity and why?\n",
        "    3. Can you find another example in the above list in which the sentence can be disambguited knowing the POS tag of the word? \n",
        "\n"
      ]
    }
  ]
}