CZ4045 Natural Language Processing - Assignment 1

Assignment

Requirements of this assignment can be found here

Report for this assignment can be found here

Files

Files are labelled according to the portion in the Report.

Installation manual

All python libraries need for the individual sections will be installed in the respective .ipynb file.

For 1.2, 1.3, 1.4, 1.5, 3 - only python libraries are needed.

For 2. Extraction of indicative adjective phrase.ipynb, a stanford core server is required to run in the background.

Stanford Core server

To download and install stanford core nlp

1. Please visit https://stanfordnlp.github.io/CoreNLP/download.html to download stanford core nlp and unzip the downloaded file
2. Download and install Java
   For windows: https://www.java.com/en/download/help/windows_manual_download.html
   For macs: https://www.java.com/en/download/help/mac_install.html
   For linux: https://java.com/en/download/help/linux_x64_install.html
   

To run background server

1. open a terminal at the directory of stanfore core nlp file
2. run the following command in terminal

java -mx4g -cp "*" edu.stanford.nlp.pipeline.StanfordCoreNLPServer -preload tokenize,ssplit,pos,lemma,ner,parse,depparse -status_port 9000 -port 9000 -timeout 100000000000000000000000 &

With this, the server should be started and you should be able to see [main] INFO CoreNLP - StanfordCoreNLPServer listening at /0:0:0:0:0:0:0:0:9000

After this, 2. Extraction of indicative adjective phrase.ipynb can be run.

Results for 3.4 Application

Python dependencies

1. VADER
2. Natural Language Toolkit
3. spaCy
4. pandas
5. Natural Language Toolkit
6. BeautifulSoup
7. numpy
8. requests
9. docx2text
10. language_tool_python
11. CNA Post Documents