The aim of this project is to create a challenge dataset for the semantic role labelling task.
- Semantic role labeling (SRL) is a task concerning the classification of the linguistic phenomena.
- The aim of such classification is to gain more information on a given text input. SRL enables one to obtain meaningful inferences by representing a text in a form: who did what to whom, how, with what, when and where.
- To achieve such representation the system identifies predicates together with their arguments that belong to a specified thematic roles.
- Syntactic variation
- Statement vs question
- Active vs passive
- Marked vs unmarked
- Lexicalizations of arguments
- frequent vs infrequent words
- Proper names
- Negation
- structured-prediction-srl-bert
- structured-prediction-srl
The models come from AllenNLP project https://github.com/allenai/allennlp-models
- Execute run_evaluation_results.py to see the summarized performance of all of the tests
- Go to challenge_tests/sentences/ to see the challenge sets per each test
- Go to run_srl_challenge/ to investigate how the tests were created
- Go to outcome/ to see the outcome per each test and each specific example