Skip to content

Commit

Permalink
complete the abstract in the paper (2023.genbench-1.11) (#3952)
Browse files Browse the repository at this point in the history
  • Loading branch information
xiulinyang authored Oct 11, 2024
1 parent e56eb21 commit b7dda3c
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion data/xml/2023.genbench.xml
Original file line number Diff line number Diff line change
Expand Up @@ -151,7 +151,7 @@
<author><first>Amélie</first><last>Reymond</last><affiliation>University of Washington</affiliation></author>
<author><first>Shane</first><last>Steinert-Threlkeld</last><affiliation>University of Washington</affiliation></author>
<pages>143-151</pages>
<abstract>Language models achieve remarkable results on a variety of tasks, yet still struggle on compositional generalisation benchmarks. The majority of these benchmarks evaluate performance in English only, leaving us with the question of whether these results generalise to other languages. As an initial step to answering this question, we introduce mSCAN, a multilingual adaptation of the SCAN dataset. It was produced by a rule-based translation, developed in cooperation with native speakers. We then showcase this novel dataset on some in-context learning experiments, and GPT3.5 and the multilingual large language model BLOOM</abstract>
<abstract>Language models achieve remarkable results on a variety of tasks, yet still struggle on compositional generalisation benchmarks. The majority of these benchmarks evaluate performance in English only, leaving us with the question of whether these results generalise to other languages. As an initial step to answering this question, we introduce mSCAN, a multilingual adaptation of the SCAN dataset. It was produced by a rule-based translation, developed in cooperation with native speakers. We then showcase this novel dataset on some in-context learning experiments, and GPT3.5 and the multilingual large language model BLOOM as well as gpt3.5-turbo.</abstract>
<url hash="d36cfbec">2023.genbench-1.11</url>
<bibkey>reymond-steinert-threlkeld-2023-mscan</bibkey>
<doi>10.18653/v1/2023.genbench-1.11</doi>
Expand Down

0 comments on commit b7dda3c

Please sign in to comment.