问题咨询：关于ceval数据集以及新增类似ceval数据集的问题

[niexufei]

描述该错误

1、data/ceval/包含formal_ceval和release_ceval两个文件夹，分别都是什么用途？评测internLM-7b时，使用的是formal_ceval下的数据。使用release_ceval的数据在数据集合配置glm文件夹下，是评测GLM的模型的时候使用release_ceval数据吗？
2、data/ceval/formal_ceval下分别有dev，test，val三个文件夹，CEvalDataset类将这些数据load到了下面变量：
return DatasetDict({ 'val': val_dataset, 'dev': dev_dataset, 'test': test_dataset })
这块后续是如何评测的呢？测试集、开发集，验证集这三类数据如何使用？
按照ceval官方的说法，测试机没有答案，需要将测试结果提交他们网站才能拿到得分，opencompass是如何操作的呢？
3、opencompass是否支持问答题的评测，如果支持，哪个数据集是问答题的例子？

建议opencompass开个论坛，我们可以到论坛里去交流相关技术问题。

感谢~

环境信息

无。

其他信息

No response

[Leymore]

1. formatl_ceval is the released version of C-Eval, the same as the one downloaded from C-Eval official . release_ceval, on the other hand, is legacy data, which should be deleted. Configuration in the glm folder is out-of-date and buggy, please use formatl_ceval in all scenarios.
2. dev is used as few shot examples. Scores of 5-shot dev and val is reported on the opencompass official site. test is used only for inference mode. As the answer to val is not released, the trial to evaluate on test will get 0s.
3. All datasets which have qa in the name is probably a question-answer datasets. You may take TriviaQA & NatualQuestions as examples.
4. Thank you for your suggestion on the discourse thing. We will look into it.

[gaotongxiao]

We just activated the Discussion panel for OpenCompass. You are welcome to discuss general topics then.