Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[R-310] Test set generation improvements #1577

Open
shahules786 opened this issue Oct 25, 2024 · 0 comments
Open

[R-310] Test set generation improvements #1577

shahules786 opened this issue Oct 25, 2024 · 0 comments
Assignees
Labels
enhancement New feature or request module-testsetgen Module testset generation

Comments

@shahules786
Copy link
Member

shahules786 commented Oct 25, 2024

For Ragas 0.2, we released our third iteration of synthetic test generation for RAG. While developing this new approach we have kept some important feedback that we gathered from earlier versions

  1. Ability to generalize over more data formats and domains.
  2. Ability to customize and create scenarios related to one's own domain.
  3. Ability to persist and resample Q&A from same list of documents w/o redoing preprocessing
  4. Lower cost and faster iteration.

We have already noted some feedback from the community and we want to welcome more, hence this discussion thread. Please feel free share your thoughts and queries on the same. This will help us greatly improve the feature in coming weeks.

R-310

Known issues

Quality enhancement

Documentation improvements

@shahules786 shahules786 added the enhancement New feature or request label Oct 25, 2024
@shahules786 shahules786 self-assigned this Oct 25, 2024
@shahules786 shahules786 changed the title Test set generation improvements [R-310] Test set generation improvements Oct 25, 2024
@shahules786 shahules786 pinned this issue Oct 25, 2024
@dosubot dosubot bot added the module-testsetgen Module testset generation label Oct 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request module-testsetgen Module testset generation
Projects
None yet
Development

No branches or pull requests

1 participant