Skip to content

Issues: explodinggradients/ragas

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

248 Open 591 Closed
248 Open 591 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

SemanticSimilarity metric does not use specified embeddings model bug Something isn't working module-metrics this is part of metrics module
#1586 opened Oct 26, 2024 by ahgraber
RAGAs compatibility with GISKARD bug Something isn't working
#1582 opened Oct 25, 2024 by aranchod490
[R-310] Test set generation improvements enhancement New feature or request module-testsetgen Module testset generation
#1577 opened Oct 25, 2024 by shahules786
@shahules786
Testset generation issue with ollama, gets stuck at Generating common concepts bug Something isn't working module-testsetgen Module testset generation
#1576 opened Oct 24, 2024 by rajuptvs
2
Tried model using lanchain_together answered 🤖 The question has been answered. Will be closed automatically if no new comments bug Something isn't working question Further information is requested
#1573 opened Oct 24, 2024 by Eknathabhiram
3
Poor indicators for the assessment of long texts module-metrics this is part of metrics module question Further information is requested
#1572 opened Oct 24, 2024 by Superyeahh
Testset generation feedback module-testsetgen Module testset generation question Further information is requested
#1568 opened Oct 24, 2024 by tuan3w
@shahules786
1
[Bug] AgentGoalAccuracyWithoutReference is not implementing _ascore method bug Something isn't working
#1565 opened Oct 23, 2024 by luqmansen
1 task done
1
[R-309] add llamaindex support for evaluation and testset generatin in v0.2 linear Created by Linear-GitHub Sync question Further information is requested
#1557 opened Oct 23, 2024 by vikrantdeshpande09876 v.27
3
Add BERTScore as potential "non-LLM" metric / MetricWithEmbedding for context-recall and context-precision enhancement New feature or request module-metrics this is part of metrics module
#1555 opened Oct 22, 2024 by ahgraber
4
[R-304] support more providers for is_finished() logic bug Something isn't working
#1548 opened Oct 22, 2024 by jjmachan
1
[BUG] Issue with the Test Generation bug Something isn't working module-testsetgen Module testset generation
#1546 opened Oct 21, 2024 by sarahzouinina
2
ValidationError bug Something isn't working question Further information is requested
#1545 opened Oct 21, 2024 by amin-kh96
15
ValidationError: 1 validation error for ChatMessage role Input should be a valid string [type=string_type, input_value=None, input_type=NoneType] bug Something isn't working question Further information is requested
#1544 opened Oct 21, 2024 by miaoxiaolv
5
How to check my embedding llama3 model is running for evaluate? Is Wrapper necessary? question Further information is requested
#1540 opened Oct 19, 2024 by RyanTree-HS
2
[R-301] make llm and embeddigns required
#1537 opened Oct 19, 2024 by jjmachan v0.3
1
@jjmachan
4
[R-302] Support function-calling / json mode / structured generation for testset generation enhancement New feature or request linear Created by Linear-GitHub Sync module-testsetgen Module testset generation
#1532 opened Oct 18, 2024 by ahgraber v.26
@jjmachan
1
[R-298] Support Passing Multiple Evaluator Models to reduce bias enhancement New feature or request module-metrics this is part of metrics module
#1528 opened Oct 18, 2024 by bdytx5 v0.3
1
[R-305] Incorrect API key provided Azure OpenAI for TestsetGenerator bug Something isn't working linear Created by Linear-GitHub Sync module-testsetgen Module testset generation question Further information is requested
#1526 opened Oct 17, 2024 by llvll0hsen v.26
@jjmachan
11
periodically save current progress and restart from last save enhancement New feature or request
#1522 opened Oct 16, 2024 by trevorbowen
8
support OpenAI's new batch API interface enhancement New feature or request
#1521 opened Oct 16, 2024 by trevorbowen
evaluate() needs optional delay to prevent rate limiting by API answered 🤖 The question has been answered. Will be closed automatically if no new comments enhancement New feature or request
#1519 opened Oct 16, 2024 by bdytx5
6
[R-306] Frequent openai.APIConnectionError when running testset generator bug Something isn't working linear Created by Linear-GitHub Sync module-testsetgen Module testset generation
#1512 opened Oct 15, 2024 by alexander-zuev v.26
@jjmachan
11
using the embeddings module-metrics this is part of metrics module question Further information is requested
#1509 opened Oct 15, 2024 by amin-kh96
9
Add cost tracking to dataset generator and allow dataset size control enhancement New feature or request module-testsetgen Module testset generation
#1506 opened Oct 15, 2024 by alexander-zuev
2
ProTip! Adding no:label will show everything without a label.