Tried model using lanchain_together

[Eknathabhiram]

[ ] I checked the documentation and related resources and couldn't find an answer to my question.

Your Question
Getting Error: AttributeError: 'Generation' object has no attribute 'message'

Code Examples

from ragas.dataset_schema import SingleTurnSample
from ragas.metrics import NoiseSensitivity
from langchain_together import Together
import asyncio
from ragas.llms import LangchainLLMWrapper
from custom_embeddings import LocalEmbeddings

generator_llm = Together(
        model="mistralai/Mistral-7B-Instruct-v0.2",
        temperature=0.7,
        top_k=50,
        top_p=0.7,
        repetition_penalty=1,
    )
evaluator_llm = LangchainLLMWrapper(Together(
        model="mistralai/Mistral-7B-Instruct-v0.2"
    ))
evaluator_embedding = LocalEmbeddings(model_path="")


sample = SingleTurnSample(
    user_input="What is the Life Insurance Corporation of India (LIC) known for?",
    response="The Life Insurance Corporation of India (LIC) is the largest insurance company in India, known for its vast portfolio of investments. LIC contributes to the financial stability of the country.",
    reference="The Life Insurance Corporation of India (LIC) is the largest insurance company in India, established in 1956 through the nationalization of the insurance industry. It is known for managing a large portfolio of investments.",
    retrieved_contexts=[
        "The Life Insurance Corporation of India (LIC) was established in 1956 following the nationalization of the insurance industry in India.",
        "LIC is the largest insurance company in India, with a vast network of policyholders and huge investments.",
        "As the largest institutional investor in India, LIC manages substantial funds, contributing to the financial stability of the country.",
        "The Indian economy is one of the fastest-growing major economies in the world, thanks to sectors like finance, technology, manufacturing etc."
    ]
)

scorer = NoiseSensitivity(llm=evaluator_llm)

async def main():
    response = await scorer.single_turn_ascore(sample)
    return response
asyncio.run(main())

Additional context
Anything else you want to share with us?

[jjmachan]

hey @Eknathabhiram use ChatTogether - it only works with that - could you try it out and let me know if it works.

also can you run evaluate() with raise_exceptions=True? and share you're entire trace? I'll add a better error message so others don't get confused here 🙂

[Eknathabhiram]

raise LLMDidNotFinishException()

ragas.exceptions.LLMDidNotFinishException: The LLM generation was not completed. Please increase try increasing the max_tokens and try again.

I have used chatTogther but still I am facing this issue.., Tried max tokens 2096 too,

[jjmachan]

can you increase it to the max possible - 8k preferably