Getting an LLMResult from RetrievalQA

[sommohapatra]

The great thing about using .generate() with LLMChain directly is that I'm able to include tracking tags (like those with Promptlayer) into my generation, with the function returning an LLMResult rather than plain text. With RetrievalQA, though, it seems that the default output is text. It looks like part of the underlying for the RetrievalQA chain is LLMChain

You can create a chain from an index like this:

from langchain.llms.loading import load_llm
from langchain.chains.question_answering import load_qa_chain
from langchain.chat_models import ChatOpenAI
from langchain.chains import RetrievalQA

def create_chain_from_index(index):
    llm = load_llm("llm.json")
    qa = RetrievalQA.from_chain_type(llm=llm, chain_type="stuff", return_source_documents=True,
                                     retriever=index.as_retriever())
    return qa

With the above function returning a RetrievalQA() object. When digging into the object's structure, you can see:

retrievalqa_object.combine_documents_chain.llm_chain

which is an LLMChain object. You can then run something like

retrievalqa_object.combine_documents_chain.llm_chain.generate([{"question": "your question", "context": "document context"}])

And boom! Running this returns us an LLMResult that we can then do other fun stuff with.

This is something that I haven't seen a Github or other discussion on yet, so figure I'd add it to the community. Questions I have for folks that have more experience here:

• How can I add the context in a non-hardcoded way?
  • How do I do this by utilizing the value of the index that I've created? IE, for a very long document, I'd rather not put in the entire document, but rather only the pieces that most closely answer the question.