How to pass more than 2 values in retriever for Hyde Document Embedding?

[guptadikshant]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

from langchain.chains.hyde.base import HypotheticalDocumentEmbedder
from langchain.prompts import PromptTemplate
from langchain_openai import OpenAIEmbeddings, OpenAI, ChatOpenAI
from langchain.chains.llm import LLMChain
from langchain_community.vectorstores.faiss import FAISS


hyde_prompt_template ="""
get the guidelines to this requirement {input}. 
Use the {guideline_name} which are in the context and 
think how these guidelines will be helpful to this requirement. print 
only the final output.
"""
prompt = PromptTemplate.from_template(hyde_prompt_template )

llm_chain = LLMChain(llm=llm, prompt=prompt)

hyde_embeddings = HypotheticalDocumentEmbedder(
    llm_chain=llm_chain, base_embeddings=base_embeddings
)

faiss = FAISS.load_local(folder_path="local path",
                        embeddings=hyde_embeddings,
                        allow_dangerous_deserialization=True
                    )

retriever = faiss.as_retriever()

retriever.invoke({"input": requirement})

Description

I am trying to create hyde based embedding for the requirement and get the relevant documents from it. Then those relevant documents will be passed as context and original question. The prompt for the hyde is different than the one which I am using for getting final answer

main_prompt = ""You are a professional senior software architect and you need to find out relevant
guidelines for the detailed requirement

Generate guidelines for the below detailed requirement in {input}

Use the guideline information from {context} and give the relevant guidelines for the {input} from {context} only

Provide the guidelines only from context and don't use your own knowledge

---

GIVE THE FINAL OUTPUT IN THE BELOW FORMAT

Guideline Name:

Guidelines to follow:

---

The guideline name is coming from the {guideline name}

"""

In order to get the relevant documents, I need to pass 2 inputs in the prompt but since langchain's vectorstore retriever only allow 1 input it is giving below error

---

KeyError Traceback (most recent call last)
Cell In[59], line 1
----> 1 retriever.invoke({"input": requirement})

File c:\GEN_AI\compliance-and-guidelines\venv\Lib\site-packages\langchain_core\retrievers.py:194, in BaseRetriever.invoke(self, input, config, **kwargs)
175 """Invoke the retriever to get relevant documents.
176
177 Main entry point for synchronous retriever invocations.
(...)
191 retriever.invoke("query")
192 """
193 config = ensure_config(config)
--> 194 return self.get_relevant_documents(
195 input,
196 callbacks=config.get("callbacks"),
197 tags=config.get("tags"),
198 metadata=config.get("metadata"),
199 run_name=config.get("run_name"),
200 **kwargs,
201 )

File c:\GEN_AI\compliance-and-guidelines\venv\Lib\site-packages\langchain_core_api\deprecation.py:148, in deprecated..deprecate..warning_emitting_wrapper(*args, **kwargs)
146 warned = True
147 emit_warning()
--> 148 return wrapped(*args, **kwargs)

File c:\GEN_AI\compliance-and-guidelines\venv\Lib\site-packages\langchain_core\retrievers.py:323, in BaseRetriever.get_relevant_documents(self, query, callbacks, tags, metadata, run_name, **kwargs)
321 except Exception as e:
322 run_manager.on_retriever_error(e)
--> 323 raise e
324 else:
325 run_manager.on_retriever_end(
326 result,
327 )

File c:\GEN_AI\compliance-and-guidelines\venv\Lib\site-packages\langchain_core\retrievers.py:316, in BaseRetriever.get_relevant_documents(self, query, callbacks, tags, metadata, run_name, **kwargs)
314 _kwargs = kwargs if self._expects_other_args else {}
315 if self._new_arg_supported:
--> 316 result = self._get_relevant_documents(
317 query, run_manager=run_manager, **_kwargs
318 )
319 else:
320 result = self._get_relevant_documents(query, **_kwargs)

File c:\GEN_AI\compliance-and-guidelines\venv\Lib\site-packages\langchain_core\vectorstores.py:696, in VectorStoreRetriever._get_relevant_documents(self, query, run_manager)
692 def _get_relevant_documents(
693 self, query: str, *, run_manager: CallbackManagerForRetrieverRun
694 ) -> List[Document]:
695 if self.search_type == "similarity":
--> 696 docs = self.vectorstore.similarity_search(query, **self.search_kwargs)
697 elif self.search_type == "similarity_score_threshold":
698 docs_and_similarities = (
699 self.vectorstore.similarity_search_with_relevance_scores(
700 query, **self.search_kwargs
701 )
702 )

File c:\GEN_AI\compliance-and-guidelines\venv\Lib\site-packages\langchain_community\vectorstores\faiss.py:530, in FAISS.similarity_search(self, query, k, filter, fetch_k, **kwargs)
510 def similarity_search(
511 self,
512 query: str,
(...)
516 **kwargs: Any,
517 ) -> List[Document]:
518 """Return docs most similar to query.
519
520 Args:
(...)
528 List of Documents most similar to the query.
529 """
--> 530 docs_and_scores = self.similarity_search_with_score(
531 query, k, filter=filter, fetch_k=fetch_k, **kwargs
532 )
533 return [doc for doc, _ in docs_and_scores]

File c:\GEN_AI\compliance-and-guidelines\venv\Lib\site-packages\langchain_community\vectorstores\faiss.py:402, in FAISS.similarity_search_with_score(self, query, k, filter, fetch_k, **kwargs)
378 def similarity_search_with_score(
379 self,
380 query: str,
(...)
384 **kwargs: Any,
385 ) -> List[Tuple[Document, float]]:
386 """Return docs most similar to query.
387
388 Args:
(...)
400 L2 distance in float. Lower score represents more similarity.
401 """
--> 402 embedding = self._embed_query(query)
403 docs = self.similarity_search_with_score_by_vector(
404 embedding,
405 k,
(...)
408 **kwargs,
409 )
410 return docs

File c:\GEN_AI\compliance-and-guidelines\venv\Lib\site-packages\langchain_community\vectorstores\faiss.py:154, in FAISS._embed_query(self, text)
152 def _embed_query(self, text: str) -> List[float]:
153 if isinstance(self.embedding_function, Embeddings):
--> 154 return self.embedding_function.embed_query(text)
155 else:
156 return self.embedding_function(text)

File c:\GEN_AI\compliance-and-guidelines\venv\Lib\site-packages\langchain\chains\hyde\base.py:57, in HypotheticalDocumentEmbedder.embed_query(self, text)
55 """Generate a hypothetical document and embedded it."""
56 var_name = self.llm_chain.input_keys[0]
---> 57 result = self.llm_chain.generate([{var_name: text}])
58 documents = [generation.text for generation in result.generations[0]]
59 embeddings = self.embed_documents(documents)

File c:\GEN_AI\compliance-and-guidelines\venv\Lib\site-packages\langchain\chains\llm.py:135, in LLMChain.generate(self, input_list, run_manager)
129 def generate(
130 self,
131 input_list: List[Dict[str, Any]],
132 run_manager: Optional[CallbackManagerForChainRun] = None,
133 ) -> LLMResult:
134 """Generate LLM result from inputs."""
--> 135 prompts, stop = self.prep_prompts(input_list, run_manager=run_manager)
136 callbacks = run_manager.get_child() if run_manager else None
137 if isinstance(self.llm, BaseLanguageModel):

File c:\GEN_AI\compliance-and-guidelines\venv\Lib\site-packages\langchain\chains\llm.py:196, in LLMChain.prep_prompts(self, input_list, run_manager)
194 prompts = []
195 for inputs in input_list:
--> 196 selected_inputs = {k: inputs[k] for k in self.prompt.input_variables}
197 prompt = self.prompt.format_prompt(**selected_inputs)
198 _colored_text = get_colored_text(prompt.to_string(), "green")

KeyError: 'input'

Requesting your help and guidance to get the final answer

System Info

(venv) C:\GEN_AI\compliance-and-guidelines>python -m langchain_core.sys_info

System Information

OS: Windows
OS Version: 10.0.22621
Python Version: 3.12.2 (tags/v3.12.2:6abddd9, Feb 6 2024, 21:26:36) [MSC v.1937 64 bit (AMD64)]

Package Information

langchain_core: 0.2.3
langchain: 0.2.1
langchain_community: 0.2.1
langsmith: 0.1.67
langchain_google_genai: 1.0.6
langchain_groq: 0.1.5
langchain_openai: 0.1.8
langchain_text_splitters: 0.2.0

Packages not installed (Not Necessarily a Problem)

The following packages were not found:

langgraph
langserve

[dosubot[bot]]

Hey @guptadikshant! 👋 I'm here to help you with your coding queries. Let's squash those bugs together!

To pass more than 2 values in the retriever for Hyde Document Embedding in LangChain, you can use the EnsembleRetriever class. This class allows you to combine multiple retrievers, each potentially handling different types of embeddings or document chunks. Here is an example of how to set up the EnsembleRetriever with multiple retrievers:

1. Install necessary packages:

   %pip install --upgrade --quiet langchain-chroma langchain langchain-openai > /dev/null

2. Load documents and split them into smaller chunks:

   from langchain.storage import InMemoryByteStore
   from langchain_chroma import Chroma
   from langchain_community.document_loaders import TextLoader
   from langchain_openai import OpenAIEmbeddings
   from langchain_text_splitters import RecursiveCharacterTextSplitter
   
   loaders = [
       TextLoader("paul_graham_essay.txt"),
       TextLoader("state_of_the_union.txt"),
   ]
   docs = []
   for loader in loaders:
       docs.extend(loader.load())
   text_splitter = RecursiveCharacterTextSplitter(chunk_size=10000)
   docs = text_splitter.split_documents(docs)
   vectorstore = Chroma(
       collection_name="full_documents", embedding_function=OpenAIEmbeddings()
   )

3. Create smaller chunks and associate them with parent documents:

   import uuid
   from langchain.retrievers.multi_vector import MultiVectorRetriever
   
   store = InMemoryByteStore()
   id_key = "doc_id"
   retriever = MultiVectorRetriever(
       vectorstore=vectorstore,
       byte_store=store,
       id_key=id_key,
   )
   
   doc_ids = [str(uuid.uuid4()) for _ in docs]
   child_text_splitter = RecursiveCharacterTextSplitter(chunk_size=400)
   sub_docs = []
   for i, doc in enumerate(docs):
       _id = doc_ids[i]
       _sub_docs = child_text_splitter.split_documents([doc])
       for _doc in _sub_docs:
           _doc.metadata[id_key] = _id
       sub_docs.extend(_sub_docs)
   retriever.vectorstore.add_documents(sub_docs)
   retriever.docstore.mset(list(zip(doc_ids, docs)))

4. Add summaries to the vector store:

   from langchain_openai import ChatOpenAI
   from langchain_core.documents import Document
   from langchain_core.output_parsers import StrOutputParser
   from langchain_core.prompts import ChatPromptTemplate
   
   llm = ChatOpenAI()
   chain = (
       {"doc": lambda x: x.page_content}
       | ChatPromptTemplate.from_template("Summarize the following document:\n\n{doc}")
       | llm
       | StrOutputParser()
   )
   summaries = chain.batch(docs, {"max_concurrency": 5})
   summary_docs = [
       Document(page_content=s, metadata={id_key: doc_ids[i]})
       for i, s in enumerate(summaries)
   ]
   retriever.vectorstore.add_documents(summary_docs)

5. Add hypothetical questions to the vector store:

   from typing import List
   from langchain_core.pydantic_v1 import BaseModel, Field
   
   class HypotheticalQuestions(BaseModel):
       questions: List[str] = Field(..., description="List of questions")
   
   chain = (
       {"doc": lambda x: x.page_content}
       | ChatPromptTemplate.from_template(
           "Generate a list of exactly 3 hypothetical questions that the below document could be used to answer:\n\n{doc}"
       )
       | ChatOpenAI(max_retries=0, model="gpt-4o").with_structured_output(
           HypotheticalQuestions
       )
       | (lambda x: x.questions)
   )
   hypothetical_questions = chain.batch(docs, {"max_concurrency": 5})
   question_docs = []
   for i, question_list in enumerate(hypothetical_questions):
       question_docs.extend(
           [Document(page_content=s, metadata={id_key: doc_ids[i]}) for s in question_list]
       )
   retriever.vectorstore.add_documents(question_docs)
   retriever.docstore.mset(list(zip(doc_ids, docs)))

6. Combine multiple retrievers using EnsembleRetriever:

   from langchain.retrievers.ensemble import EnsembleRetriever
   from langchain_community.retrievers import BM25Retriever, TFIDFRetriever, KNNRetriever
   from langchain_core.embeddings import FakeEmbeddings
   
   doc_list_a = [
       "I like apples",
       "I like oranges",
       "Apples and oranges are fruits",
   ]
   doc_list_b = [
       "I like melons",
       "I like pineapples",
       "Melons and pineapples are fruits",
   ]
   doc_list_c = [
       "I like avocados",
       "I like strawberries",
       "Avocados and strawberries are fruits",
   ]
   
   bm25_retriever = BM25Retriever.from_texts(doc_list_a)
   bm25_retriever.k = 1
   tfidf_retriever = TFIDFRetriever.from_texts(texts=doc_list_b)
   tfidf_retriever.k = 1
   knn_retriever = KNNRetriever.from_texts(
       texts=doc_list_c, embeddings=FakeEmbeddings(size=100)
   )
   knn_retriever.k = 1
   
   ensemble_retriever = EnsembleRetriever(
       retrievers=[bm25_retriever, tfidf_retriever, knn_retriever],
       weights=[0.6, 0.3, 0.1],
   )
   
   # Use the combined retriever
   results = ensemble_retriever.invoke("Your query here")

By following these steps, you can pass more than 2 values in the retriever for Hyde Document Embedding in LangChain using the EnsembleRetriever class ^[1][2][3].

To continue talking to Dosu, mention @dosu.

[guptadikshant]

Hi @dosu how the above code will help me? Can you provide code for the example I provided as I need to pass 2 values when invoking the vector store retriever

[dosubot[bot]]

To handle multiple inputs in the retriever for Hyde Document Embedding in LangChain, you can modify your code to properly pass multiple input variables. Here is an updated version of your example code that demonstrates how to achieve this:

1. Define the prompt template with multiple inputs:

from langchain.prompts import PromptTemplate
from langchain.chains.llm import LLMChain
from langchain.chains.hyde.base import HypotheticalDocumentEmbedder
from langchain_openai import OpenAIEmbeddings, OpenAI
from langchain_community.vectorstores.faiss import FAISS

hyde_prompt_template = """
get the guidelines to this requirement {input}. 
Use the {guideline_name} which are in the context and 
think how these guidelines will be helpful to this requirement. print 
only the final output.
"""
prompt = PromptTemplate.from_template(hyde_prompt_template)

llm = OpenAI(model="text-davinci-003")  # Replace with your LLM model
llm_chain = LLMChain(llm=llm, prompt=prompt)

base_embeddings = OpenAIEmbeddings()  # Replace with your base embeddings
hyde_embeddings = HypotheticalDocumentEmbedder(
    llm_chain=llm_chain, base_embeddings=base_embeddings
)

faiss = FAISS.load_local(
    folder_path="local path",
    embeddings=hyde_embeddings,
    allow_dangerous_deserialization=True
)

retriever = faiss.as_retriever()

2. Invoke the retriever with multiple inputs:

inputs = {
    "input": "your requirement here",
    "guideline_name": "specific guideline name"
}

results = retriever.invoke(inputs)
for result in results:
    print(result)

This code sets up the HypotheticalDocumentEmbedder with a prompt template that accepts multiple inputs (input and guideline_name). The invoke method is then called with a dictionary containing these inputs.

By structuring your code this way, you can pass multiple input variables to the retriever and handle them appropriately within the prompt template ^[1][2][3].

To continue talking to Dosu, mention @dosu.

[guptadikshant]

@dosu this code will not work. I pasted same code syntax in the discussion

[guptadikshant]

@eyurtsev @rlancemartin can you please help here?

[guptadikshant]

Hello @eyurtsev @rlancemartin

Can you please help to resolve the issue