How to have conditional LLM action depending on what the user asks?

[geekyme]

This is what I'm trying to achieve but I'm not sure which part of langchain solves my problem:

1. User sends in a question
2. LLM detects the user’s intent
3. LLM either performs a retrieval to a vector store or escalate user’s question to a human by making an API call to a customer service tool.

At the moment I'm able to achieve my goals somewhat with the conversation retrieval chain, but this has the side effect of making a retrieval call for every input the user sends, even if it's something like “ok”, “I need help”.

Are agents the way?

[lucas-labs]

I have a similar scenario and solved it like this:

1. user sends a message
2. I call a completion LLM with a prompt that categorizes the user input into several "categories" including "greetings", "farewell", "question", "thanks". It works pretty fine and as it's just completion it's quite fast and cheap
3. Based on the output of the "categorizer" LLM, I decide whether I follow the path of retrival + chat LLM or I just respond with one of many pre-defined answers to each category (I basically have a collection of hellos, goodbyes and you're welcome messages that I pick up randomly just to keep it "natural")

I don't know if that's a good solution, I'm not exactly an expert... so I'm also curious about how others are handling this situations

[znb899]

Can you please share the prompt you used?

[sergioigual]

Hey im trying to do something like that which llm did you use to categorize the user input into several "categories"? And which prompt did you use.
Thanks

[hinthornw]

Have you tried routing approaches? https://python.langchain.com/docs/expression_language/how_to/routing

[Arslan-Mehmood1]

Hi, I don't know if my answer is much relevant to your scenario or not, but I would like to share something.
The goal was to engage user and ask questions about three attributes of wine and then a wine should be recommended to user based on vectordb store.

I tried many things but couldn't. Finally, I ended up creating a custom langchain agent with access to 4 tools.

1. tool1: instructs the agent to ask user about attribute1 if it doesn't already exist in the chat history.
2. tool2: instructs the agent to ask user about attribute2 if it doesn't already exist in the chat history.
3. tool3: instructs the agent to ask user about attribute3 if it doesn't already exist in the chat history.
4. tool4: instructs the agent to use a self-query retriever to retreive and recommed a wine based on three attributes provided by user.

The good thing is that, the agent first ask all three attributes from user and then move to call self-query retrieval tool.

One more thing, I learned form this, we can have any langchain function etc inside a custom tool, and tool is not only for using external apis etc.

[eyurtsev]

You could try combining this into a single tool that requires all three attributes. The agent should be instructed to use the tool only if it has full information and otherwise to try and collect the necessary information. I've done this before for a situation where the agent is asking a user to enter some personal information (e.g., name, d/o/b, etc).

[Arslan-Mehmood1]

Yes, but I used 3 tools to engage user in a conversation. Asking for 3 attributes in single query somewhat feels overwhelming.