How to add control over the caching of a user rprompt and the value generated by my custom agent? #23897
Unanswered
Ikshan-Tango
asked this question in
Q&A
Replies: 1 comment
-
To add control over the caching of a user prompt and the value generated by your custom agent, you can modify your code to include a check for the substring 'error' in the response before caching it. Here's how you can do it:
Here's the updated code: import json
from langchain_core.prompts import (
ChatPromptTemplate,
FewShotChatMessagePromptTemplate,
)
from langchain_community.chat_models import ChatOllama
from langchain_community.cache import SQLiteCache
from langchain.globals import set_llm_cache
import time
def serialize_llm_config(llm):
params = llm._get_parameters(stop=None)
return json.dumps(params)
def conditional_cache(query, result, llm):
llm_string = serialize_llm_config(llm)
if "error" not in result.content:
cache.set((query, llm_string), result.content)
def ddts_priority_agent(query: str, LLM = None) -> str:
set_llm_cache(SQLiteCache(database_path=".ddts_priority_cache.db"))
details_examples = [
{
"input": "status of CSCvx83805",
"output": '{"intent_type": "details", "ddts_id": "CSCvx83805"}'
},
{
"input": "Status of DDTS CSCvx83805",
"output": '{"intent_type": "details", "ddts_id": "CSCvx83805"}'
}
]
priority_increase_examples = [
{
"input": "increase priority of <run_id>",
"output": '{"intent_type": "increase", "run_id": "<run_id>"}'
},
{
"input": "please increase priority of run <id> as soon as possible",
"output": '{"intent_type": "increase", "run_id": "<id>"}'
},
]
examples = details_examples + priority_increase_examples
example_prompt = ChatPromptTemplate.from_messages(
[
("human", "{input}"),
("ai", "{output}"),
]
)
few_shot_prompt = FewShotChatMessagePromptTemplate(
example_prompt=example_prompt,
examples=examples,
)
system_prompt = """
"You are an expert english support agent with the name Jeeves for a software company who specializes in identifying nouns. ",
"Your task is to ONLY AND ONLY correctly identify the intent_type of the query and extract the extra params from the query. ",
"The query given to you will be either of intent_type as 1. details 2. increase, ",
"'details' if the query has details word in it, ",
"'increase' if the query is asking to 'increase priority', ",
"Extract the ddts_id from the query if the intent_type is details, ",
"Extract the run_id from the query if the intent_type is priority. ",
"You need to provide the output in json format. ",
"As a tool you're not supposed to know the answer to the query, ",
"rather you just have to extract the intent_type and extra params correctly. ",
"IF YOU CANT EXTRACT OR YOU'RE CONFUSED THEN RETURN RESPONSE AS '{{\"error\": <YOUR MESSAGE HERE>}}' ",
"The only extra params that you're allowed to send are 'run_id', 'ddts_id'."
"""
final_prompt = ChatPromptTemplate.from_messages(
[
("system", system_prompt),
few_shot_prompt,
("human", "{input}"),
]
)
if LLM is None:
LLM = ChatOllama(
model="llama3",
temperature= 0,
num_gpu= 1,
top_k = 20,
top_p = 0.6,
verbose= True,
keep_alive = "3600",
)
llm_string = str(LLM)
print(llm_string)
chain = final_prompt | LLM
result = chain.invoke({"input": query})
result = result.content
dict_result = json.loads(result)
# Conditional caching
conditional_cache(query, result, LLM)
return result
def process_query(query):
start_time = time.time()
(ddts_priority_agent(query))
print(f"Time taken: {time.time() - start_time}")
if __name__ == "__main__":
process_query("jeeves, please increase the priority of run with id as 3242342")
process_query("hii jeeves please tell me how are you??")
process_query("can you please tell me details of ddts CSCvx83805")
process_query("jeeves, please increase the priority of run with id as 3242342") In this code:
This ensures that only responses without the substring 'error' are cached. |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Checked other resources
Commit to Help
Example Code
Description
What's happening right now is that for EVERY user query the output generated by the agent is being Cached.
I dont want this to happen, I want to check first if the response from the agent i.e
result.content
does not have any subtring'error'
present inside it.If there is a keyword error present inside it then dont cache it, else we can cache it.
Psudo logic should be something like this :-
System Info
System Information
Package Information
Packages not installed (Not Necessarily a Problem)
The following packages were not found:
Beta Was this translation helpful? Give feedback.
All reactions