Gemini 1.5 flash start generating repeated text until reach max token limit condition

[kashish63]

Description of the bug:

I have developed a chatbot with gpt4o that have a lot of down time then i switch to gemini and now i am having this issue that model is generating same text again and again

like this

Here are the relevant HSN codes for "honey, lemon, dry fruits":

HSN Codes:

a. 04090000: Natural honey;
b. 08051000: Citrus fruit, fresh or dried; Lemons and limes
c. 08135000: Nuts, fresh or dried; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other; Other

what is this behavior of the model this is huge bug that is happening and its impacting very bad on production level please some one suggest me how can i improve it.

Actual vs expected behavior:

No response

Any other information you'd like to share?

No response

[gmKeshari]

Hi @kashish63,

Are you using Json mode in your code?? Can you provide your code, so i can reproduce the issue from my end if possible?

[kashish63]

i am using function calling for external API in that case when data from external API does not give relevant result it start giving response by its own and behaving like this. This is very big issue on production. Please help me.

[gmKeshari]

Hey @kashish63,

You can try below suggestions in your code :

Use proper error handling, include fallback responses, check external API reliability, improve your prompt.

Robust Error Handling: Implement comprehensive error handling in your code:

Check API Responses: Always validate the API response. Check for HTTP status codes (e.g., 200 OK vs. 404 Not Found, 500 Internal Server Error). Ensure the response data is in the expected format and contains the expected fields.
Handle Empty or Invalid Data: If the API returns an empty response or invalid data, design your code to explicitly handle this. Don't just let the LLM try to process it.

Fallback Responses: Instead of letting the LLM generate its own response, design specific fallback responses for different error scenarios. For example:

"I'm sorry, I couldn't find the relevant HSN codes for that item. Please try again later or check the official website."
"The API is temporarily unavailable. Please try your request again in a few minutes."

Improved Prompt Engineering: Refine your prompt to explicitly address the possibility of API failures:

Instructions for Failure: Include instructions in your prompt like: "If the API call fails to return relevant data, respond with: 'I'm sorry, I couldn't find the requested information.'"
Clear Input/Output Specification: Clearly define the expected input and output format for the function call in your prompt.
Contextual Awareness: Give the model enough context to understand the limitations of the API. For example, you could state: "This API may not have data for all items."

Here is a sample code example :

import requests

def get_hsn_codes(item):
    try:
        response = requests.get(f"your_api_url?item={item}")
        response.raise_for_status()  # Raise HTTPError for bad responses (4xx or 5xx)
        data = response.json()
        if data and data["codes"]:  #Check for valid data
            return data["codes"]
        else:
            return None #Handle empty or invalid data
    except requests.exceptions.RequestException as e:
        print(f"API error: {e}")
        return None

# ... in your chatbot function ...
codes = get_hsn_codes(user_input)
if codes:
    # Process codes and generate response
    pass
else:
    # Use a pre-defined fallback response
    print("I'm sorry, I couldn't find the requested information.")

Please let me know if this resolves your issue. If not, I'll escalate it to the internal team.

[kashish63]

Thank you for Response and yes i implemented all of these and trying to manage this kind of response but for any case literally for any case model should not behave like this. This is serious issue and should be resolve immediately

[kashish63]

@gmKeshari their is one more problem. I made a instruction based chatbot that make api call using function call. I give it instruction to answer only logistics based question and refuse answer to anything else. I used temperature 0 and less top_p but still some times it forget its instruction and give response out of logistics field. what next step should I made to improve this behavior.

[gmKeshari]

This problem is common with LLMs, they can sometimes hallucinate or deviate from instructions, even with low temperature and top_p settings.

Here I can suggest few changes :

1. More explicit instructions: Instead of just saying "answer only logistics-based questions," be much more specific and provide examples. For instance:

"You are a logistics expert chatbot. You will only answer questions related to shipping, warehousing, transportation, supply chain management, and order fulfillment. If a question is not related to these areas, respond with: 'This question is outside the scope of my logistics expertise.'"

2. Include examples of both appropriate and inappropriate questions and responses. Show it what good answers look like (within the logistics domain) and what a refusal response should look like.

3. Reiterate the core instruction within the prompt. Frequent reminders help reinforce the model's focus.

[Giom-V]

Have you tried with Gemini 2.0? You could also try with a higher temperature (if you were using temperature=0)