What is the default request retry logic of model AzureChatOpenAI()?

[axiangcoding]

• What is the default request retry logic of AzureChatOpenAI()?
• Can i custom the retry logic?

[dosubot[bot]]

Answer generated by a 🤖

Answer

I understand that you're inquiring about the default request retry logic of the AzureChatOpenAI() model in the LangChain framework and whether it's possible to customize this logic.

The default retry logic is encapsulated in the _create_retry_decorator function. This function uses the tenacity library to manage retries. By default, it will retry if exceptions such as openai.error.Timeout, openai.error.APIError, openai.error.APIConnectionError, openai.error.RateLimitError, or openai.error.ServiceUnavailableError occur. The number of retry attempts is determined by the self.max_retries attribute.

The wait time between each retry starts at 1 second and increases exponentially (2^x * 1 second) up to a maximum of 60 seconds. A warning message is logged before each sleep period.

If you wish to customize this retry logic, you can modify the _create_retry_decorator function to suit your needs. You can change the exceptions that trigger a retry, adjust the number of retry attempts, or alter the wait time between retries. The retry logic is applied to the completion_with_retry function, which uses OpenAI's API to create a completion. This function can also be customized if needed.

Please note that this is one potential solution based on the current implementation of the LangChain framework. If you find this solution helpful and believe it could benefit other users, we encourage you to make a pull request to update the documentation. This way, other users who encounter similar issues can benefit from your contribution.

Sources

• langchain/chat_models/jinachat.py
• langchain/chat_models/openai.py

---

This response is meant to be useful, save you time, and share context. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

[raf-madrigal]

Is there a way for us to log the warning message via callbacks or some other logging mechanism?