Langchain with AOAI with APIM

[ltivmitre]

Any thoughts or ideas on how we could configure fronting AOIA with APIM to work with langchain? The langchain framework looks for an endpoint and key directly to the LLM.

[paullizer]

Hello, Can you provide example of the code?

[ltivmitre]

Below is a python script I've used to test with. We have come up with a work around by takeing the content and pipe it through the python requests library to make the calls.

import os

from langchain_core.messages import HumanMessage
from langchain_openai import AzureChatOpenAI

os.environ["AZURE_OPENAI_API_KEY"] = ""
os.environ["AZURE_OPENAI_ENDPOINT"] = ""

model = AzureChatOpenAI(
openai_api_version="2023-05-15",
azure_deployment="gpt-35-turbo-16k",
)

message = HumanMessage(
content="Translate this sentence from English to French. I love programming."
)
model([message])

[paullizer]

Please share your current solution for others to see your work around.

Have you tried to use the APIM subscription key and APIM url for the OS.environ variables?

The following may work, I have not tested it though. If you can test it, let me know. If you modify it to get it to work, please share. I will test later today/tomorrow.

openai.ChatCompletion.azure_api_prefix = ""
chat_model = AzureChatOpenAI(
    openai_api_base=apim_url,
    openai_api_version=api_version,
    deployment_name="deployment_name",
    openai_api_key=apim_subscription_key,
    openai_api_type="azure",
    model="gpt-35-turbo",
    headers = {
    "Ocp-Apim-Subscription-Key": apim_subscription_key,
    "Content-Type': 'application/json"
    },
)

[ltivmitre]

Will give it a try today, thank you!

[ltivmitre]

From my testing, the headers is not a valid parameter.

TypeError: create() got an unexpected keyword argument 'headers'