Performance/Processing Times

[rdhillbb]

First, let me apologize, as I am new to this arena. N I am in the early stages of creating an app that will look up information regarding courses offered at the university. As a test, I created five documents, all having less than 800 words. 4 of the text files are about course offerings, and the other provides an overview of the university. (Sorry for the long intro)
The result is good.It takes roughly 39 seconds to produce the results for the following question I was thinking less than 2 seconds.
I am using a Bookbook pro I9 2015 model.

I would like information on the social media marketing class. Please provide me the background on the trainer also."

code Snippets
docsearch = Chroma.from_documents(texts, embeddings)
qa = RetrievalQA.from_chain_type(llm=ChatOpenAI(max_tokens=2043, model_name='gpt-4', temperature=.4, request_timeout=120),
chain_type="stuff",
retriever=docsearch.as_retriever(),chain_type_kwargs=chain_type_kwargs)

start = timer()
query = "I would like information on the social media marketing class. Please provide me the background on the trainer also."
msg= qa.run(query)
end = timer()
print(msg)
print(end - start)

[rdhillbb]

Did some testing on the time_out. If the timeout is less than 30. I get the error message at the end of this message
If the timeout is set to 30 seconds or more there is no issue. However, it takes 30+ seconds for a query.

qa = RetrievalQA.from_chain_type(llm=ChatOpenAI(max_tokens=2043, model_name='gpt-4', temperature=.4, request_timeout=29),
chain_type="stuff",
retriever=docsearch.as_retriever(),chain_type_kwargs=chain_type_kwargs)
start = timer()
query = "I would like information on the social media marketing class. Please provide me the background on the trainer also."
msg= qa.run(query)
end = timer()
print(msg)
print(end - start)

ERROR------>

Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 1.0 seconds as it raised Timeout: Request timed out: HTTPSConnectionPool(host='api.openai.com', port=443): Read timed out. (read timeout=29).