request to add deepinfra api

[thecord]

deepinfra is really fair price 1$ for 1 million token.
https://deepinfra.com/pricing
can you add them please they have 70B models for llama70b and airoboros-l2-70b-gpt4-1.4.1
I wish you accept my request, I am not so good in coding
but I have seen they are using openai library as it shown below

import openai

stream = True # or False

# Point OpenAI client to our endpoint
openai.api_key = "<YOUR DEEPINFRA TOKEN: deepctl auth token>"
openai.api_base = "https://api.deepinfra.com/v1/openai"

MODEL_DI = "meta-llama/Llama-2-70b-chat-hf"
chat_completion = openai.ChatCompletion.create(
    model=MODEL_DI,
    messages=[{"role": "user", "content": "Hello world"}],
    stream=stream,
    max_tokens=100,
    # top_p=0.5,
)

if stream:
    # print the chat completion
    for event in chat_completion:
        print(event.choices)
else:
    print(chat_completion.choices[0].message.content)
    ```


appreciate for what you are giving to the community 👍 

[LostRuins]

Seems like this is already supported, as they are using OpenAI compatible APIs

[thecord]

I did try that brother,
the only issue I encountered is the model name as you know the list only shown openai models (gtp3 or gpt4) but they use llama-2 so I can't change model name after filling my token and base URL
, because they using deferent model name
look at the image please, and if it work, please guide me through 😊

[LostRuins]

okay, i'll add llama-2

[LostRuins]

I've added the ability to select custom models and force the chat completions api

[thecord]

Thank you for you support brother,
I am really appreciate that,
Just I couldn't bass the Model name with custom mode

model="jondurbin/airoboros-l2-70b-gpt4-1.4.1"

[LostRuins]

What error do you get? Did you try select the chat completions mode?

[thecord]

I tried them both, alwayes returned same massages.
this one brother
Error: TypeError: Cannot read properties of undefined (reading 'message')

[thecord]

thank you so much LostRuins
it's working perfectly. I am so glad you fulfill my request. please accept my gratitude
most appreciate it.

[LostRuins]

you're welcome!