You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Allow the user to switch the model without necessarily starting a new session.
Do you have thoughts on the technical implementation?
I'm not sure why we wouldn't allow switching model. There's no reason in the backend why switching model wouldn't just work, I don't think. If I recall correctly, this used to work with the UI too. I'll note that the user will lose history if we force a new session. Why not let them try a better/different model for their current project?
To be sure, there are settings which have to trigger a runtime reload, which we might call new session, but model/base url/api key are not. In fact, I think all LLMConfig settings would work just fine, backend-wise, in the same session.
The text was updated successfully, but these errors were encountered:
What problem or use case are you trying to solve?
Allow the user to switch the model without necessarily starting a new session.
Do you have thoughts on the technical implementation?
I'm not sure why we wouldn't allow switching model. There's no reason in the backend why switching model wouldn't just work, I don't think. If I recall correctly, this used to work with the UI too. I'll note that the user will lose history if we force a new session. Why not let them try a better/different model for their current project?
To be sure, there are settings which have to trigger a runtime reload, which we might call new session, but model/base url/api key are not. In fact, I think all LLMConfig settings would work just fine, backend-wise, in the same session.
The text was updated successfully, but these errors were encountered: