Releases: BerriAI/litellm
Releases · BerriAI/litellm
v1.53.3-dev1
What's Changed
- Litellm dev 11 30 2024 by @krrishdholakia in #6974
- LiteLLM Minor Fixes & Improvements (12/02/2024) by @krrishdholakia in #6994
- Litellm test ci cd by @krrishdholakia in #6997
- (fix) logging Auth errors on datadog by @ishaan-jaff in #6995
- (fixes) datadog logging - handle 1MB max log size on DD by @ishaan-jaff in #6996
- Litellm dbrx structured outputs support by @krrishdholakia in #6993
Full Changelog: v1.53.2...v1.53.3-dev1
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.53.3-dev1
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Failed ❌ | 270.0 | 322.8250497930263 | 5.940031623578464 | 0.0 | 1778 | 0 | 227.83484099994666 | 3640.05648899996 |
| Aggregated | Failed ❌ | 270.0 | 322.8250497930263 | 5.940031623578464 | 0.0 | 1778 | 0 | 227.83484099994666 | 3640.05648899996 |
Contributors
krrishdholakia and ishaan-jaff
Assets 4
v1.53.3
What's Changed
- Litellm dev 11 30 2024 by @krrishdholakia in #6974
- LiteLLM Minor Fixes & Improvements (12/02/2024) by @krrishdholakia in #6994
- Litellm test ci cd by @krrishdholakia in #6997
- (fix) logging Auth errors on datadog by @ishaan-jaff in #6995
- (fixes) datadog logging - handle 1MB max log size on DD by @ishaan-jaff in #6996
- Litellm dbrx structured outputs support by @krrishdholakia in #6993
Full Changelog: v1.53.2...v1.53.3
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.53.3
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 260.0 | 295.3963784342538 | 6.049806369807933 | 0.0 | 1810 | 0 | 224.3657600000688 | 2447.638761999997 |
| Aggregated | Passed ✅ | 260.0 | 295.3963784342538 | 6.049806369807933 | 0.0 | 1810 | 0 | 224.3657600000688 | 2447.638761999997 |
Contributors
krrishdholakia and ishaan-jaff
Assets 4
v1.53.2
What's Changed
- fix(key_management_endpoints.py): support 'tags' param on
/key/updateby @krrishdholakia in #6945 - LiteLLM Minor Fixes & Improvements (11/29/2024) by @krrishdholakia in #6965
- Update team_endpoints.py by @superpoussin22 in #6983
Full Changelog: v1.53.1...v1.53.2
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.53.2
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Failed ❌ | 250.0 | 318.2618058818948 | 6.0033656688808605 | 0.003344493408847276 | 1795 | 1 | 225.67902299999787 | 55505.375238 |
| Aggregated | Failed ❌ | 250.0 | 318.2618058818948 | 6.0033656688808605 | 0.003344493408847276 | 1795 | 1 | 225.67902299999787 | 55505.375238 |
Contributors
krrishdholakia and superpoussin22
Assets 4
v1.53.1.dev1
Full Changelog: v1.53.1...v1.53.1.dev1
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.53.1.dev1
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 250.0 | 285.15501328346113 | 6.138794444114975 | 0.0 | 1838 | 0 | 223.90917799998533 | 2684.1706850000264 |
| Aggregated | Passed ✅ | 250.0 | 285.15501328346113 | 6.138794444114975 | 0.0 | 1838 | 0 | 223.90917799998533 | 2684.1706850000264 |
v1.53.1
🚨 Relevant Changes:
- LiteLLM Proxy Virtual Keys: Unique Key Aliases will be enforced on /key/generate and /key/update requests
- datadog integration will use StandardLoggingPayload (from LiteLLM v1.53.0+) & also supports logging failures #6929
If you need to use the v1 of the payload (not recommended), you can set this in your config
litellm_settings:
datadog_use_v1: TrueBenefits of using StandardLoggingPayload for datadog
- It's a standard logging object so should be consistent over time across our logging integrations
- Added support for logging LLM failures
- Has additional info like cache_hit , request_tags etc. Full payload is here https://docs.litellm.ai/docs/proxy/logging#what-gets-logged
What's Changed
- LiteLLM Minor Fixes & Improvements (11/24/2024) by @krrishdholakia in #6890
- (feat) pass through llm endpoints - add
PATCHsupport (vertex context caching requires for update ops) by @ishaan-jaff in #6924 - sonnet supports pdf, haiku does not by @paul-gauthier in #6928
- (feat) DataDog Logger - Add Failure logging + use Standard Logging payload by @ishaan-jaff in #6929
- (feat) log proxy auth errors on datadog by @ishaan-jaff in #6931
- (feat) Allow using include to include external YAML files in a config.yaml by @ishaan-jaff in #6922
- (feat) dd logger - set tags according to the values set by those env vars by @ishaan-jaff in #6933
- LiteLLM Minor Fixes & Improvements (11/26/2024) by @krrishdholakia in #6913
- LiteLLM Minor Fixes & Improvements (11/27/2024) by @krrishdholakia in #6943
- Update Argilla integration documentation by @sdiazlor in #6923
- (bug fix) /key/update was not storing
budget_durationin the DB by @ishaan-jaff in #6941 - (fix) handle json decode errors for DD exception logging by @ishaan-jaff in #6934
- (docs + fix) Add docs on Moderations endpoint, Text Completion by @ishaan-jaff in #6947
- (feat) add enforcement for unique key aliases on /key/update and /key/generate by @ishaan-jaff in #6944
- (fix) tag merging / aggregation logic by @ishaan-jaff in #6932
- (feat) Allow disabling ErrorLogs written to the DB by @ishaan-jaff in #6940
New Contributors
Full Changelog: v1.52.16...v1.53.1
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.53.1
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 200.0 | 215.7709455547284 | 6.292082946554957 | 0.0 | 1882 | 0 | 178.3981389999667 | 2851.1550680000255 |
| Aggregated | Passed ✅ | 200.0 | 215.7709455547284 | 6.292082946554957 | 0.0 | 1882 | 0 | 178.3981389999667 | 2851.1550680000255 |
Contributors
krrishdholakia, ishaan-jaff, and 2 other contributors
Assets 4
v1.52.15.staging1
Full Changelog: v1.52.15...v1.52.15.staging1
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.52.15.staging1
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 250.0 | 278.6646185965574 | 6.211416620977041 | 0.0033412676820747935 | 1859 | 1 | 217.41687699994827 | 3149.612769999976 |
| Aggregated | Passed ✅ | 250.0 | 278.6646185965574 | 6.211416620977041 | 0.0033412676820747935 | 1859 | 1 | 217.41687699994827 | 3149.612769999976 |
v1.52.15-stable
Full Changelog: v1.52.15...v1.52.15-stable
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:litellm_stable_nov27-stable
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 200.0 | 221.52349298020383 | 6.244722043862887 | 0.0 | 1869 | 0 | 181.6640519999737 | 2200.3593760000513 |
| Aggregated | Passed ✅ | 200.0 | 221.52349298020383 | 6.244722043862887 | 0.0 | 1869 | 0 | 181.6640519999737 | 2200.3593760000513 |
v1.52.16.dev4
What's Changed
- LiteLLM Minor Fixes & Improvements (11/24/2024) by @krrishdholakia in #6890
- (feat) pass through llm endpoints - add
PATCHsupport (vertex context caching requires for update ops) by @ishaan-jaff in #6924 - sonnet supports pdf, haiku does not by @paul-gauthier in #6928
- (feat) DataDog Logger - Add Failure logging + use Standard Logging payload by @ishaan-jaff in #6929
- (feat) log proxy auth errors on datadog by @ishaan-jaff in #6931
- (feat) Allow using include to include external YAML files in a config.yaml by @ishaan-jaff in #6922
Full Changelog: v1.52.16...v1.52.16.dev4
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.52.16.dev4
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 200.0 | 223.2904107688168 | 6.215640236973433 | 0.0 | 1860 | 0 | 174.75808199998255 | 3944.991313999992 |
| Aggregated | Passed ✅ | 200.0 | 223.2904107688168 | 6.215640236973433 | 0.0 | 1860 | 0 | 174.75808199998255 | 3944.991313999992 |
Contributors
krrishdholakia, ishaan-jaff, and paul-gauthier
Assets 4
v1.52.16.dev1
What's Changed
- LiteLLM Minor Fixes & Improvements (11/24/2024) by @krrishdholakia in #6890
- (feat) pass through llm endpoints - add
PATCHsupport (vertex context caching requires for update ops) by @ishaan-jaff in #6924 - sonnet supports pdf, haiku does not by @paul-gauthier in #6928
- (feat) DataDog Logger - Add Failure logging + use Standard Logging payload by @ishaan-jaff in #6929
- (feat) log proxy auth errors on datadog by @ishaan-jaff in #6931
- (feat) Allow using include to include external YAML files in a config.yaml by @ishaan-jaff in #6922
- (feat) dd logger - set tags according to the values set by those env vars by @ishaan-jaff in #6933
Full Changelog: v1.52.16...v1.52.16.dev1
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.52.16.dev1
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.52.16.dev1
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 250.0 | 285.0974372649336 | 6.039486955708498 | 0.0 | 1808 | 0 | 224.19419400000606 | 3263.23956899995 |
| Aggregated | Passed ✅ | 250.0 | 285.0974372649336 | 6.039486955708498 | 0.0 | 1808 | 0 | 224.19419400000606 | 3263.23956899995 |
Contributors
krrishdholakia, ishaan-jaff, and paul-gauthier
Assets 4
v1.52.16
What's Changed
- feat - allow sending
tagson vertex pass through requests by @ishaan-jaff in #6876 - (feat) Add support for using @google/generative-ai JS with LiteLLM Proxy by @ishaan-jaff in #6899
- (UI fix) UI does not reload when you login / open a new tab by @ishaan-jaff in #6909
- (fix) pass through endpoints - run logging async + use thread pool executor for sync logging callbacks by @ishaan-jaff in #6907
- (redis fix) - fix
AbstractConnection.__init__() got an unexpected keyword argument 'ssl'by @ishaan-jaff in #6908 - (docs) Simplify
/vertex_ai/pass through docs by @ishaan-jaff in #6910
Full Changelog: v1.52.15...v1.52.16
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.52.16
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 230.0 | 261.12838541230485 | 6.194778256251999 | 0.0 | 1853 | 0 | 206.21302299997524 | 2167.8605710000056 |
| Aggregated | Passed ✅ | 230.0 | 261.12838541230485 | 6.194778256251999 | 0.0 | 1853 | 0 | 206.21302299997524 | 2167.8605710000056 |
Contributors
ishaan-jaff