Releases: BerriAI/litellm
Releases · BerriAI/litellm
v1.52.2
What's Changed
- chore: comment for maritalk by @nobu007 in #6607
- Update gpt-4o-2024-08-06, and o1-preview, o1-mini models in model cost map by @emerzon in #6654
- (QOL improvement) add unit testing for all static_methods in litellm_logging.py by @ishaan-jaff in #6640
- (feat) log error class, function_name on prometheus service failure hook + only log DB related failures on DB service hook by @ishaan-jaff in #6650
- Update several Azure AI models in model cost map by @emerzon in #6655
- ci(conftest.py): reset conftest.py for local_testing/ by @krrishdholakia in #6657
- Litellm dev 11 07 2024 by @krrishdholakia in #6649
New Contributors
Full Changelog: v1.52.1...v1.52.2
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.52.2
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 230.0 | 251.09411961031876 | 6.087114215107422 | 0.0 | 1822 | 0 | 198.72582000004968 | 1667.4085729999888 |
| Aggregated | Passed ✅ | 230.0 | 251.09411961031876 | 6.087114215107422 | 0.0 | 1822 | 0 | 198.72582000004968 | 1667.4085729999888 |
Contributors
nobu007, krrishdholakia, and 2 other contributors
Assets 4
v1.52.1
What's Changed
- (DB fix) don't run apply_db_fixes on startup by @ishaan-jaff in #6604
- LiteLLM Minor Fixes & Improvements (11/04/2024) by @krrishdholakia in #6572
- ci: remove redundant lint.yml workflow by @krrishdholakia in #6622
- LiteLLM Minor Fixes & Improvements (11/05/2024) by @krrishdholakia in #6590
- LiteLLM Minor Fixes & Improvements (11/06/2024) by @krrishdholakia in #6624
- (feat) GCS Bucket logging. Allow using IAM auth for logging to GCS by @ishaan-jaff in #6628
- Update opentelemetry_integration.md - Fix typos by @ronensc in #6618
- (fix) ProxyStartup - Check that prisma connection is healthy when starting an instance of LiteLLM by @ishaan-jaff in #6627
- Update team_budgets.md by @superpoussin22 in #6611
- (feat) Allow failed DB connection requests to allow virtual keys with
allow_failed_db_requestsby @ishaan-jaff in #6605 - fix(pattern_match_deployments.py): default to user input if unable to… by @krrishdholakia in #6646
- fix(pattern_match_deployments.py): default to user input if unable to… by @krrishdholakia in #6632
New Contributors
Full Changelog: v1.52.0...v1.52.1
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.52.1
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 260.0 | 292.8286898309638 | 6.110969302244283 | 0.0 | 1828 | 0 | 230.12115400001676 | 2643.3588609999674 |
| Aggregated | Passed ✅ | 260.0 | 292.8286898309638 | 6.110969302244283 | 0.0 | 1828 | 0 | 230.12115400001676 | 2643.3588609999674 |
Contributors
krrishdholakia, superpoussin22, and 2 other contributors
Assets 4
v1.52.0-stable
What's Changed
- LiteLLM Minor Fixes & Improvements (11/01/2024) by @krrishdholakia in #6551
- Litellm dev 11 02 2024 by @krrishdholakia in #6561
- build(deps): bump cookie and express in /docs/my-website by @dependabot in #6566
- docs(virtual_keys.md): update Dockerfile reference by @emmanuel-ferdman in #6554
- (proxy fix) - call connect on prisma client when running setup by @ishaan-jaff in #6534
- Add 3.5 haiku by @paul-gauthier in #6588
- Litellm perf improvements 3 by @krrishdholakia in #6573
- (fix) /image/generation - ImageObject conversion when
content_filter_resultsexists by @ishaan-jaff in #6584 - (fix) litellm.text_completion raises a non-blocking error on simple usage by @ishaan-jaff in #6546
- (feat) add
Predicted Outputsfor OpenAI by @ishaan-jaff in #6594 - (fix) Vertex Improve Performance when using
image_urlby @ishaan-jaff in #6593 - fix(lowest_tpm_rpm_routing.py): fix parallel rate limit check by @krrishdholakia in #6577
New Contributors
- @emmanuel-ferdman made their first contribution in #6554
Full Changelog: v1.51.3...v1.52.0-stable
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.52.0-stable
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 240.0 | 270.29554346208295 | 6.124428891308869 | 0.0 | 1833 | 0 | 212.83106800001406 | 1622.2440090000418 |
| Aggregated | Passed ✅ | 240.0 | 270.29554346208295 | 6.124428891308869 | 0.0 | 1833 | 0 | 212.83106800001406 | 1622.2440090000418 |
Contributors
krrishdholakia, dependabot, and 3 other contributors
Assets 4
v1.52.0
What's Changed
- LiteLLM Minor Fixes & Improvements (11/01/2024) by @krrishdholakia in #6551
- Litellm dev 11 02 2024 by @krrishdholakia in #6561
- build(deps): bump cookie and express in /docs/my-website by @dependabot in #6566
- docs(virtual_keys.md): update Dockerfile reference by @emmanuel-ferdman in #6554
- (proxy fix) - call connect on prisma client when running setup by @ishaan-jaff in #6534
- Add 3.5 haiku by @paul-gauthier in #6588
- Litellm perf improvements 3 by @krrishdholakia in #6573
- (fix) /image/generation - ImageObject conversion when
content_filter_resultsexists by @ishaan-jaff in #6584 - (fix) litellm.text_completion raises a non-blocking error on simple usage by @ishaan-jaff in #6546
- (feat) add
Predicted Outputsfor OpenAI by @ishaan-jaff in #6594 - (fix) Vertex Improve Performance when using
image_urlby @ishaan-jaff in #6593 - fix(lowest_tpm_rpm_routing.py): fix parallel rate limit check by @krrishdholakia in #6577
New Contributors
- @emmanuel-ferdman made their first contribution in #6554
Full Changelog: v1.51.3...v1.52.0
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.52.0
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 210.0 | 231.0704959909717 | 6.291122309918775 | 0.0 | 1883 | 0 | 180.74613400000317 | 2618.3897060000163 |
| Aggregated | Passed ✅ | 210.0 | 231.0704959909717 | 6.291122309918775 | 0.0 | 1883 | 0 | 180.74613400000317 | 2618.3897060000163 |
Contributors
krrishdholakia, dependabot, and 3 other contributors
Assets 4
v1.51.3-dev1
What's Changed
- LiteLLM Minor Fixes & Improvements (11/01/2024) by @krrishdholakia in #6551
- Litellm dev 11 02 2024 by @krrishdholakia in #6561
Full Changelog: v1.51.3...v1.51.3-dev1
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.51.3-dev1
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 250.0 | 292.3714877928421 | 6.163980651581093 | 0.0 | 1844 | 0 | 226.11442700002726 | 2207.691740000001 |
| Aggregated | Passed ✅ | 250.0 | 292.3714877928421 | 6.163980651581093 | 0.0 | 1844 | 0 | 226.11442700002726 | 2207.691740000001 |
Contributors
krrishdholakia
Assets 4
v1.51.3
What's Changed
- Support specifying exponential backoff retry strategy when calling completions() by @dbczumar in #6520
- (fix) slack alerting - don't spam the failed cost tracking alert for the same model by @ishaan-jaff in #6543
- (feat) add XAI ChatCompletion Support by @ishaan-jaff in #6373
- LiteLLM Minor Fixes & Improvements (10/30/2024) by @krrishdholakia in #6519
Full Changelog: v1.51.2...v1.51.3
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.51.3
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 200.0 | 220.3819331893052 | 6.253936592654308 | 0.0 | 1870 | 0 | 179.7343989999831 | 3185.1700670000014 |
| Aggregated | Passed ✅ | 200.0 | 220.3819331893052 | 6.253936592654308 | 0.0 | 1870 | 0 | 179.7343989999831 | 3185.1700670000014 |
Contributors
krrishdholakia, ishaan-jaff, and dbczumar
Assets 4
v1.51.1-stable
What's Changed
- (UI) Delete Internal Users on Admin UI by @ishaan-jaff in #6442
- (testing) increase prometheus.py test coverage to 90% by @ishaan-jaff in #6466
- (Feat) New Logging integration - add Datadog LLM Observability support by @ishaan-jaff in #6449
- (feat) add pricing for amazon.titan-embed-image-v1 by @ishaan-jaff in #6444
- LiteLLM Minor Fixes & Improvements (10/24/2024) by @krrishdholakia in #6441
- Litellm dev 10 26 2024 by @krrishdholakia in #6472
- (Testing) Add unit testing for DualCache - ensure in memory cache is used when expected by @ishaan-jaff in #6471
- redis otel tracing + async support for latency routing by @krrishdholakia in #6452
- (fix) Prometheus - Log Postgres DB latency, status on prometheus by @ishaan-jaff in #6484
- (router_strategy/) ensure all async functions use async cache methods by @ishaan-jaff in #6489
- (fix) proxy - fix when
STORE_MODEL_IN_DBshould be set by @ishaan-jaff in #6492 - (fix)
PrometheusServicesLogger_get_metricshould return metric in Registry by @ishaan-jaff in #6486 - Add
azure/gpt-4o-mini-2024-07-18to model_prices_and_context_window… by @xingyaoww in #6477 - Update utils.py by @vibhanshu-ob in #6468
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:litellm_stable_nov1-v1.51.1
Full Changelog: v1.51.2...v1.51.1-stable
Contributors
krrishdholakia, ishaan-jaff, and 2 other contributors
Assets 2
custom-docker-image-test-dev
What's Changed
- Support specifying exponential backoff retry strategy when calling completions() by @dbczumar in #6520
Full Changelog: v1.51.2...custom-docker-image-test-dev
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-custom-docker-image-test-dev
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-custom-docker-image-test-dev
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-custom-docker-image-test-dev
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-custom-docker-image-test-dev
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-custom-docker-image-test-dev
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 250.0 | 277.2980743547866 | 6.178135563258271 | 0.0 | 1849 | 0 | 222.02958399998352 | 3219.9342000000115 |
| Aggregated | Passed ✅ | 250.0 | 277.2980743547866 | 6.178135563258271 | 0.0 | 1849 | 0 | 222.02958399998352 | 3219.9342000000115 |
Contributors
dbczumar
Assets 4
v1.51.2
What's Changed
- (perf) Litellm redis router fix - ~100ms improvement by @krrishdholakia in #6483
- LiteLLM Minor Fixes & Improvements (10/28/2024) by @krrishdholakia in #6475
- Litellm dev 10 29 2024 by @krrishdholakia in #6502
- Litellm router max depth by @krrishdholakia in #6501
- (UI) fix bug with rendering max budget = 0 by @ishaan-jaff in #6506
- (UI) fix + test displaying number of keys an internal user owns by @ishaan-jaff in #6507
- (UI) Fix viewing members, keys in a team + added testing by @ishaan-jaff in #6514
Full Changelog: v1.51.1...v1.51.2
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.51.2
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 250.0 | 287.73103211135674 | 6.039141910660648 | 0.0 | 1805 | 0 | 213.5724959999834 | 2930.6253560000073 |
| Aggregated | Passed ✅ | 250.0 | 287.73103211135674 | 6.039141910660648 | 0.0 | 1805 | 0 | 213.5724959999834 | 2930.6253560000073 |
Contributors
krrishdholakia and ishaan-jaff
Assets 4
v1.51.1-staging
What's Changed
- (perf) Litellm redis router fix - ~100ms improvement by @krrishdholakia in #6483
- LiteLLM Minor Fixes & Improvements (10/28/2024) by @krrishdholakia in #6475
- Litellm dev 10 29 2024 by @krrishdholakia in #6502
- Litellm router max depth by @krrishdholakia in #6501
- (UI) fix bug with rendering max budget = 0 by @ishaan-jaff in #6506
- (UI) fix + test displaying number of keys an internal user owns by @ishaan-jaff in #6507
- (UI) Fix viewing members, keys in a team + added testing by @ishaan-jaff in #6514
Full Changelog: v1.51.1...v1.51.1-staging
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.51.1-staging
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Failed ❌ | 270.0 | 311.93605914725106 | 6.080288332872121 | 0.0033408177653143525 | 1820 | 1 | 117.93499300000576 | 3293.080912999983 |
| Aggregated | Failed ❌ | 270.0 | 311.93605914725106 | 6.080288332872121 | 0.0033408177653143525 | 1820 | 1 | 117.93499300000576 | 3293.080912999983 |
Contributors
krrishdholakia and ishaan-jaff