BricksLLM: AI Gateway For Putting LLM In Production

Tip

A managed version of BricksLLM is also available! It is production ready, and comes with a dashboard to make interacting with BricksLLM easier. Try us out for free today!

BricksLLM is a cloud native AI gateway written in Go. Currently, it provide native support for OpenAI, Anthropic and Azure OpenAI. We let you create API keys that have rate limits, cost limits and TTLs. The API keys can be used in both development and production to achieve fine-grained access control that is not provided by any of the foundational model providers. The proxy is designed to be 100% compatible with existing SDKs.

Features

Roadmap

Access control via API key with rate limit, cost limit and ttl
Logging integration
Statsd integration
Custom Provider Integration
PII detection and masking 🚧

Getting Started

The easiest way to get started with BricksLLM is through BricksLLM-Docker.

Step 1 - Clone BricksLLM-Docker repository

git clone https://github.com/bricks-cloud/BricksLLM-Docker

Step 2 - Change to BricksLLM-Docker directory

cd BricksLLM-Docker

Step 3 - Deploy BricksLLM locally with Postgresql and Redis

docker-compose up

You can run this in detach mode use the -d flag: docker-compose up -d

Step 4 - Create a provider setting

curl -X PUT http://localhost:8001/api/provider-settings \
   -H "Content-Type: application/json" \
   -d '{
          "provider":"openai",
          "setting": {
             "apikey": "YOUR_OPENAI_KEY"
          }
      }'

Copy the id from the response.

Step 5 - Create a Bricks API key

Use id from the previous step as settingId to create a key with a rate limit of 2 req/min and a spend limit of 25 cents.

curl -X PUT http://localhost:8001/api/key-management/keys \
   -H "Content-Type: application/json" \
   -d '{
	      "name": "My Secret Key",
	      "key": "my-secret-key",
	      "tags": ["mykey"],
        "settingIds": ["ID_FROM_STEP_FOUR"],
        "rateLimitOverTime": 2,
        "rateLimitUnit": "m",
        "costLimitInUsd": 0.25
      }'

Congratulations you are done!!!

Then, just redirect your requests to us and use OpenAI as you would normally. For example:

curl -X POST http://localhost:8002/api/providers/openai/v1/chat/completions \
   -H "Authorization: Bearer my-secret-key" \
   -H "Content-Type: application/json" \
   -d '{
          "model": "gpt-3.5-turbo",
          "messages": [
              {
                  "role": "system",
                  "content": "hi"
              }
          ]
      }'

Or if you're using an SDK, you could change its baseURL to point to us. For example:

// OpenAI Node SDK v4
import OpenAI from 'openai';

const openai = new OpenAI({
  apiKey: "some-secret-key", // key created earlier
  baseURL: "http://localhost:8002/api/providers/openai/v1", // redirect to us
});

How to Update?

For updating to the latest version

docker pull luyuanxin1995/bricksllm:latest

For updating to a particular version

docker pull luyuanxin1995/bricksllm:1.4.0

Documentation

Environment variables

Name type description default

POSTGRESQL_HOSTS required Hosts for Postgresql DB. Separated by , localhost

POSTGRESQL_DB_NAME optional Name for Postgresql DB.

POSTGRESQL_USERNAME required Postgresql DB username

POSTGRESQL_PASSWORD required Postgresql DB password

POSTGRESQL_SSL_MODE optional Postgresql SSL mode disable

POSTGRESQL_PORT optional The port that Postgresql DB runs on 5432

POSTGRESQL_READ_TIME_OUT optional Timeout for Postgresql read operations 2s

POSTGRESQL_WRITE_TIME_OUT optional Timeout for Postgresql write operations 1s

REDIS_HOSTS required Host for Redis. Separated by , localhost

REDIS_PASSWORD optional Redis Password

REDIS_PORT optional The port that Redis DB runs on 6379

REDIS_READ_TIME_OUT optional Timeout for Redis read operations 1s

REDIS_WRITE_TIME_OUT optional Timeout for Redis write operations 500ms

IN_MEMORY_DB_UPDATE_INTERVAL optional The interval BricksLLM API gateway polls Postgresql DB for latest key configurations 1s

STATS_PROVIDER optional This value can only be datadog. Required for integration with Datadog.

PROXY_TIMEOUT optional Timeout for proxy HTTP requests.

ADMIN_PASS optional Simple password authentication for admin endpoints.

Configuration Endpoints

The configuration server runs on Port 8001.

Headers

name type data type description

X-API-KEY optional string Key authentication header.

Get keys: GET /api/key-management/keys

Description

This endpoint is set up for retrieving key configurations using a query param called tag.

Query Parameters

name type data type description

tag optional string Identifier attached to a key configuration

tags optional []string Identifiers attached to a key configuration

provider optional string Provider attached to a key provider configuration. Its value can only be openai.

keyIds optional []string Unique identifiers for keys.

Error Response

http code content-type

400, 500 application/json

Field type example

status int 400

title string request body reader error

type string /errors/request-body-read

detail string something is wrong

instance string /api/key-management/keys

Response

Response Body

[]KeyConfiguration

Fields of KeyConfiguration

Field type example description

name string spike's developer key Name of the API key.

createdAt int64 1257894000 Key configuration creation time in unix.

updatedAt int64 1257894000 Key configuration update time in unix.

revoked boolean true Indicator for whether the key is revoked.

revokedReason string The key has expired Reason for why the key is revoked.

tags []string ["org-tag-12345"] Identifiers associated with the key.

keyId string 550e8400-e29b-41d4-a716-446655440000 Unique identifier for the key.

costLimitInUsd float64 5.5 Total spend limit of the API key.

costLimitInUsdOverTime float64 2 Total spend within period of time. This field is required if costLimitInUsdUnit is specified.

costLimitInUsdUnit enum d Time unit for costLimitInUsdOverTime. Possible values are [m, h, d, mo].

rateLimitOverTime int 2 rate limit over period of time. This field is required if rateLimitUnit is specified.

rateLimitUnit string m Time unit for rateLimitOverTime. Possible values are [h, m, s, d]

ttl string 2d time to live. Available units are [s, m, h]

allowedPaths []PathConfig [{ "path": "/api/providers/openai/v1/chat/completion", "method": "POST"}] Allowed paths that can be accessed using the key.

settingId string 98daa3ae-961d-4253-bf6a-322a32fdca3d This field is DEPERCATED. Use settingIds field instead.

settingIds string [98daa3ae-961d-4253-bf6a-322a32fdca3d] Setting ids associated with the key.

shouldLogRequest bool false Should request be stored.

shouldLogResponse bool true Should response be stored.

rotationEnabled bool false Should key rotate setting used to access third party endpoints in order to circumvent rate limits.

Create key: PUT /api/key-management/keys

Description

This endpoint is set up for retrieving key configurations using a query param called tag.

Request

PathConfig

Field required type example description

path required string /api/providers/openai/v1/chat/completion Allowed path

method required string POST HTTP Method

Field required type example description

name required string spike's developer key Name of the API key.

tags optional []string ["org-tag-12345"] Identifiers associated with the key.

key required string abcdef12345 API key.

settingId depercated string 98daa3ae-961d-4253-bf6a-322a32fdca3d This field is DEPERCATED. Use settingIds field instead.

settingIds required string 98daa3ae-961d-4253-bf6a-322a32fdca3d Setting ids associated with the key.

costLimitInUsd optional float64 5.5 Total spend limit of the API key.

costLimitInUsdOverTime optional float64 2 Total spend within period of time. This field is required if costLimitInUsdUnit is specified.

costLimitInUsdUnit optional enum d Time unit for costLimitInUsdOverTime. Possible values are [m, h, d, mo].

rateLimitOverTime optional int 2 rate limit over period of time. This field is required if rateLimitUnit is specified.

rateLimitUnit optional enum m Time unit for rateLimitOverTime. Possible values are [h, m, s, d]

ttl optional string 2d time to live. Available units are [s, m, h].

allowedPaths optional []PathConfig 2d Pathes allowed for access.

shouldLogRequest optional bool false Should request be stored.

shouldLogResponse optional bool true Should response be stored.

rotationEnabled optional bool false Should key rotate setting used to access third party endpoints in order to circumvent rate limits.

Error Response

http code content-type

400, 500 application/json

Field type example

status int 400

title string request body reader error

type string /errors/request-body-read

detail string something is wrong

instance string /api/key-management/keys

Responses

Field type example description

name string spike's developer key Name of the API key.

createdAt int64 1257894000 Key configuration creation time in unix.

updatedAt int64 1257894000 Key configuration update time in unix.

revoked boolean true Indicator for whether the key is revoked.

revokedReason string The key has expired Reason for why the key is revoked.

tags []string ["org-tag-12345"] Identifiers associated with the key.

keyId string 550e8400-e29b-41d4-a716-446655440000 Unique identifier for the key.

costLimitInUsd float64 5.5 Total spend limit of the API key.

costLimitInUsdOverTime float64 2 Total spend within period of time. This field is required if costLimitInUsdUnit is specified.

costLimitInUsdUnit enum d Time unit for costLimitInUsdOverTime. Possible values are [m, h, d, mo].

rateLimitOverTime int 2 rate limit over period of time. This field is required if rateLimitUnit is specified.

rateLimitOverTime int 2 rate limit over period of time. This field is required if rateLimitUnit is specified.

rateLimitUnit string m Time unit for rateLimitOverTime. Possible values are [h, m, s, d].

ttl string 2d time to live. Available units are [s, m, h]

allowedPaths []PathConfig [{ "path": "/api/providers/openai/v1/chat/completion", method: "POST"}] Allowed paths that can be accessed using the key.

settingId string 98daa3ae-961d-4253-bf6a-322a32fdca3d This field is DEPERCATED. Use settingIds field instead.

settingIds string [98daa3ae-961d-4253-bf6a-322a32fdca3d] Setting ids associated with the key.

shouldLogRequest bool false Should request be stored.

shouldLogResponse bool true Should response be stored.

rotationEnabled bool false Should key rotate setting used to access third party endpoints in order to circumvent rate limits.

Update key: PATCH /api/key-management/keys/{keyId}

Description

This endpoint is set up for updating key configurations using key id.

Parameters

name type data type description

keyId required string Unique key configuration identifier.

Request

PathConfig

Field required type example description

path required string /api/providers/openai/v1/chat/completion Allowed path

method required string POST HTTP Method

Field required type example description

settingId optional string 98daa3ae-961d-4253-bf6a-322a32fdca3d This field is DEPERCATED. Use settingIds field instead.

settingIds optional string 98daa3ae-961d-4253-bf6a-322a32fdca3d Setting ids associated with the key.

name optional string spike's developer key Name of the API key.

tags optional []string ["org-tag-12345"] Identifiers associated with the key.

revoked optional boolean true Indicator for whether the key is revoked.

revokedReason optional string The key has expired Reason for why the key is revoked.

costLimitInUsd optional float64 5.5 Total spend limit of the API key.

costLimitInUsdOverTime optional float64 2 Total spend within period of time. This field is required if costLimitInUsdUnit is specified.

costLimitInUsdUnit optional enum d Time unit for costLimitInUsdOverTime. Possible values are [m, h, d, mo].

rateLimitOverTime optional int 2 rate limit over period of time. This field is required if rateLimitUnit is specified.

rateLimitUnit optional string m Time unit for rateLimitOverTime. Possible values are [h, m, s, d]

allowedPaths optional [{ "path": "/api/providers/openai/v1/chat/completions", "method": "POST"}] `` Pathes allowed for access.

shouldLogRequest optional bool false Should request be stored.

shouldLogResponse optional bool true Should response be stored.

rotationEnabled optional bool false Should key rotate setting used to access third party endpoints in order to circumvent rate limits.

Error Response

http code content-type

400, 500 application/json

Field type example

status int 400

title string request body reader error

type string /errors/request-body-read

detail string something is wrong

instance string /api/key-management/keys

Response

Field type example description

name string spike's developer key Name of the API key.

createdAt int64 1257894000 Key configuration creation time in unix.

updatedAt int64 1257894000 Key configuration update time in unix.

revoked boolean true Indicator for whether the key is revoked.

revokedReason string The key has expired Reason for why the key is revoked.

tags []string ["org-tag-12345"] Identifiers associated with the key.

keyId string 550e8400-e29b-41d4-a716-446655440000 Unique identifier for the key.

costLimitInUsd float64 5.5 Total spend limit of the API key.

costLimitInUsdOverTime float64 2 Total spend within period of time. This field is required if costLimitInUsdUnit is specified.

costLimitInUsdUnit enum d Time unit for costLimitInUsdOverTime. Possible values are [m, h, d, mo].

rateLimitOverTime int 2 rate limit over period of time. This field is required if rateLimitUnit is specified.

rateLimitUnit string m Time unit for rateLimitOverTime. Possible values are [h, m, s, d]

ttl string 2d time to live. Available units are [s, m, h]

allowedPaths []PathConfig [{ "path": "/api/providers/openai/v1/chat/completion", method: "POST"}] Allowed paths that can be accessed using the key.

settingId string 98daa3ae-961d-4253-bf6a-322a32fdca3d This field is DEPERCATED. Use settingIds field instead.

settingIds string [98daa3ae-961d-4253-bf6a-322a32fdca3d] Setting ids associated with the key.

shouldLogRequest bool false Should request be stored.

shouldLogResponse bool true Should response be stored.

rotationEnabled bool false Should key rotate setting used to access third party endpoints in order to circumvent rate limits.

Create a provider setting: POST /api/provider-settings

Description

This endpoint is creating a provider setting.

Request

Field required type example description

provider required enum openai This value can only be openai, anthropic and azure as for now.

setting required Setting { "apikey": "YOUR_OPENAI_KEY" } A map of values used for authenticating with the selected provider.

name optional string YOUR_PROVIDER_SETTING_NAME This field is used for giving a name to provider setting

allowedModels []string ["text-embedding-ada-002"] Allowed models for this provider setting.

Setting

Field required type example description

apiKey required string xx-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx This value is required.

resourceName required string YOUR_AZURE_RESOURCE_NAME This value is required when the provider is azure.

Error Response

http code content-type

400, 500 application/json

Field type example

status int 400

title string request body reader error

type string /errors/request-body-read

detail string something is wrong

instance string /api/provider-settings

Response

Field type example description

createdAt int64 1699933571 Unix timestamp for creation time.

updatedAt int64 1699933571 Unix timestamp for update time.

provider enum openai This value can only be openai as for now.

id string 98daa3ae-961d-4253-bf6a-322a32fdca3d This value is a unique identifier.

name string YOUR_PROVIDER_SETTING_NAME Provider setting name.

allowedModels []string ["text-embedding-ada-002"] Allowed models for this provider setting.

Get provider settings: GET /api/provider-settings

Description

This endpoint is getting provider settings.

Query Parameters

name type data type description

ids optional []string Provider setting ids

Error Response

http code content-type

500 application/json

Field type example

status int 400

title string request body reader error

type string /errors/request-body-read

detail string something is wrong

instance string /api/provider-settings

Response

[]ProviderSetting

ProviderSetting

Field type example description

createdAt int64 1699933571 Unix timestamp for creation time.

updatedAt int64 1699933571 Unix timestamp for update time.

provider enum openai This value can only be openai as for now.

id string 98daa3ae-961d-4253-bf6a-322a32fdca3d This value is a unique identifier.

name string YOUR_PROVIDER_SETTING_NAME Provider setting name.

allowedModels []string ["text-embedding-ada-002"] Allowed models for this provider setting.

Update a provider setting: PATCH /api/provider-settings/:id

Description

This endpoint is updating a provider setting .

Parameters

name type data type description

id required string Unique identifier for the provider setting that you want to update.

Request

Field required type example description

setting required Setting { "apikey": "YOUR_OPENAI_KEY" } A map of values used for authenticating with the selected provider.

name optional string YOUR_PROVIDER_SETTING_NAME This field is used for giving a name to provider setting

allowedModels []string ["text-embedding-ada-002"] Allowed models for this provider setting.

Setting

Field required type example description

apiKey required string xx-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx This value is required.

resourceName required string YOUR_AZURE_RESOURCE_NAME This value is required when the provider is azure.

Error Response

http code content-type

400, 500 application/json

Field type example

status int 400

title string request body reader error

type string /errors/request-body-read

detail string something is wrong

instance string /api/provider-settings

Response

Field type example description

createdAt int64 1699933571 Unix timestamp for creation time.

updatedAt int64 1699933571 Unix timestamp for update time.

provider enum openai This value can only be openai as for now.

id string 98daa3ae-961d-4253-bf6a-322a32fdca3d This value is a unique identifier

name string YOUR_PROVIDER_SETTING_NAME Provider setting name.

allowedModels []string ["text-embedding-ada-002"] Allowed models for this provider setting.

Retrieve Metrics: POST /api/reporting/events

Description

This endpoint is retrieving aggregated metrics given an array of key ids and tags.

Request

Field required type example description

keyIds required []string ["key-1", "key-2", "key-3" ] Array of ids that specicify the keys that you want to aggregate stats from.

tags required []string ["tag-1", "tag-2"] Array of tags that specicify the key tags that you want to aggregate stats from.

customIds required []string ["customId-1", "customId-2"] A list of custom IDs that you want to aggregate stats from.

filters required []string ["model", "keyId"] Group by data points through different filters(model,keyId or customId).

start required int64 1699933571 Start timestamp for the requested timeseries data.

end required int64 1699933571 End timestamp for the requested timeseries data.

increment required int 60 This field is the increment in seconds for the requested timeseries data.

Error Response

http code content-type

500 application/json

Field type example

status int 400

title string request body reader error

type string /errors/request-body-read

detail string something is wrong

instance string /api/provider-settings

Response

Field type example description

dataPoints []dataPoint [{ "timeStamp": 1699933571, "numberOfRequests": 1, "costInUsd": 0.8, "latencyInMs": 600, "promptTokenCount": 0, "completionTokenCount": 0, "successCount": 1 }] Unix timestamp for creation time.

latencyInMsMedian float64 656.7 Median latency for the given time period.

latencyInMs99th float64 555.7 99th percentile latency for the given time period.

Datapoint

Field type example description

timeStamp int64 1702504746 Unix timestamp for the data point

numberOfRequests int64 100 Aggregated number of http requests over the given time increment.

costInUsd float64 1.7 Aggregated cost of proxied requests in USD over the given time increment.

latencyInMs int 555 Aggregated latency in milliseconds of http requests over the given time increment.

promptTokenCount int 25 Aggregated prompt token counts over the given time increment.

completionTokenCount int 4000 Aggregated completion token counts over the given time increment.

successCount int 555 Aggregated number of successful http requests over the given time increment.

keyId int 555.7 key Id associated with the event.

model string gpt-3.5-turbo model associated with the event.

customId string customId customId associated with the event.

Get events: GET /api/events

Description

This endpoint is for getting events.

Query Parameters

name type data type description

customId optional string Custom identifier attached to an event.

keyIds optional []string A list of key IDs.

start required if keyIds is specified int64 Start timestamp.

end required if keyIds is specified int64 End timestamp.

Error Response

http code content-type

500 application/json

Field type example

status int 400

title string request body reader error

type string /errors/request-body-read

detail string something is wrong

instance string /api/provider-settings

Response

[]Event

Event

Field type example description

id int64 1699933571 Unique identifier associated with the event.

created_at int64 1699933571 Unix timestamp for creation time.

tags int64 ["YOUR_TAG"] Tags of the key.

key_id string YOUR_KEY_ID Key Id associated with the proxy request.

cost_in_usd float64 0.0004 Cost incured by the proxy request.

model string gpt-4-1105-preview Model used in the proxy request.

provider string openai Provider for the proxy request.

status int 200 Http status.

prompt_token_count int 8 Prompt token count of the proxy request.

completion_token_count int 16 Completion token counts of the proxy request.

latency_in_ms int 160 Provider setting name.

path string /api/v1/chat/completion Provider setting name.

method string POST Http method for the assoicated proxu request.

custom_id string YOUR_CUSTOM_ID Custom Id passed by the user in the headers of proxy requests.

request []byte {} Request in bytes to providers.

custom_id string YOUR_CUSTOM_ID Custom Id passed by the user in the headers of proxy requests.

response []byte {} Response in bytes from providers.

Create custom provider: POST /api/custom/providers

Description

This endpoint is creating custom providers.

RouteConfig

Field required type example description

path required string /chat/completion Path associated with the custom provider route. It must be unique within the custom provider.

target_url required string https://api.openai.com/v1/chat/completions Proxy destination URL for the custom provider route.

model_location required string model JSON field for the model in the HTTP request.

request_prompt_location required string messages.#.content JSON field for the prompt request in the HTTP request.

response_completion_location required string choices.#.message.content JSON field for the completion content in the HTTP response.

stream_location required string stream JSON field for the stream boolean in the HTTP request.

stream_end_word required string [DONE] End word for the stream.

stream_response_completion_location required string choices.#.delta.content JSON field for the completion content in the streaming response.

stream_max_empty_messages required int 10 Number of max empty messages in stream.

Request

Field required type example description

provider required string bricks Unique identifier associated with the route config.

route_configs required []RouteConfig {{ "path": "/chat/completions", "target_url": "https://api.openai.com/v1/chat/completions" }} Route configurations for the custom provider.

authentication_param optional string apikey The authentication parameter required for.

Error Response

http code content-type

500, 400 application/json

Field type example

status int 400

title string request body reader error

type string /errors/request-body-read

detail string something is wrong

instance string /api/custom/providers

Response

Field type example description

id int64 1699933571 Unique identifier associated with the event.

created_at int64 1699933571 Unix timestamp for creation time.

updated_at int64 1699933571 Unix timestamp for update time.

provider string bricks Unique identifier associated with the route config.

route_configs []RouteConfig {{ "path": "/chat/completions", "target_url": "https://api.openai.com/v1/chat/completions" }} Start timestamp for the requested timeseries data.

authentication_param string apikey The authentication parameter required for.

Update custom provider: PATCH /api/custom/providers/:id

Description

This endpoint is updating a custom provider.

RouteConfig

Field required type example description

path required string /chat/completion Path associated with the custom provider route. It must be unique within the custom provider.

target_url required string https://api.openai.com/v1/chat/completions Proxy destination URL for the custom provider route.

model_location required string model JSON field for the model in the HTTP request.

request_prompt_location required string messages.#.content JSON field for the prompt request in the HTTP request.

response_completion_location required string choices.#.message.content JSON field for the completion content in the HTTP response.

stream_location required string stream JSON field for the stream boolean in the HTTP request.

stream_end_word required string [DONE] End word for the stream.

stream_response_completion_location required string choices.#.delta.content JSON field for the completion content in the streaming response.

stream_max_empty_messages required int 10 Number of max empty messages in stream.

Request

Field required type example description

route_configs optional []RouteConfig {{ "path": "/chat/completions", "target_url": "https://api.openai.com/v1/chat/completions" }} Route configurations for the custom provider.

authentication_param optional string apikey The authentication parameter required for.

Error Response

http code content-type

500, 404, 400 application/json

Field type example

status int 400

title string request body reader error

type string /errors/request-body-read

detail string something is wrong

instance string /api/custom/providers

Response

Field type example description

id int64 1699933571 Unique identifier associated with the event.

created_at int64 1699933571 Unix timestamp for creation time.

updated_at int64 1699933571 Unix timestamp for update time.

provider string bricks Unique identifier associated with the route config.

route_configs []RouteConfig {{ "path": "/chat/completions", "target_url": "https://api.openai.com/v1/chat/completions" }} Start timestamp for the requested timeseries data.

authentication_param string apikey The authentication parameter required for.

Get custom providers: GET /api/custom/providers

Description

This endpoint is for getting custom providers.

RouteConfig

Field required type example description

path required string /chat/completion Path associated with the custom provider route. It must be unique within the custom provider.

target_url required string https://api.openai.com/v1/chat/completions Proxy destination URL for the custom provider route.

model_location required string model JSON field for the model in the HTTP request.

request_prompt_location required string messages.#.content JSON field for the prompt request in the HTTP request.

response_completion_location required string choices.#.message.content JSON field for the completion content in the HTTP response.

stream_location required string stream JSON field for the stream boolean in the HTTP request.

stream_end_word required string [DONE] End word for the stream.

stream_response_completion_location required string choices.#.delta.content JSON field for the completion content in the streaming response.

stream_max_empty_messages required int 10 Number of max empty messages in stream.

Request

Field required type example description

route_configs optional []RouteConfig {{ "path": "/chat/completions", "target_url": "https://api.openai.com/v1/chat/completions" }} Route configurations for the custom provider.

authentication_param optional string apikey The authentication parameter required for.

Error Response

http code content-type

500 application/json

Field type example

status int 400

title string request body reader error

type string /errors/request-body-read

detail string something is wrong

instance string /api/custom/providers

Response

[]Provider

Provider

Field type example description

id int64 9e6e8b27-2ce0-4ef0-bdd7-1ed3916592eb Unique identifier associated with the event.

created_at int64 1699933571 Unix timestamp for creation time.

updated_at int64 1699933571 Unix timestamp for update time.

provider string bricks Unique identifier associated with the route config.

route_configs []RouteConfig {{ "path": "/chat/completions", "target_url": "https://api.openai.com/v1/chat/completions" }} Start timestamp for the requested timeseries data.

authentication_param string apikey The authentication parameter required for.

Create routes: POST /api/routes

Description

This endpoint is for creating routes.

StepConfig

Field required type example description

provider required enum azure Provider for the step. Can only be either azure or openai.

model required string gpt-3.5-turbo Model that the step should call. Can only be chat completion or embedding models from OpenAI or Azure OpenAI.

retries optional int 2 Number of retries.

params optional object { deploymentId: "ada-test",apiVersion: "2022-12-01" } Params required for maing API requests to desired modela and provider combo. Required if the provider is azure

timeout optional string 5s Timeout desired for each request. Default value is 5m.

CacheConfig

Field required type example description

enabled required bool false Boolean flag indicating whether caching is enabled.

ttl optional string 5s TTL for the cache. Default value is 168h.

Request

Field required type example description

name required string staging-openai-azure-completion-route Name for the route.

path required string / Unique identifier for.

steps required []StepConfig apikey The authentication parameter required for.

keyIds required []string [] The authentication parameter required for.

cacheConfig required CacheConfig [] The authentication parameter required for.

Error Response

http code content-type

500, 400 application/json

Field type example

status int 400

title string request body reader error

type string /errors/request-body-read

detail string something is wrong

instance string /api/routes

Response

Field type example description

id required string 9e6e8b27-2ce0-4ef0-bdd7-1ed3916592eb

createdAt required string 1699933571

updatedAt required string 1699933571

name required string staging-openai-azure-completion-route

path required string /production/chat/completion

steps required []StepConfig [{"retries": 2, "provider": "openai", "params": {}, "model": "gpt-3.5-turbo", "timeout": "1s"}]

keyIds required []string ["9e6e8b27-2ce0-4ef0-bdd7-1ed3916592eb"]

cacheConfig required CacheConfig { "enabled": false, "ttl": "5s" }

Retrieve a route: GET /api/routes/:id

Description

This endpoint is for retrieving a route.

Parameters

name type data type description

id required string Unique identifier for the route.

Error Response

http code content-type

500, 404 application/json

Field type example

status int 404

title string request body reader error

type string /errors/request-body-read

detail string something is wrong

instance string /api/routes/:id

Response

Field type example description

id required string 9e6e8b27-2ce0-4ef0-bdd7-1ed3916592eb

createdAt required string 1699933571

updatedAt required string 1699933571

name required string staging-openai-azure-completion-route

path required string /production/chat/completion

steps required []StepConfig [{"retries": 2, "provider": "openai", "params": {}, "model": "gpt-3.5-turbo", "timeout": "1s"}]

keyIds required []string ["9e6e8b27-2ce0-4ef0-bdd7-1ed3916592eb"]

cacheConfig required CacheConfig { "enabled": false, "ttl": "5s" }

Retrieve routes: GET /api/routes

Description

This endpoint is for retrieving routes.

Error Response

http code content-type

500 application/json

Field type example

status int 404

title string request body reader error

type string /errors/request-body-read

detail string something is wrong

instance string /api/routes/:id

CacheConfig

Field required type example description

enabled required bool false Boolean flag indicating whether caching is enabled.

ttl optional string 5s TTL for the cache. Default value is 168h.

StepConfig

Field required type example description

provider required enum azure Provider for the step. Can only be either azure or openai.

model required string gpt-3.5-turbo Model that the step should call. Can only be chat completion or embedding models from OpenAI or Azure OpenAI.

retries optional int 2 Number of retries.

params optional object { deploymentId: "ada-test",apiVersion: "2022-12-01" } Params required for maing API requests to desired modela and provider combo. Required if the provider is azure

timeout optional string 5s Timeout desired for each request. Default value is 5m.

RouteConfig

Field type example description

id required string 9e6e8b27-2ce0-4ef0-bdd7-1ed3916592eb

createdAt required string 1699933571

updatedAt required string 1699933571

name required string staging-openai-azure-completion-route

path required string /production/chat/completion

steps required []StepConfig [{"retries": 2, "provider": "openai", "params": {}, "model": "gpt-3.5-turbo", "timeout": "1s"}]

keyIds required []string ["9e6e8b27-2ce0-4ef0-bdd7-1ed3916592eb"]

cacheConfig required CacheConfig { "enabled": false, "ttl": "5s" }

Response

[]RouteConfig

OpenAI Proxy

The OpenAI proxy runs on Port 8002.

Headers

name type data type description

x-custom-event-id optional string Custom Id that can be used to retrieve an event associated with each proxy request.

Chat Completion

Call OpenAI chat completions: POST /api/providers/openai/v1/chat/completions

Description

This endpoint is set up for proxying OpenAI chat completion requests. Documentation for this endpoint can be found here.

Embeddings

Call OpenAI embeddings: POST /api/providers/openai/v1/embeddings

Description

This endpoint is set up for proxying OpenAI embedding requests. Documentation for this endpoint can be found here.

Moderations

Call OpenAI moderations: POST /api/providers/openai/v1/moderations

Description

This endpoint is set up for proxying OpenAI moderation requests. Documentation for this endpoint can be found here.

Models

Get OpenAI models: GET /api/providers/openai/v1/models

Description

This endpoint is set up for retrieving OpenAI models. Documentation for this endpoint can be found here.

Retrieve an OpenAI model: GET /api/providers/openai/v1/models/:model

Description

This endpoint is set up for retrieving an OpenAI model. Documentation for this endpoint can be found here.

Files

List files: GET /api/providers/openai/v1/files

Description

This endpoint is set up for list OpenAI files. Documentation for this endpoint can be found here.

Upload a file: POST /api/providers/openai/v1/files

Description

This endpoint is set up for creating an OpenAI file. Documentation for this endpoint can be found here.

Delete a file: POST /api/providers/openai/v1/files/:file_id

Description

This endpoint is set up for creating an OpenAI file. Documentation for this endpoint can be found here.

Retrieve a file: GET /api/providers/openai/v1/files/:file_id

Description

This endpoint is set up for retrieving an OpenAI file. Documentation for this endpoint can be found here.

Retrieve file content: GET /api/providers/openai/v1/files/:file_id/content

Description

This endpoint is set up for retrieving an OpenAI file content. Documentation for this endpoint can be found here.

Images

Generate images: POST /api/providers/openai/v1/images/generations

Description

This endpoint is set up for generating OpenAI images. Documentation for this endpoint can be found here.

Edit images: POST /api/providers/openai/v1/images/edits

Description

This endpoint is set up for editting OpenAI generated images. Documentation for this endpoint can be found here.

Create image variations: POST /api/providers/openai/v1/images/variations

Description

This endpoint is set up for creating OpenAI image variations. Documentation for this endpoint can be found here.

Voices

Create speech: POST /api/providers/openai/v1/audio/speech

Description

This endpoint is set up for creating speeches. Documentation for this endpoint can be found here.

Create transcriptions: POST /api/providers/openai/v1/audio/transcriptions

Description

This endpoint is set up for editting generated images. Documentation for this endpoint can be found here.

Create translations: POST /api/providers/openai/v1/audios/translations

Description

This endpoint is set up for creating translations. Documentation for this endpoint can be found here.

Assistants

Create assistant: POST /api/providers/openai/v1/assistants

Description

This endpoint is set up for creating an OpenAI assistant. Documentation for this endpoint can be found here.

Retrieve assistant: GET /api/providers/openai/v1/assistants/:assistant_id

Description

This endpoint is set up for retrieving an OpenAI assistant. Documentation for this endpoint can be found here.

Modify assistant: POST /api/providers/openai/v1/assistants/:assistant_id

Description

This endpoint is set up for modifying an OpenAI assistant. Documentation for this endpoint can be found here.

Delete assistant: DELETE /api/providers/openai/v1/assistants/:assistant_id

Description

This endpoint is set up for deleting an OpenAI assistant. Documentation for this endpoint can be found here.

List assistants: GET /api/providers/openai/v1/assistants

Description

This endpoint is set up for listing OpenAI assistants. Documentation for this endpoint can be found here.

Create assistant file: POST /api/providers/openai/v1/assistants/:assistant_id/files

Description

This endpoint is set up for creating an OpenAI assistant file. Documentation for this endpoint can be found here.

Retrieve assistant file: GET /api/providers/openai/v1/assistants/:assistant_id/files/:file_id

Description

This endpoint is set up for retrieving an OpenAI assistant file. Documentation for this endpoint can be found here.

Delete assistant file: DELETE /api/providers/openai/v1/assistants/:assistant_id/files/:file_id

Description

This endpoint is set up for deleting an OpenAI assistant file. Documentation for this endpoint can be found here.

List assistant files: GET /api/providers/openai/v1/assistants/:assistant_id/files

Description

This endpoint is set up for retrieving OpenAI assistant files. Documentation for this endpoint can be found here.

Create thread: POST /api/providers/openai/v1/threads

Description

This endpoint is set up for creating an OpenAI thread. Documentation for this endpoint can be found here.

Retrieve thread: GET /api/providers/openai/v1/threads/:thread_id

Description

This endpoint is set up for retrieving an OpenAI thread. Documentation for this endpoint can be found here.

Modify thread: POST /api/providers/openai/v1/threads/:thread_id

Description

This endpoint is set up for modifying an OpenAI thread. Documentation for this endpoint can be found here.

Delete thread: DELETE /api/providers/openai/v1/threads/:thread_id

Description

This endpoint is set up for deleting an OpenAI thread. Documentation for this endpoint can be found here.

Create message: POST /api/providers/openai/v1/threads/:thread_id/messages

Description

This endpoint is set up for creating an OpenAI message. Documentation for this endpoint can be found here.

Retrieve message: GET /api/providers/openai/v1/threads/:thread_id/messages/:message_id

Description

This endpoint is set up for retrieving an OpenAI message. Documentation for this endpoint can be found here.

Modify message: POST /api/providers/openai/v1/files/:file_id/content

Description

This endpoint is set up for modifying an OpenAI message. Documentation for this endpoint can be found here.

List messages: GET /api/providers/openai/v1/threads/:thread_id/messages

Description

This endpoint is set up for listing OpenAI messages. Documentation for this endpoint can be found here.

Retrieve message file: GET /api/providers/openai/v1/threads/:thread_id/messages/:message_id/files/:file_id

Description

This endpoint is set up for retrieving an OpenAI message file. Documentation for this endpoint can be found here.

List message files: GET /api/providers/openai/v1/threads/:thread_id/messages/:message_id/files

Description

This endpoint is set up for retrieving OpenAI message files. Documentation for this endpoint can be found here.

Create run: POST /api/providers/openai/v1/threads/:thread_id/runs

Description

This endpoint is set up for creating an OpenAI run. Documentation for this endpoint can be found here.

Retrieve run: GET /api/providers/openai/v1/threads/:thread_id/runs/:run_id

Description

This endpoint is set up for retrieving an OpenAI run. Documentation for this endpoint can be found here.

Modify run: POST /api/providers/openai/v1/threads/:thread_id/runs/:run_id

Description

This endpoint is set up for modifying an OpenAI run. Documentation for this endpoint can be found here.

List runs: GET /api/providers/openai/v1/threads/runs

Description

This endpoint is set up for retrieving OpenAI runs. Documentation for this endpoint can be found here.

Submit tool outputs to run: POST /api/providers/openai/v1/threads/runs

Description

This endpoint is set up for submitting tool outputs to an OpenAI run. Documentation for this endpoint can be found here.

Cancel a run: POST /api/providers/openai/v1/threads/:thread_id/runs/:run_id/cancel

Description

This endpoint is set up for cancellling an OpenAI run. Documentation for this endpoint can be found here.

Create thread and run: POST /api/providers/openai/v1/threads/runs

Description

This endpoint is set up for creating an OpenAI thread and run. Documentation for this endpoint can be found here.

Retrieve run step: GET /api/providers/openai/v1/threads/:thread_id/runs/:run_id/steps/:step_id

Description

This endpoint is set up for retrieving an OpenAI run step. Documentation for this endpoint can be found here.

List run steps: GET /api/providers/openai/v1/threads/:thread_id/runs/:run_id/steps

Description

This endpoint is set up for listing OpenAI run steps. Documentation for this endpoint can be found here.

Azure OpenAI Proxy

The custom provider proxy runs on Port 8002.

Create Azure OpenAI chat completion: POST /api/providers/azure/openai/deployments/:deployment_id/chat/completions?api-version={API_VERSION}

Description

This endpoint is set up for proxying Anthropic completion requests. Documentation for this endpoint can be found here.

Create Azure OpenAI embeddings: POST /api/providers/azure/openai/deployments/:deployment_id/embeddings?api-version={API_VERSION}

Description

This endpoint is set up for proxying Azure OpenAI completion requests. Documentation for this endpoint can be found here.

Anthropic Proxy

The custom provider proxy runs on Port 8002.

Create Anthropic completion: POST /api/providers/anthropic/v1/complete

Description

This endpoint is set up for proxying Anthropic completion requests. Documentation for this endpoint can be found here.

Custom Provider Proxy

The custom provider proxy runs on Port 8002.

Call custom providers: POST /api/custom/providers/:provider/*

Description

First you need to use create custom providers endpoint to create custom providers. Then create corresponding provider setting for the newly created custom provider. Afterward, you can start creating keys associated with the custom provider, and use the keys to access this endpoint by placing the created key in Authorization: Bearer YOUR_BRICKSLLM_KEY as part of your HTTP request headers.

Route Proxy

The custom provider proxy runs on Port 8002.

Call a route: POST /api/route/*

Description

Route helps you interpolate different models (embeddings or chat completion models) and providers (OpenAI or Azure OpenAI) to gurantee API responses.

First you need to use create route endpoint to create routes. If the route uses both Azure and OpenAI, you need to create API keys with corresponding provider settings as well. If the route is for chat completion, just call the route using the OpenAI chat completion format. On the other hand, if the route is for embeddings, just call the route using the embeddings format.

Name		Name	Last commit message	Last commit date
Latest commit History 456 Commits
.github/workflows		.github/workflows
assets		assets
cmd/bricksllm		cmd/bricksllm
cookbook		cookbook
internal		internal
scripts		scripts
.gitignore		.gitignore
.goreleaser.yml		.goreleaser.yml
CHANGELOG.md		CHANGELOG.md
Dockerfile.datadog		Dockerfile.datadog
Dockerfile.dev		Dockerfile.dev
Dockerfile.prod		Dockerfile.prod
LICENSE		LICENSE
README.md		README.md
docker-compose.yml		docker-compose.yml
go.mod		go.mod
go.sum		go.sum

Name	type	description	default
`POSTGRESQL_HOSTS`	required	Hosts for Postgresql DB. Separated by ,	`localhost`
`POSTGRESQL_DB_NAME`	optional	Name for Postgresql DB.
`POSTGRESQL_USERNAME`	required	Postgresql DB username
`POSTGRESQL_PASSWORD`	required	Postgresql DB password
`POSTGRESQL_SSL_MODE`	optional	Postgresql SSL mode	`disable`
`POSTGRESQL_PORT`	optional	The port that Postgresql DB runs on	`5432`
`POSTGRESQL_READ_TIME_OUT`	optional	Timeout for Postgresql read operations	`2s`
`POSTGRESQL_WRITE_TIME_OUT`	optional	Timeout for Postgresql write operations	`1s`
`REDIS_HOSTS`	required	Host for Redis. Separated by ,	`localhost`
`REDIS_PASSWORD`	optional	Redis Password
`REDIS_PORT`	optional	The port that Redis DB runs on	`6379`
`REDIS_READ_TIME_OUT`	optional	Timeout for Redis read operations	`1s`
`REDIS_WRITE_TIME_OUT`	optional	Timeout for Redis write operations	`500ms`
`IN_MEMORY_DB_UPDATE_INTERVAL`	optional	The interval BricksLLM API gateway polls Postgresql DB for latest key configurations	`1s`
`STATS_PROVIDER`	optional	This value can only be datadog. Required for integration with Datadog.
`PROXY_TIMEOUT`	optional	Timeout for proxy HTTP requests.
`ADMIN_PASS`	optional	Simple password authentication for admin endpoints.

name	type	data type	description
`tag`	optional	`string`	Identifier attached to a key configuration
`tags`	optional	`[]string`	Identifiers attached to a key configuration
`provider`	optional	`string`	Provider attached to a key provider configuration. Its value can only be `openai`.
`keyIds`	optional	`[]string`	Unique identifiers for keys.

Field	type	example
status	`int`	400
title	`string`	request body reader error
type	`string`	/errors/request-body-read
detail	`string`	something is wrong
instance	`string`	/api/key-management/keys

Field	type	example	description
name	`string`	spike's developer key	Name of the API key.
createdAt	`int64`	1257894000	Key configuration creation time in unix.
updatedAt	`int64`	1257894000	Key configuration update time in unix.
revoked	`boolean`	true	Indicator for whether the key is revoked.
revokedReason	`string`	The key has expired	Reason for why the key is revoked.
tags	`[]string`	["org-tag-12345"]	Identifiers associated with the key.
keyId	`string`	550e8400-e29b-41d4-a716-446655440000	Unique identifier for the key.
costLimitInUsd	`float64`	`5.5`	Total spend limit of the API key.
costLimitInUsdOverTime	`float64`	`2`	Total spend within period of time. This field is required if costLimitInUsdUnit is specified.
costLimitInUsdUnit	`enum`	d	Time unit for costLimitInUsdOverTime. Possible values are [`m`, `h`, `d`, `mo`].
rateLimitOverTime	`int`	`2`	rate limit over period of time. This field is required if rateLimitUnit is specified.
rateLimitUnit	`string`	m	Time unit for rateLimitOverTime. Possible values are [`h`, `m`, `s`, `d`]
ttl	`string`	2d	time to live. Available units are [`s`, `m`, `h`]
allowedPaths	`[]PathConfig`	`[{ "path": "/api/providers/openai/v1/chat/completion", "method": "POST"}]`	Allowed paths that can be accessed using the key.
settingId	`string`	`98daa3ae-961d-4253-bf6a-322a32fdca3d`	This field is DEPERCATED. Use `settingIds` field instead.
settingIds	`string`	`[98daa3ae-961d-4253-bf6a-322a32fdca3d]`	Setting ids associated with the key.
shouldLogRequest	`bool`	`false`	Should request be stored.
shouldLogResponse	`bool`	`true`	Should response be stored.
rotationEnabled	`bool`	`false`	Should key rotate setting used to access third party endpoints in order to circumvent rate limits.

Field	required	type	example	description
path	required	`string`	/api/providers/openai/v1/chat/completion	Allowed path
method	required	`string`	POST	HTTP Method

Field	type	example
status	`int`	`400`
title	`string`	request body reader error
type	`string`	/errors/request-body-read
detail	`string`	something is wrong
instance	`string`	/api/key-management/keys

Field	required	type	example	description
apiKey	required	`string`	`xx-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx`	This value is required.
resourceName	required	`string`	`YOUR_AZURE_RESOURCE_NAME`	This value is required when the provider is `azure`.

Field	type	example	description
createdAt	`int64`	`1699933571`	Unix timestamp for creation time.
updatedAt	`int64`	`1699933571`	Unix timestamp for update time.
provider	`enum`	`openai`	This value can only be `openai` as for now.
id	`string`	`98daa3ae-961d-4253-bf6a-322a32fdca3d`	This value is a unique identifier.
name	`string`	`YOUR_PROVIDER_SETTING_NAME`	Provider setting name.
allowedModels	`[]string`	`["text-embedding-ada-002"]`	Allowed models for this provider setting.

Field	required	type	example	description
setting	required	`Setting`	`{ "apikey": "YOUR_OPENAI_KEY" }`	A map of values used for authenticating with the selected provider.
name	optional	`string`	`YOUR_PROVIDER_SETTING_NAME`	This field is used for giving a name to provider setting
allowedModels	`[]string`	`["text-embedding-ada-002"]`	Allowed models for this provider setting.

Field	required	type	example	description
keyIds	required	`[]string`	`["key-1", "key-2", "key-3" ]`	Array of ids that specicify the keys that you want to aggregate stats from.
tags	required	`[]string`	`["tag-1", "tag-2"]`	Array of tags that specicify the key tags that you want to aggregate stats from.
customIds	required	`[]string`	`["customId-1", "customId-2"]`	A list of custom IDs that you want to aggregate stats from.
filters	required	`[]string`	`["model", "keyId"]`	Group by data points through different filters(`model`,`keyId` or `customId`).
start	required	`int64`	`1699933571`	Start timestamp for the requested timeseries data.
end	required	`int64`	`1699933571`	End timestamp for the requested timeseries data.
increment	required	`int`	`60`	This field is the increment in seconds for the requested timeseries data.

Field	type	example	description
dataPoints	`[]dataPoint`	`[{ "timeStamp": 1699933571, "numberOfRequests": 1, "costInUsd": 0.8, "latencyInMs": 600, "promptTokenCount": 0, "completionTokenCount": 0, "successCount": 1 }]`	Unix timestamp for creation time.
latencyInMsMedian	`float64`	`656.7`	Median latency for the given time period.
latencyInMs99th	`float64`	`555.7`	99th percentile latency for the given time period.

Field	type	example	description
timeStamp	`int64`	`1702504746`	Unix timestamp for the data point
numberOfRequests	`int64`	`100`	Aggregated number of http requests over the given time increment.
costInUsd	`float64`	`1.7`	Aggregated cost of proxied requests in USD over the given time increment.
latencyInMs	`int`	`555`	Aggregated latency in milliseconds of http requests over the given time increment.
promptTokenCount	`int`	`25`	Aggregated prompt token counts over the given time increment.
completionTokenCount	`int`	`4000`	Aggregated completion token counts over the given time increment.
successCount	`int`	`555`	Aggregated number of successful http requests over the given time increment.
keyId	`int`	`555.7`	key Id associated with the event.
model	`string`	`gpt-3.5-turbo`	model associated with the event.
customId	`string`	`customId`	customId associated with the event.

Field	type	example	description
id	`int64`	`1699933571`	Unique identifier associated with the event.
created_at	`int64`	`1699933571`	Unix timestamp for creation time.
tags	`int64`	`["YOUR_TAG"]`	Tags of the key.
key_id	`string`	`YOUR_KEY_ID`	Key Id associated with the proxy request.
cost_in_usd	`float64`	`0.0004`	Cost incured by the proxy request.
model	`string`	`gpt-4-1105-preview`	Model used in the proxy request.
provider	`string`	`openai`	Provider for the proxy request.
status	`int`	`200`	Http status.
prompt_token_count	`int`	`8`	Prompt token count of the proxy request.
completion_token_count	`int`	`16`	Completion token counts of the proxy request.
latency_in_ms	`int`	`160`	Provider setting name.
path	`string`	`/api/v1/chat/completion`	Provider setting name.
method	`string`	`POST`	Http method for the assoicated proxu request.
custom_id	`string`	`YOUR_CUSTOM_ID`	Custom Id passed by the user in the headers of proxy requests.
request	`[]byte`	`{}`	Request in bytes to providers.
custom_id	`string`	`YOUR_CUSTOM_ID`	Custom Id passed by the user in the headers of proxy requests.
response	`[]byte`	`{}`	Response in bytes from providers.

Field	required	type	example	description
route_configs	optional	`[]RouteConfig`	`{{ "path": "/chat/completions", "target_url": "https://api.openai.com/v1/chat/completions" }}`	Route configurations for the custom provider.
authentication_param	optional	`string`	`apikey`	The authentication parameter required for.

Field	type	example	description
id	`int64`	`9e6e8b27-2ce0-4ef0-bdd7-1ed3916592eb`	Unique identifier associated with the event.
created_at	`int64`	`1699933571`	Unix timestamp for creation time.
updated_at	`int64`	`1699933571`	Unix timestamp for update time.
provider	`string`	`bricks`	Unique identifier associated with the route config.
route_configs	`[]RouteConfig`	`{{ "path": "/chat/completions", "target_url": "https://api.openai.com/v1/chat/completions" }}`	Start timestamp for the requested timeseries data.
authentication_param	`string`	`apikey`	The authentication parameter required for.

Field	required	type	example	description
enabled	required	`bool`	`false`	Boolean flag indicating whether caching is enabled.
ttl	optional	`string`	`5s`	TTL for the cache. Default value is `168h`.

Field	type	example
status	`int`	`404`
title	`string`	`request body reader error`
type	`string`	`/errors/request-body-read`
detail	`string`	`something is wrong`
instance	`string`	`/api/routes/:id`

License

Rocavall/BricksLLM

Folders and files

Latest commit

History

Repository files navigation

BricksLLM: AI Gateway For Putting LLM In Production

Features

Roadmap

Getting Started

Step 1 - Clone BricksLLM-Docker repository

Step 2 - Change to BricksLLM-Docker directory

Step 3 - Deploy BricksLLM locally with Postgresql and Redis

Step 4 - Create a provider setting

Step 5 - Create a Bricks API key

Congratulations you are done!!!

How to Update?

Documentation

Environment variables

Configuration Endpoints

Headers

Description

Query Parameters

Error Response

Response

Description

Request

Error Response

Responses

Description

Parameters

Request

Error Response

Response

Description

Request

Error Response

Response

Description

Query Parameters

Error Response

Response

Description

Parameters

Request

Error Response

Response

Description

Request

Error Response

Response

Description

Query Parameters

Error Response

Response

Description

RouteConfig

Request

Error Response

Response

Description

RouteConfig

Request

Error Response

Response

Description

RouteConfig

Request

Error Response

Response

Description

StepConfig

CacheConfig

Request

Error Response

Response

Description

Parameters

Error Response

Response