- Basic Skeleton App configured with
openai
API - A ChatBot using LangChain and Chainlit
- Docker Support with Optimisation Cache etc
- Deployment on Google Cloud App Engine
- Deployment on Google Cloud using
Cloud Run
Reference repository: https://github.com/amjadraza/langchain-streamlit-docker-template
This repo contains an main.py
file which has a template for a chatbot implementation.
To add your chain, you need to change the load_chain
function in main.py
.
Depending on the type of your chain, you may also need to change the inputs/outputs that occur later on.
- Clone the repository📂
git clone https://github.com/amjadraza/langchain-chainlit-docker-deployment-template
- Install dependencies with Poetry and activate virtual environment🔨
poetry install
poetry shell
- Run the Chainlit server🚀
chainlit run demo_app/main.py
This project includes Dockerfile
to run the app in Docker container. In order to optimise the Docker Image
size and building time with cache techniques, I have follow tricks in below Article
https://medium.com/@albertazzir/blazing-fast-python-docker-builds-with-poetry-a78a66f5aed0
Build the docker container
docker build . -t langchain-chainlit-chat-app:latest
To generate Image with DOCKER_BUILDKIT
, follow below command
DOCKER_BUILDKIT=1 docker build --target=runtime . -t langchain-chainlit-chat-app:latest
- Run the docker container directly
docker run -d --name langchain-chainlit-chat-app -p 8000:8000 langchain-chainlit-chat-app
- Run the docker container using docker-compose (Recommended)
docker-compose up
This app can be deployed on Google App Engine following below steps.
Follow below guide on basic Instructions. How to deploy Streamlit apps to Google App Engine
We added below tow configurations files
app.yaml
: A Configuration file forgcloud
.gcloudignore
: Configure the file to ignore file / folders to be uploaded
I have adopted Dockerfile
to deploy the app on GCP APP Engine.
- Initialise & Configure the App
gcloud app create --project=[YOUR_PROJECT_ID]
- Deploy the App using
gcloud app deploy
- Access the App using
https://langchain-chat-app-ex6cbrefpq-ts.a.run.app/
This app can be deployed on Google Cloud using Cloud Run following below steps.
Follow below guide on basic Instructions. How to deploy Streamlit apps to Google App Engine
We added below tow configurations files
cloudbuild.yaml
: A Configuration file forgcloud
.gcloudignore
: Configure the file to ignore file / folders to be uploaded
we are going to use Dockerfile
to deploy the app using Google Cloud Run.
- Initialise & Configure the Google Project using Command Prompt
gcloud app create --project=[YOUR_PROJECT_ID]
- Enable Services for the Project
gcloud services enable cloudbuild.googleapis.com
gcloud services enable run.googleapis.com
- Create Service Account
gcloud iam service-accounts create langchain-app-cr \
--display-name="langchain-app-cr"
gcloud projects add-iam-policy-binding langchain-chat \
--member="serviceAccount:langchain-app-cr@langchain-chat.iam.gserviceaccount.com" \
--role="roles/run.invoker"
gcloud projects add-iam-policy-binding langchain-chat \
--member="serviceAccount:langchain-app-cr@langchain-chat.iam.gserviceaccount.com" \
--role="roles/serviceusage.serviceUsageConsumer"
gcloud projects add-iam-policy-binding langchain-chat \
--member="serviceAccount:langchain-app-cr@langchain-chat.iam.gserviceaccount.com" \
--role="roles/run.admin"
- Generate the Docker
DOCKER_BUILDKIT=1 docker build --target=runtime . -t australia-southeast1-docker.pkg.dev/langchain-chat/clapp/langchain-chainlit-chat-app:latest
- Push Image to Google Artifact's Registry
Create the repository with name clapp
gcloud artifacts repositories create clapp \
--repository-format=docker \
--location=australia-southeast1 \
--description="A Langachain Chainlit App" \
--async
Configure-docker
gcloud auth configure-docker australia-southeast1-docker.pkg.dev
In order to push the docker-image
to Artifact registry, first create app in the region of choice.
Check the artifacts locations
gcloud artifacts locations list
Once ready, let us push the image to location
docker push australia-southeast1-docker.pkg.dev/langchain-chat/clapp/langchain-chainlit-chat-app:latest
- Deploy using Cloud Run
Once image is pushed to Google Cloud Artifacts Registry. Let us deploy the image.
gcloud run deploy langchain-chat-app --image=australia-southeast1-docker.pkg.dev/langchain-chat/clapp/langchain-chainlit-chat-app:latest \
--region=australia-southeast1 \
--service-account=langchain-app-cr@langchain-chat.iam.gserviceaccount.com \
--port=8000
- Test the App Yourself
You can try the app using below link
https://langchain-chat-app-ex6cbrefpq-ts.a.run.app/
As langchain-chainlit-docker-deployment-template
is a template project with minimal example. Report issues if you face any.
This is a template App, when using with openai_api key, you will be charged a nominal fee depending on number of prompts etc.