Architecture
Synapse is built on a resource-oriented architecture, inspired by Kubernetes, to optimize scalability, resilience, and maintainability. Each component is specialized to manage distinct system resources, ensuring efficient operation and seamless integration. This document provides an in-depth overview of Synapse's core components and offers best practices for their effective usage.
The API Server is the central hub for managing Synapse resources and provides the HTTP API that facilitates communication with other components and users. It also optionally serves the Dashboard, which is Synapse's graphical user interface (GUI).
- Resource Management: Handles CRUD operations for Synapse resources such as workflows, correlators, and operators.
- Dashboard (Optional): Offers a web-based interface for visual interaction with Synapse components.
- Secure API Access: Implement and manage authentication and authorization to protect access to the API.
- Monitor API Performance: Use monitoring tools to track the performance and health of the API server, including response times and error rates.
- Optimize Dashboard Usage: Regularly review and update the Dashboard to ensure it meets user needs and reflects current system states.
| Name | Required | Description |
|---|---|---|
| CONNECTIONSTRINGS__REDIS | yes |
The connection string to the REDIS database to use. |
| SYNAPSE_API_AUTH_TOKEN_FILE | yes |
The path to the file that defines the static tokens used to authenticate on the API. |
| SYNAPSE_API_JWT_AUTHORITY | yes |
The API's JWT authority. Must be the URI used by other Synapse applications to connect to the API. |
| SYNAPSE_API_CLOUD_EVENTS_ENDPOINT | no |
An absolute URI that references the endpoint the API publishes cloud events to. If not set, the API won't publish any event. |
| SYNAPSE_DASHBOARD_SERVE | no |
Configures whether or not the API should serve the Dashboard. Supported values are: true and false.Defaults to false.
|
| SYNAPSE_DATABASE_SEEDING_RESET | no |
A boolean indicating whether or not to reset the database upon starting the API server. Defaults to false.
|
| SYNAPSE_DATABASE_SEEDING_DIRECTORY | no |
The directory from which to load the static resources used to seed the database. Defaults to SYNAPSE_DIRECTORY/data/seed.
|
| SYNAPSE_DATABASE_SEEDING_OVERWRITE | no |
A boolean indicating whether or not to overwrite existing resources. Defaults to false.
|
| SYNAPSE_DATABASE_SEEDING_FILE_PATTERN | no |
GLOB pattern used to match the static resource files to use to seed the database. Defaults to *.*.
|
The Operator manages the execution and lifecycle of workflows and workflow instances within Synapse.
- Workflow Management: Controls the initiation and execution of workflows.
- Workflow Instance Control: Oversees the lifecycle of workflow instances, including starting, stopping, and monitoring.
- Ensure Workflow Resilience: Monitor workflow execution and implement strategies to handle failures or retries effectively.
- Balance Load: Optimize scheduling and resource allocation to prevent bottlenecks and maintain system performance.
- Regular Monitoring: Continuously monitor the status and performance of workflows to ensure smooth operation.
| Name | Required | Description |
|---|---|---|
| CONNECTIONSTRINGS__REDIS | yes |
The connection string to the REDIS database to use. |
| SYNAPSE_OPERATOR_NAMESPACE | yes |
The operator's namespace. |
| SYNAPSE_OPERATOR_NAME | yes |
The operator's name. |
| SYNAPSE_RUNNER_API | yes |
The absolute URI that references the Synapse API used by runners spawned by the operator. |
| SYNAPSE_RUNNER_LIFECYCLE_EVENTS | no |
Configures whether or not runners spawned by the operator should publish lifecycle events. Supported values are: true and false.Defaults to true.
|
| SYNAPSE_RUNNER_CONTAINER_PLATFORM | no |
Configures the platform used by runners spawned by the operator to create and manage containers. Supported values are: docker and kubernetes.Defaults to docker.
|
| SYNAPSE_RUNTIME_MODE | no |
Configures the runtime to use. Supported values are: docker, kubernetes and native.Defaults to docker.
|
| Name | Required | Description |
|---|---|---|
| SYNAPSE_RUNTIME_DOCKER_API_ENDPOINT | no |
The endpoint of the Docker API to use to spawn runners. *Defaults to npipe://./pipe/docker_engine in Windows, and to unix:/var/run/docker.sock in Unix systems. |
| SYNAPSE_RUNTIME_DOCKER_API_VERSION | no |
The version of the Docker API to use to spawn runners. |
| SYNAPSE_RUNTIME_DOCKER_CONTAINER | no |
The path to the YAML file used to configure the template of runner containers. If not set, Synapse's default container template is used. |
| SYNAPSE_RUNTIME_DOCKER_IMAGE_REGISTRY | no |
The name of the image registry to use when pulling the runner image. |
| SYNAPSE_RUNTIME_DOCKER_IMAGE_PULL_POLICY | no |
The Docker image pull policy to use. Supported values are: Always, IfNotPresent and Never.Defaults to Always.
|
| SYNAPSE_RUNTIME_DOCKER_NETWORK | no |
The name of the network, if any, to connect Runner containers to. Defaults to synapse.
|
| SYNAPSE_RUNTIME_DOCKER_SECRETS_DIRECTORY | no |
The path to the directory that contains the secrets to mount onto runner containers. Defaults to %USERPROFILE%/.synapse/secrets.
|
| SYNAPSE_RUNTIME_DOCKER_SECRETS_MOUNT_PATH | no |
The path to the directory to mount the secrets volume to. Defaults to /run/secrets/synapse.
|
| Name | Required | Description |
|---|---|---|
| SYNAPSE_RUNTIME_K8S_KUBECONFIG | no |
The path to the Kubeconfig file to use, if any. Defaults to InCluster configuration.
|
| SYNAPSE_RUNTIME_K8S_POD | no |
The template to use to create runner containers. If not set, Synapse's default pod template is used. |
| SYNAPSE_RUNTIME_K8S_SECRETS_VOLUME_NAME | no |
The name of the volume on which to mounts secrets. Defaults to secrets.
|
| SYNAPSE_RUNTIME_K8S_SECRETS_MOUNT_PATH | no |
The path to the directory to mount the secrets volume to. Defaults to /run/secrets/synapse.
|
| SYNAPSE_RUNTIME_K8S_NAMESPACE | no |
The namespace into which to create runner pods. If not set, defaults to the namespace defined by the pod template. |
| SYNAPSE_RUNTIME_K8S_SERVICE_ACCOUNT | no |
The name of the service account that grants the runner the ability to spawn containers when its container platform has been set to kubernetes.*Required if the runner's platform has been set to kubernetes. |
| Name | Required | Description |
|---|---|---|
| SYNAPSE_RUNTIME_NATIVE_DIRECTORY | yes |
The path to the directory that contains the runner binaries. |
| SYNAPSE_RUNTIME_NATIVE_EXECUTABLE | yes |
The path to the runner executable file. |
| SYNAPSE_RUNTIME_NATIVE_SECRETS_DIRECTORY | no |
The path to the directory that contains the secrets to mount onto runner containers. Defaults to %USERPROFILE%/.synapse/secrets.
|
The Correlator performs Complex Event Processing (CEP) by analyzing and correlating ingested events based on predefined rules.
- Event Correlation: Detects and processes events to identify patterns and relationships.
- Pattern Recognition: Recognizes complex patterns within the event stream.
- Define Effective Rules: Regularly review and update correlation rules to ensure accurate event detection and processing.
- Monitor Event Streams: Use monitoring tools to track the performance of event processing and identify any anomalies.
- Optimize Performance: Fine-tune correlation rules and processing settings to handle high volumes of events efficiently.
| Name | Required | Description |
|---|---|---|
| CONNECTIONSTRINGS__REDIS | yes |
The connection string to the REDIS database to use. |
| SYNAPSE_CORRELATOR_NAMESPACE | yes |
The correlator's namespace. |
| SYNAPSE_CORRELATOR_NAME | yes |
The correlator's name. |
The Runner executes individual instances of workflows, handling the tasks and transitions defined in the workflow.
- Task Execution: Processes tasks according to workflow definitions.
- Instance Isolation: Ensures that each workflow instance operates independently to maintain data integrity.
- Monitor Task Processing: Keep track of task execution metrics to identify and resolve potential issues.
Since runners are spawned by operators, their configuration is indirectly determined by the operators that spawn them.
See operator configuration for more information.
The Command Line Interface (CLI) allows users to interact with Synapse's API directly from the command line. It supports a range of operations including resource management and workflow control.
The CLI is configured using specific commands.
See configuration commands for more information.