Global Alliance for Genomics and Health (GA4GH) is an organization that sets policies and technical standards related to genomics data sharing and processing. They release the Task Execution Schema (TES), an API contract to permit cloud-agnostic execution of tasks in the cloud.
The TES API Server for Azure Compute provides a GA4GH TES (Task Execution Schema) compatible API and leverages Azure compute services on its backend for task execution. Today workloads can be performed by Azure Batch, and Azure Kubernetes Services (AKS) support is planned.
The TES endpoint can be used standalone, or integrated into various existing workflow tools such as Cromwell or cwl-tes.
- Plugin-based architecture for compute backends with Azure Batch supported today
- Support for standalone operation or authentication with multiple users (via AAD+OAuth), with per-tenant or per-user task isolation
- Automatic file transfer of task inputs and outputs (reads from Azure Blob, HTTP, HTTPS, SFTP, FTP+SSL, AWS S3, GCP and writes to Azure Blob, SFTP, FTP+SSL)
- Key Vault stores all sensitive secrets
- App logging & tracing sent to Application Insights
- Simple, Docker-based deployment provisioned with Terraform; backend resources are also automatically provisioned through a REST API
- Tasks run using containers sourced from Docker hub or a private container registry
| Component Name | Purpose |
|---|---|
| api-server | Flask-based API implementing TES with multiple modular Azure compute backends |
| container-filetransfer | Provides input/output file marshalling for containers running in the backend compute |
| local-forwarder | Local telemetry forwarded for Application Insights |
- User guide - getting started deploying and using the TES API Server for Azure Compute
- Developer documentation - developing & local environment setup
- Design docs - architecture description and design decisions
The project and the code base is maintained by Microsoft Commercial Engineering (CSE) Healthcare Industry team on a best-effort basis.
Please note that the code provided in this repository is intended as a code sample for kickstarting development, and that you may need to customize and test the code for your intended use case. It does not constitute a product officially supported by Microsoft nor bound to an SLA. If you intend to leverage these code samples, it is your responsibility to deploy them in a manner consistent with the availability and uptime requirements of your project.
Licensed under the MIT License.
If you have found any issues, or you want to request a missing feature, please do so by opening an issue.
This project has adopted the Microsoft Open Source Code of Conduct and welcomes contributions and suggestions. For more details on contributing, please see the guidance for contributors.