Example Go service using go-swagger and The Clean Architecture

This project shows an example of how to use go-swagger accordingly to Uncle Bob's "Clean Architecture".

Also it includes go-swagger JSON Schema support cheatsheet, which list all validations/annotations for JSON body actually implemented by go-swagger v0.18.0.

Table of Contents

• Overview
  • The Clean Architecture
  • The hexagonal architecture, or ports and adapters architecture
  • Structure of Go packages
  • Features
• Development
  • Requirements
  • Setup
  • Usage
    • Cheatsheet
• Run
  • Docker
  • Source
• TODO

Overview

The Clean Architecture

It's not a complete example of The Clean Architecture itself (business-logic of this example is too trivial, so "Use Cases" layer in package app embeds "Entities" layer), but it does show the most relevant part: how to create "API Controller" layer in package srv/openapi between code auto-generated by go-swagger and "Use Cases" layer in package app. Also it includes "DB Gateway" layer in packages dal/memory (provided trivial in-memory implementation is "DB" and "Gateway" layers at once) and dal/mysql (just "Gateway" layer).

The hexagonal architecture, or ports and adapters architecture

It may be even easier to understand implemented architecture as "ports and adapters":

• "ports" are defined as interfaces in app/app.go - they make it possible to easily test business-logic in app without any external dependencies by mocking all these interfaces.
• "adapters" are implemented in srv/* (serve project APIs), dal/* (access DB) and svc/* (use external services) packages - they can't be tested that easy, so they should be as thin and straightforward as possible and try hard to do nothing than convert ("adapt") data between format used by external world and our business-logic (package app).

Structure of Go packages

• api/* - definitions of own and 3rd-party (in api/ext-*) APIs/protocols and related auto-generated code
• cmd/* - main application(s)
• internal/config - configuration(s) (default values, env, flags) for application(s) subcommands and tests
• internal/app - define interfaces ("ports") and implements business-logic
• internal/srv/* - adapters for served APIs/UI
• internal/dal/* - adapters for data storage
• internal/migrations/* - DB migrations (in both SQL and Go)
• internal/svc/* - adapters for accessing external services
• pkg/* - helper packages, not related to architecture and business-logic (may be later moved to own modules and/or replaced by external dependencies), e.g.:
  • pkg/def/ - project-wide defaults

Features

• Project structure (mostly) follows Standard Go Project Layout.
• Strict but convenient golangci-lint configuration.
• Easily testable code (thanks to The Clean Architecture).
• Avoids (and resists to) using global objects (to make it possible to embed such microservices into modular monolith).
• CLI subcommands support using cobra.
• Graceful shutdown support.
• Configuration defaults can be overwritten by env vars and flags.
• CORS support, so you can play with API using Swagger Editor tool.
• Example go-swagger authentication and authorization.
• Example DAL (data access layer):
  • MySQL 5.6 (strictest SQL mode).
• Example tests, both unit and integration.
• Production logging using structlog.
• Production metrics using Prometheus.
• Docker and docker-compose support.
• Smart test coverage report, with optional support for coveralls.io.
• Linters for Dockerfile and shell scripts.
• CI/CD setup for GitHub Actions and CircleCI.

Development

Requirements

• Go 1.16
• Docker 19.03+
• Docker Compose 1.25+

Setup

1. After cloning the repo copy env.sh.dist to env.sh.
2. Review env.sh and update for your system as needed.
3. It's recommended to add shell alias alias dc="if test -f env.sh; then source env.sh; fi && docker-compose" and then run dc instead of docker-compose - this way you won't have to run source env.sh after changing it.

Usage

To develop this project you'll need only standard tools: go generate, go test, go build, docker build. Provided scripts are for convenience only.

• Always load env.sh in every terminal used to run any project-related commands (including go test): source env.sh.
  • When env.sh.dist change (e.g. by git pull) next run of source env.sh will fail and remind you to manually update env.sh to match current env.sh.dist.
• go generate ./... - do not forget to run after making changes related to auto-generated code
• go test ./... - test project (excluding integration tests), fast
• ./scripts/test - thoroughly test project, slow
• ./scripts/test-ci-circle - run tests locally like CircleCI will do
• ./scripts/cover - analyse and show coverage
• ./scripts/build - build docker image and binaries in bin/
  • Then use mentioned above dc (or docker-compose) to run and control the project.
  • Access project at host/port(s) defined in env.sh.

Cheatsheet

dc up -d --remove-orphans               # (re)start all project's services
dc logs -f -t                           # view logs of all services
dc logs -f SERVICENAME                  # view logs of some service
dc ps                                   # status of all services
dc restart SERVICENAME
dc exec SERVICENAME COMMAND             # run command in given container
dc stop && dc rm -f                     # stop the project
docker volume rm PROJECT_SERVICENAME    # remove some service's data

It's recommended to avoid docker-compose down - this command will also remove docker's network for the project, and next dc up -d will create a new network… repeat this many enough times and docker will exhaust available networks, then you'll have to restart docker service or reboot.

Run

Docker

$ docker run -i -t --rm ghcr.io/powerman/go-service-example -v
address-book version 0894daa 2020-09-13_19:44:26 go1.15.2

$ dc up -d mysql
$ docker run -i -t --rm \
    -p 8000:8000 \
    --net=go-service-example_default \
    -e EXAMPLE_APIKEY_ADMIN=secret \
    -e EXAMPLE_MYSQL_ADDR_HOST=mysql \
    -e EXAMPLE_MYSQL_AUTH_LOGIN=root \
    -e EXAMPLE_MYSQL_AUTH_PASS= \
    ghcr.io/powerman/go-service-example
 address-book: inf      main: `started` version 0894daa 2020-09-13_19:44:26
 address-book: inf   openapi: `OpenAPI protocol` version 0.2.0
 address-book: inf     serve: `serve` b3ecd12369c3:9000 [Prometheus metrics]
 address-book: inf     serve: `serve` 172.19.0.6:8000 [OpenAPI]
 address-book: inf   swagger: `Serving address book at http://172.19.0.6:8000`
^C
 address-book: inf   swagger: `Shutting down... `
 address-book: inf   swagger: `HTTP server Shutdown: context deadline exceeded`
 address-book: inf   swagger: `Stopped serving address book at http://172.19.0.6:8000`
 address-book: inf     serve: `shutdown` [OpenAPI]
 address-book: inf     serve: `shutdown` [Prometheus metrics]
 address-book: inf      main: `finished` version 0894daa 2020-09-13_19:44:26

Source

Use of the ./scripts/build script is optional (it's main feature is embedding git version into compiled binary), you can use usual go get|install|build to get the application instead.

$ ./scripts/build
$ ./bin/address-book -h
Example microservice with OpenAPI

Usage:
  address-book [flags]
  address-book [command]

Available Commands:
  help        Help about any command
  serve       Starts microservice

Flags:
  -h, --help                    help for address-book
      --log.level OneOfString   log level [debug|info|warn|err] (default debug)
  -v, --version                 version for address-book

Use "address-book [command] --help" for more information about a command.

$ ./bin/address-book serve -h
Starts microservice

Usage:
  address-book serve [flags]

Flags:
  -h, --help                        help for serve
      --host NotEmptyString         host to serve OpenAPI (default localhost)
      --metrics.port Port           port to serve Prometheus metrics (default 9000)
      --port Port                   port to serve OpenAPI (default 8000)
      --timeout.shutdown Duration   must be less than 10s used by 'docker stop' between SIGTERM and SIGKILL (default 9s)
      --timeout.startup Duration    must be less than swarm's deploy.update_config.monitor (default 3s)

Global Flags:
      --log.level OneOfString   log level [debug|info|warn|err] (default debug)

$ ./bin/address-book -v
address-book version v1.0.0 5e45f44 2020-09-03_15:15:53 go1.15.1

$ ./bin/address-book serve
 address-book: inf      main: `started` version v1.0.0 5e45f44 2020-09-03_15:15:53
 address-book: inf   openapi: `OpenAPI protocol` version 0.2.0
 address-book: inf     serve: `serve` localhost:9000 [Prometheus metrics]
 address-book: inf     serve: `serve` 127.0.0.1:8000 [OpenAPI]
 address-book: inf   swagger: `Serving address book at http://127.0.0.1:8000`
 address-book: dbg   openapi: 127.0.0.1:36500           POST    contacts: `calling AddContact` admin
 address-book: dbg       dal: 127.0.0.1:36500           POST    contacts: `contact added` admin
 address-book: inf   openapi: 127.0.0.1:36500       201 POST    contacts: `handled` admin
 address-book: dbg   openapi: 127.0.0.1:36502           POST    contacts: `calling AddContact` admin
 address-book: dbg       dal: 127.0.0.1:36502           POST    contacts: `contact added` admin
 address-book: inf   openapi: 127.0.0.1:36502       201 POST    contacts: `handled` admin
 address-book: inf   openapi: 127.0.0.1:36504       200 GET     contacts: `handled` admin
 address-book: inf   openapi: 127.0.0.1:36508       200 GET     contacts: `handled` user
 address-book: inf   openapi: 127.0.0.1:36510       401 GET     contacts: `handled`
 address-book: inf   openapi: 127.0.0.1:36518       403 POST    contacts: `handled`
^C
 address-book: inf   swagger: `Shutting down... `
 address-book: inf   swagger: `HTTP server Shutdown: context deadline exceeded`
 address-book: inf   swagger: `Stopped serving address book at http://127.0.0.1:8000`
 address-book: inf     serve: `shutdown` [OpenAPI]
 address-book: inf     serve: `shutdown` [Prometheus metrics]
 address-book: inf      main: `finished` version v1.0.0 5e45f44 2020-09-03_15:15:53

TODO

• Update JSON Schema support cheatsheet to latest go-swagger version.
• Add alternative DAL implementation for Postgresql.
• Add cookie-based auth with CSRF middleware.
• Add an example of adapter for external service in svc/something.