Project license validator for Athens proxy

Why?

Direct or transitive dependencies may have license like AGPL-3.0 that enforces user to open-source product. So this project should help to deal with such situations. Related articles:

• https://www.agwa.name/blog/post/always_review_your_dependencies

How this project can help me?

It's a web-server that handles Athens proxy server validation webhook requests. This hook called each time when user tries to download module through Athens. This project receives request and performs validation according to settings. If validation fails Athens receives HTTP 403 (Forbidden) status and doesn't allow module downloading.

Features

• Flexible rule system:
  • Blacklist modules by name or version constraint (i.e. forbid modules with version less than 1.0.0)
  • Whitelist modules by name or version (i.e. always allow modules from your internal repos)
  • Allow only modules licensed by configured licenses
  • Deny modules licensed by configured licenses
  • License can be defined by SPDX License List id or human-readable name.
• Configurable behaviour for modules with non-determined license:
  • Allow such modules
  • Deny such modules
  • Notifying about such modules. Currently it's a configurable http request.
• Dealing with vanity servers (servers needed for decoupling module name from repository like gopkg.in). Project supports gopkg.in, golang.org/x and go.googlesource.com out of the box. Other rewrite rules can be added through config
• Multiple sources of license detection:
  • Github for modules hosted on it. Has fallback to go-license-detector
  • Detection using module zip from proxy.golang.org with go-license-detector without downloading whole zip
• In-memory (plain or LRU) and Redis-based caching
• Opentelemetry support (Zipkin, Jaeger exporters onboard) and metrics (prometheus handler at /metrics)
• Ability to switch log level "on the fly". Just supply level at PUT /loglevel in form {"level": "<level>"}. Supported levels:
  • debug - logs are typically voluminous, and are usually disabled in production.
  • info - is the default logging priority.
  • warn - logs are more important than info, but don't need individual human review.
  • error - logs are high-priority.
  • dpanic - logs are particularly important errors. In development (selected by Debug parameter in config) the logger panics after writing the message.
  • panic - logs a message, then panics.
  • fatal - logs a message, then exits with code 1.

Running

• Direct install:
  • go install github.com/xakep666/licensevalidator/cmd/licensevalidator
  • Generate config example and tune it licensevalidator sample-config > config.toml
  • Start a service licensevalidator -c config.toml
• Use pre-build docker image. Configuration can be bind-mount to /etc/licensevalidator.toml

You can manually check if module allowed to use by running making HTTP POST to /athens/admission with body

{
    "Module": "github.com/stretchr/testify",
    "Version": "v1.5.1"
}

Note that header Content-Type: application/json is required.

Configuration

Example config can be received by running licensevalidator sample-config Here it is with some comments (more comments in config.go).

# enable debug logging
Debug = true

# Cache for some heavy operations (currently license resolution operation).
# It's not recommended to disable it.
[Cache]
  Type = "memory"

[Github]
  # Provide github access token to decrease rate-limit
  AccessToken = "test-github-token"

[GoProxy]
  # URL of goproxy server that will be used for license detection
  # Obviously it should not be address of Athens server which calls this app.
  BaseURL = "https://proxy.golang.org"

# Path overrides for vanity servers
# This example holds rule for modules published by Uber
[[PathOverrides]]
  Match = "^go.uber.org/(.*)$"
  Replace = "github.com/uber-go/$1"

# Web server settings
[Server]
  ListenAddr = ":8080"
  EnablePprof = true # adds pprof handlers at /pprof

# Health check server
# Contains two endpoints:
# * GET /live - for liveness probes
# * Get /ready - for readiness probes
[HealthServer]
  ListenAddr = ":8081"

[Validation]
  # Some ways of license detection doesn't produce 100% accurate result.
  # This parameter holds lower-bound threshold of license matching confidence.
  ConfidenceThreshold = 0.8

  # How to deal with unknown licenses: allow or deny
  UnknownLicenseAction = "allow"

  [Validation.RuleSet]

    # Allowed licenses list. If not empty only modules with provided licenses can be used.
    [[Validation.RuleSet.AllowedLicenses]]
      SPDXID = "MIT"

    # If module will be matched by these rules it will be blocked anyway.
    [[Validation.RuleSet.BlacklistedModules]]
      Name = "rsc.io/pdf"
      # for constraint syntax see https://github.com/Masterminds/semver/#checking-version-constraints
      VersionConstraint = "<1.0.0"

    # Module with denied licenses will be blocked.
    [[Validation.RuleSet.DeniedLicenses]]
      SPDXID = "AGPL-3.0"

    # Modules matching whitelist always allowed.
    [[Validation.RuleSet.WhitelistedModules]]
      Name = "^gitlab.mycorp.com/.*"

    [[Validation.RuleSet.WhitelistedModules]]
      Name = "github.com/user/repo"
      VersionConstraint = ">=1.0.0"

Athens proxy should be configured properly by setting ATHENS_PROXY_VALIDATOR environment variable or ValidatorHook parameter in config to <base-url of app>/athens/admission

Running tests

This project contains integration tests that uses testcontainers-go. They can be skipped using -short flag. Correct running requires working Docker. For running tests inside container be sure that management is available inside container i.e docker socket bind-mounted into container -v /var/run/docker.socket:/var/run/docker.socket and network mode is host.