confpb
is a Go library that leverages Protocol Buffers (protobuf) to define, manage, and load configuration structures. It provides tools to map environment variables, set default values, merge configurations, and parse multiple formats like YAML, JSON, and protobuf files.
protoc-gen-env
: Automatically map environment variables to protobuf fields in Go code.protoc-gen-default
: Define default values for protobuf fields directly in your.proto
files.protoc-gen-jsonschema
: Generate JSON Schema files for your configuration, enabling IDE autocompletion.- Merging & Overlaying: Combine multiple protobuf files, allowing one to overlay values from another.
- Multi-format Parsing: Seamlessly parse configurations from various formats such as YAML, JSON, binary protobuf, and text protobuf.
- Simplified configuration management: Centralize and standardize configuration using protobuf, version your schemas, generate typed code from schemas.
- Standardisation of config: Use the wide ecosystem of protobuf tooling to lint your schemas, generate docs, detect breaking changes and validate values.
- Flexibility: environment binding, default generation and file loading are separated, allowing you to choose what features to implement.
Below is an example of how to define your application's configuration schema using protobuf. It includes environment variable mapping and default value definitions, though you can use either feature independently.
syntax = "proto3";
package config.v1;
import "confpb/v1/field.proto"; // Import confpb field options.
option go_package = "sample-app/gen/config/v1;configv1";
message ApplicationConfig {
message ServerConfig {
string host = 1 [(confpb.v1.env) = "HOST"];
uint32 port = 2 [(confpb.v1.env) = "PORT"];
}
// Map environment variables:
// SERVER_HOST=127.0.0.1
// SERVER_PORT=8080
ServerConfig server = 1 [(confpb.v1.env) = "SERVER"];
string some_string = 2 [(confpb.v1.default).string = "foo"];
// Set environment variables for repeated fields:
// SOME_LIST_1 = "item1"
// SOME_LIST_2 = "item2"
// SOME_LIST_3 = "item3"
// ...
repeated string some_list = 3 [
(confpb.v1.env) = "SOME_LIST",
(confpb.v1.default).repeated_string = {
values: ["default1", "default2"]
}
];
}
Here’s how to load configuration in your Go application from default values, environment variables, and config files.
package main
import (
"github.com/marnixbouhuis/confpb/pkg/config"
configv1 "sample-app/gen/config/v1"
)
func main() {
// Start with default configuration
conf := configv1.ApplicationConfigFromDefault()
// Overlay with environment variables
envConfig, err := configv1.ApplicationConfigFromEnv()
if err != nil {
panic(err)
}
conf = config.Overlay(conf, envConfig)
// Overlay with configuration from a YAML file
fileConfig, err := config.FromFile[configv1.ApplicationConfig]("./config.yaml")
if err != nil {
panic(err)
}
conf = config.Overlay(conf, fileConfig)
// The config is now complete, with precedence in this order:
// 1. Config file
// 2. Environment variables
// 3. Default values
// Use the final configuration
// fmt.Println(conf.SomeList[0])
}
Load values from a YAML file:
"@type": "type.googleapis.com/config.v1.ApplicationConfig"
server:
host: 127.0.0.1
port: 8443
some_string: example
some_list:
- item1
- item2
For a complete example, check out the sample-app.
There are two ways to install the generation plugins.
- Download prebuilt binaries (Recommended)
- Build from Source
Installing a prebuilt binary from release is recommended as this binary contains some extra version information.
- Head to the releases page to download the appropriate binary for your platform.
- Download
protoc-gen-env
,protoc-gen-default
andprotoc-gen-jsonschema
(depending on what features you require).
- Download
- Extract the downloaded archives.
- Move the binaries to a directory in your
$PATH
, such as/usr/local/bin
or/bin
.
$ go install github.com/marnixbouhuis/confpb/cmd/protoc-gen-default@latest
$ go install github.com/marnixbouhuis/confpb/cmd/protoc-gen-env@latest
$ go install github.com/marnixbouhuis/confpb/cmd/protoc-gen-jsonschema@latest