Skip to content

Latest commit

 

History

History
254 lines (185 loc) · 7.21 KB

README.md

File metadata and controls

254 lines (185 loc) · 7.21 KB
Raw

infra-shared

Software Defined Git Operated Infrastructure

Reusables of a learning project by rewriting parts of my home infrastructure as

a Pulumi and Fedora Coreos based Gitops Project in Python.

  • See safe for usage in an example project

Quick start

create a base project, lock and install build requirements, install and configure a simulation of the targets

mkdir -p example; cd example; git init
git submodule add https://github.com/wuxxin/infra-shared.git infra
infra/scripts/create_skeleton.sh --yes
make sim-up

Congratulations!

You have just created two TLS Certificates and an SSH Keypair in a very fancy way!

See the examples for code of what else can be done with it

Features

  • Fedora-CoreOS Linux - updating, minimal, monolithic, container-focused operating system
    • Setup: Bootstrap and Reconfiguration of CoreOS with Jinja templated butane files
    • Reconfiguration: update-system-config*
      • Fast (~4s) reconfiguration using saltstack and butane to salt translation
    • Single Container: podman-systemd.unit
      • container* - run systemd container units using podman-quadlet
    • Compose Container: compose.yml
      • compose* - run multi-container applications defined using a compose file
    • nSpawn OS-Container: systemd-nspawn
      • nspawn* - run any linux OS in a light-weight system container
    • tls/http/web FrontEnd: traefik
      • using container, compose and nspawn labels for dynamic configuration
    • DNS Resolver: unbound
      • using container for local DNSSEC capable recursive DNS-Resolver
  • TLS Certificate-Authority, TLS Certificates and SSH-Certificates
  • SSH copy/deploy/execute functions, local and remote Salt-Call
  • serve configuration HTTPS payloads, request a port forwarding
  • write image to removable storage specified by serial_number
  • build Embedded-OS Images and IOT Images
    • Raspberry PI Extras - Eeprom, U-Boot and UEFI bios files
    • Openwrt Linux - Network Device Distribution for Router and other network devices

Technologies

Need to know technologies (to write Deployment and Docs):

  • Basic Knowledge of Python, Yaml, Jinja, Systemd Service, Containerfile, Markdown

Advanced functionality available with knowledge of:

  • Pulumi, Butane, more Systemd, Fcos, Saltstack, Podman, compose.yml, makefile, Pipfile, libvirt, Bash, Mkdocs, Mermaid, Marimo Notebooks

Provision can be run on Arch Linux, Manjaro Linux or as Container Image.

Tools used

  • pulumi - imperativ infrastructure delaration using python
  • fcos - Fedora-CoreOS, minimal OS with clevis (sss,tang,tpm) storage unlock
  • butane - create fcos ignition configs using jinja enhanced butane yaml
  • systemd - service, socker, path, timer, nspawn machine container
  • podman - build Container and NSpawn images, run Container using quadlet systemd container
  • saltstack
    • local build environments and local services
    • remote fcos config update using butane to saltstack translation and execution
  • mkdocs - documentation using markdown and mermaid
  • libvirt - simulation of machines using the virtualization api supporting qemu and kvm
  • tang - server used for getting a key shard for unattended encrypted storage unlock on boot
  • age - ssh keys based encryption of production files and pulumi master password
  • uv- virtualenv management using pyproject.toml and uv.lock

Usage

List available Makefile targets/commands

make

Bootstrap skeleton files to a new repo

  • from current directory, eg. pwd=~/code
project_name=example
current_dir=$(pwd)
project_dir=${current_dir}/${project_name}
mkdir -p ${project_dir}
cd ${project_dir}
git init
git submodule add https://github.com/wuxxin/infra-shared.git infra
infra/create_skeleton.sh --yes
  • create_skeleton.sh creates default dirs and files in the project_dir
    • use cat infra/create_skeleton.sh to inspect script before running it
    • directories created:
      • docs, state, target with an empty .gitkeep file inside
    • files created:
      • README.md, __main__.py, Pulumi.yaml, Makefile, Pipfile
      • config-template.yaml, .gitignore, mkdocs.yml, empty authorized_keys

Install build requirements

  • on arch or manjaro linux
make install-requirements
  • on other linux, use a provision container.

This needs podman or docker already installed on host.

For the simulation environment with libvirt the host system must also have a configured libvirt.

# Either: build container using `sudo podman build`
make provision-client

# Or: build container using any other container tool
# - replace "docker" with the preferred container build call
cd infra/Containerfile/provision-client && \
    docker build -t provision-client:latest $(pwd)

# call provision shell(defaults to /usr/bin/bash interactive shell)
# defaults to podman, but can be overriden with DOCKER_CMD=executable
DOCKER_CMD=docker infra/scripts/provision_shell.sh
# use exit to return to base shell

Build documentation

make docs
# build infra-shared documentation
make docs-infra

Create/build/install simulation target

make sim-up

Show/use root and provision cert

make sim-show args="ca_factory" | jq ".root_cert_pem" -r | \
    openssl x509 -in /dev/stdin -noout -text
make sim-show args="ca_factory" | jq ".provision_cert_pem" -r | \
    openssl x509 -in /dev/stdin -noout -text

Manual pulumi invocation

export PULUMI_SKIP_UPDATE_CHECK=1
export PULUMI_CONFIG_PASSPHRASE=sim

pulumi stack select sim
pulumi about

Execute in provision python environment

uv run ipython

Sim stack: destroy, cleanup, re/create

make sim-clean
# in case something happend while destroying sim stack
make sim__ args="stack rm --force"; rm Pulumi.sim.yaml
# recreate stack
make sim-create

test if changes would compute before applying

make sim-preview
# if list of changes looks good, apply them
make sim-up

cancel an currently running/stuck pulumi update

# "error: the stack is currently locked by 1 lock(s)."
# "Either wait for the other process(es) to end or delete the lock file with `pulumi cancel`."
make sim__ args="cancel"

show resource output as json

make sim-show

show resource output key list as yaml

make sim-list

show resource output data as colorized formatted json or yaml

# use highlight and less
make sim-show | highlight --syntax json -O ansi | less
# use bat for integrated highlight plus pager
make sim-show | bat -l json

Production

Add SSH Keys of GitOps Developer

# eg. add the own ssh public key in project_dir/authorized_keys
cat ~/.ssh/id_rsa.pub >> authorized_keys

Create stack

make prod-create
make prod__ args="preview --suppress-outputs"
make prod__ args=up

Credits

License

All code in this repository is covered by the terms of the Apache 2.0 License,
the full text of which can be found in the LICENSE file.