This is a mono repository for my home infrastructure and Kubernetes cluster. I try to adhere to Infrastructure as Code (IaC) and GitOps practices using tools like Kubernetes, Flux, Renovate, and GitHub Actions.
There is a template over at onedr0p/cluster-template that I used to create this repository. It is a good starting point for creating a new repository for your own cluster. Also consider joining the Home Operations discord!
My Kubernetes cluster is deployed with Talos. This is a semi-hyper-converged cluster - workloads and block storage are sharing the same available resources on my nodes, while I have a separate truenas server using ZFS for NFS/SMB shares, bulk file storage and backups.
- cert-manager: Creates SSL certificates for services in my cluster.
- cilium: Internal Kubernetes container networking interface.
- cloudflared: Enables Cloudflare secure access to certain ingresses.
- external-dns: Automatically syncs ingress DNS records to a DNS provider.
- SOPS: encrypted Kubernetes secrets using age
- ingress-nginx: Kubernetes ingress controller using NGINX as a reverse proxy and load balancer.
- sops: Managed secrets for Kubernetes and Terraform which are commited to Git.
- spegel: Stateless cluster local OCI registry mirror.
- volsync: Backup and recovery of persistent volume claims.
Flux watches the clusters in my kubernetes folder (see Directories below) and makes the changes to my clusters based on the state of my Git repository.
The way Flux works for me here is it will recursively search the kubernetes/apps folder until it finds the most top level kustomization.yaml per directory and then apply all the resources listed in it. That aforementioned kustomization.yaml will generally only have a namespace resource and one or many Flux kustomizations (ks.yaml). Under the control of those Flux kustomizations there will be a HelmRelease or other resources related to the application which will be applied.
Renovate watches my entire repository looking for dependency updates, when they are found a PR is automatically created. When some PRs are merged Flux applies the changes to my cluster.
This Git repository contains the following directories under Kubernetes.
📁 kubernetes
├── 📁 apps # applications
├── 📁 bootstrap # bootstrap procedures
├── 📁 flux # core flux configuration
└── 📁 templates # re-useable componentsI'm using Cilium as my CNI, pod-gateway for routing traffic through VPNs when needed, and cloudflared to get my services exposed without needing to port forward. (although i do anyways, lol)
While most of my infrastructure and workloads are self-hosted I do rely upon the cloud for certain key parts of my setup. This saves me from having to worry about two things. (1) Dealing with chicken/egg scenarios and (2) services I critically need whether my cluster is online or not.
The alternative solution to these two problems would be to host a Kubernetes cluster in the cloud and deploy applications like Vault, Vaultwarden, ntfy, and Gatus. However, maintaining another cluster and monitoring another group of workloads is a lot more time and effort than I am willing to put in.
| Service | Use | Cost |
|---|---|---|
| Cloudflare | DNS, Argo Tunnels | Free |
| GitHub | Hosting this repository and continuous integration/deployments | Free |
| NextDNS | My router DNS server which includes AdBlocking | ~$20/yr |
| Total: ~$20/yr |
I'm using external-dns to manage my dns records in cloudflare. Works great!
| Device | Count | Disk Size | Ram | Operating System | Purpose |
|---|---|---|---|---|---|
| Intel NUC7i5BNK | 1 | 1TB SSD | 32GB | Talos | getting started node |
| Intel NUC8i7BEH | 1 | 1TB SSD | 32GB | Talos | thank u klipper ❤️ |
| Homebuilt PC (Ryzen 5600G) | 1 | 2TB SSD | 32GB | Talos | the real workhorse |
| HP T740 Thin Client | 2 | 1TB SSD | varies | Talos | what a steal for $50! |
Special thanks goes out to onedr0p for his awesome template and readme which i have shamelessly stolen and tweaked.
Thanks to all the people who donate their time to the Home Operations Discord community. Be sure to check out kubesearch.dev for ideas on how to deploy applications or get ideas on what you may deploy.