v8s-router

A highly available Kubernetes operator built to simplify service exposing. Service is a resource on Kubernetes cluster which defines policies to reach Pod. Although there are multiple methods (i.e. ClusterIP, NodePort, LoadBalancer, ExternalName) for exposing Service to external applications, technical know-how is required for familiar usage. v8s-router delivers Route as a higher level wrapper to Service and encapsulates those methods, while respecting Ingress from the ever-growing Kubernetes community.

Versions

Name	Version
Golang	1.15.2
Operator-sdk	1.0.1
Helm	3.3.3
bitnami/metallb	0.1.21
haproxytech/kubernetes-ingress	1.6.0
ingress-nginx/ingress-nginx	3.3.0

Development

This project is a Kubernetes native application (Operator) running on Operator Framework.

To install Custom Resource into cluster

make install

To run controller locally and reconcile Route on cluster

make run

API

v8s-router offers a Custom Resource as Route to expose its functionalities. The documentations of Route are presented as inline API docs inside route_types.go.

Deploying to Kubernetes

Despite v8s-router is an infrastructural component equipped on v8s cluster, it can be deployed to vanilla Kubernetes with prerequisites setup as followings.

MetalLB should be installed to provide high level loadbalancing for on-premises cluster. Installation guide for MetalLB can be found here.

As v8s-router uses Ingress to support CNI traffic routing, it is deemed to use ingress controller for actual implementation on ingress mode. Currently HAProxy and Nginx are supported as backing ingress controller, and these two ingress controllers should be deployed for v8s-router to adopt.

HAProxy Ingress Controller can be installed following here.

Nginx Ingress Controller can be installed following here.

After installing listed prerequisites, configurations are required for all the components to integrate with v8s-router. MetalLB should provide specific address pools and ingress controllers (i.e. HAProxy and Nginx) should be annotated with regarding ingress.class. You can refer to route_types.go and route_controller.go for detailed configurations.

To get v8s-router, you can first add the chart repository

helm repo add v8s-router https://cedio.github.io/v8s-router

You can verify the charts within the added chart repository

helm search repo v8s-router

After that you can install the chart using helm

helm install my-router v8s-router/v8s-router \
  --namespace my-namespace \
  --set controller.kind=Deployment \
  --set controller.extraArgs."cluster-domain"="example.com"

Detailed helm chart values can be found here.

Delivery

Helm Chart

1. Update Chart.yaml with regarding appVersion and version. Set appVersion for controller version on Docker Hub and version for chart version

2. Commit and Pull Request to master branch

3. Github Action pipeline will publish latest chart on cedio.github.io

Container Image

1. Commit and Pull Request to master branch

2. Create Release with Tag /^controller-([0-9.]+)$/

3. Docker Hub pipeline will build and publish latest image based on tagged Release

Known Issues

• Both HAProxy Ingress Controller and Nginx Ingress Controller hangs sometime during backend update (i.e. Ingress creation/deletion), use kubectl rollout restart daemonsets/ingress-controller to restart

• TLS Re-encryption only supports certificate changing, but not backend certificate verification. Thus pseudo re-encryption

• Blue/Green not supported

Example

Loadbalancer External

kind: Route
apiVersion: router.v8s.cedio.dev/v1beta1
metadata:
  name: nginx-route
  namespace: route-test
spec:
  serviceName: nginx
  type: loadbalancer
  loadbalancer:
    addressPool: external

HAProxy Ingress Without TLS

kind: Route
apiVersion: router.v8s.cedio.dev/v1beta1
metadata:
  name: nginx-route
  namespace: route-test
spec:
  serviceName: nginx
  type: ingress
  ingress:
    class: haproxy
    servicePort: 80
    host: nginx.apps1.v8s.lab