We will see how to deploy KBS (with builtin Attestation Service) on a Kubernetes cluster.
⚠️ Be aware that the manifests and instructions below do not account for all stateful resources in a KBS deployment. Changes to a deployment may be lost if the pod is restarted or rescheduled and service replication might yield inconsistent behaviour. For a production deployment, consider using a persistent volume.
Create a secret that you want to be served using this instance of KBS:
echo "This is my super secret" > overlays/$(uname -m)/key.bin
If you have more than one secret, copy them over to the config/kubernetes/overlays
directory and add those to the overlays/kustomization.yaml
file after as shown below:
...
- name: keys
files:
- key.bin
- secret.key
- password.txt
...
With the default configuration the keys will be stored in reponame/workload_key/
. If you wish to define a different repository make necessary changes to the overlays/patch.yaml
file.
If you would like to expose KBS using Ingress, then run the following commands:
Note
If you are using AKS then set the KBS_INGRESS_CLASS
to addon-http-application-routing
and get the CLUSTER_SPECIFIC_DNS_ZONE
by following the instructions here.
export KBS_INGRESS_CLASS="REPLACE_ME"
export CLUSTER_SPECIFIC_DNS_ZONE="REPLACE_ME"
export KBS_INGRESS_HOST="kbs.${CLUSTER_SPECIFIC_DNS_ZONE}"
pushd overlays
envsubst <ingress.yaml >ingress.yaml.tmp && mv ingress.yaml.tmp ingress.yaml
kustomize edit add resource ingress.yaml
popd
Sometimes it may be desirable to deploy KBS with an image that is not what is set in the repo (typically
the latest release image). To change the deployment to use a staging build, set the image using kustomize
:
pushd base
kustomize edit set image kbs-container-image=ghcr.io/confidential-containers/staged-images/kbs:65ee7e1acccd13dcb515058e71c5f8bfb4281e35
popd
The available image tags can be found in the CoCo packages listing.
If you would like to expose KBS service using Nodeport then export the following environment variable:
export DEPLOYMENT_DIR=nodeport
Once you deploy the KBS, you can use the services' nodeport and the Kubernetes node's IP to reach out to the KBS. You can generate the KBS URL by running the following command:
echo $(kubectl get nodes -o jsonpath='{.items[0].status.addresses[0].address}'):$(kubectl get svc kbs -n coco-tenant -o jsonpath='{.spec.ports[0].nodePort}')
If you would like to override the default sgx_default_qcnl.conf
in the KBS/AS images, copy/configure one into custom_pccs/
directory and deploy using:
export DEPLOYMENT_DIR=custom_pccs
NB: this currently builds on nodeport
kustomization.
Deploy KBS by running the following command:
./deploy-kbs.sh
When deploying trustee on an IBM Secure Execution
enabled environment, where the IBM SE verifier verifier is needed,
an environment variable IBM_SE_CREDS_DIR
is needed that points to a directory containing extra files required for
attestation on IBM Secure Execution:
$ export IBM_SE_CREDS_DIR=/path/to/your/directory
$ tree $IBM_SE_CREDS_DIR
/path/to/your/directory
├── certs
│ ├── DigiCertCA.crt
│ └── ibm-z-host-key-signing-gen2.crt
├── crls
│ └── ibm-z-host-key-gen2.crl
├── hdr
│ └── hdr.bin
├── hkds
│ └── HKD-3931-0275D38.crt
└── rsa
├── encrypt_key.pem
└── encrypt_key.pub
5 directories, 7 files
Please check out the documentation for details.
Note
For running trustee on non-TEE s390x environment using the sample verifier for non-production environments, this extra
IBM_SE_CREDS_DIR
environment variable is not required.
Run the following command to check if the KBS is deployed successfully:
kubectl -n coco-tenant get pods
A successful run will look like the following:
$ kubectl -n coco-tenant get pods
NAME READY STATUS RESTARTS AGE
kbs-bdffc8dd4-jv2kr 1/1 Running 0 7m30s
A Kuberentes service is also deployed as a part of this deployment, you can reach the KBS:
$ kubectl -n coco-tenant get service
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kbs ClusterIP 10.0.210.190 <none> 8080/TCP 4s
$ kubectl delete -k ${DEPLOYMENT_DIR}/$(uname -m)