Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[release-1.5] 📖 Add a guide describing ownerReference usage in CAPI #9154

Merged
merged 1 commit into from
Aug 9, 2023
Merged

[release-1.5] 📖 Add a guide describing ownerReference usage in CAPI #9154

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/book/src/SUMMARY.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -115,3 +115,4 @@
- [Code Review in Cluster API](./REVIEWING.md)
- [Version Support](./reference/versions.md)
- [Supported Labels and Annotations](./reference/labels_and_annotations.md)
- [Owner References](./reference/owner_references.md)
80 changes: 80 additions & 0 deletions docs/book/src/reference/owner_references.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
# Owner References


Cluster API uses [Kubernetes owner references](https://kubernetes.io/docs/concepts/overview/working-with-objects/owners-dependents/) to track relationships between objects. These references are used for Kubernetes garbage collection, which is the basis of Cluster deletion in CAPI. They are also used places where the ownership hierarchy is important, for example when using `clusterctl move`.

CAPI uses owner references in an opinionated way. The following guidelines should be considered:
1. Objects should always be created with an owner reference to prevent leaking objects. Initial ownerReferences can be replaced later where another object is a more appropriate owner.
2. Owner references should be re-reconciled if they are lost for an object. This is required as some tools - e.g. velero - may delete owner references on objects.
3. Owner references should be kept to the most recent apiVersion.
- This ensures garbage collection still works after an old apiVersion is no longer served.
4. Owner references should not be added unless required.
- Multiple owner references on a single object should be exceptional.




## Owner reference relationships in Cluster API

The below tables map out the a reference for ownership relationships for the objects in a Cluster API cluster.
Providers may implement their own ownership relationships which may or may not map directly to the below tables.
These owner references are almost all tested in an [end-to-end test](https://github.com/kubernetes-sigs/cluster-api/blob/caaa74482b51fae777334cd7a29595da1c06481e/test/e2e/quick_start_test.go#L31). Lack of testing is noted where this is not the case. CAPI Providers can take advantage of the e2e test framework to ensure their owner references are predictable, documented and stable.

Kubernetes core types

| type | Owner | Note |
|-----------|---------------------|------------------------------------------|
| Secret | KubeadmControlPlane | For cluster certificates |
| Secret | KubeadmConfig | For bootstrap secrets |
| Secret | ClusterResourceSet | When created by CRS. Not covered in e2e. |
| ConfigMap | ClusterResourceSet | When created by CRS |

## Core types

| type | Owner | Note |
|---------------------|---------------------|----------------------------|
| ExtensionConfig | None | |
| ClusterClass | None | |
| Cluster | None | |
| MachineDeployments | Cluster | |
| MachineSet | MachineDeployment | |
| Machine | MachineSet | When created by MachineSet |
| Machine | KubeadmControlPlane | When created by KCP |
| MachineHealthChecks | Cluster | |



## Experimental types
| type | Owner | Note |
|----------------------------|--------------------|------|
| ClusterResourcesSet | None | |
| ClusterResourcesSetBinding | ClusterResourceSet | |
| MachinePool | Cluster | |


## KubeadmControlPlane types
| type | Owner | Note |
|-----------------------------|--------------|------|
| KubeadmControlPlane | Cluster | |
| KubeadmControlPlaneTemplate | ClusterClass | |


## Kubeadm bootstrap types
| type | Owner | Note |
|-----------------------|--------------|-------------------------------------------|
| KubeadmConfig | Machine | When created for Machine |
| KubeadmConfig | MachinePool | When created for MachinePool |
| KubeadmConfigTemplate | Cluster | When referenced in MachineDeployment spec |
| KubeadmConfigTemplate | ClusterClass | When referenced in ClusterClass |

## Infrastructure provider types
| type | Owner | Note |
|-------------------------------|--------------|---------------------------------------------|
| InfrastructureMachine | Machine | |
| InfrastructureMachineTemplate | Cluster | When created by cluster topology controller |
| InfrastructureMachineTemplate | ClusterClass | When referenced in a ClusterClass |
| InfrastructureCluster | Cluster | |
| InfrastructureClusterTemplate | ClusterClass | |
| InfrastructureMachinePool | MachinePool | |


12 changes: 12 additions & 0 deletions test/framework/ownerreference_helpers.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -118,6 +118,8 @@ var (

// CoreOwnerReferenceAssertion maps Cluster API core types to functions which return an error if the passed
// OwnerReferences aren't as expected.
// Note: These relationships are documented in https://github.com/kubernetes-sigs/cluster-api/tree/main/docs/book/src/reference/owner_references.md.
// That document should be updated if these references change.
var CoreOwnerReferenceAssertion = map[string]func([]metav1.OwnerReference) error{
extensionConfigKind: func(owners []metav1.OwnerReference) error {
// ExtensionConfig should have no owners.
Expand Down Expand Up @@ -161,6 +163,8 @@ var (

// ExpOwnerReferenceAssertions maps experimental types to functions which return an error if the passed OwnerReferences
// aren't as expected.
// Note: These relationships are documented in https://github.com/kubernetes-sigs/cluster-api/tree/main/docs/book/src/reference/owner_references.md.
// That document should be updated if these references change.
var ExpOwnerReferenceAssertions = map[string]func([]metav1.OwnerReference) error{
clusterResourceSetKind: func(owners []metav1.OwnerReference) error {
// ClusterResourcesSet doesn't have ownerReferences (it is a clusterctl move-hierarchy root).
Expand All @@ -184,6 +188,8 @@ var (

// KubernetesReferenceAssertions maps Kubernetes types to functions which return an error if the passed OwnerReferences
// aren't as expected.
// Note: These relationships are documented in https://github.com/kubernetes-sigs/cluster-api/tree/main/docs/book/src/reference/owner_references.md.
// That document should be updated if these references change.
var KubernetesReferenceAssertions = map[string]func([]metav1.OwnerReference) error{
secretKind: func(owners []metav1.OwnerReference) error {
// Secrets for cluster certificates must be owned by the KubeadmControlPlane. The bootstrap secret should be owned by a KubeadmControlPlane.
Expand All @@ -205,6 +211,8 @@ var (

// KubeadmControlPlaneOwnerReferenceAssertions maps Kubeadm control plane types to functions which return an error if the passed
// OwnerReferences aren't as expected.
// Note: These relationships are documented in https://github.com/kubernetes-sigs/cluster-api/tree/main/docs/book/src/reference/owner_references.md.
// That document should be updated if these references change.
var KubeadmControlPlaneOwnerReferenceAssertions = map[string]func([]metav1.OwnerReference) error{
kubeadmControlPlaneKind: func(owners []metav1.OwnerReference) error {
// The KubeadmControlPlane must be owned by a Cluster.
Expand All @@ -226,6 +234,8 @@ var (

// KubeadmBootstrapOwnerReferenceAssertions maps KubeadmBootstrap types to functions which return an error if the passed OwnerReferences
// aren't as expected.
// Note: These relationships are documented in https://github.com/kubernetes-sigs/cluster-api/tree/main/docs/book/src/reference/owner_references.md.
// That document should be updated if these references change.
var KubeadmBootstrapOwnerReferenceAssertions = map[string]func([]metav1.OwnerReference) error{
kubeadmConfigKind: func(owners []metav1.OwnerReference) error {
// The KubeadmConfig must be owned by a Cluster or by a MachinePool.
Expand All @@ -248,6 +258,8 @@ var (

// DockerInfraOwnerReferenceAssertions maps Docker Infrastructure types to functions which return an error if the passed
// OwnerReferences aren't as expected.
// Note: These relationships are documented in https://github.com/kubernetes-sigs/cluster-api/tree/main/docs/book/src/reference/owner_references.md.
// That document should be updated if these references change.
var DockerInfraOwnerReferenceAssertions = map[string]func([]metav1.OwnerReference) error{
dockerMachineKind: func(owners []metav1.OwnerReference) error {
// The DockerMachine must be owned by a Machine.
Expand Down