Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs(controllers): adds a section on controllers quorums and limitations #10864

Merged
merged 2 commits into from
Nov 26, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 34 additions & 3 deletions documentation/assemblies/deploying/assembly-kraft-mode.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -50,11 +50,42 @@ In the following example, three nodes perform a dual role and two nodes act only
.Example cluster with dual-role nodes and dedicated broker nodes
image::kraft-dual-role-quorum.png[KRaft cluster with nodes that combine roles]

== KRaft limitations
== Controller scaling
PaulRMellor marked this conversation as resolved.
Show resolved Hide resolved

Currently, the KRaft mode in Strimzi has the following major limitations:
KRaft mode supports two types of controller quorums:

* Scaling of KRaft controller nodes up or down is not supported.
* *Static controller quorums* +
In this mode, the number of controllers is fixed, and scaling requires downtime.
* *Dynamic controller quorums* +
This mode enables dynamic scaling of controllers without downtime.
New controllers join as observers, replicate the metadata log, and eventually become eligible to join the quorum.
If a controller being removed is the active controller, it will step down from the quorum only after the new quorum is confirmed.

Scaling is useful not only for adding or removing controllers, but supports the following operations:

* Replacing controllers because of hardware failure
* Migrating clusters onto new hardware
* Moving nodes from dedicated controller roles to dual roles or vice versa
PaulRMellor marked this conversation as resolved.
Show resolved Hide resolved

Dynamic controller quorums provide the flexibility to make these operations significantly easier to perform.

== Limitations with static controller quorums

Strimzi currently supports only static controller quorums.
This limitation means dynamic scaling of controller quorums cannot be used to support the following:

* Scaling up or down of node pools that have controller roles
* Adding or removing controller roles from node pools
* Switching node pools between dedicated controller roles and dual roles
PaulRMellor marked this conversation as resolved.
Show resolved Hide resolved

Static controller quorums also limit operations that require scaling.
For example, changing the storage type for a node pool with a controller role is not possible because it involves scaling the controller quorum. For non-JBOD storage, creating a new node pool with the desired storage type, adding it to the cluster, and removing the old one would require scaling operations, which are not supported.
In some cases, workarounds are possible.
For instance, when modifying node pool roles to combine controller and broker functions, you can xref:proc-joining-node-pools-roles-str[add broker roles to controller nodes] instead of adding controller roles to broker nodes to avoid controller scaling.
However, this approach would require reassigning more data, temporarily affecting cluster performance.

Migration between static and dynamic controller quorums is not currently supported, though it is expected to be introduced in a future Kafka release.
PaulRMellor marked this conversation as resolved.
Show resolved Hide resolved
PaulRMellor marked this conversation as resolved.
Show resolved Hide resolved
Once migration is possible, Strimzi plans to assess introducing support for dynamic quorums.

//migrating to KRaft
include::../../modules/deploying/proc-deploy-migrate-kraft.adoc[leveloffset=+1]
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,9 @@ Storage management in Strimzi is usually straightforward, and requires little ch
Node pools simplify this process, because you can set up separate node pools that specify your new storage requirements.

In this procedure we create and manage storage for a node pool called `pool-a` containing three nodes.
The steps require a scaling operation to add a new node pool.
Currently, scaling is only possible for broker-only node pools containing nodes that run as dedicated brokers.

We show how to change the storage class (`volumes.class`) that defines the type of persistent storage it uses.
You can use the same steps to change the storage size (`volumes.size`).
This approach is particularly useful if you want to reduce disk sizes.
Expand Down Expand Up @@ -38,6 +41,8 @@ metadata:
labels:
strimzi.io/cluster: my-cluster
spec:
roles:
- broker
replicas: 3
storage:
type: jbod
Expand Down
Loading