Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

KAFKA-16580: Enable dynamic quorum reconfiguration for raft simulation tests pt 1 #18987

Open
wants to merge 6 commits into
base: trunk
Choose a base branch
from

Conversation

kevin-wu24
Copy link
Contributor

@kevin-wu24 kevin-wu24 commented Feb 20, 2025

What

  • Remove global concept of voters from Cluster, instead this state now is just an accurate view of the initial voters.
  • endpointsFromIds, which is used to set up network partitions for these tests, cannot rely on voters anymore, and must instead use Cluster#running. This means that tests must call Cluster#start() before setting up network partitions.
  • Enable KIP-853 by writing a bootstrap snapshot to each of the initial voters similar to tests that use RaftClientTestContext.
  • Refactor the MajorityReachedHighWatermark invariant for KIP-853: since the voter set can change, the size of the majority that needs to reach the HWM also changes. For clusters running in KIP-853 mode, we check that a majority of at least one of: leader's voter set at the HWM or it's lastVoterSet(), has reached the HWM. We need to perform a more elaborate check here because in clusters where a Add/RemoveVoter request increases/decreases the majority of voters value by 1, the leader could have used either majority value to update its HWM value depending on whether the leader read the most recent VotersRecord prior to updating its HWM value. This matters for invariant checking because we perform this verification on every message delivery.
  • Added RaftTestUtils to house shared functions between these simulation tests and existing integration tests.
  • Add the AddVoterAction and RemoveVoterAction to add or remove voters within simulation tests.

Committer Checklist (excluded from commit message)

  • Verify design and implementation
  • Verify test coverage and CI build status
  • Verify documentation (including upgrade notes)

@github-actions github-actions bot added triage PRs from the community kraft labels Feb 20, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kraft triage PRs from the community
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant