Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Modified Manager.run startup to account for upgrade stages #5376

Merged
merged 8 commits into from
Mar 10, 2025

Conversation

dlmarion
Copy link
Contributor

@dlmarion dlmarion commented Mar 3, 2025

Updated Manager to account for the status of an upgrate when starting processes. For example, the new fate table is created during the 4.0 upgrade, so we can't start Fate or anything that uses Fate (e.g. Splitter) until after the table has been created.

Closes #5368

Updated Manager to account for the status of an upgrate
when starting processes. For example, the new fate table
is created during the 4.0 upgrade, so we can't start Fate
or anything that uses Fate (e.g. Splitter) until after
the table has been created.

Closes apache#5368
@dlmarion dlmarion added this to the 4.0.0 milestone Mar 3, 2025
@dlmarion dlmarion self-assigned this Mar 3, 2025
Copy link
Member

@ctubbsii ctubbsii left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reordering some things and tracking extra state might work, but it also seems that some of this could be solved more easily by an upgrade step that runs before the manager. It is an extra step we've tried very hard to avoid in the past... for user convenience, but maybe it's necessary for the sake of overall simplicity. I'm not sure.

@dlmarion
Copy link
Contributor Author

dlmarion commented Mar 4, 2025

Reordering some things and tracking extra state might work, but it also seems that some of this could be solved more easily by an upgrade step that runs before the manager. It is an extra step we've tried very hard to avoid in the past... for user convenience, but maybe it's necessary for the sake of overall simplicity. I'm not sure.

That would work for updating ZooKeeper, but I think you also need a TabletServer up at some point to host the root and metadata tablets so that they can be upgraded.

@dlmarion dlmarion marked this pull request as ready for review March 5, 2025 16:33
// Start processing the metadata table
metadataTableTGW.start();
metaTGWStarted = true;
break;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The upgrade code may or may not block waiting on the TGW for each level, it depends on if it reads something or not. So this code may not see each UPGRADED_XXX. Maybe it needs to check the earlier TGW to be safe.

Suggested change
break;
// TODO check if rootTableTGW needs to be started
break;

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed this in cceb9fb

@dlmarion dlmarion merged commit f99847c into apache:main Mar 10, 2025
8 checks passed
@dlmarion dlmarion deleted the 5368-manager-upgrade-start-order branch March 10, 2025 20:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

TableDeletedException in Manager during 4.0.0 upgrade
3 participants