Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(l1): full-sync with both consensus and execution node #2081

Draft
wants to merge 4 commits into
base: fix/full-sync
Choose a base branch
from

Conversation

avilagaston9
Copy link
Contributor

@avilagaston9 avilagaston9 commented Feb 25, 2025

Motivation

Currently, during full-sync, we download, execute and store blocks as canonical in chunks of MAX_BLOCK_BODIES_TO_REQUEST=128. However, we do not verify whether we are ahead of the consensus head, which results in invalid forkChoice responses.

Description

  • Filters the downloaded headers to request only the needed bodies to peers.

Closes #2066

@avilagaston9 avilagaston9 self-assigned this Feb 25, 2025
Copy link

github-actions bot commented Feb 25, 2025

| File                                                                            | Lines | Diff  |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/cmd/ethrex/cli.rs                               | 151   | -34   |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/cmd/ethrex/ethrex.rs                            | 393   | -37   |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/cmd/ethrex_l2/src/cli.rs                        | 53    | -3    |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/cmd/ethrex_l2/src/commands/mod.rs               | 7     | -1    |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/blockchain/fork_choice.rs                | 179   | +40   |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/common/types/transaction.rs              | 2486  | -27   |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/l2/proposer/errors.rs                    | 153   | -2    |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/l2/prover/bench/src/main.rs              | 101   | -3    |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/networking/p2p/peer_handler.rs           | 516   | +1    |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/networking/p2p/rlpx/connection.rs        | 460   | -41   |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/networking/p2p/rlpx/eth/transactions.rs  | 289   | -8    |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/networking/p2p/sync.rs                   | 470   | +37   |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/networking/rpc/clients/auth/mod.rs       | 155   | +1    |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/networking/rpc/engine/fork_choice.rs     | 315   | -42   |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/networking/rpc/engine/payload.rs         | 567   | -58   |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/networking/rpc/eth/filter.rs             | 548   | -24   |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/networking/rpc/eth/gas_price.rs          | 121   | -8    |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/networking/rpc/eth/max_priority_fee.rs   | 106   | -8    |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/networking/rpc/eth/transaction.rs        | 542   | -34   |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/networking/rpc/rpc.rs                    | 520   | -102  |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/networking/rpc/utils.rs                  | 277   | -12   |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/storage/store/api.rs                     | 207   | +207  |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/storage/store/error.rs                   | 47    | +47   |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/storage/store/lib.rs                     | 11    | +11   |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/storage/store/rlp.rs                     | 90    | +90   |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/storage/store/store.rs                   | 1272  | +1272 |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/storage/store/store_db/in_memory.rs      | 516   | +516  |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/storage/store/store_db/libmdbx.rs        | 927   | +927  |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/storage/store/store_db/mod.rs            | 5     | +5    |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/storage/store/store_db/redb.rs           | 918   | +918  |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/storage/store/trie_db/libmdbx.rs         | 169   | +169  |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/storage/store/trie_db/libmdbx_dupsort.rs | 85    | +85   |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/storage/store/trie_db/mod.rs             | 11    | +11   |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/storage/store/trie_db/redb.rs            | 65    | +65   |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/storage/store/trie_db/redb_multitable.rs | 81    | +81   |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/storage/store/trie_db/test_utils.rs      | 23    | +23   |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/storage/store/trie_db/utils.rs           | 11    | +11   |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/storage/store/utils.rs                   | 56    | +56   |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/storage/trie/db.rs                       | 42    | +42   |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/storage/trie/error.rs                    | 15    | +15   |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/storage/trie/nibbles.rs                  | 224   | +224  |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/storage/trie/node.rs                     | 150   | +150  |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/storage/trie/node/branch.rs              | 536   | +536  |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/storage/trie/node/extension.rs           | 422   | +422  |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/storage/trie/node/leaf.rs                | 265   | +265  |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/storage/trie/node_hash.rs                | 105   | +105  |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/storage/trie/rlp.rs                      | 107   | +107  |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/storage/trie/src/main.rs                 | 3     | +3    |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/storage/trie/state.rs                    | 73    | +73   |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/storage/trie/test_utils.rs               | 68    | +68   |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/storage/trie/trie.rs                     | 808   | +808  |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/storage/trie/trie_iter.rs                | 91    | +91   |
+---------------------------------------------------------------------------------+-------+-------+
| /home/runner/work/ethrex/ethrex/crates/storage/trie/verify_range.rs             | 620   | +620  |
+---------------------------------------------------------------------------------+-------+-------+

Total lines added: +8102
Total lines removed: 444
Total lines changed: 8546

@@ -265,7 +265,7 @@ impl SyncManager {
store.add_block_headers(block_hashes.clone(), block_headers)?;

if self.sync_mode == SyncMode::Full {
self.download_and_run_blocks(&mut block_hashes, store.clone())
self.download_and_run_blocks(&mut block_hashes, store.clone(), sync_head)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe, instead of passing sync_head , you should filter block_hashes so that every block after sync_head is filtered out. Does this make sense?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes! Actually, my approach for some reason wasn't working. This is simpler and it is working in the server. I will update the PR later

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated!

@avilagaston9 avilagaston9 marked this pull request as ready for review February 26, 2025 21:36
@avilagaston9 avilagaston9 requested a review from a team as a code owner February 26, 2025 21:36
@avilagaston9 avilagaston9 marked this pull request as draft February 26, 2025 21:37
Comment on lines 268 to 275
if sync_head_found {
// Filter out everything after the sync_head
block_hashes = block_hashes
.iter()
.take_while(|&hash| *hash != sync_head)
.cloned()
.collect();
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This logic should be on both sync modes: snap and full.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated in 973719d!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants