Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci(l2): automate flamegraph tests #1366

Merged
merged 89 commits into from
Jan 20, 2025
Merged
Show file tree
Hide file tree
Changes from 81 commits
Commits
Show all changes
89 commits
Select commit Hold shift + click to select a range
52da6ca
script that watches balance of account
dsocolobsky Nov 29, 2024
441fc6a
Run flamegraph reporting in Github CI
dsocolobsky Dec 2, 2024
1ce321d
feat(cli/ethrex_l2): allow --default parameter in "ethrex_l2 config c…
dsocolobsky Dec 2, 2024
f9cb7bc
fix: create ethrex_l2 config in flamegraph_reporter.yaml
dsocolobsky Dec 2, 2024
d3679ec
Make load test retry if connection failed
dsocolobsky Dec 2, 2024
37fcfd1
Use addr2line for faster perf usage
dsocolobsky Dec 3, 2024
122a4a8
Use addr2line for faster perf usage 2
dsocolobsky Dec 3, 2024
830159f
Run perf with --no-inline to make it faster
dsocolobsky Dec 3, 2024
967e79e
Run perf with --no-inline to make it faster (2)
dsocolobsky Dec 3, 2024
fbb456f
Run inferno-collapse-perf with --tid parameter
dsocolobsky Dec 3, 2024
803a72f
Run inferno-collapse-perf without --tid parameter
dsocolobsky Dec 3, 2024
0b783ea
Run flamegraph for reth as well
dsocolobsky Dec 4, 2024
6349a7a
Run flamegraph for reth in parallel in a separate job
dsocolobsky Dec 4, 2024
89799a3
Add balance to account in genesis-load-test.json
dsocolobsky Dec 4, 2024
5d3429c
Run cargo flamegraph with perf options
dsocolobsky Dec 4, 2024
0dd0e05
Adjust paths in workflow
dsocolobsky Dec 4, 2024
71b2c5d
Adjust paths in workflow 2
dsocolobsky Dec 4, 2024
73c5edf
Deploy page workflow
dsocolobsky Dec 4, 2024
b128f48
Update deployed page with generated flamegraphs
dsocolobsky Dec 5, 2024
9a7c430
Update deployed page with generated flamegraphs 2
dsocolobsky Dec 5, 2024
29d50a4
Fix .svg paths
dsocolobsky Dec 5, 2024
2d2160b
Add elapsed time for each test to static page
dsocolobsky Dec 5, 2024
320acec
Use Github Action Outputs for test times
dsocolobsky Dec 5, 2024
bc4b4f1
Merge branch 'main' into automate-perf-tests
dsocolobsky Dec 5, 2024
11c6696
Merge branch 'main' into automate-perf-tests
dsocolobsky Dec 6, 2024
5f6d547
Run a 1000 iterations for the load tests.
dsocolobsky Dec 6, 2024
201ab7a
Minor adjustments and cleanup for flamegraph_reporter.yaml
dsocolobsky Dec 6, 2024
6b0c5e2
Simplify flamegraph script into a single one.
dsocolobsky Dec 6, 2024
5db6105
Metadata for Slack link previews
dsocolobsky Dec 6, 2024
43689a8
Send daily flamegraph report via Slack
dsocolobsky Dec 6, 2024
3ff3063
Measure load test time more accurately
dsocolobsky Dec 6, 2024
19e8b78
Fix local script
dsocolobsky Dec 6, 2024
91e0b87
Send direct links to .svg files in Slack
dsocolobsky Dec 6, 2024
dbec9c7
Fix local script "make flamegraph"
dsocolobsky Dec 6, 2024
a28971f
refactor: small improvements
fborello-lambda Jan 7, 2025
dab2121
Merge branch 'main' into automate-perf-tests
fborello-lambda Jan 7, 2025
f55c741
rm: unneeded step in ci
fborello-lambda Jan 7, 2025
9e54c2e
fix: bash scripts duration time
fborello-lambda Jan 8, 2025
3cbb899
Merge branch 'main' into automate-perf-tests
fborello-lambda Jan 8, 2025
9392999
chore: lower the amount of txs
fborello-lambda Jan 8, 2025
d8410ca
Merge branch 'automate-perf-tests' of github.com:lambdaclass/ethrex i…
fborello-lambda Jan 8, 2025
968d951
chore: suppress output
fborello-lambda Jan 8, 2025
15567c9
fix: test pgrep -f
fborello-lambda Jan 8, 2025
0ba0191
fix: test pgrep -l shorter
fborello-lambda Jan 8, 2025
1b8a3a4
fix: test if full_path works
fborello-lambda Jan 8, 2025
8bb0077
fix: test lowering the iterations and wait for perf
fborello-lambda Jan 8, 2025
274c7e4
chore: spinner
fborello-lambda Jan 8, 2025
18e96a5
chore: rm spinner for ci
fborello-lambda Jan 8, 2025
3cca963
fix: calculate time to process the balance
fborello-lambda Jan 9, 2025
a82cb9c
Merge branch 'main' into automate-perf-tests
dsocolobsky Jan 15, 2025
3c39c68
chore: add .idea/ to .gitignore
dsocolobsky Jan 15, 2025
b73b2d4
chore: Bump workflow's CI Rust version to 1.84.0
dsocolobsky Jan 15, 2025
28344d5
chore: Attempt to cache cargo binaries in CI
dsocolobsky Jan 15, 2025
d93c90b
chore: ci: try running reth without profile=profiling
dsocolobsky Jan 15, 2025
9e65249
chore: ci: Use gimli's addr2line for faster perf
dsocolobsky Jan 16, 2025
e1f9063
chore: ci: Cache reth repo
dsocolobsky Jan 16, 2025
dcf001e
chore: ci: Read kernel symbols in perf
dsocolobsky Jan 16, 2025
4552225
chore: ci: Cache ethrex_l2 in "flamegraph-reth" as well
dsocolobsky Jan 16, 2025
56b57b9
chore: ci: Add CARGO_PROFILE_RELEASE_DEBUG=true to reth call
dsocolobsky Jan 16, 2025
a7392f2
chore: ci: Modify slightly how we measure load test time
dsocolobsky Jan 16, 2025
4b47cb6
chore: ci: Try compiling reth with maxperf
dsocolobsky Jan 16, 2025
47c4b8a
chore: ci: fix ethrex_l2 for flamegraph-reth
dsocolobsky Jan 16, 2025
f93037c
chore: ci: Test sleeping 20min before running the perf tests on reth
dsocolobsky Jan 16, 2025
fde9afd
chore: ci: Proper wait for compilation to finish before executing loa…
dsocolobsky Jan 16, 2025
69f5800
chore: Improve slightly the layout of the flamegraph website
dsocolobsky Jan 16, 2025
8c97beb
chore: ci: restore wait for compilation time by minutes
dsocolobsky Jan 16, 2025
01b540b
fix: ci: fix svg path
dsocolobsky Jan 16, 2025
b300c57
chore: ci: cache reth binary
dsocolobsky Jan 16, 2025
1e65609
chore: ci: attempt to fix cache
dsocolobsky Jan 16, 2025
a0a3bce
chore: ci: fix addr2line cache & some improvements
dsocolobsky Jan 16, 2025
d60eb15
chore: ci: bump up iterations
dsocolobsky Jan 16, 2025
6d25f4a
chore: ci: fix cargo install
dsocolobsky Jan 16, 2025
9043e78
chore: ci: fix ethrex and rex cache.
dsocolobsky Jan 16, 2025
18dc60b
Merge branch 'main' into automate-perf-tests
dsocolobsky Jan 16, 2025
687f0da
chore: ci: fix merge issue in daily_reports.yaml
dsocolobsky Jan 16, 2025
f72cacc
chore: ci: bump up iterations for testing cache
dsocolobsky Jan 16, 2025
1950130
chore: ci: Separate into 2 CI jobs for better cache
dsocolobsky Jan 16, 2025
71b8389
Revert "chore: ci: Separate into 2 CI jobs for better cache"
dsocolobsky Jan 16, 2025
adcfeb6
chore: ci: Just use Swatinem/rust-cache@v2 for caching
dsocolobsky Jan 16, 2025
ae1bb5c
chore: ci: Change iterations to test CI cache
dsocolobsky Jan 16, 2025
82782f9
chore: ci: Attempt to cache binaries
dsocolobsky Jan 16, 2025
ff0553f
chore: ci: Change action name & only trigger on commits to main.
dsocolobsky Jan 17, 2025
e00c153
chore: ci: Fix link unfurling in Slack message
dsocolobsky Jan 17, 2025
5735f72
chore: ci: Actually post links to slack
dsocolobsky Jan 17, 2025
f1532cf
fix: ci: Fix cache paths
dsocolobsky Jan 20, 2025
25e8148
fix: ci: Update gas limit
dsocolobsky Jan 20, 2025
785f32b
chore: ci: Start measuring time before executing the test load
dsocolobsky Jan 20, 2025
534d9d0
Merge branch 'main' into automate-perf-tests
dsocolobsky Jan 20, 2025
f7b0a10
Merge branch 'main' into automate-perf-tests
dsocolobsky Jan 20, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 38 additions & 0 deletions .github/scripts/flamegraph_watcher.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
#!/bin/bash

# This script sends 171 * <iterations> transactions to a test account, per defined private key
# then polls the account balance until the expected balance has been reached
# and then kills the process. It also measures the elapsed time of the test and
# outputs it to Github Action's outputs.
iterations=500
value=1
account=0x33c6b73432B3aeA0C1725E415CC40D04908B85fd
end_val=$((171 * $iterations * $value))

ethrex_l2 test load --path /home/runner/work/ethrex/ethrex/test_data/private_keys.txt -i $iterations -v --value $value --to $account >/dev/null

start_time=$(date +%s)
output=$(ethrex_l2 info -b -a $account --wei 2>&1)
while [[ $output -lt $end_val ]]; do
sleep 5
output=$(ethrex_l2 info -b -a $account --wei 2>&1)
done
end_time=$(date +%s)
elapsed=$((end_time - start_time))

minutes=$((elapsed / 60))
seconds=$((elapsed % 60))
output=$(ethrex_l2 info -b -a $account --wei 2>&1)
echo "Balance of $output reached in $minutes min $seconds s, killing process"

echo killing "$PROGRAM"
sudo pkill "$PROGRAM"

while pgrep -l "perf" >/dev/null; do
echo "perf still alive, waiting for it to exit..."
sleep 10
done
echo "perf exited"

# We need this for the following job, to add to the static page
echo "time=$minutes minutes $seconds seconds" >>"$GITHUB_OUTPUT"
23 changes: 23 additions & 0 deletions .github/scripts/publish_link_flamegraphs.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
curl -XPOST -H "Content-type: application/json" -d '{
"blocks": [
{
"type": "header",
"text": {
"type": "plain_text",
"text": "Daily Flamegraph Report"
}
},
{
"type": "divider"
},
{
"type": "section",
"text": {
"type": "mrkdwn",
"text": "🔥 Flamegraphs are available at *<https://lambdaclass.github.io/ethrex/|lambdaclass.github.io/ethrex/>*\n
• *<https://lambdaclass.github.io/ethrex/flamegraph_ethrex.svg/flamegraph_ethrex.svg|Ethrex Flamegraph>*\n
• *<https://lambdaclass.github.io/ethrex/flamegraph_reth.svg/flamegraph_reth.svg|Reth Flamegraph>*\n"
}
}
]
}' "$1"
29 changes: 19 additions & 10 deletions .github/workflows/daily_reports.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -15,19 +15,19 @@ jobs:
runs-on: ubuntu-latest
strategy:
matrix:
vm: [levm, revm]
vm: [ levm, revm ]
test:
- {
name: "Rpc Compat tests",
file_name: rpc-compat,
simulation: ethereum/rpc-compat,
}
name: "Rpc Compat tests",
file_name: rpc-compat,
simulation: ethereum/rpc-compat,
}
- { name: "Devp2p eth tests", file_name: devp2p, simulation: devp2p }
- {
name: "Cancun Engine tests",
file_name: engine,
simulation: ethereum/engine,
}
name: "Cancun Engine tests",
file_name: engine,
simulation: ethereum/engine,
}
- { name: "Sync tests", file_name: sync, simulation: ethereum/sync }

steps:
Expand Down Expand Up @@ -67,7 +67,7 @@ jobs:
runs-on: ubuntu-latest
strategy:
matrix:
vm: [levm, revm]
vm: [ levm, revm ]
steps:
- name: Checkout sources
uses: actions/checkout@v4
Expand Down Expand Up @@ -193,3 +193,12 @@ jobs:
|| secrets.LEVM_SLACK_WEBHOOK
}}
run: sh .github/scripts/publish_levm_ef_tests.sh $SLACK_WEBHOOK

flamegraphs-page:
name: Post to Slack link to Flamegraphs Page
runs-on: ubuntu-latest
steps:
- name: Post message to slack
env:
SLACK_WEBHOOK: ${{ secrets.TEST_CHANNEL_SLACK }}
run: sh .github/scripts/publish_link_flamegraphs.sh $SLACK_WEBHOOK
Loading
Loading