Skip to content

Commit a7ec479

Browse files
committed
Merge #894: Update benchmarking docs
f8a9976 docs: [#770] update benchmarking docs (Jose Celano) Pull request description: We are now using criterion to benchmark the torrent repository implementations. ACKs for top commit: josecelano: ACK f8a9976 Tree-SHA512: 1ac09a5d4d6853a519762e16cf9d43fe7a32ffd1db8109c463d557cb6a411509e9288b1fdbffca82faf14be1ed771afe5c2887c47bbf8aa79c320f3ed1be3527
2 parents e50934e + f8a9976 commit a7ec479

File tree

2 files changed

+63
-33
lines changed

2 files changed

+63
-33
lines changed

docs/benchmarking.md

+63-33
Original file line numberDiff line numberDiff line change
@@ -96,7 +96,7 @@ Announce responses per info hash:
9696
- p100: 361
9797
```
9898

99-
> IMPORTANT: The performance of th Torrust UDP Tracker is drastically decreased with these log levels: `info`, `debug`, `trace`.
99+
> IMPORTANT: The performance of the Torrust UDP Tracker is drastically decreased with these log levels: `info`, `debug`, `trace`.
100100
101101
```output
102102
Requests out: 40719.21/second
@@ -226,46 +226,76 @@ Using a PC with:
226226

227227
## Repository benchmarking
228228

229+
### Requirements
230+
231+
You need to install the `gnuplot` package.
232+
233+
```console
234+
sudo apt install gnuplot
235+
```
236+
237+
### Run
238+
229239
You can run it with:
230240

231241
```console
232-
cargo run --release -p torrust-torrent-repository-benchmarks -- --threads 4 --sleep 0 --compare true
242+
cargo bench -p torrust-tracker-torrent-repository
233243
```
234244

235-
It tests the different implementation for the internal torrent storage.
245+
It tests the different implementations for the internal torrent storage. The output should be something like this:
236246

237247
```output
238-
tokio::sync::RwLock<std::collections::BTreeMap<InfoHash, Entry>>
239-
add_one_torrent: Avg/AdjAvg: (60ns, 59ns)
240-
update_one_torrent_in_parallel: Avg/AdjAvg: (10.909457ms, 0ns)
241-
add_multiple_torrents_in_parallel: Avg/AdjAvg: (13.88879ms, 0ns)
242-
update_multiple_torrents_in_parallel: Avg/AdjAvg: (7.772484ms, 7.782535ms)
243-
244-
std::sync::RwLock<std::collections::BTreeMap<InfoHash, Entry>>
245-
add_one_torrent: Avg/AdjAvg: (43ns, 39ns)
246-
update_one_torrent_in_parallel: Avg/AdjAvg: (4.020937ms, 4.020937ms)
247-
add_multiple_torrents_in_parallel: Avg/AdjAvg: (5.896177ms, 5.768448ms)
248-
update_multiple_torrents_in_parallel: Avg/AdjAvg: (3.883823ms, 3.883823ms)
249-
250-
std::sync::RwLock<std::collections::BTreeMap<InfoHash, Arc<std::sync::Mutex<Entry>>>>
251-
add_one_torrent: Avg/AdjAvg: (51ns, 49ns)
252-
update_one_torrent_in_parallel: Avg/AdjAvg: (3.252314ms, 3.149109ms)
253-
add_multiple_torrents_in_parallel: Avg/AdjAvg: (8.411094ms, 8.411094ms)
254-
update_multiple_torrents_in_parallel: Avg/AdjAvg: (4.106086ms, 4.106086ms)
255-
256-
tokio::sync::RwLock<std::collections::BTreeMap<InfoHash, Arc<std::sync::Mutex<Entry>>>>
257-
add_one_torrent: Avg/AdjAvg: (91ns, 90ns)
258-
update_one_torrent_in_parallel: Avg/AdjAvg: (3.542378ms, 3.435695ms)
259-
add_multiple_torrents_in_parallel: Avg/AdjAvg: (15.651172ms, 15.651172ms)
260-
update_multiple_torrents_in_parallel: Avg/AdjAvg: (4.368189ms, 4.257572ms)
261-
262-
tokio::sync::RwLock<std::collections::BTreeMap<InfoHash, Arc<tokio::sync::Mutex<Entry>>>>
263-
add_one_torrent: Avg/AdjAvg: (111ns, 109ns)
264-
update_one_torrent_in_parallel: Avg/AdjAvg: (6.590677ms, 6.808535ms)
265-
add_multiple_torrents_in_parallel: Avg/AdjAvg: (16.572217ms, 16.30488ms)
266-
update_multiple_torrents_in_parallel: Avg/AdjAvg: (4.073221ms, 4.000122ms)
248+
Running benches/repository_benchmark.rs (target/release/deps/repository_benchmark-2f7830898bbdfba4)
249+
add_one_torrent/RwLockStd
250+
time: [60.936 ns 61.383 ns 61.764 ns]
251+
Found 24 outliers among 100 measurements (24.00%)
252+
15 (15.00%) high mild
253+
9 (9.00%) high severe
254+
add_one_torrent/RwLockStdMutexStd
255+
time: [60.829 ns 60.937 ns 61.053 ns]
256+
Found 1 outliers among 100 measurements (1.00%)
257+
1 (1.00%) high severe
258+
add_one_torrent/RwLockStdMutexTokio
259+
time: [96.034 ns 96.243 ns 96.545 ns]
260+
Found 6 outliers among 100 measurements (6.00%)
261+
4 (4.00%) high mild
262+
2 (2.00%) high severe
263+
add_one_torrent/RwLockTokio
264+
time: [108.25 ns 108.66 ns 109.06 ns]
265+
Found 2 outliers among 100 measurements (2.00%)
266+
2 (2.00%) low mild
267+
add_one_torrent/RwLockTokioMutexStd
268+
time: [109.03 ns 109.11 ns 109.19 ns]
269+
Found 4 outliers among 100 measurements (4.00%)
270+
1 (1.00%) low mild
271+
1 (1.00%) high mild
272+
2 (2.00%) high severe
273+
Benchmarking add_one_torrent/RwLockTokioMutexTokio: Collecting 100 samples in estimated 1.0003 s (7.1M iterationsadd_one_torrent/RwLockTokioMutexTokio
274+
time: [139.64 ns 140.11 ns 140.62 ns]
275+
```
276+
277+
After running it you should have a new directory containing the criterion reports:
278+
279+
```console
280+
target/criterion/
281+
├── add_multiple_torrents_in_parallel
282+
├── add_one_torrent
283+
├── report
284+
├── update_multiple_torrents_in_parallel
285+
└── update_one_torrent_in_parallel
267286
```
268287

288+
You can see one report for each of the operations we are considering for benchmarking:
289+
290+
- Add multiple torrents in parallel.
291+
- Add one torrent.
292+
- Update multiple torrents in parallel.
293+
- Update one torrent in parallel.
294+
295+
Each report look like the following:
296+
297+
![Torrent repository implementations benchmarking report](./media/torrent-repository-implementations-benchmarking-report.png)
298+
269299
## Other considerations
270300

271-
We are testing new repository implementations that allow concurrent writes. See <https://github.com/torrust/torrust-tracker/issues/565>.
301+
If you are interested in knowing more about the tracker performance or contribute to improve its performance you ca join the [performance optimizations discussion](https://github.com/torrust/torrust-tracker/discussions/774).
Loading

0 commit comments

Comments
 (0)