Skip to content

Commit 92772cb

Browse files
josecelanojosecelano
committed
Merge #732: Docs: benchmarking
45c77c3 fix: linter error, unused code (Jose Celano) d32a748 docs: [#87] benchmarking (Jose Celano) fd0ad1b test: [#87] remove lua script for HTTP tracker benchmarking (Jose Celano) Pull request description: - How to run E2E load tests with aquatic load test commands. - How to run storage implementations benchmarking report. ACKs for top commit: josecelano: ACK 45c77c3 Tree-SHA512: 7523262981adf59dadc638366e7bc2b5bc5999517dd97bf68a1341cacdfdc4c9418f9852bdf904e0e401b2a4758795ebdf8e0b33d148f15458382b25c1ac6186
2 parents 930a50d + 45c77c3 commit 92772cb

File tree

7 files changed

+294
-84
lines changed

7 files changed

+294
-84
lines changed

‎README.md

+4
Original file line numberDiff line numberDiff line change
@@ -126,6 +126,10 @@ The following services are provided by the default configuration:
126126
- [Tracker (HTTP/TLS)][HTTP]
127127
- [Tracker (UDP)][UDP]
128128

129+
## Benchmarking
130+
131+
- [Benchmarking](./docs/benchmarking.md)
132+
129133
## Contributing
130134

131135
We are happy to support and welcome new people to our project. Please consider our [contributor guide][guide.md].</br>

‎cSpell.json

+1
Original file line numberDiff line numberDiff line change
@@ -50,6 +50,7 @@
5050
"hexlify",
5151
"hlocalhost",
5252
"Hydranode",
53+
"hyperthread",
5354
"Icelake",
5455
"imdl",
5556
"impls",

‎docs/benchmarking.md

+252
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,252 @@
1+
# Benchmarking
2+
3+
We have two types of benchmarking:
4+
5+
- E2E benchmarking running the service (HTTP or UDP tracker).
6+
- Internal torrents repository benchmarking.
7+
8+
## E2E benchmarking
9+
10+
We are using the scripts provided by [aquatic](https://github.com/greatest-ape/aquatic).
11+
12+
Installing both commands:
13+
14+
```console
15+
cargo install aquatic_udp_load_test
16+
cargo install aquatic_http_load_test
17+
```
18+
19+
### Run UDP load test
20+
21+
Run the tracker with UDP service enabled on port 3000 and set log level to `error`.
22+
23+
```toml
24+
log_level = "error"
25+
26+
[[udp_trackers]]
27+
bind_address = "0.0.0.0:3000"
28+
enabled = true
29+
```
30+
31+
Run the load test with:
32+
33+
```console
34+
aquatic_udp_load_test
35+
```
36+
37+
Output:
38+
39+
```output
40+
Starting client with config: Config {
41+
server_address: 127.0.0.1:3000,
42+
log_level: Error,
43+
workers: 1,
44+
duration: 0,
45+
network: NetworkConfig {
46+
multiple_client_ipv4s: true,
47+
first_port: 45000,
48+
poll_timeout: 276,
49+
poll_event_capacity: 2877,
50+
recv_buffer: 6000000,
51+
},
52+
requests: RequestConfig {
53+
number_of_torrents: 10000,
54+
scrape_max_torrents: 50,
55+
weight_connect: 0,
56+
weight_announce: 100,
57+
weight_scrape: 1,
58+
torrent_gamma_shape: 0.2,
59+
torrent_gamma_scale: 100.0,
60+
peer_seeder_probability: 0.25,
61+
additional_request_probability: 0.5,
62+
},
63+
}
64+
65+
Requests out: 32632.43/second
66+
Responses in: 24239.33/second
67+
- Connect responses: 7896.91
68+
- Announce responses: 16327.01
69+
- Scrape responses: 15.40
70+
- Error responses: 0.00
71+
Peers per announce response: 33.10
72+
```
73+
74+
### Run HTTP load test
75+
76+
Run the tracker with UDP service enabled on port 3000 and set log level to `error`.
77+
78+
```toml
79+
[[udp_trackers]]
80+
bind_address = "0.0.0.0:3000"
81+
enabled = true
82+
```
83+
84+
Run the load test with:
85+
86+
```console
87+
aquatic_http_load_test
88+
```
89+
90+
Output:
91+
92+
```output
93+
Starting client with config: Config {
94+
server_address: 127.0.0.1:3000,
95+
log_level: Error,
96+
num_workers: 1,
97+
num_connections: 128,
98+
connection_creation_interval_ms: 10,
99+
url_suffix: "",
100+
duration: 0,
101+
keep_alive: true,
102+
torrents: TorrentConfig {
103+
number_of_torrents: 10000,
104+
peer_seeder_probability: 0.25,
105+
weight_announce: 5,
106+
weight_scrape: 0,
107+
torrent_gamma_shape: 0.2,
108+
torrent_gamma_scale: 100.0,
109+
},
110+
cpu_pinning: CpuPinningConfigDesc {
111+
active: false,
112+
direction: Descending,
113+
hyperthread: System,
114+
core_offset: 0,
115+
},
116+
}
117+
```
118+
119+
### Comparing UDP tracker with other Rust implementations
120+
121+
#### Torrust UDP Tracker
122+
123+
Running the tracker:
124+
125+
```console
126+
git@github.com:torrust/torrust-tracker.git
127+
cd torrust-tracker
128+
cargo build --release
129+
TORRUST_TRACKER_PATH_CONFIG="./share/default/config/tracker.udp.benchmarking.toml" ./target/release/torrust-tracker
130+
```
131+
132+
Running the test: `aquatic_udp_load_test`.
133+
134+
```output
135+
Requests out: 13075.56/second
136+
Responses in: 12058.38/second
137+
- Connect responses: 1017.18
138+
- Announce responses: 11035.00
139+
- Scrape responses: 6.20
140+
- Error responses: 0.00
141+
Peers per announce response: 41.13
142+
```
143+
144+
#### Aquatic UDP Tracker
145+
146+
Running the tracker:
147+
148+
```console
149+
git clone git@github.com:greatest-ape/aquatic.git
150+
cd aquatic
151+
cargo build --release -p aquatic_udp
152+
./target/release/aquatic_udp -c "aquatic-udp-config.toml"
153+
./target/release/aquatic_udp -c "aquatic-udp-config.toml"
154+
```
155+
156+
Running the test: `aquatic_udp_load_test`.
157+
158+
```output
159+
Requests out: 383873.14/second
160+
Responses in: 383440.35/second
161+
- Connect responses: 429.19
162+
- Announce responses: 379249.22
163+
- Scrape responses: 3761.93
164+
- Error responses: 0.00
165+
Peers per announce response: 15.33
166+
```
167+
168+
#### Torrust-Actix UDP Tracker
169+
170+
Run the tracker with UDP service enabled on port 3000 and set log level to `error`.
171+
172+
```toml
173+
[[udp_trackers]]
174+
bind_address = "0.0.0.0:3000"
175+
enabled = true
176+
```
177+
178+
```console
179+
git clone https://github.com/Power2All/torrust-actix.git
180+
cd torrust-actix
181+
cargo build --release
182+
./target/release/torrust-actix --create-config
183+
./target/release/torrust-actix
184+
```
185+
186+
Running the test: `aquatic_udp_load_test`.
187+
188+
```output
189+
Requests out: 3072.94/second
190+
Responses in: 2395.15/second
191+
- Connect responses: 556.79
192+
- Announce responses: 1821.16
193+
- Scrape responses: 17.20
194+
- Error responses: 0.00
195+
Peers per announce response: 133.88
196+
```
197+
198+
### Results
199+
200+
Announce request per second:
201+
202+
| Tracker | Announce |
203+
|---------------|-----------|
204+
| Aquatic | 379,249 |
205+
| Torrust | 11,035 |
206+
| Torrust-Actix | 1,821 |
207+
208+
## Repository benchmarking
209+
210+
You can run it with:
211+
212+
```console
213+
cargo run --release -p torrust-torrent-repository-benchmarks -- --threads 4 --sleep 0 --compare true
214+
```
215+
216+
It tests the different implementation for the internal torrent storage.
217+
218+
```output
219+
tokio::sync::RwLock<std::collections::BTreeMap<InfoHash, Entry>>
220+
add_one_torrent: Avg/AdjAvg: (60ns, 59ns)
221+
update_one_torrent_in_parallel: Avg/AdjAvg: (10.909457ms, 0ns)
222+
add_multiple_torrents_in_parallel: Avg/AdjAvg: (13.88879ms, 0ns)
223+
update_multiple_torrents_in_parallel: Avg/AdjAvg: (7.772484ms, 7.782535ms)
224+
225+
std::sync::RwLock<std::collections::BTreeMap<InfoHash, Entry>>
226+
add_one_torrent: Avg/AdjAvg: (43ns, 39ns)
227+
update_one_torrent_in_parallel: Avg/AdjAvg: (4.020937ms, 4.020937ms)
228+
add_multiple_torrents_in_parallel: Avg/AdjAvg: (5.896177ms, 5.768448ms)
229+
update_multiple_torrents_in_parallel: Avg/AdjAvg: (3.883823ms, 3.883823ms)
230+
231+
std::sync::RwLock<std::collections::BTreeMap<InfoHash, Arc<std::sync::Mutex<Entry>>>>
232+
add_one_torrent: Avg/AdjAvg: (51ns, 49ns)
233+
update_one_torrent_in_parallel: Avg/AdjAvg: (3.252314ms, 3.149109ms)
234+
add_multiple_torrents_in_parallel: Avg/AdjAvg: (8.411094ms, 8.411094ms)
235+
update_multiple_torrents_in_parallel: Avg/AdjAvg: (4.106086ms, 4.106086ms)
236+
237+
tokio::sync::RwLock<std::collections::BTreeMap<InfoHash, Arc<std::sync::Mutex<Entry>>>>
238+
add_one_torrent: Avg/AdjAvg: (91ns, 90ns)
239+
update_one_torrent_in_parallel: Avg/AdjAvg: (3.542378ms, 3.435695ms)
240+
add_multiple_torrents_in_parallel: Avg/AdjAvg: (15.651172ms, 15.651172ms)
241+
update_multiple_torrents_in_parallel: Avg/AdjAvg: (4.368189ms, 4.257572ms)
242+
243+
tokio::sync::RwLock<std::collections::BTreeMap<InfoHash, Arc<tokio::sync::Mutex<Entry>>>>
244+
add_one_torrent: Avg/AdjAvg: (111ns, 109ns)
245+
update_one_torrent_in_parallel: Avg/AdjAvg: (6.590677ms, 6.808535ms)
246+
add_multiple_torrents_in_parallel: Avg/AdjAvg: (16.572217ms, 16.30488ms)
247+
update_multiple_torrents_in_parallel: Avg/AdjAvg: (4.073221ms, 4.000122ms)
248+
```
249+
250+
## Other considerations
251+
252+
We are testing new repository implementations that allow concurrent writes. See <https://github.com/torrust/torrust-tracker/issues/565>.

‎share/default/config/tracker.udp.benchmarking.toml

+37
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
announce_interval = 120
2+
db_driver = "Sqlite3"
3+
db_path = "./storage/tracker/lib/database/sqlite3.db"
4+
external_ip = "0.0.0.0"
5+
inactive_peer_cleanup_interval = 600
6+
log_level = "error"
7+
max_peer_timeout = 900
8+
min_announce_interval = 120
9+
mode = "public"
10+
on_reverse_proxy = false
11+
persistent_torrent_completed_stat = false
12+
remove_peerless_torrents = true
13+
tracker_usage_statistics = true
14+
15+
[[udp_trackers]]
16+
bind_address = "0.0.0.0:3000"
17+
enabled = true
18+
19+
[[http_trackers]]
20+
bind_address = "0.0.0.0:7070"
21+
enabled = false
22+
ssl_cert_path = ""
23+
ssl_enabled = false
24+
ssl_key_path = ""
25+
26+
[http_api]
27+
bind_address = "127.0.0.1:1212"
28+
enabled = false
29+
ssl_cert_path = ""
30+
ssl_enabled = false
31+
ssl_key_path = ""
32+
33+
[http_api.access_tokens]
34+
admin = "MyAccessToken"
35+
36+
[health_check_api]
37+
bind_address = "127.0.0.1:1313"

‎src/console/clients/udp/responses.rs

-7
Original file line numberDiff line numberDiff line change
@@ -68,13 +68,6 @@ impl From<ScrapeResponse> for ScrapeResponseDto {
6868
}
6969
}
7070

71-
#[derive(Serialize)]
72-
struct Peer {
73-
seeders: i32,
74-
completed: i32,
75-
leechers: i32,
76-
}
77-
7871
#[derive(Serialize)]
7972
struct TorrentStats {
8073
seeders: i32,

‎tests/README.md

-9
This file was deleted.

0 commit comments

Comments
 (0)