Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Start byte-based backpressure earlier #1179

Merged
merged 1 commit into from
Feb 26, 2024

Conversation

mkeeter
Copy link
Contributor

@mkeeter mkeeter commented Feb 23, 2024

This PR implements a suggestion from RFD 445 of having byte-based backpressure start earlier (see the section titled Whither latency?).

It also caps the max byte-based backpressure, so that we'd timeout sooner if a Downstairs goes away during a large-write workload (see Problematic Bytes-In-Flight Delay)

Previously, the system stabilized at about 1.8 GiB of in-flight data. With the tuning in this PR, it stabilizes at around 500-600 MiB instead. This means that when we switch from writes to reads, we doesn't need to wait as long!

I see no performance impacts. For reference, here's a quick fio test run:

1M WRITE: bw=723MiB/s (758MB/s), 723MiB/s-723MiB/s (758MB/s-758MB/s), io=42.5GiB (45.6GB), run=60099-60099msec
4K WRITE: bw=28.3MiB/s (29.7MB/s), 28.3MiB/s-28.3MiB/s (29.7MB/s-29.7MB/s), io=1698MiB (1781MB), run=60009-60009msec
1M WRITE: bw=772MiB/s (810MB/s), 772MiB/s-772MiB/s (810MB/s-810MB/s), io=45.3GiB (48.6GB), run=60025-60025msec
4M WRITE: bw=754MiB/s (791MB/s), 754MiB/s-754MiB/s (791MB/s-791MB/s), io=44.3GiB (47.5GB), run=60107-60107msec
4K READ: bw=27.5MiB/s (28.9MB/s), 27.5MiB/s-27.5MiB/s (28.9MB/s-28.9MB/s), io=1653MiB (1733MB), run=60003-60003msec
1M READ: bw=629MiB/s (659MB/s), 629MiB/s-629MiB/s (659MB/s-659MB/s), io=36.9GiB (39.6GB), run=60036-60036msec
4M READ: bw=737MiB/s (773MB/s), 737MiB/s-737MiB/s (773MB/s-773MB/s), io=43.3GiB (46.5GB), run=60130-60130msec

This is roughly the same as numbers from main here.

(#1167 remains unaddressed by these changes)

@mkeeter mkeeter requested review from jmpesp and leftwo February 23, 2024 21:39
@mkeeter mkeeter merged commit bf25e38 into oxidecomputer:main Feb 26, 2024
18 checks passed
@mkeeter mkeeter deleted the earlier-byte-backpressure branch February 26, 2024 16:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants