Consolidate ack checking (#1561)

There are a bunch of places where we check to see if jobs should be
acked and/or retired:

- `Downstairs::enqueue` (if skipped on all three Downstairs)
- `Downstairs::skip_all_jobs` (for skipped jobs only)
- `DownstairsClient::process_io_completion` (lots of conditionals to
decide if we're ready to ack; returns a flag to
`Downstairs::process_io_completion_inner`)

This PR consolidates ack and retire checks into a single place:
`Downstairs::ack_check`. This new function is responsible for deciding
when to ack and retire jobs; it's called in all of the places where
you'd expect.

`DownstairsClient::process_io_completion` is reorganized, with one minor
logical change: non-fatal read errors now increment the
`downstairs_errors` counter. It's no longer responsible for deciding
when to ack a job; that's now in `Downstairs::ack_check`.

The `up-to-ds-*-done` DTrace probes are removed, because (after #1552)
they fire immediately before the equivalent `gw-*-done` probes; there's
no delay waiting for ackable jobs to be acked.

Author: mkeeter

tools/dtrace/perf-upstairs-wf.d

@@ -24,19 +24,10 @@ crucible_upstairs*:::up-to-ds-write-start
     substart[arg0] = timestamp;
 }
 
-crucible_upstairs*:::up-to-ds-flush-done,
-crucible_upstairs*:::up-to-ds-write-done
+crucible_upstairs*:::gw-flush-done,
+crucible_upstairs*:::gw-write-done
 /substart[arg0]/
 {
     @[probename] = quantize(timestamp - substart[arg0]);
     substart[arg0] = 0;
-    final[arg0] = timestamp;
-}
-
-crucible_upstairs*:::gw-flush-done,
-crucible_upstairs*:::gw-write-done
-/final[arg0]/
-{
-    @[probename] = quantize(timestamp - final[arg0]);
-    final[arg0] = 0;
 }