Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[ereport] intial types and dropshot API crate #7833

Open
wants to merge 14 commits into
base: main
Choose a base branch
from
Open

[ereport] intial types and dropshot API crate #7833

wants to merge 14 commits into from

Conversation

hawkw
Copy link
Member

@hawkw hawkw commented Mar 19, 2025 •
edited
Loading

This branch adds an initial version of the HTTP API for the control
plane's ereport ingestion mechanism, as described in RFD 520. In
particularr, this branch adds the following:

  • an ereport-types crate, containing the shared type definitions for
    ereport data
  • an ereporter-api crate defining the Dropshot HTTP API that will be
    implemented by MGS and sled-agents in order to expose ereports to
    Nexus
  • the generated ereporter-client crate for that API, which Nexus will
    use to ingest ereports from reporters.

Actually implementing the API in MGS and sled-agent, and implementing
the Nexus RPW for collecting ereports using this interface, is left as
an exercise for the reader future Eliza. I also intend to add a
simulated version of this API to the simulated sled-agent and MGS, to to
use for testing the ingestion bits in Nexus.

The ereporter HTTP API has changed a bit from the strawman version
proposed in RFD 520, as the request to collect new ereports and the
request to flush already-committed ereports are now contained in the
same HTTP request, rather than requiring separate endpoints for these
operations. This is due to @cbiffle: in his proposal for
the MGS-to-SP hop in the ereport pipeline in RFD 545, Cliff has
combined these into one message, which I thought seemed nice to do here
as well.

hawkw added a commit that referenced this pull request Mar 20, 2025
@hawkw
[ls-apis] support APIs exposed by multiple servers
564ce96 
Currently, `cargo xtask ls-apis` will fail when it encounters an API
that is served by multiple entities.

For example, when adding the `ereporter-api` in #7833, which is served
by both MGS and sled-agent, `ls-apis` currently fails with an error like
this one:

```console
eliza@theseus ~/Code/oxide/omicron $ cargo xtask ls-apis apis
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.45s
     Running `target/debug/xtask ls-apis apis`
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 1.01s
     Running `target/debug/ls-apis apis`
loading metadata for workspace omicron from current workspace
loading metadata for workspace lldp from /home/eliza/.cargo/git/checkouts/lldp-d47de417041f191b/ce952e6/Cargo.toml
loading metadata for workspace propolis from /home/eliza/.cargo/git/checkouts/propolis-d68c8bd1bc59c9bd/6b5f2af/Cargo.toml
loading metadata for workspace crucible from /home/eliza/.cargo/git/checkouts/crucible-0a48bd218bc2bbbc/81a3528/Cargo.toml
loading metadata for workspace maghemite from /home/eliza/.cargo/git/checkouts/maghemite-c0236f0fd3d582b6/caafd88/Cargo.toml

loading metadata for workspace dendrite from /home/eliza/.cargo/git/checkouts/dendrite-ae9f1715c17fc765/a66561e/Cargo.toml
note: ignoring Cargo dependency from crucible-pantry -> ... -> crucible-control-client
note: ignoring Cargo dependency from omicron-sled-agent -> dns-server
error: API for client ereporter-client appears to be exported by multiple components: at least omicron-sled-agent and omicron-gateway (DepPath([PackageId { repr: "path+file:///home/eliza/Code/oxide/omicron/gateway#omicron-gateway@0.1.0" }]))
Error: found at least one API exported by multiple servers

# ... backtrace snipped for brevity ...

```

This is unfortunate, as it would be nice to be able to have such APIs
without breaking `ls-apis`. Therefore, this commit adds support for APIs
with multiple server components. This change ends up being pretty
straightforward; it's mainly just changing the `api_producers` map from
a map of client package names to server components to a map of client
package names to *a map of* server components, and updating the code
that consumes this information to loop over those server components as
appropriate.

Now, the various `ls-apis` commands can handle the new `ereproter-api`.
For example:

```console
eliza@theseus ~/Code/oxide/omicron $ cargo xtask ls-apis apis
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.37s
     Running `target/debug/xtask ls-apis apis`
   Compiling omicron-ls-apis v0.1.0 (/home/eliza/Code/oxide/omicron/dev-tools/ls-apis)
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 1.69s
     Running `target/debug/ls-apis apis`
loading metadata for workspace omicron from current workspace
loading metadata for workspace lldp from /home/eliza/.cargo/git/checkouts/lldp-d47de417041f191b/ce952e6/Cargo.toml
loading metadata for workspace maghemite from /home/eliza/.cargo/git/checkouts/maghemite-c0236f0fd3d582b6/caafd88/Cargo.toml
loading metadata for workspace propolis from /home/eliza/.cargo/git/checkouts/propolis-d68c8bd1bc59c9bd/6b5f2af/Cargo.toml
loading metadata for workspace crucible from /home/eliza/.cargo/git/checkouts/crucible-0a48bd218bc2bbbc/81a3528/Cargo.toml
loading metadata for workspace dendrite from /home/eliza/.cargo/git/checkouts/dendrite-ae9f1715c17fc765/a66561e/Cargo.toml
note: ignoring Cargo dependency from crucible-pantry -> ... -> crucible-control-client
note: ignoring Cargo dependency from omicron-sled-agent -> dns-server
Bootstrap Agent (client: bootstrap-agent-client)
    consumed by: omicron-sled-agent (omicron/sled-agent) via 1 path
    consumed by: wicketd (omicron/wicketd) via 2 paths

# ... snipped irrelevant APIs for brevity ...

Ereporter (client: ereporter-client)
    consumed by: omicron-nexus (omicron/nexus) via 1 path

# ... snipped irrelevant APIs for brevity ...

eliza@theseus ~/Code/oxide/omicron $ cargo xtask ls-apis check
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.32s
     Running `target/debug/xtask ls-apis check`
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.52s
     Running `target/debug/ls-apis check`
loading metadata for workspace omicron from current workspace
loading metadata for workspace lldp from /home/eliza/.cargo/git/checkouts/lldp-d47de417041f191b/ce952e6/Cargo.toml
loading metadata for workspace maghemite from /home/eliza/.cargo/git/checkouts/maghemite-c0236f0fd3d582b6/caafd88/Cargo.toml
loading metadata for workspace propolis from /home/eliza/.cargo/git/checkouts/propolis-d68c8bd1bc59c9bd/6b5f2af/Cargo.toml
loading metadata for workspace crucible from /home/eliza/.cargo/git/checkouts/crucible-0a48bd218bc2bbbc/81a3528/Cargo.toml
loading metadata for workspace dendrite from /home/eliza/.cargo/git/checkouts/dendrite-ae9f1715c17fc765/a66561e/Cargo.toml
note: ignoring Cargo dependency from crucible-pantry -> ... -> crucible-control-client
note: ignoring Cargo dependency from omicron-sled-agent -> dns-server

Server-managed APIs:

    Clickhouse Cluster Admin for Keepers (clickhouse-admin-keeper-client, exposed by omicron-clickhouse-admin)
    Clickhouse Cluster Admin for Servers (clickhouse-admin-server-client, exposed by omicron-clickhouse-admin)
    Clickhouse Single-Node Cluster Admin (clickhouse-admin-single-client, exposed by omicron-clickhouse-admin)
    CockroachDB Cluster Admin (cockroach-admin-client, exposed by omicron-cockroach-admin)
    Crucible Agent (crucible-agent-client, exposed by crucible-agent)
    Crucible Control (for testing only) (crucible-control-client, exposed by propolis-server)
    Crucible Pantry (crucible-pantry-client, exposed by crucible-pantry)
    Maghemite DDM Admin (ddm-admin-client, exposed by ddmd)
    DNS Server (dns-service-client, exposed by dns-server)
    Ereporter (ereporter-client, exposed by omicron-gateway, omicron-sled-agent)
    Management Gateway Service (gateway-client, exposed by omicron-gateway)
    LLDP daemon (lldpd-client, exposed by lldpd)
    Maghemite MG Admin (mg-admin-client, exposed by mgd)
    External API (oxide-client, exposed by omicron-nexus)
    Oximeter (oximeter-client, exposed by oximeter-collector)
    Propolis (propolis-client, exposed by propolis-server)
    Sled Agent (sled-agent-client, exposed by omicron-sled-agent)
    Wicketd (wicketd-client, exposed by wicketd)

Client-managed API:

    Bootstrap Agent (bootstrap-agent-client, exposed by omicron-sled-agent)
        reason: depends on itself (i.e., instances call each other)
    Dendrite DPD (dpd-client, exposed by dpd)
        reason: Sled Agent calling DPD creates a circular dependency
    Wicketd Installinator (installinator-client, exposed by wicketd)
        reason: client is provided implicitly by the operator
    Nexus Internal API (nexus-client, exposed by omicron-nexus)
        reason: Circular dependencies between Nexus and other services
    Crucible Repair (repair-client, exposed by crucible-downstairs)
        reason: depends on itself (i.e., instances call each other)
    Repo Depot API (repo-depot-client, exposed by omicron-sled-agent)
        reason: depends on itself (i.e., instances call each other)

APIs with unknown version management: none

````
@hawkw hawkw mentioned this pull request Mar 20, 2025
Merged
hawkw added a commit that referenced this pull request Mar 20, 2025
@hawkw
[ls-apis] support APIs exposed by multiple servers (#7843)
67a8984 
Currently, `cargo xtask ls-apis` will fail when it encounters an API
that is served by multiple entities.

For example, when adding the `ereporter-api` in #7833, which is served
by both MGS and sled-agent, `ls-apis` currently fails with an error like
this one:

```console
eliza@theseus ~/Code/oxide/omicron $ cargo xtask ls-apis apis
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.45s
     Running `target/debug/xtask ls-apis apis`
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 1.01s
     Running `target/debug/ls-apis apis`
loading metadata for workspace omicron from current workspace
loading metadata for workspace lldp from /home/eliza/.cargo/git/checkouts/lldp-d47de417041f191b/ce952e6/Cargo.toml
loading metadata for workspace propolis from /home/eliza/.cargo/git/checkouts/propolis-d68c8bd1bc59c9bd/6b5f2af/Cargo.toml
loading metadata for workspace crucible from /home/eliza/.cargo/git/checkouts/crucible-0a48bd218bc2bbbc/81a3528/Cargo.toml
loading metadata for workspace maghemite from /home/eliza/.cargo/git/checkouts/maghemite-c0236f0fd3d582b6/caafd88/Cargo.toml

loading metadata for workspace dendrite from /home/eliza/.cargo/git/checkouts/dendrite-ae9f1715c17fc765/a66561e/Cargo.toml
note: ignoring Cargo dependency from crucible-pantry -> ... -> crucible-control-client
note: ignoring Cargo dependency from omicron-sled-agent -> dns-server
error: API for client ereporter-client appears to be exported by multiple components: at least omicron-sled-agent and omicron-gateway (DepPath([PackageId { repr: "path+file:///home/eliza/Code/oxide/omicron/gateway#omicron-gateway@0.1.0" }]))
Error: found at least one API exported by multiple servers

# ... backtrace snipped for brevity ...

```

This is unfortunate, as it would be nice to be able to have such APIs
without breaking `ls-apis`. Therefore, this commit adds support for APIs
with multiple server components. This change ends up being pretty
straightforward; it's mainly just changing the `api_producers` map from
a map of client package names to server components to a map of client
package names to *a map of* server components, and updating the code
that consumes this information to loop over those server components as
appropriate.

Now, the various `ls-apis` commands can handle the new `ereproter-api`.
For example:

```console
eliza@theseus ~/Code/oxide/omicron $ cargo xtask ls-apis apis
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.37s
     Running `target/debug/xtask ls-apis apis`
   Compiling omicron-ls-apis v0.1.0 (/home/eliza/Code/oxide/omicron/dev-tools/ls-apis)
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 1.69s
     Running `target/debug/ls-apis apis`
loading metadata for workspace omicron from current workspace
loading metadata for workspace lldp from /home/eliza/.cargo/git/checkouts/lldp-d47de417041f191b/ce952e6/Cargo.toml
loading metadata for workspace maghemite from /home/eliza/.cargo/git/checkouts/maghemite-c0236f0fd3d582b6/caafd88/Cargo.toml
loading metadata for workspace propolis from /home/eliza/.cargo/git/checkouts/propolis-d68c8bd1bc59c9bd/6b5f2af/Cargo.toml
loading metadata for workspace crucible from /home/eliza/.cargo/git/checkouts/crucible-0a48bd218bc2bbbc/81a3528/Cargo.toml
loading metadata for workspace dendrite from /home/eliza/.cargo/git/checkouts/dendrite-ae9f1715c17fc765/a66561e/Cargo.toml
note: ignoring Cargo dependency from crucible-pantry -> ... -> crucible-control-client
note: ignoring Cargo dependency from omicron-sled-agent -> dns-server
Bootstrap Agent (client: bootstrap-agent-client)
    consumed by: omicron-sled-agent (omicron/sled-agent) via 1 path
    consumed by: wicketd (omicron/wicketd) via 2 paths

# ... snipped irrelevant APIs for brevity ...

Ereporter (client: ereporter-client)
    consumed by: omicron-nexus (omicron/nexus) via 1 path

# ... snipped irrelevant APIs for brevity ...

eliza@theseus ~/Code/oxide/omicron $ cargo xtask ls-apis check
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.32s
     Running `target/debug/xtask ls-apis check`
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.52s
     Running `target/debug/ls-apis check`
loading metadata for workspace omicron from current workspace
loading metadata for workspace lldp from /home/eliza/.cargo/git/checkouts/lldp-d47de417041f191b/ce952e6/Cargo.toml
loading metadata for workspace maghemite from /home/eliza/.cargo/git/checkouts/maghemite-c0236f0fd3d582b6/caafd88/Cargo.toml
loading metadata for workspace propolis from /home/eliza/.cargo/git/checkouts/propolis-d68c8bd1bc59c9bd/6b5f2af/Cargo.toml
loading metadata for workspace crucible from /home/eliza/.cargo/git/checkouts/crucible-0a48bd218bc2bbbc/81a3528/Cargo.toml
loading metadata for workspace dendrite from /home/eliza/.cargo/git/checkouts/dendrite-ae9f1715c17fc765/a66561e/Cargo.toml
note: ignoring Cargo dependency from crucible-pantry -> ... -> crucible-control-client
note: ignoring Cargo dependency from omicron-sled-agent -> dns-server

Server-managed APIs:

    Clickhouse Cluster Admin for Keepers (clickhouse-admin-keeper-client, exposed by omicron-clickhouse-admin)
    Clickhouse Cluster Admin for Servers (clickhouse-admin-server-client, exposed by omicron-clickhouse-admin)
    Clickhouse Single-Node Cluster Admin (clickhouse-admin-single-client, exposed by omicron-clickhouse-admin)
    CockroachDB Cluster Admin (cockroach-admin-client, exposed by omicron-cockroach-admin)
    Crucible Agent (crucible-agent-client, exposed by crucible-agent)
    Crucible Control (for testing only) (crucible-control-client, exposed by propolis-server)
    Crucible Pantry (crucible-pantry-client, exposed by crucible-pantry)
    Maghemite DDM Admin (ddm-admin-client, exposed by ddmd)
    DNS Server (dns-service-client, exposed by dns-server)
    Ereporter (ereporter-client, exposed by omicron-gateway, omicron-sled-agent)
    Management Gateway Service (gateway-client, exposed by omicron-gateway)
    LLDP daemon (lldpd-client, exposed by lldpd)
    Maghemite MG Admin (mg-admin-client, exposed by mgd)
    External API (oxide-client, exposed by omicron-nexus)
    Oximeter (oximeter-client, exposed by oximeter-collector)
    Propolis (propolis-client, exposed by propolis-server)
    Sled Agent (sled-agent-client, exposed by omicron-sled-agent)
    Wicketd (wicketd-client, exposed by wicketd)

Client-managed API:

    Bootstrap Agent (bootstrap-agent-client, exposed by omicron-sled-agent)
        reason: depends on itself (i.e., instances call each other)
    Dendrite DPD (dpd-client, exposed by dpd)
        reason: Sled Agent calling DPD creates a circular dependency
    Wicketd Installinator (installinator-client, exposed by wicketd)
        reason: client is provided implicitly by the operator
    Nexus Internal API (nexus-client, exposed by omicron-nexus)
        reason: Circular dependencies between Nexus and other services
    Crucible Repair (repair-client, exposed by crucible-downstairs)
        reason: depends on itself (i.e., instances call each other)
    Repo Depot API (repo-depot-client, exposed by omicron-sled-agent)
        reason: depends on itself (i.e., instances call each other)

APIs with unknown version management: none

eliza@theseus ~/Code/oxide/omicron $ cargo xtask ls-apis servers
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.35s
     Running `target/debug/xtask ls-apis servers`
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.52s
     Running `target/debug/ls-apis servers`
loading metadata for workspace omicron from current workspace
loading metadata for workspace lldp from /home/eliza/.cargo/git/checkouts/lldp-d47de417041f191b/ce952e6/Cargo.toml
loading metadata for workspace maghemite from /home/eliza/.cargo/git/checkouts/maghemite-c0236f0fd3d582b6/caafd88/Cargo.toml
loading metadata for workspace propolis from /home/eliza/.cargo/git/checkouts/propolis-d68c8bd1bc59c9bd/6b5f2af/Cargo.toml
loading metadata for workspace crucible from /home/eliza/.cargo/git/checkouts/crucible-0a48bd218bc2bbbc/81a3528/Cargo.toml
loading metadata for workspace dendrite from /home/eliza/.cargo/git/checkouts/dendrite-ae9f1715c17fc765/a66561e/Cargo.toml
note: ignoring Cargo dependency from crucible-pantry -> ... -> crucible-control-client
note: ignoring Cargo dependency from omicron-sled-agent -> dns-server
omicron-clickhouse-admin (omicron/clickhouse-admin)
    exposes: Clickhouse Cluster Admin for Keepers (client = clickhouse-admin-keeper-client)
    exposes: Clickhouse Cluster Admin for Servers (client = clickhouse-admin-server-client)
    exposes: Clickhouse Single-Node Cluster Admin (client = clickhouse-admin-single-client)

omicron-cockroach-admin (omicron/cockroach-admin)
    exposes: CockroachDB Cluster Admin (client = cockroach-admin-client)

crucible-agent (crucible/agent)
    exposes: Crucible Agent (client = crucible-agent-client)

crucible-downstairs (crucible/downstairs)
    exposes: Crucible Repair (client = repair-client)
    consumes: repair-client

crucible-pantry (crucible/pantry)
    exposes: Crucible Pantry (client = crucible-pantry-client)
    consumes: nexus-client

dns-server (omicron/dns-server)
    exposes: DNS Server (client = dns-service-client)

omicron-sled-agent (omicron/sled-agent)
    exposes: Bootstrap Agent (client = bootstrap-agent-client)
    exposes: Ereporter (client = ereporter-client)
    exposes: Repo Depot API (client = repo-depot-client)
    exposes: Sled Agent (client = sled-agent-client)
    consumes: bootstrap-agent-client
    consumes: ddm-admin-client
    consumes: dns-service-client
    consumes: dpd-client
    consumes: gateway-client
    consumes: mg-admin-client
    consumes: nexus-client
    consumes: propolis-client
    consumes: repo-depot-client

propolis-server (propolis/bin/propolis-server)
    exposes: Crucible Control (for testing only) (client = crucible-control-client)
    exposes: Propolis (client = propolis-client)
    consumes: nexus-client

ddmd (maghemite/ddmd)
    exposes: Maghemite DDM Admin (client = ddm-admin-client)
    consumes: dpd-client

dpd (dendrite/dpd)
    exposes: Dendrite DPD (client = dpd-client)
    consumes: gateway-client
    consumes: nexus-client

lldpd (lldp/lldpd)
    exposes: LLDP daemon (client = lldpd-client)

mgd (maghemite/mgd)
    exposes: Maghemite MG Admin (client = mg-admin-client)
    consumes: ddm-admin-client
    consumes: dpd-client

omicron-gateway (omicron/gateway)
    exposes: Ereporter (client = ereporter-client)
    exposes: Management Gateway Service (client = gateway-client)

tfportd (dendrite/tfportd)
    consumes: dpd-client
    consumes: lldpd-client

wicketd (omicron/wicketd)
    exposes: Wicketd Installinator (client = installinator-client)
    exposes: Wicketd (client = wicketd-client)
    consumes: bootstrap-agent-client
    consumes: ddm-admin-client
    consumes: dpd-client
    consumes: gateway-client

installinator (omicron/installinator)
    consumes: ddm-admin-client
    consumes: installinator-client

omicron-nexus (omicron/nexus)
    exposes: Nexus Internal API (client = nexus-client)
    exposes: External API (client = oxide-client)
    consumes: clickhouse-admin-keeper-client
    consumes: clickhouse-admin-server-client
    consumes: clickhouse-admin-single-client
    consumes: cockroach-admin-client
    consumes: crucible-agent-client
    consumes: crucible-pantry-client
    consumes: dns-service-client
    consumes: dpd-client
    consumes: ereporter-client
    consumes: gateway-client
    consumes: lldpd-client
    consumes: mg-admin-client
    consumes: oximeter-client
    consumes: propolis-client
    consumes: sled-agent-client

oximeter-collector (omicron/oximeter/collector)
    exposes: Oximeter (client = oximeter-client)
    consumes: nexus-client


````
@hawkw hawkw force-pushed the eliza/ereporter-api-2 branch from 0d40c8f to f42577c Compare March 20, 2025 21:17
@hawkw hawkw marked this pull request as ready for review March 20, 2025 21:17
davepacheco
davepacheco reviewed Mar 21, 2025
View reviewed changes
Cargo.toml
@@ -49,6 +50,8 @@ members = [
"dns-server",
"dns-server-api",
"end-to-end-tests",
"ereport/api",
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: seems like it'd be nice to be consistent about ereport vs. ereporter. Is there a reason they're different?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@davepacheco davepacheco davepacheco left review comments

@smklein smklein Awaiting requested review from smklein

@andrewjstone andrewjstone Awaiting requested review from andrewjstone

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@hawkw @davepacheco