[ls-apis] support APIs exposed by multiple servers #7843

hawkw · 2025-03-20T17:49:16Z

Currently, cargo xtask ls-apis will fail when it encounters an API that is served by multiple entities.

For example, when adding the ereporter-api in #7833, which is served by both MGS and sled-agent, ls-apis currently fails with an error like this one:

eliza@theseus ~/Code/oxide/omicron $ cargo xtask ls-apis apis
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.45s
     Running `target/debug/xtask ls-apis apis`
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 1.01s
     Running `target/debug/ls-apis apis`
loading metadata for workspace omicron from current workspace
loading metadata for workspace lldp from /home/eliza/.cargo/git/checkouts/lldp-d47de417041f191b/ce952e6/Cargo.toml
loading metadata for workspace propolis from /home/eliza/.cargo/git/checkouts/propolis-d68c8bd1bc59c9bd/6b5f2af/Cargo.toml
loading metadata for workspace crucible from /home/eliza/.cargo/git/checkouts/crucible-0a48bd218bc2bbbc/81a3528/Cargo.toml
loading metadata for workspace maghemite from /home/eliza/.cargo/git/checkouts/maghemite-c0236f0fd3d582b6/caafd88/Cargo.toml

loading metadata for workspace dendrite from /home/eliza/.cargo/git/checkouts/dendrite-ae9f1715c17fc765/a66561e/Cargo.toml
note: ignoring Cargo dependency from crucible-pantry -> ... -> crucible-control-client
note: ignoring Cargo dependency from omicron-sled-agent -> dns-server
error: API for client ereporter-client appears to be exported by multiple components: at least omicron-sled-agent and omicron-gateway (DepPath([PackageId { repr: "path+file:///home/eliza/Code/oxide/omicron/gateway#omicron-gateway@0.1.0" }]))
Error: found at least one API exported by multiple servers

# ... backtrace snipped for brevity ...

This is unfortunate, as it would be nice to be able to have such APIs without breaking ls-apis. Therefore, this commit adds support for APIs with multiple server components. This change ends up being pretty straightforward; it's mainly just changing the api_producers map from a map of client package names to server components to a map of client package names to a map of server components, and updating the code that consumes this information to loop over those server components as appropriate.

Now, the various ls-apis commands can handle the new ereproter-api. For example:

eliza@theseus ~/Code/oxide/omicron $ cargo xtask ls-apis apis
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.37s
     Running `target/debug/xtask ls-apis apis`
   Compiling omicron-ls-apis v0.1.0 (/home/eliza/Code/oxide/omicron/dev-tools/ls-apis)
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 1.69s
     Running `target/debug/ls-apis apis`
loading metadata for workspace omicron from current workspace
loading metadata for workspace lldp from /home/eliza/.cargo/git/checkouts/lldp-d47de417041f191b/ce952e6/Cargo.toml
loading metadata for workspace maghemite from /home/eliza/.cargo/git/checkouts/maghemite-c0236f0fd3d582b6/caafd88/Cargo.toml
loading metadata for workspace propolis from /home/eliza/.cargo/git/checkouts/propolis-d68c8bd1bc59c9bd/6b5f2af/Cargo.toml
loading metadata for workspace crucible from /home/eliza/.cargo/git/checkouts/crucible-0a48bd218bc2bbbc/81a3528/Cargo.toml
loading metadata for workspace dendrite from /home/eliza/.cargo/git/checkouts/dendrite-ae9f1715c17fc765/a66561e/Cargo.toml
note: ignoring Cargo dependency from crucible-pantry -> ... -> crucible-control-client
note: ignoring Cargo dependency from omicron-sled-agent -> dns-server
Bootstrap Agent (client: bootstrap-agent-client)
    consumed by: omicron-sled-agent (omicron/sled-agent) via 1 path
    consumed by: wicketd (omicron/wicketd) via 2 paths

# ... snipped irrelevant APIs for brevity ...

Ereporter (client: ereporter-client)
    consumed by: omicron-nexus (omicron/nexus) via 1 path

# ... snipped irrelevant APIs for brevity ...

eliza@theseus ~/Code/oxide/omicron $ cargo xtask ls-apis check
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.32s
     Running `target/debug/xtask ls-apis check`
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.52s
     Running `target/debug/ls-apis check`
loading metadata for workspace omicron from current workspace
loading metadata for workspace lldp from /home/eliza/.cargo/git/checkouts/lldp-d47de417041f191b/ce952e6/Cargo.toml
loading metadata for workspace maghemite from /home/eliza/.cargo/git/checkouts/maghemite-c0236f0fd3d582b6/caafd88/Cargo.toml
loading metadata for workspace propolis from /home/eliza/.cargo/git/checkouts/propolis-d68c8bd1bc59c9bd/6b5f2af/Cargo.toml
loading metadata for workspace crucible from /home/eliza/.cargo/git/checkouts/crucible-0a48bd218bc2bbbc/81a3528/Cargo.toml
loading metadata for workspace dendrite from /home/eliza/.cargo/git/checkouts/dendrite-ae9f1715c17fc765/a66561e/Cargo.toml
note: ignoring Cargo dependency from crucible-pantry -> ... -> crucible-control-client
note: ignoring Cargo dependency from omicron-sled-agent -> dns-server

Server-managed APIs:

    Clickhouse Cluster Admin for Keepers (clickhouse-admin-keeper-client, exposed by omicron-clickhouse-admin)
    Clickhouse Cluster Admin for Servers (clickhouse-admin-server-client, exposed by omicron-clickhouse-admin)
    Clickhouse Single-Node Cluster Admin (clickhouse-admin-single-client, exposed by omicron-clickhouse-admin)
    CockroachDB Cluster Admin (cockroach-admin-client, exposed by omicron-cockroach-admin)
    Crucible Agent (crucible-agent-client, exposed by crucible-agent)
    Crucible Control (for testing only) (crucible-control-client, exposed by propolis-server)
    Crucible Pantry (crucible-pantry-client, exposed by crucible-pantry)
    Maghemite DDM Admin (ddm-admin-client, exposed by ddmd)
    DNS Server (dns-service-client, exposed by dns-server)
    Ereporter (ereporter-client, exposed by omicron-gateway, omicron-sled-agent)
    Management Gateway Service (gateway-client, exposed by omicron-gateway)
    LLDP daemon (lldpd-client, exposed by lldpd)
    Maghemite MG Admin (mg-admin-client, exposed by mgd)
    External API (oxide-client, exposed by omicron-nexus)
    Oximeter (oximeter-client, exposed by oximeter-collector)
    Propolis (propolis-client, exposed by propolis-server)
    Sled Agent (sled-agent-client, exposed by omicron-sled-agent)
    Wicketd (wicketd-client, exposed by wicketd)

Client-managed API:

    Bootstrap Agent (bootstrap-agent-client, exposed by omicron-sled-agent)
        reason: depends on itself (i.e., instances call each other)
    Dendrite DPD (dpd-client, exposed by dpd)
        reason: Sled Agent calling DPD creates a circular dependency
    Wicketd Installinator (installinator-client, exposed by wicketd)
        reason: client is provided implicitly by the operator
    Nexus Internal API (nexus-client, exposed by omicron-nexus)
        reason: Circular dependencies between Nexus and other services
    Crucible Repair (repair-client, exposed by crucible-downstairs)
        reason: depends on itself (i.e., instances call each other)
    Repo Depot API (repo-depot-client, exposed by omicron-sled-agent)
        reason: depends on itself (i.e., instances call each other)

APIs with unknown version management: none

eliza@theseus ~/Code/oxide/omicron $ cargo xtask ls-apis servers
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.35s
     Running `target/debug/xtask ls-apis servers`
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.52s
     Running `target/debug/ls-apis servers`
loading metadata for workspace omicron from current workspace
loading metadata for workspace lldp from /home/eliza/.cargo/git/checkouts/lldp-d47de417041f191b/ce952e6/Cargo.toml
loading metadata for workspace maghemite from /home/eliza/.cargo/git/checkouts/maghemite-c0236f0fd3d582b6/caafd88/Cargo.toml
loading metadata for workspace propolis from /home/eliza/.cargo/git/checkouts/propolis-d68c8bd1bc59c9bd/6b5f2af/Cargo.toml
loading metadata for workspace crucible from /home/eliza/.cargo/git/checkouts/crucible-0a48bd218bc2bbbc/81a3528/Cargo.toml
loading metadata for workspace dendrite from /home/eliza/.cargo/git/checkouts/dendrite-ae9f1715c17fc765/a66561e/Cargo.toml
note: ignoring Cargo dependency from crucible-pantry -> ... -> crucible-control-client
note: ignoring Cargo dependency from omicron-sled-agent -> dns-server
omicron-clickhouse-admin (omicron/clickhouse-admin)
    exposes: Clickhouse Cluster Admin for Keepers (client = clickhouse-admin-keeper-client)
    exposes: Clickhouse Cluster Admin for Servers (client = clickhouse-admin-server-client)
    exposes: Clickhouse Single-Node Cluster Admin (client = clickhouse-admin-single-client)

omicron-cockroach-admin (omicron/cockroach-admin)
    exposes: CockroachDB Cluster Admin (client = cockroach-admin-client)

crucible-agent (crucible/agent)
    exposes: Crucible Agent (client = crucible-agent-client)

crucible-downstairs (crucible/downstairs)
    exposes: Crucible Repair (client = repair-client)
    consumes: repair-client

crucible-pantry (crucible/pantry)
    exposes: Crucible Pantry (client = crucible-pantry-client)
    consumes: nexus-client

dns-server (omicron/dns-server)
    exposes: DNS Server (client = dns-service-client)

omicron-sled-agent (omicron/sled-agent)
    exposes: Bootstrap Agent (client = bootstrap-agent-client)
    exposes: Ereporter (client = ereporter-client)
    exposes: Repo Depot API (client = repo-depot-client)
    exposes: Sled Agent (client = sled-agent-client)
    consumes: bootstrap-agent-client
    consumes: ddm-admin-client
    consumes: dns-service-client
    consumes: dpd-client
    consumes: gateway-client
    consumes: mg-admin-client
    consumes: nexus-client
    consumes: propolis-client
    consumes: repo-depot-client

propolis-server (propolis/bin/propolis-server)
    exposes: Crucible Control (for testing only) (client = crucible-control-client)
    exposes: Propolis (client = propolis-client)
    consumes: nexus-client

ddmd (maghemite/ddmd)
    exposes: Maghemite DDM Admin (client = ddm-admin-client)
    consumes: dpd-client

dpd (dendrite/dpd)
    exposes: Dendrite DPD (client = dpd-client)
    consumes: gateway-client
    consumes: nexus-client

lldpd (lldp/lldpd)
    exposes: LLDP daemon (client = lldpd-client)

mgd (maghemite/mgd)
    exposes: Maghemite MG Admin (client = mg-admin-client)
    consumes: ddm-admin-client
    consumes: dpd-client

omicron-gateway (omicron/gateway)
    exposes: Ereporter (client = ereporter-client)
    exposes: Management Gateway Service (client = gateway-client)

tfportd (dendrite/tfportd)
    consumes: dpd-client
    consumes: lldpd-client

wicketd (omicron/wicketd)
    exposes: Wicketd Installinator (client = installinator-client)
    exposes: Wicketd (client = wicketd-client)
    consumes: bootstrap-agent-client
    consumes: ddm-admin-client
    consumes: dpd-client
    consumes: gateway-client

installinator (omicron/installinator)
    consumes: ddm-admin-client
    consumes: installinator-client

omicron-nexus (omicron/nexus)
    exposes: Nexus Internal API (client = nexus-client)
    exposes: External API (client = oxide-client)
    consumes: clickhouse-admin-keeper-client
    consumes: clickhouse-admin-server-client
    consumes: clickhouse-admin-single-client
    consumes: cockroach-admin-client
    consumes: crucible-agent-client
    consumes: crucible-pantry-client
    consumes: dns-service-client
    consumes: dpd-client
    consumes: ereporter-client
    consumes: gateway-client
    consumes: lldpd-client
    consumes: mg-admin-client
    consumes: oximeter-client
    consumes: propolis-client
    consumes: sled-agent-client

oximeter-collector (omicron/oximeter/collector)
    exposes: Oximeter (client = oximeter-client)
    consumes: nexus-client

Currently, `cargo xtask ls-apis` will fail when it encounters an API that is served by multiple entities. For example, when adding the `ereporter-api` in #7833, which is served by both MGS and sled-agent, `ls-apis` currently fails with an error like this one: ```console eliza@theseus ~/Code/oxide/omicron $ cargo xtask ls-apis apis Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.45s Running `target/debug/xtask ls-apis apis` Finished `dev` profile [unoptimized + debuginfo] target(s) in 1.01s Running `target/debug/ls-apis apis` loading metadata for workspace omicron from current workspace loading metadata for workspace lldp from /home/eliza/.cargo/git/checkouts/lldp-d47de417041f191b/ce952e6/Cargo.toml loading metadata for workspace propolis from /home/eliza/.cargo/git/checkouts/propolis-d68c8bd1bc59c9bd/6b5f2af/Cargo.toml loading metadata for workspace crucible from /home/eliza/.cargo/git/checkouts/crucible-0a48bd218bc2bbbc/81a3528/Cargo.toml loading metadata for workspace maghemite from /home/eliza/.cargo/git/checkouts/maghemite-c0236f0fd3d582b6/caafd88/Cargo.toml loading metadata for workspace dendrite from /home/eliza/.cargo/git/checkouts/dendrite-ae9f1715c17fc765/a66561e/Cargo.toml note: ignoring Cargo dependency from crucible-pantry -> ... -> crucible-control-client note: ignoring Cargo dependency from omicron-sled-agent -> dns-server error: API for client ereporter-client appears to be exported by multiple components: at least omicron-sled-agent and omicron-gateway (DepPath([PackageId { repr: "path+file:///home/eliza/Code/oxide/omicron/gateway#omicron-gateway@0.1.0" }])) Error: found at least one API exported by multiple servers # ... backtrace snipped for brevity ... ``` This is unfortunate, as it would be nice to be able to have such APIs without breaking `ls-apis`. Therefore, this commit adds support for APIs with multiple server components. This change ends up being pretty straightforward; it's mainly just changing the `api_producers` map from a map of client package names to server components to a map of client package names to *a map of* server components, and updating the code that consumes this information to loop over those server components as appropriate. Now, the various `ls-apis` commands can handle the new `ereproter-api`. For example: ```console eliza@theseus ~/Code/oxide/omicron $ cargo xtask ls-apis apis Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.37s Running `target/debug/xtask ls-apis apis` Compiling omicron-ls-apis v0.1.0 (/home/eliza/Code/oxide/omicron/dev-tools/ls-apis) Finished `dev` profile [unoptimized + debuginfo] target(s) in 1.69s Running `target/debug/ls-apis apis` loading metadata for workspace omicron from current workspace loading metadata for workspace lldp from /home/eliza/.cargo/git/checkouts/lldp-d47de417041f191b/ce952e6/Cargo.toml loading metadata for workspace maghemite from /home/eliza/.cargo/git/checkouts/maghemite-c0236f0fd3d582b6/caafd88/Cargo.toml loading metadata for workspace propolis from /home/eliza/.cargo/git/checkouts/propolis-d68c8bd1bc59c9bd/6b5f2af/Cargo.toml loading metadata for workspace crucible from /home/eliza/.cargo/git/checkouts/crucible-0a48bd218bc2bbbc/81a3528/Cargo.toml loading metadata for workspace dendrite from /home/eliza/.cargo/git/checkouts/dendrite-ae9f1715c17fc765/a66561e/Cargo.toml note: ignoring Cargo dependency from crucible-pantry -> ... -> crucible-control-client note: ignoring Cargo dependency from omicron-sled-agent -> dns-server Bootstrap Agent (client: bootstrap-agent-client) consumed by: omicron-sled-agent (omicron/sled-agent) via 1 path consumed by: wicketd (omicron/wicketd) via 2 paths # ... snipped irrelevant APIs for brevity ... Ereporter (client: ereporter-client) consumed by: omicron-nexus (omicron/nexus) via 1 path # ... snipped irrelevant APIs for brevity ... eliza@theseus ~/Code/oxide/omicron $ cargo xtask ls-apis check Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.32s Running `target/debug/xtask ls-apis check` Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.52s Running `target/debug/ls-apis check` loading metadata for workspace omicron from current workspace loading metadata for workspace lldp from /home/eliza/.cargo/git/checkouts/lldp-d47de417041f191b/ce952e6/Cargo.toml loading metadata for workspace maghemite from /home/eliza/.cargo/git/checkouts/maghemite-c0236f0fd3d582b6/caafd88/Cargo.toml loading metadata for workspace propolis from /home/eliza/.cargo/git/checkouts/propolis-d68c8bd1bc59c9bd/6b5f2af/Cargo.toml loading metadata for workspace crucible from /home/eliza/.cargo/git/checkouts/crucible-0a48bd218bc2bbbc/81a3528/Cargo.toml loading metadata for workspace dendrite from /home/eliza/.cargo/git/checkouts/dendrite-ae9f1715c17fc765/a66561e/Cargo.toml note: ignoring Cargo dependency from crucible-pantry -> ... -> crucible-control-client note: ignoring Cargo dependency from omicron-sled-agent -> dns-server Server-managed APIs: Clickhouse Cluster Admin for Keepers (clickhouse-admin-keeper-client, exposed by omicron-clickhouse-admin) Clickhouse Cluster Admin for Servers (clickhouse-admin-server-client, exposed by omicron-clickhouse-admin) Clickhouse Single-Node Cluster Admin (clickhouse-admin-single-client, exposed by omicron-clickhouse-admin) CockroachDB Cluster Admin (cockroach-admin-client, exposed by omicron-cockroach-admin) Crucible Agent (crucible-agent-client, exposed by crucible-agent) Crucible Control (for testing only) (crucible-control-client, exposed by propolis-server) Crucible Pantry (crucible-pantry-client, exposed by crucible-pantry) Maghemite DDM Admin (ddm-admin-client, exposed by ddmd) DNS Server (dns-service-client, exposed by dns-server) Ereporter (ereporter-client, exposed by omicron-gateway, omicron-sled-agent) Management Gateway Service (gateway-client, exposed by omicron-gateway) LLDP daemon (lldpd-client, exposed by lldpd) Maghemite MG Admin (mg-admin-client, exposed by mgd) External API (oxide-client, exposed by omicron-nexus) Oximeter (oximeter-client, exposed by oximeter-collector) Propolis (propolis-client, exposed by propolis-server) Sled Agent (sled-agent-client, exposed by omicron-sled-agent) Wicketd (wicketd-client, exposed by wicketd) Client-managed API: Bootstrap Agent (bootstrap-agent-client, exposed by omicron-sled-agent) reason: depends on itself (i.e., instances call each other) Dendrite DPD (dpd-client, exposed by dpd) reason: Sled Agent calling DPD creates a circular dependency Wicketd Installinator (installinator-client, exposed by wicketd) reason: client is provided implicitly by the operator Nexus Internal API (nexus-client, exposed by omicron-nexus) reason: Circular dependencies between Nexus and other services Crucible Repair (repair-client, exposed by crucible-downstairs) reason: depends on itself (i.e., instances call each other) Repo Depot API (repo-depot-client, exposed by omicron-sled-agent) reason: depends on itself (i.e., instances call each other) APIs with unknown version management: none ````

davepacheco

Nice!

dev-tools/ls-apis/src/bin/ls-apis.rs

davepacheco · 2025-03-20T17:59:25Z

dev-tools/ls-apis/src/bin/ls-apis.rs

-                Some (name) if name == s
-            )
-        }) {
+        for api in metadata


Could you post to the PR the output of this when there are multiple producers of an API?

I've added the output for ls-apis servers to the PR description. Here's the relevant bits (omicron-gateway and omicron-sled-agent both include the ereporter API):

... omicron-sled-agent (omicron/sled-agent) exposes: Bootstrap Agent (client = bootstrap-agent-client) exposes: Ereporter (client = ereporter-client) exposes: Repo Depot API (client = repo-depot-client) exposes: Sled Agent (client = sled-agent-client) consumes: bootstrap-agent-client consumes: ddm-admin-client consumes: dns-service-client consumes: dpd-client consumes: gateway-client consumes: mg-admin-client consumes: nexus-client consumes: propolis-client consumes: repo-depot-client ... omicron-gateway (omicron/gateway) exposes: Ereporter (client = ereporter-client) exposes: Management Gateway Service (client = gateway-client) ...

davepacheco · 2025-03-20T18:00:09Z

dev-tools/ls-apis/src/bin/ls-apis.rs

-            api.client_package_name,
-            apis.api_producer(&api.client_package_name).unwrap()
-        );
+        print_api_and_producers(api, apis);


Same -- would love to see the change in output for this.

I included the output from ls-apis check in the PR description here. It's a bit easy to miss the relevant bit in that output (the API with multiple producers), so I'll extract just that bit:

Server-managed APIs: ... Ereporter (ereporter-client, exposed by omicron-gateway, omicron-sled-agent) ...

dev-tools/ls-apis/src/system_apis.rs

hawkw requested a review from davepacheco March 20, 2025 17:49

hawkw self-assigned this Mar 20, 2025

davepacheco reviewed Mar 20, 2025

View reviewed changes

hawkw added 2 commits March 20, 2025 12:03

[ls-apis] output bulleted list of servers in table cell

adf046e

[ls-apis] allow multiple dep paths for servers

dc84a1b

hawkw requested a review from davepacheco March 20, 2025 19:07

davepacheco approved these changes Mar 20, 2025

View reviewed changes

hawkw merged commit 67a8984 into main Mar 20, 2025
16 checks passed

hawkw deleted the eliza/multiple-servers-in-ls-apis branch March 20, 2025 21:16

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[ls-apis] support APIs exposed by multiple servers #7843

[ls-apis] support APIs exposed by multiple servers #7843

hawkw commented Mar 20, 2025 •

edited

Loading

davepacheco left a comment

davepacheco Mar 20, 2025

hawkw Mar 20, 2025

davepacheco Mar 20, 2025

hawkw Mar 20, 2025

[ls-apis] support APIs exposed by multiple servers #7843

[ls-apis] support APIs exposed by multiple servers #7843

Conversation

hawkw commented Mar 20, 2025 • edited Loading

davepacheco left a comment

Choose a reason for hiding this comment

davepacheco Mar 20, 2025

Choose a reason for hiding this comment

hawkw Mar 20, 2025

Choose a reason for hiding this comment

davepacheco Mar 20, 2025

Choose a reason for hiding this comment

hawkw Mar 20, 2025

Choose a reason for hiding this comment

hawkw commented Mar 20, 2025 •

edited

Loading