Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ereport] intial types and dropshot API crate #7833

Open
wants to merge 17 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
51 changes: 51 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

9 changes: 9 additions & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ members = [
"clients/ddm-admin-client",
"clients/dns-service-client",
"clients/dpd-client",
"clients/ereporter-client",
"clients/gateway-client",
"clients/installinator-client",
"clients/nexus-client",
Expand Down Expand Up @@ -49,6 +50,8 @@ members = [
"dns-server",
"dns-server-api",
"end-to-end-tests",
"ereport/api",
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: seems like it'd be nice to be consistent about ereport vs. ereporter. Is there a reason they're different?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was perhaps due to me being unnecessarily cutesy with it. I was expecting to use stuff from the -types crate anywhere that we deal with ereports, including in the collector and eventually diagnosis engines, while the things that actually implement the APIs are reporters, thus "ereporter". I can change them to be consistent.

"ereport/types",
"gateway",
"gateway-api",
"gateway-cli",
Expand Down Expand Up @@ -150,6 +153,7 @@ default-members = [
"clients/ddm-admin-client",
"clients/dns-service-client",
"clients/dpd-client",
"clients/ereporter-client",
"clients/gateway-client",
"clients/installinator-client",
"clients/nexus-client",
Expand Down Expand Up @@ -187,6 +191,8 @@ default-members = [
"dns-server",
"dns-server-api",
"end-to-end-tests",
"ereport/api",
"ereport/types",
"gateway",
"gateway-api",
"gateway-cli",
Expand Down Expand Up @@ -385,6 +391,9 @@ dpd-client = { path = "clients/dpd-client" }
dropshot = { version = "0.16.0", features = [ "usdt-probes" ] }
dyn-clone = "1.0.19"
either = "1.14.0"
ereporter-api = { path = "ereport/api" }
ereporter-client = { path = "clients/ereporter-client" }
ereport-types = { path = "ereport/types" }
expectorate = "1.1.0"
fatfs = "0.3.6"
filetime = "0.2.25"
Expand Down
22 changes: 22 additions & 0 deletions clients/ereporter-client/Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
[package]
name = "ereporter-client"
version = "0.1.0"
edition = "2024"

[lints]
workspace = true

[dependencies]
chrono.workspace = true
expectorate.workspace = true
ereport-types.workspace = true
http.workspace = true
progenitor.workspace = true
reqwest = { workspace = true, features = ["json", "rustls-tls", "stream"] }
schemars.workspace = true
serde.workspace = true
serde_json.workspace = true
slog.workspace = true
omicron-uuid-kinds.workspace = true
omicron-workspace-hack.workspace = true
uuid.workspace = true
23 changes: 23 additions & 0 deletions clients/ereporter-client/src/lib.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
// This Source Code Form is subject to the terms of the Mozilla Public
// License, v. 2.0. If a copy of the MPL was not distributed with this
// file, You can obtain one at https://mozilla.org/MPL/2.0/.

progenitor::generate_api!(
spec = "../../openapi/ereporter/ereporter-latest.json",
inner_type = slog::Logger,
derives = [schemars::JsonSchema, Clone, Eq, PartialEq],
pre_hook = (|log: &slog::Logger, request: &reqwest::Request| {
slog::debug!(log, "client request";
"method" => %request.method(),
"uri" => %request.url(),
"body" => ?&request.body(),
);
}),
post_hook = (|log: &slog::Logger, result: &Result<_, _>| {
slog::debug!(log, "client response"; "result" => ?result);
}),
replace = {
Ena = ereport_types::Ena,
TypedUuidForEreporterGenerationKind = omicron_uuid_kinds::EreporterGenerationUuid,
}
);
12 changes: 12 additions & 0 deletions dev-tools/ls-apis/api-manifest.toml
Original file line number Diff line number Diff line change
Expand Up @@ -270,6 +270,18 @@ and exists as a client library within omicron. This is because the Dendrite \
repo is not currently open source.
"""

[[apis]]
client_package_name = "ereporter-client"
label = "Ereporter"
server_package_name = "ereporter-api"
versioned_how = "server"
notes = """
Implemented by sled-agents and by MGS, and consumed by Nexus to collect \
ereports.

The sled-agent and MGS APIs are server-versioned, so this can be as well.
"""

[[apis]]
client_package_name = "lldpd-client"
label = "LLDP daemon"
Expand Down
3 changes: 3 additions & 0 deletions dev-tools/ls-apis/tests/api_dependencies.out
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,9 @@ Dendrite DPD (client: dpd-client)

Downstairs Controller (debugging only) (client: dsc-client)

Ereporter (client: ereporter-client)
consumed by: omicron-nexus (omicron/nexus) via 1 path

Management Gateway Service (client: gateway-client)
consumed by: dpd (dendrite/dpd) via 1 path
consumed by: omicron-nexus (omicron/nexus) via 3 paths
Expand Down
1 change: 1 addition & 0 deletions dev-tools/openapi-manager/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ cockroach-admin-api.workspace = true
debug-ignore.workspace = true
dns-server-api.workspace = true
dropshot.workspace = true
ereporter-api.workspace = true
hex.workspace = true
fs-err.workspace = true
gateway-api.workspace = true
Expand Down
12 changes: 12 additions & 0 deletions dev-tools/openapi-manager/src/omicron.rs
Original file line number Diff line number Diff line change
Expand Up @@ -85,6 +85,18 @@ pub fn all_apis() -> Vec<ManagedApiConfig> {
ident: "dns-server",
extra_validation: None,
},
ManagedApiConfig {
title: "Ereporter API",
versions: Versions::new_versioned(
ereporter_api::supported_versions()
),
description: "API for ereport producers",
boundary: ApiBoundary::Internal,
api_description:
ereporter_api::ereporter_api_mod::stub_api_description,
ident: "ereporter",
extra_validation: None,
},
ManagedApiConfig {
title: "Installinator API",
versions: Versions::new_lockstep(semver::Version::new(0,0,1)),
Expand Down
19 changes: 19 additions & 0 deletions ereport/api/Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
[package]
name = "ereporter-api"
version = "0.1.0"
edition = "2024"

[dependencies]
ereport-types.workspace = true
dropshot.workspace = true
openapi-manager-types.workspace = true
schemars.workspace = true
semver.workspace = true
serde.workspace = true
serde_json.workspace = true
thiserror.workspace = true
uuid.workspace = true
omicron-workspace-hack.workspace = true

[lints]
workspace = true
116 changes: 116 additions & 0 deletions ereport/api/src/lib.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,116 @@
// This Source Code Form is subject to the terms of the Mozilla Public
// License, v. 2.0. If a copy of the MPL was not distributed with this
// file, You can obtain one at https://mozilla.org/MPL/2.0/.

//! HTTP API for ereport producers.

use dropshot::HttpError;
use dropshot::HttpResponseOk;
use dropshot::Path;
use dropshot::Query;
use dropshot::RequestContext;
use dropshot::ResultsPage;
pub use ereport_types::Ena;
pub use ereport_types::Ereport;
pub use ereport_types::EreporterGenerationUuid;
pub use ereport_types::Reporter;
use openapi_manager_types::{
SupportedVersion, SupportedVersions, api_versions,
};
use schemars::JsonSchema;
use serde::Deserialize;
use serde::Serialize;
use std::num::NonZeroU32;
use uuid::Uuid;

api_versions!([
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

First time I've actually seen this. Looks nice.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i just followed the instructions from here --- shoutout to @davepacheco and @sunshowers for a very easy to follow guide!

// WHEN CHANGING THE API (part 1 of 2):
//
// +- Pick a new semver and define it in the list below. The list MUST
// | remain sorted, which generally means that your version should go at
// | the very top.
// |
// | Duplicate this line, uncomment the *second* copy, update that copy for
// | your new API version, and leave the first copy commented out as an
// | example for the next person.
// v
// (next_int, IDENT),
(1, INITIAL),
]);

// WHEN CHANGING THE API (part 2 of 2):
//
// The call to `api_versions!` above defines constants of type
// `semver::Version` that you can use in your Dropshot API definition to specify
// the version when a particular endpoint was added or removed. For example, if
// you used:
//
// (2, ADD_FOOBAR)
//
// Then you could use `VERSION_ADD_FOOBAR` as the version in which endpoints
// were added or removed.

/// API for ereport producers.
#[dropshot::api_description]
pub trait EreporterApi {
type Context;

/// Collect a tranche of ereports from this reporter.
#[endpoint {
method = POST,
path = "/ereports/{reporter_id}",
}]
async fn ereports_collect(
rqctx: RequestContext<Self::Context>,
path: Path<ReporterPath>,
query: Query<EreportQuery>,
) -> Result<HttpResponseOk<Ereports>, HttpError>;
}

#[derive(Clone, Debug, Serialize, Deserialize, JsonSchema)]
pub struct ReporterPath {
/// The UUID of the reporter from which to collct ereports.
pub reporter_id: Uuid,
}

#[derive(Clone, Debug, Serialize, Deserialize, JsonSchema)]
pub struct EreportQuery {
/// The generation (restart nonce) of the reporter at which all other query
/// parameters are valid.
///
/// If this value does not match the reporter's current generation, the
/// reporter's response will include the current generation, and will start
/// at the earliest known ENA, rather than the provided `last_seen` ENA.`
pub generation: EreporterGenerationUuid,

/// If present, the reporter should not include ENAs earlier than this one
/// in its response, provided that the query's requested generation matches
/// the current generation.
pub start_at: Option<Ena>,

/// The ENA of the last ereport committed to persistent storage from the
/// requested reporter generation.
///
/// If the generation parameter matches the reporter's current generation,
/// it is permitted to discard any ereports with ENAs up to and including
/// this value. If the generation has changed from the provided generation,
/// the reporter will not discard data.
pub committed: Option<Ena>,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm a big fan of the "committed counter being overloaded in the normal request" pattern. Very raft like.


/// Maximum number of ereports to return in this tranche.
pub limit: NonZeroU32,
}

/// A tranche of ereports received from a reporter.
#[derive(Debug, Serialize, Deserialize, JsonSchema)]
pub struct Ereports {
/// The reporter's current generation ID.
///
/// If this is not equal to the current known generation, then the reporter
/// has restarted.
pub generation: EreporterGenerationUuid,
/// The ereports in this tranche, and the ENA of the next page of ereports
/// (if one exists).)
#[serde(flatten)]
pub reports: ResultsPage<Ereport>,
}
15 changes: 15 additions & 0 deletions ereport/types/Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
[package]
name = "ereport-types"
version = "0.1.0"
edition = "2024"

[dependencies]
omicron-uuid-kinds.workspace = true
schemars.workspace = true
serde.workspace = true
serde_json.workspace = true
uuid.workspace = true
omicron-workspace-hack.workspace = true

[lints]
workspace = true
Loading
Loading