Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

(7/5) [nexus-db-queries] Benchmark for VMM reservation #7498

Merged
merged 129 commits into from
Mar 4, 2025
Merged
Show file tree
Hide file tree
Changes from 105 commits
Commits
Show all changes
129 commits
Select commit Hold shift + click to select a range
c9fb7a6
[nexus] Add Affinity/Anti-Affinity Groups to API (unimplemented)
smklein Jan 30, 2025
4020517
[nexus] Add Affinity/Anti-Affinity groups to database
smklein Jan 30, 2025
8f1d37c
[nexus] Add CRUD implementations for Affinity/Anti-Affinity Groups
smklein Jan 30, 2025
772e64f
[nexus] Consider Affinity/Anti-Affinity Groups during instance placement
smklein Jan 30, 2025
d8cff32
[nexus] Implement Affinity/Anti-Affinity Groups in external API
smklein Jan 30, 2025
161f9d6
fix policy tests
smklein Jan 30, 2025
df119b6
Merge branch 'affinity-db-crud' into affinity-instance-integration
smklein Jan 30, 2025
83a26a4
Merge branch 'affinity-instance-integration' into affinity-integration
smklein Jan 30, 2025
8dc0825
fmt
smklein Jan 30, 2025
e3113ff
Merge branch 'affinity-api' into affinity-db-model
smklein Jan 30, 2025
789bc97
Merge branch 'affinity-db-model' into affinity-db-crud
smklein Jan 30, 2025
5e21f34
Merge branch 'affinity-db-crud' into affinity-instance-integration
smklein Jan 30, 2025
fa9461b
Merge branch 'affinity-instance-integration' into affinity-integration
smklein Jan 30, 2025
4e9cebc
tags
smklein Jan 30, 2025
6cfca2d
doc comments
smklein Jan 31, 2025
a1c97d4
Merge branch 'affinity-api' into affinity-db-model
smklein Jan 31, 2025
050b4c5
Merge branch 'main' into affinity-api
smklein Jan 31, 2025
4b08032
typed UUID
smklein Jan 31, 2025
8bc8f0c
Merge branch 'affinity-api' into affinity-db-model
smklein Jan 31, 2025
900f09c
Typed UUID
smklein Jan 31, 2025
195e167
Merge branch 'affinity-db-model' into affinity-db-crud
smklein Jan 31, 2025
f2ebe31
Typed UUID
smklein Jan 31, 2025
62c38ec
Merge branch 'affinity-db-crud' into affinity-instance-integration
smklein Jan 31, 2025
85985c1
Merge branch 'affinity-instance-integration' into affinity-integration
smklein Jan 31, 2025
1326116
UUID typing
smklein Jan 31, 2025
aba9596
comments
smklein Jan 31, 2025
a271f1d
Merge branch 'affinity-db-model' into affinity-db-crud
smklein Jan 31, 2025
1ad0101
review feedback
smklein Feb 1, 2025
4d26262
comment
smklein Feb 1, 2025
6ae1910
clippy
smklein Feb 1, 2025
58ebd65
Merge branch 'affinity-db-crud' into affinity-instance-integration
smklein Feb 1, 2025
ab817bc
Merge branch 'affinity-instance-integration' into affinity-integration
smklein Feb 1, 2025
9daf923
Converting sled_resource to sled_resource_vmm
smklein Feb 3, 2025
1b39c51
schema, more renaming
smklein Feb 3, 2025
e21125f
Merge branch 'main' into affinity-api
smklein Feb 4, 2025
bd95b03
Merge branch 'affinity-api' into affinity-db-model
smklein Feb 4, 2025
76b0af9
Merge branch 'affinity-db-model' into affinity-db-crud
smklein Feb 4, 2025
1179919
Merge branch 'affinity-db-crud' into affinity-instance-integration
smklein Feb 4, 2025
a07c46e
review feedback
smklein Feb 4, 2025
fdccd6b
review feedback (grammatical)
smklein Feb 5, 2025
db49c67
Merge branch 'affinity-instance-integration' into affinity-integration
smklein Feb 5, 2025
6e44392
anti-affinity group description
smklein Feb 5, 2025
8e6fe1a
Merge branch 'affinity-integration' into sled-resource-vmm
smklein Feb 5, 2025
d5b951f
Update comment
smklein Feb 5, 2025
fdfb78d
Merge branch 'affinity-integration' into sled-resource-vmm
smklein Feb 5, 2025
f57a96a
Fix schema migration
smklein Feb 5, 2025
fb8976d
Redo regression test for OID poisoning, and move it
smklein Feb 5, 2025
17dc409
clippppyyyyyy
smklein Feb 6, 2025
97b9b2e
Eliza's feedback
smklein Feb 6, 2025
d949890
Merge branch 'affinity-instance-integration' into affinity-integration
smklein Feb 6, 2025
fa2550e
Merge branch 'affinity-integration' into sled-resource-vmm
smklein Feb 6, 2025
79b4252
[nexus-db-queries] Benchmark for VMM reservation
smklein Feb 6, 2025
04a4b98
Tweak usable hardware threads to make instance placement less flaky
smklein Feb 6, 2025
db40d05
Normalize reservation time, only benchmark creation pathway
smklein Feb 6, 2025
b68239b
Normalize
smklein Feb 7, 2025
ca8f890
cleanup
smklein Feb 7, 2025
5406dd4
Better contention info
smklein Feb 10, 2025
bb0f349
Better contention information
smklein Feb 11, 2025
6704be1
restructure benchmark
smklein Feb 11, 2025
127285c
more refactoring
smklein Feb 11, 2025
acf3a64
test naming
smklein Feb 12, 2025
43f4437
benchmark updates
smklein Feb 14, 2025
9a3ef20
fmt
smklein Feb 14, 2025
f028351
clippy
smklein Feb 14, 2025
45b993a
Merge branch 'main' into affinity-api
smklein Feb 18, 2025
05c8b60
Merge branch 'affinity-api' into affinity-db-model
smklein Feb 18, 2025
eee1d6f
Merge branch 'affinity-db-model' into affinity-db-crud
smklein Feb 18, 2025
10aaed9
Merge branch 'affinity-db-crud' into affinity-instance-integration
smklein Feb 18, 2025
f8b9b5d
Merge branch 'affinity-instance-integration' into affinity-integration
smklein Feb 18, 2025
1032128
Merge branch 'affinity-integration' into sled-resource-vmm
smklein Feb 18, 2025
859f099
Merge branch 'sled-resource-vmm' into vmm-reserve-bench
smklein Feb 18, 2025
3edc5ed
Fix mismerge
smklein Feb 18, 2025
8566395
Merge branch 'sled-resource-vmm' into vmm-reserve-bench
smklein Feb 18, 2025
393a925
Merge branch 'main' into affinity-api
smklein Feb 19, 2025
7a99a96
Merge branch 'affinity-api' into affinity-db-model
smklein Feb 19, 2025
ee657cc
Merge branch 'affinity-db-model' into affinity-db-crud
smklein Feb 19, 2025
931801d
Merge branch 'affinity-db-crud' into affinity-instance-integration
smklein Feb 19, 2025
5f75a2b
Merge branch 'affinity-instance-integration' into affinity-integration
smklein Feb 19, 2025
b1689c8
Merge branch 'affinity-integration' into sled-resource-vmm
smklein Feb 19, 2025
fd09de7
Merge branch 'sled-resource-vmm' into vmm-reserve-bench
smklein Feb 19, 2025
c18449c
Merge branch 'main' into affinity-api
smklein Feb 19, 2025
4bf0e48
Merge branch 'affinity-api' into affinity-db-model
smklein Feb 19, 2025
1a7e7e0
Merge branch 'main' into affinity-api
smklein Feb 19, 2025
01b8f03
Merge branch 'affinity-api' into affinity-db-model
smklein Feb 19, 2025
a35a33c
Merge branch 'main' into affinity-api
smklein Feb 20, 2025
eb9682f
Merge branch 'affinity-api' into affinity-db-model
smklein Feb 20, 2025
0e6eaf2
Merge branch 'affinity-db-model' into affinity-db-crud
smklein Feb 20, 2025
45a97a3
Merge branch 'affinity-db-crud' into affinity-instance-integration
smklein Feb 20, 2025
bd1f2cc
Merge branch 'affinity-instance-integration' into affinity-integration
smklein Feb 20, 2025
7ba08b0
Merge branch 'affinity-integration' into sled-resource-vmm
smklein Feb 20, 2025
1add3c1
Merge branch 'sled-resource-vmm' into vmm-reserve-bench
smklein Feb 20, 2025
4a00ddd
Merge branch 'main' into affinity-api
smklein Feb 21, 2025
7a8da16
Merge branch 'affinity-api' into affinity-db-model (schema)
smklein Feb 21, 2025
081e8b4
Merge branch 'main' into affinity-api
smklein Feb 21, 2025
7733054
Merge branch 'affinity-api' into affinity-db-model
smklein Feb 21, 2025
e9cc19c
Merge branch 'affinity-db-model' into affinity-db-crud
smklein Feb 21, 2025
01914c3
Merge branch 'affinity-db-crud' into affinity-instance-integration
smklein Feb 21, 2025
e6634dc
Merge branch 'affinity-instance-integration' into affinity-integration
smklein Feb 21, 2025
565ffc8
Merge branch 'main' into affinity-api
smklein Feb 21, 2025
e15460c
Merge branch 'affinity-api' into affinity-db-model
smklein Feb 21, 2025
9190d80
Merge branch 'affinity-db-model' into affinity-db-crud
smklein Feb 21, 2025
8d90161
Merge branch 'affinity-db-crud' into affinity-instance-integration
smklein Feb 21, 2025
9f6ad7b
Merge branch 'affinity-instance-integration' into affinity-integration
smklein Feb 21, 2025
49196d2
Merge branch 'affinity-integration' into sled-resource-vmm
smklein Feb 21, 2025
7193c88
Merge branch 'sled-resource-vmm' into vmm-reserve-bench
smklein Feb 21, 2025
16b3df3
Merge branch 'main' into affinity-api
smklein Feb 24, 2025
f5bbe28
Merge branch 'affinity-api' into affinity-db-model
smklein Feb 24, 2025
beaedfc
Merge branch 'affinity-db-model' into affinity-db-crud
smklein Feb 24, 2025
afbc4ad
Merge branch 'affinity-db-crud' into affinity-instance-integration
smklein Feb 24, 2025
25a6e5e
Merge branch 'affinity-instance-integration' into affinity-integration
smklein Feb 24, 2025
7c90072
Merge branch 'affinity-integration' into sled-resource-vmm
smklein Feb 24, 2025
701106d
Merge branch 'sled-resource-vmm' into vmm-reserve-bench
smklein Feb 24, 2025
0419d37
review feedback
smklein Feb 24, 2025
2ad7bb7
Merge branch 'affinity-integration' into sled-resource-vmm
smklein Feb 24, 2025
da625b9
Merge branch 'sled-resource-vmm' into vmm-reserve-bench
smklein Feb 24, 2025
36ddafc
Relocate some functions into pub_test_utils
smklein Feb 25, 2025
b8c5024
Code review feedback
smklein Feb 25, 2025
bba32a6
Merge branch 'main' into affinity-instance-integration
smklein Feb 25, 2025
e893250
Merge branch 'affinity-instance-integration' into affinity-integration
smklein Feb 25, 2025
f5ec80d
Merge branch 'affinity-integration' into sled-resource-vmm
smklein Feb 25, 2025
fba0614
Merge branch 'sled-resource-vmm' into vmm-reserve-bench
smklein Feb 25, 2025
e79fa5f
README
smklein Feb 25, 2025
faed1c4
Merge branch 'main' into vmm-reserve-bench
smklein Feb 25, 2025
01029f1
Add issue
smklein Feb 25, 2025
3c0cc80
Merge branch 'main' into vmm-reserve-bench
smklein Feb 25, 2025
462afc7
Merge branch 'main' into vmm-reserve-bench
smklein Feb 27, 2025
93d4ac3
Merge branch 'main' into vmm-reserve-bench
smklein Feb 28, 2025
75acd32
Merge branch 'main' into vmm-reserve-bench
smklein Mar 3, 2025
a3e885a
Merge branch 'main' into vmm-reserve-bench
smklein Mar 4, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

13 changes: 13 additions & 0 deletions common/src/api/external/http_pagination.rs
Original file line number Diff line number Diff line change
Expand Up @@ -312,6 +312,19 @@ pub type PaginatedByNameOrId<Selector = ()> = PaginationParams<
pub type PageSelectorByNameOrId<Selector = ()> =
PageSelector<ScanByNameOrId<Selector>, NameOrId>;

pub fn id_pagination<'a, Selector>(
pag_params: &'a DataPageParams<Uuid>,
scan_params: &'a ScanById<Selector>,
) -> Result<PaginatedBy<'a>, HttpError>
where
Selector:
Clone + Debug + DeserializeOwned + JsonSchema + PartialEq + Serialize,
{
match scan_params.sort_by {
IdSortMode::IdAscending => Ok(PaginatedBy::Id(pag_params.clone())),
}
}

pub fn name_or_id_pagination<'a, Selector>(
pag_params: &'a DataPageParams<NameOrId>,
scan_params: &'a ScanByNameOrId<Selector>,
Expand Down
10 changes: 9 additions & 1 deletion common/src/api/external/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -293,6 +293,12 @@ impl<'a> From<&'a Name> for &'a str {
}
}

impl From<Name> for String {
fn from(name: Name) -> Self {
name.0
}
}

/// `Name` instances are comparable like Strings, primarily so that they can
/// be used as keys in trees.
impl<S> PartialEq<S> for Name
Expand Down Expand Up @@ -1335,7 +1341,9 @@ pub enum InstanceAutoRestartPolicy {
/// Affinity policy used to describe "what to do when a request cannot be satisfied"
///
/// Used for both Affinity and Anti-Affinity Groups
#[derive(Clone, Copy, Debug, Deserialize, Serialize, PartialEq, JsonSchema)]
#[derive(
Clone, Copy, Debug, Deserialize, Hash, Eq, Serialize, PartialEq, JsonSchema,
)]
#[serde(rename_all = "snake_case")]
pub enum AffinityPolicy {
/// If the affinity request cannot be satisfied, allow it anyway.
Expand Down
13 changes: 6 additions & 7 deletions dev-tools/omdb/src/bin/omdb/db.rs
Original file line number Diff line number Diff line change
Expand Up @@ -6403,7 +6403,7 @@ async fn cmd_db_vmm_info(
&VmmInfoArgs { uuid }: &VmmInfoArgs,
) -> Result<(), anyhow::Error> {
use db::schema::migration::dsl as migration_dsl;
use db::schema::sled_resource::dsl as resource_dsl;
use db::schema::sled_resource_vmm::dsl as resource_dsl;
use db::schema::vmm::dsl as vmm_dsl;

let vmm = vmm_dsl::vmm
Expand Down Expand Up @@ -6439,14 +6439,13 @@ async fn cmd_db_vmm_info(
);

fn prettyprint_reservation(
resource: db::model::SledResource,
resource: db::model::SledResourceVmm,
include_sled_id: bool,
) {
use db::model::ByteCount;
let db::model::SledResource {
let db::model::SledResourceVmm {
id: _,
sled_id,
kind: _,
resources:
db::model::Resources {
hardware_threads,
Expand All @@ -6468,10 +6467,10 @@ async fn cmd_db_vmm_info(
println!(" {RESERVOIR:>WIDTH$}: {reservoir}");
}

let reservations = resource_dsl::sled_resource
let reservations = resource_dsl::sled_resource_vmm
.filter(resource_dsl::id.eq(uuid))
.select(db::model::SledResource::as_select())
.load_async::<db::model::SledResource>(
.select(db::model::SledResourceVmm::as_select())
.load_async::<db::model::SledResourceVmm>(
&*datastore.pool_connection_for_tests().await?,
)
.await
Expand Down
16 changes: 16 additions & 0 deletions nexus/auth/src/authz/api_resources.rs
Original file line number Diff line number Diff line change
Expand Up @@ -714,6 +714,22 @@ authz_resource! {
polar_snippet = InProject,
}

authz_resource! {
name = "AffinityGroup",
parent = "Project",
primary_key = Uuid,
roles_allowed = false,
polar_snippet = InProject,
}

authz_resource! {
name = "AntiAffinityGroup",
parent = "Project",
primary_key = Uuid,
roles_allowed = false,
polar_snippet = InProject,
}

authz_resource! {
name = "InstanceNetworkInterface",
parent = "Instance",
Expand Down
2 changes: 2 additions & 0 deletions nexus/auth/src/authz/oso_generic.rs
Original file line number Diff line number Diff line change
Expand Up @@ -125,6 +125,8 @@ pub fn make_omicron_oso(log: &slog::Logger) -> Result<OsoInit, anyhow::Error> {
Disk::init(),
Snapshot::init(),
ProjectImage::init(),
AffinityGroup::init(),
AntiAffinityGroup::init(),
Instance::init(),
IpPool::init(),
InstanceNetworkInterface::init(),
Expand Down
6 changes: 2 additions & 4 deletions nexus/db-model/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -92,8 +92,7 @@ mod silo_user_password_hash;
mod sled;
mod sled_instance;
mod sled_policy;
mod sled_resource;
mod sled_resource_kind;
mod sled_resource_vmm;
mod sled_state;
mod sled_underlay_subnet_allocation;
mod snapshot;
Expand Down Expand Up @@ -203,8 +202,7 @@ pub use silo_user_password_hash::*;
pub use sled::*;
pub use sled_instance::*;
pub use sled_policy::to_db_sled_policy; // Do not expose DbSledPolicy
pub use sled_resource::*;
pub use sled_resource_kind::*;
pub use sled_resource_vmm::*;
pub use sled_state::*;
pub use sled_underlay_subnet_allocation::*;
pub use snapshot::*;
Expand Down
5 changes: 2 additions & 3 deletions nexus/db-model/src/schema.rs
Original file line number Diff line number Diff line change
Expand Up @@ -954,13 +954,12 @@ table! {
}

table! {
sled_resource (id) {
sled_resource_vmm (id) {
id -> Uuid,
sled_id -> Uuid,
hardware_threads -> Int8,
rss_ram -> Int8,
reservoir_ram -> Int8,
kind -> crate::SledResourceKindEnum,
instance_id -> Nullable<Uuid>,
}
}
Expand Down Expand Up @@ -2103,7 +2102,7 @@ allow_tables_to_appear_in_same_query!(
identity_provider,
console_session,
sled,
sled_resource,
sled_resource_vmm,
support_bundle,
router_route,
vmm,
Expand Down
3 changes: 2 additions & 1 deletion nexus/db-model/src/schema_versions.rs
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ use std::{collections::BTreeMap, sync::LazyLock};
///
/// This must be updated when you change the database schema. Refer to
/// schema/crdb/README.adoc in the root of this repository for details.
pub const SCHEMA_VERSION: SemverVersion = SemverVersion::new(126, 0, 0);
pub const SCHEMA_VERSION: SemverVersion = SemverVersion::new(127, 0, 0);

/// List of all past database schema versions, in *reverse* order
///
Expand All @@ -28,6 +28,7 @@ static KNOWN_VERSIONS: LazyLock<Vec<KnownVersion>> = LazyLock::new(|| {
// | leaving the first copy as an example for the next person.
// v
// KnownVersion::new(next_int, "unique-dirname-with-the-sql-files"),
KnownVersion::new(127, "sled-resource-for-vmm"),
KnownVersion::new(126, "affinity"),
KnownVersion::new(125, "blueprint-disposition-expunged-cleanup"),
KnownVersion::new(124, "support-read-only-region-replacement"),
Expand Down
19 changes: 0 additions & 19 deletions nexus/db-model/src/sled_resource_kind.rs

This file was deleted.

Original file line number Diff line number Diff line change
Expand Up @@ -2,22 +2,22 @@
// License, v. 2.0. If a copy of the MPL was not distributed with this
// file, You can obtain one at https://mozilla.org/MPL/2.0/.

use crate::schema::sled_resource;
use crate::schema::sled_resource_vmm;
use crate::typed_uuid::DbTypedUuid;
use crate::{ByteCount, SledResourceKind, SqlU32};
use omicron_uuid_kinds::GenericUuid;
use crate::{ByteCount, SqlU32};
use omicron_uuid_kinds::InstanceKind;
use omicron_uuid_kinds::InstanceUuid;
use omicron_uuid_kinds::PropolisKind;
use omicron_uuid_kinds::PropolisUuid;
use omicron_uuid_kinds::SledKind;
use omicron_uuid_kinds::SledUuid;
use uuid::Uuid;

type DbInstanceUuid = DbTypedUuid<InstanceKind>;
type DbPropolisUuid = DbTypedUuid<PropolisKind>;
type DbSledUuid = DbTypedUuid<SledKind>;

#[derive(Clone, Selectable, Queryable, Insertable, Debug)]
#[diesel(table_name = sled_resource)]
#[diesel(table_name = sled_resource_vmm)]
pub struct Resources {
pub hardware_threads: SqlU32,
pub rss_ram: ByteCount,
Expand All @@ -34,32 +34,30 @@ impl Resources {
}
}

/// Describes sled resource usage by services
/// Describes sled resource usage by a VMM
#[derive(Clone, Selectable, Queryable, Insertable, Debug)]
#[diesel(table_name = sled_resource)]
pub struct SledResource {
pub id: Uuid,
#[diesel(table_name = sled_resource_vmm)]
pub struct SledResourceVmm {
pub id: DbPropolisUuid,
pub sled_id: DbSledUuid,

#[diesel(embed)]
pub resources: Resources,

pub kind: SledResourceKind,
pub instance_id: Option<DbInstanceUuid>,
}

impl SledResource {
pub fn new_for_vmm(
impl SledResourceVmm {
pub fn new(
id: PropolisUuid,
instance_id: InstanceUuid,
sled_id: SledUuid,
resources: Resources,
) -> Self {
Self {
id: id.into_untyped_uuid(),
id: id.into(),
instance_id: Some(instance_id.into()),
sled_id: sled_id.into(),
kind: SledResourceKind::Instance,
resources,
}
}
Expand Down
8 changes: 7 additions & 1 deletion nexus/db-queries/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ futures.workspace = true
internal-dns-resolver.workspace = true
internal-dns-types.workspace = true
ipnetwork.workspace = true
itertools.workspace = true
macaddr.workspace = true
oxnet.workspace = true
paste.workspace = true
Expand Down Expand Up @@ -74,6 +75,7 @@ testing = ["omicron-test-utils"]
[dev-dependencies]
assert_matches.workspace = true
camino-tempfile.workspace = true
criterion.workspace = true
expectorate.workspace = true
hyper-rustls.workspace = true
gateway-client.workspace = true
Expand All @@ -84,7 +86,7 @@ nexus-inventory.workspace = true
nexus-reconfigurator-planning.workspace = true
nexus-test-utils.workspace = true
omicron-sled-agent.workspace = true
omicron-test-utils.workspace = true
omicron-test-utils = { workspace = true, features = ["seed-gen"] }
openapiv3.workspace = true
oso.workspace = true
pem.workspace = true
Expand All @@ -96,3 +98,7 @@ regex.workspace = true
rustls.workspace = true
subprocess.workspace = true
term.workspace = true

[[bench]]
name = "sled_reservation"
harness = false
Loading
Loading