Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Integrate github app workflow with lambda. #302

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions resctl-bench/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ aws-sdk-s3 = { version = "0.28.0", optional = true, features = ["rustls", "rt-to
aws-sdk-ssm = { version = "0.28.0", optional = true, features = ["rustls", "rt-tokio"], default-features = false }
aws_lambda_events = "0.10.0"

jsonwebtoken = "8.3.0"
lambda_runtime = { version = "0.8.1", optional = true }
md5 = { version = "0.7", optional = true }
octocrab = { version = "0.28.0", optional = true, features = ["rustls"], default-features = false }
Expand Down
26 changes: 26 additions & 0 deletions resctl-bench/doc/lambda.md
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,32 @@ We still need to configure our S3 bucket to allow public read only access, so th
}
```


We also store the credential parameters required to file a github issue in AWS Systems Manager -> parameter store.
We use github app `iocost-issue-creater` to file a github issue, thus it's credentials information `App Id` and
`Private Key` are stored in parameter store.
```
{
/iocost-bot/appid : "xxxx"
/iocost-bot/privatekey : "PEM format key"
}
```

AWS lambda flow
===============
1. User generates the benchmark result on their device.
`$ resctl-bench -r "$RESULT_JSON" --logfile=$LOG_FILE run iocost-tun`
2. User uploads the result to aws lambda function url as:
`resctl-bench -r <RESULT_JSON> upload --upload-url <AWS lambda function URL>`
e.g
`$resctl-bench --result resctl-bench-result_2022_07_01-00_26_40.json.gz upload --upload-url https://ygvr6jnjckwamfao5xztg6idiu0ukjeb.lambda-url.eu-west-1.on.aws`

3. Lamda is tiggered automatically in AWS.
- It saves the benchmark result to S3 bucket.
- Then create a issue in iocost-benchmark/iocost-benchmarks project using `iocost-issue-creater` github app.
- Issue contain a link to benchmark result stored in s3 bucket.


Deploying
=========

Expand Down
48 changes: 45 additions & 3 deletions resctl-bench/src/lambda.rs
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,9 @@ use aws_lambda_events::event::lambda_function_urls::LambdaFunctionUrlRequest;
use aws_sdk_s3::error::SdkError;
use lambda_runtime::{service_fn, LambdaEvent};
use log::error;
use octocrab::models::InstallationToken;
use octocrab::params::apps::CreateInstallationAccessToken;
use octocrab::Octocrab;
use std::io::{Cursor, Read, Write};
use std::os::unix::process::CommandExt;
use std::path::Path;
Expand Down Expand Up @@ -145,10 +148,10 @@ impl LambdaHelper {
}

pub async fn create_github_issue(&self, title: &str, body: &str) -> Result<String> {
let token = self
let app_id = self
.ssm
.get_parameter()
.set_name(Some("/iocost-bot/token".to_string()))
.set_name(Some("/iocost-bot/appid".to_string()))
.send()
.await
.expect("Failed to query parameter")
Expand All @@ -157,8 +160,47 @@ impl LambdaHelper {
.value
.expect("Parameter value is None");

let pem = self
.ssm
.get_parameter()
.set_name(Some("/iocost-bot/privatekey".to_string()))
.send()
.await
.expect("Failed to query parameter")
.parameter
.expect("Could not find parameter")
.value
.expect("Parameter value is None");

let token = octocrab::auth::create_jwt(
app_id.parse::<u64>().unwrap().into(),
&jsonwebtoken::EncodingKey::from_rsa_pem(pem.as_bytes()).unwrap(),
)
.unwrap();

let octocrab = Octocrab::builder().personal_token(token).build()?;

let installations = octocrab
.apps()
.installations()
.send()
.await
.unwrap()
.take_items();

let mut create_access_token = CreateInstallationAccessToken::default();
create_access_token.repositories = vec!["iocost-benchmarks".to_string()];

let access: InstallationToken = octocrab
.post(
installations[0].access_tokens_url.as_ref().unwrap(),
Some(&create_access_token),
)
.await
.unwrap();

let issue = octocrab::OctocrabBuilder::new()
.personal_token(token)
.personal_token(access.token)
.build()?
.issues("iocost-benchmark", "iocost-benchmarks")
.create(title)
Expand Down
Loading