Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

lambda.md : Add the lambda flow section #319

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

lambda.md : Add the lambda flow section #319

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
174e371
lambda.md : Add the lambda flow section for clarity.
santoshmahto7 Jul 24, 2024
cfdcd34
Update resctl-bench/doc/README.md
santoshmahto7 Aug 4, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions resctl-bench/doc/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,10 @@ The benchmarks
* [iocost-qos](iocost-qos.md): Benchmark IO isolation with different io.cost QoS configurations
* [iocost-tune](iocost-tune.md): Benchmark storage device to determine io.cost QoS solutions

AWS lambda workflow
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
AWS lambda workflow
AWS Lambda Result Upload workflow

===================

* [lambda](lambda.md): Lambda Result Upload overview, configuration and setup

Other Notes
===========
Expand Down
18 changes: 10 additions & 8 deletions resctl-bench/doc/lambda.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -87,10 +87,8 @@ We still need to configure our S3 bucket to allow public read only access, so th
}
```


We also store the credential parameters required to file a github issue in AWS Systems Manager -> parameter store.
We use github app `iocost-issue-creater` to file a github issue, thus it's credentials information `App Id` and
`Private Key` are stored in parameter store.
Lambda uses github app [iocost-issue-creater](https://github.com/apps/iocost-issue-creater) to file a github issue, thus it's credentials information `App Id` and `Private Key` are stored in AWS parameter store.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Lambda uses github app [iocost-issue-creater](https://github.com/apps/iocost-issue-creater) to file a github issue, thus it's credentials information `App Id` and `Private Key` are stored in AWS parameter store.
The Lambda function uses a GitHub app [iocost-issue-creater](https://github.com/apps/iocost-issue-creater) to file a GitGub issue, thus it's credentials information `App Id` and `Private Key` are stored in the AWS parameter store.

```
{
/iocost-bot/appid : "xxxx"
Expand All @@ -100,18 +98,22 @@ We use github app `iocost-issue-creater` to file a github issue, thus it's crede

AWS lambda flow
===============
1. User generates the benchmark result on their device.
`$ resctl-bench -r "$RESULT_JSON" --logfile=$LOG_FILE run iocost-tun`
2. User uploads the result to aws lambda function url as:
`resctl-bench -r <RESULT_JSON> upload --upload-url <AWS lambda function URL>`
1. User generates the benchmark result on their device.
`$ resctl-bench -r "$RESULT_JSON" --logfile=$LOG_FILE run iocost-tune`
2. User uploads the result to aws lambda function url as:
`resctl-bench -r <RESULT_JSON> upload --upload-url <AWS lambda function URL>`
e.g
`$resctl-bench --result resctl-bench-result_2022_07_01-00_26_40.json.gz upload --upload-url https://ygvr6jnjckwamfao5xztg6idiu0ukjeb.lambda-url.eu-west-1.on.aws`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need to confirm the URL first.


3. Lamda is tiggered automatically in AWS.
- It saves the benchmark result to S3 bucket.
- Then create a issue in iocost-benchmark/iocost-benchmarks project using `iocost-issue-creater` github app.
- Then creates an issue in [iocost-benchmarks](https://github.com/iocost-benchmark/iocost-benchmarks) project using [iocost-issue-creater](https://github.com/apps/iocost-issue-creater) github app.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- Then creates an issue in [iocost-benchmarks](https://github.com/iocost-benchmark/iocost-benchmarks) project using [iocost-issue-creater](https://github.com/apps/iocost-issue-creater) github app.
- Then creates an issue in the [iocost-benchmarks](https://github.com/iocost-benchmark/iocost-benchmarks) project using [iocost-issue-creater](https://github.com/apps/iocost-issue-creater) GitHub app.

- Issue contain a link to benchmark result stored in s3 bucket.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- Issue contain a link to benchmark result stored in s3 bucket.
- Issue contains a link to the benchmark result stored in an S3 bucket.

e.g https://github.com/iocost-benchmark/iocost-benchmarks/issues/88

### Lambda workflow:
Client uploads the benchmark result (above steps) -> AWS Lambda runs -> save result to s3 bucket -> Create github Issue with link of result stored in s3.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Client uploads the benchmark result (above steps) -> AWS Lambda runs -> save result to s3 bucket -> Create github Issue with link of result stored in s3.
User uploads the benchmark result (following the above steps), the AWS Lambda function runs and saves the result into the S3 bucket, then the Lambda function creates a GitHub Issue with a link to the result stored in the S3 bucket.

Thereafter it's job of [iocost-benchmarks project](https://github.com/iocost-benchmark/iocost-benchmarks) to import and merge the results with existing database and generate final hwdb file.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Thereafter it's job of [iocost-benchmarks project](https://github.com/iocost-benchmark/iocost-benchmarks) to import and merge the results with existing database and generate final hwdb file.
The submitted result is further processed by the CI workflows running in the [iocost-benchmarks](https://github.com/iocost-benchmark/iocost-benchmarks/) repository, importing and merging the results with the existing database and finally generating the hwdb file.


Deploying
=========
Expand Down
Loading