Skip to content

Commit bab72ee

Browse files
peterzhuamazonpeterzhuamazon
authored
Refactor getters and pass resource into call and add codecov (#11)
* Refactor getter and pass resource into call Signed-off-by: Peter Zhu <zhujiaxi@amazon.com> * Fix existing tests Signed-off-by: Peter Zhu <zhujiaxi@amazon.com> * More tweaks Signed-off-by: Peter Zhu <zhujiaxi@amazon.com> * Add print-to-console test Signed-off-by: Peter Zhu <zhujiaxi@amazon.com> * Add tests for verify-resource and create-issue-comment call Signed-off-by: Peter Zhu <zhujiaxi@amazon.com> * Add octokit test on octokitAuth utility Signed-off-by: Peter Zhu <zhujiaxi@amazon.com> * Update Readme Signed-off-by: Peter Zhu <zhujiaxi@amazon.com> * Update codecov Signed-off-by: Peter Zhu <zhujiaxi@amazon.com> * Add codecov step Signed-off-by: Peter Zhu <zhujiaxi@amazon.com> * Add codecov step Signed-off-by: Peter Zhu <zhujiaxi@amazon.com> * Rename verifyOrgRepo to verifyResourceConfig for later extension Signed-off-by: Peter Zhu <zhujiaxi@amazon.com> --------- Signed-off-by: Peter Zhu <zhujiaxi@amazon.com>
1 parent f194866 commit bab72ee

37 files changed

+600
-263
lines changed

‎.github/CODEOWNERS

+1-1
Original file line numberDiff line numberDiff line change
@@ -1 +1 @@
1-
* @getsaurabh02 @peterzhuamazon @gaiksaya @prudhvigodithi
1+
* @getsaurabh02 @peterzhuamazon @gaiksaya @prudhvigodithi @nhtruong

‎.github/workflows/build-test.yml

+4-2
Original file line numberDiff line numberDiff line change
@@ -27,5 +27,7 @@ jobs:
2727
run: echo "Please run 'npm run format' before commiting the code!"
2828
- name: Run build
2929
run: npm run build
30-
- name: Run Test
31-
run: npm test
30+
- name: Upload results to Codecov
31+
uses: codecov/codecov-action@v4
32+
env:
33+
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}

‎.gitignore

+1
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
node_modules
22
npm-debug.log
33
*.pem
4+
*.swp*
45
!mock-cert.pem
56
.env*
67
coverage

‎MAINTAINERS.md

+1
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,7 @@ This document contains a list of maintainers in this repo. See [opensearch-proje
1414
| Peter Zhu | [peterzhuamazon](https://github.com/peterzhuamazon) | Amazon |
1515
| Prudhvi Godithi | [prudhvigodithi](https://github.com/prudhvigodithi) | Amazon |
1616
| Sayali Gaikawad | [gaiksaya](https://github.com/gaiksaya) | Amazon |
17+
| Theo Truong | [nhtruong](https://github.com/nhtruong) | Amazon |
1718

1819
## Emeritus
1920

‎README.md

+22-3
Original file line numberDiff line numberDiff line change
@@ -35,22 +35,41 @@ To create a service, you need two configuration files:
3535

3636
### Start the Service
3737

38-
Once you have created the resource and operation configuration files, follow these steps to start the service:
38+
Before starting the service, create a `.env` file to connect it to your GitHub App by copying the `.env.example` file from this repository to `.env`.
39+
40+
Once you have created the `.env` file, resource / operation configuration files, follow these steps to start the service:
3941

4042
1. Set the `RESOURCE_CONFIG` environment variable to the path of the resource configuration YAML file.
4143
1. Set the `OPERATION_CONFIG` environment variable to the path of the operation configuration YAML file.
42-
1. Run the service using the following command:
44+
1. Update the `INSTALLATION_ID` variable in `.env` file. ([How to find installation id of your GitHub App](https://docs.github.com/en/apps/creating-github-apps/authenticating-with-a-github-app/authenticating-as-a-github-app-installation#using-octokitjs-to-authenticate-with-an-installation-id))
45+
1. Update your GitHub App settings on the GitHub website by defining the app's permissions and specifying the events to monitor.
46+
1. Run the service using the following command.
47+
1. (First Time Only) If you have created a new `.env` file, you will be directed to `http://localhost:3000` as seen in the console message. Open this URL in your browser and follow the instructions to set up the necessary information.
4348

4449
```bash
45-
RESOURCE_CONFIG=configs/resources/sample-resource.yml OPERATION_CONFIG=configs/operations/sample-operation.yml npm start
50+
RESOURCE_CONFIG=configs/resources/sample-resource.yml \
51+
OPERATION_CONFIG=configs/operations/sample-operation.yml \
52+
npm run dev
4653
```
4754

55+
**Note**: You should run `npm run start` instead in production to run prettier / eslint / jest before starting the service.
56+
4857
When you run the above command, the following takes place:
4958

5059
1. The app starts a `Service` instance based on the specified configurations.
5160
1. Retrieves the [GitHub Context](https://probot.github.io/api/latest/classes/context.Context.html) (or any other defined context) for all the resources listed in the resource config file.
5261
1. Registers and listens for events, executes the `Tasks` defined in the operation config. These tasks will be executed sequentially when the corresponding events occur.
5362

63+
64+
#### List of Environment Variables (You can use them directly in the startup command, export them, or add them to the `.env` file):
65+
| Name | Type | Default | Description | Example |
66+
|-----------------------------|---------|-----------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------|
67+
| RESOURCE_CONFIG | String | '' | Path to resource config yaml file. | 'configs/resources/sample-resource.yml' |
68+
| OPERATION_CONFIG | String | '' | Path to operation config yaml file. | 'configs/operations/sample-operation.yml' |
69+
| INSTALLATION_ID | String | '' | Installation Id of your GitHub App, must install the App to repositories before retrieving the id. | '1234567890' |
70+
| ADDITIONAL_RESOURCE_CONTEXT | Boolean | false | Setting true will let each resource defined in RESOURCE_CONFIG to call GitHub Rest API and GraphQL for more detailed context (ex: node_id). Increase startup time. | true / false |
71+
| SERVICE_NAME | String | 'default' | Set Service Name | 'My Service' |'
72+
5473
## Code of Conduct
5574

5675
This project has adopted [the Open Source Code of Conduct](CODE_OF_CONDUCT.md).

‎configs/operations/sample-operation.yml

+4-3
Original file line numberDiff line numberDiff line change
@@ -11,8 +11,9 @@ tasks:
1111
- name: Print a message to the console
1212
call: print-to-console@default
1313
args:
14-
text: Hello World!
15-
- call: create-issue-comment@createIssueCommentTagUser
14+
text: This is a sample output message!
15+
- name: Print Hello World to the console
16+
call: print-to-console@printToConsoleHelloWorld
17+
- call: create-issue-comment@default
1618
args:
1719
text: This comment is created by Hello World Operation
18-
tagUser: peterzhuamazon

‎package.json

+6-4
Original file line numberDiff line numberDiff line change
@@ -13,14 +13,16 @@
1313
"automation-app"
1414
],
1515
"scripts": {
16-
"build": "npm run clean && npm run format && npm run lint && npm run compile",
16+
"build": "npm run format-dryrun && npm run clean && npm run lint && npm run compile",
17+
"postbuild": "npm run test",
1718
"compile": "tsc",
19+
"dev": "npm run clean && npm run compile && probot run ./bin/app.js",
1820
"start": "npm run build && probot run ./bin/app.js",
1921
"clean": "rm -rf ./bin/*",
20-
"format": "prettier --write \"src/**/*.{js,ts}\" \"test/**/*.ts\" \"configs/**/*.{yaml,yml}\"",
21-
"format-dryrun": "prettier --check \"src/**/*.{js,ts}\" \"test/**/*.ts\" \"configs/**/*.{yaml,yml}\"",
22+
"format": "prettier --write src/**/*.ts test/**/*.ts configs/**/*.yml",
23+
"format-dryrun": "prettier --check src/**/*.ts test/**/*.ts configs/**/*.yml",
2224
"lint": "eslint --fix \"src/**/*.ts\" --ignore-pattern \"**/*.d.ts\"",
23-
"test": "jest"
25+
"test": "jest --coverage"
2426
},
2527
"dependencies": {
2628
"@aws-sdk/client-opensearch": "^3.658.1",

‎src/app.ts

+15-6
Original file line numberDiff line numberDiff line change
@@ -10,18 +10,27 @@
1010
import { Probot } from 'probot';
1111
import { Service } from './service/service';
1212

13-
// TODO: Move Probot to the workflow folder and make it server
1413
export default async (app: Probot) => {
1514
app.log.info('OpenSearch Automation App is starting now......');
1615

17-
const srvObj = new Service('Hello World Service');
18-
const resourceConfig: string = process.env.RESOURCE_CONFIG || 'configs/resources/sample-resource.yml';
19-
const processConfig: string = process.env.OPERATION_CONFIG || 'configs/operations/sample-operation.yml';
16+
// Env Vars
17+
const resourceConfig: string = process.env.RESOURCE_CONFIG || '';
18+
const processConfig: string = process.env.OPERATION_CONFIG || '';
19+
const additionalResourceContext: boolean = Boolean(process.env.ADDITIONAL_RESOURCE_CONTEXT) || false;
20+
const serviceName: string = process.env.SERVICE_NAME || 'default';
21+
22+
// Start service
23+
const srvObj = new Service(serviceName);
2024

2125
if (resourceConfig === '' || processConfig === '') {
22-
throw new Error(`Invalid config path: RESOURCE_CONFIG=${resourceConfig} or OPERATION_CONFIG=${processConfig}`);
26+
throw new Error(`Empty config path: RESOURCE_CONFIG='${resourceConfig}' or OPERATION_CONFIG='${processConfig}'`);
27+
}
28+
29+
if (additionalResourceContext) {
30+
app.log.info('Start requesting additional resource context now, take a while......');
2331
}
24-
await srvObj.initService(app, resourceConfig, processConfig);
32+
33+
await srvObj.initService(app, resourceConfig, processConfig, additionalResourceContext);
2534

2635
app.log.info('All objects initialized, start listening events......');
2736
};

‎src/call/create-issue-comment.ts

+6-9
Original file line numberDiff line numberDiff line change
@@ -15,18 +15,15 @@
1515

1616
import { Probot } from 'probot';
1717
import dedent from 'dedent';
18+
import { Resource } from '../service/resource/resource';
19+
import { validateResourceConfig } from '../utility/verification/verify-resource';
1820

1921
export interface CreateIssueCommentParams {
2022
text: string;
21-
tagUser?: string;
2223
}
2324

24-
export default async function createIssueComment(app: Probot, context: any, { text }: CreateIssueCommentParams): Promise<void> {
25-
const comment = context.issue({ body: dedent`${text}` });
26-
context.octokit.issues.createComment(comment);
27-
}
28-
29-
export async function createIssueCommentTagUser(app: Probot, context: any, { text, tagUser }: CreateIssueCommentParams): Promise<void> {
30-
const comment = context.issue({ body: dedent`@${tagUser}: ${text}` });
31-
context.octokit.issues.createComment(comment);
25+
export default async function createIssueComment(app: Probot, context: any, resource: Resource, { text }: CreateIssueCommentParams): Promise<void> {
26+
if (!(await validateResourceConfig(app, context, resource))) return;
27+
const comment = await context.issue({ body: dedent`${text}` });
28+
await context.octokit.issues.createComment(comment);
3229
}

‎src/call/github-merged-pulls-monitor.ts

+5-1
Original file line numberDiff line numberDiff line change
@@ -11,9 +11,13 @@
1111
// Description : Monitors the CI workflows of merged pull requests, providing metrics that give an overview of whether pull requests were merged without passing CI checks.
1212

1313
import { Probot } from 'probot';
14+
import { Resource } from '../service/resource/resource';
1415
import { OpensearchClient } from '../utility/opensearch/opensearch-client';
16+
import { validateResourceConfig } from '../utility/verification/verify-resource';
17+
18+
export default async function githubMergedPullsMonitor(app: Probot, context: any, resource: Resource): Promise<void> {
19+
if (!(await validateResourceConfig(app, context, resource))) return;
1520

16-
export default async function githubMergedPullsMonitor(app: Probot, context: any): Promise<void> {
1721
const pr = context.payload.pull_request;
1822

1923
if (!pr.merged) {

‎src/call/github-workflow-runs-monitor.ts

+5-1
Original file line numberDiff line numberDiff line change
@@ -13,13 +13,17 @@
1313
// - events : The list of events to monitor and index, from https://docs.github.com/en/actions/writing-workflows/choosing-when-your-workflow-runs/events-that-trigger-workflows.
1414

1515
import { Probot } from 'probot';
16+
import { Resource } from '../service/resource/resource';
1617
import { OpensearchClient } from '../utility/opensearch/opensearch-client';
18+
import { validateResourceConfig } from '../utility/verification/verify-resource';
1719

1820
interface WorkflowRunMonitorArgs {
1921
events: string[];
2022
}
2123

22-
export default async function githubWorkflowRunsMonitor(app: Probot, context: any, { events }: WorkflowRunMonitorArgs): Promise<void> {
24+
export default async function githubWorkflowRunsMonitor(app: Probot, context: any, resource: Resource, { events }: WorkflowRunMonitorArgs): Promise<void> {
25+
if (!(await validateResourceConfig(app, context, resource))) return;
26+
2327
const job = context.payload.workflow_run;
2428

2529
if (!events.includes(job?.event)) {

‎src/call/print-to-console.ts

+9-1
Original file line numberDiff line numberDiff line change
@@ -13,11 +13,19 @@
1313
// - text : (string) the text string to be printed out
1414

1515
import { Probot } from 'probot';
16+
import { Resource } from '../service/resource/resource';
17+
import { validateResourceConfig } from '../utility/verification/verify-resource';
1618

1719
export interface PrintToConsoleParams {
1820
text: string;
1921
}
2022

21-
export default async function printToConsole(app: Probot, context: any, { text }: PrintToConsoleParams): Promise<void> {
23+
export default async function printToConsole(app: Probot, context: any, resource: Resource, { text }: PrintToConsoleParams): Promise<void> {
24+
if (!(await validateResourceConfig(app, context, resource))) return;
2225
app.log.info(text);
2326
}
27+
28+
export async function printToConsoleHelloWorld(app: Probot, context: any, resource: Resource): Promise<void> {
29+
if (!(await validateResourceConfig(app, context, resource))) return;
30+
app.log.info('Hello World');
31+
}

‎src/config/config.ts

+7-5
Original file line numberDiff line numberDiff line change
@@ -14,14 +14,16 @@ import { readFileSync, realpathSync } from 'fs';
1414
import { ResourceData, OperationData } from './types';
1515

1616
export abstract class Config {
17-
protected configType: string;
17+
protected _configType: string;
1818

19-
protected configData: ResourceData | OperationData;
20-
21-
protected configSchema: any;
19+
protected _configData: ResourceData | OperationData;
2220

2321
constructor(configType: string) {
24-
this.configType = configType;
22+
this._configType = configType;
23+
}
24+
25+
protected get configData(): ResourceData | OperationData {
26+
return this._configData;
2527
}
2628

2729
protected static readConfig(filePath: string): any {

‎src/config/operation-config.ts

+5-6
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@ import { OperationData, TaskData } from './types';
1313
import { Config } from './config';
1414

1515
export class OperationConfig extends Config {
16-
private static configSchema = {
16+
private static readonly _configSchema = {
1717
type: 'object',
1818
properties: {
1919
name: {
@@ -62,15 +62,14 @@ export class OperationConfig extends Config {
6262

6363
constructor(configPath: string) {
6464
super('OperationConfig');
65-
this.configData = OperationConfig.readConfig(configPath);
66-
this.configSchema = OperationConfig.configSchema;
67-
OperationConfig.validateConfig(this.configData, this.configSchema);
65+
this._configData = OperationConfig.readConfig(configPath);
66+
OperationConfig.validateConfig(this.configData, OperationConfig._configSchema);
6867
}
6968

7069
private static async _initTasks(taskDataArray: TaskData[]): Promise<Task[]> {
7170
const taskObjArray = taskDataArray.map((taskData) => {
7271
const taskObj = new Task(taskData.call, taskData.args, taskData.name);
73-
console.log(`Setup Task: ${taskObj.getName()}`);
72+
console.log(`Setup Task: ${taskObj.name}`);
7473
return taskObj;
7574
});
7675
return taskObjArray;
@@ -82,7 +81,7 @@ export class OperationConfig extends Config {
8281
(this.configData as OperationData).events,
8382
await OperationConfig._initTasks((this.configData as OperationData).tasks),
8483
);
85-
console.log(`Setup Operation: ${opObj.getName()}`);
84+
console.log(`Setup Operation: ${opObj.name}`);
8685
return opObj;
8786
}
8887
}

‎src/config/resource-config.ts

+13-11
Original file line numberDiff line numberDiff line change
@@ -17,9 +17,11 @@ import { Repository } from '../service/resource/repository';
1717
import { Config } from './config';
1818

1919
export class ResourceConfig extends Config {
20-
private octokit: ProbotOctokit;
20+
private readonly _octokit: ProbotOctokit;
2121

22-
private static configSchema = {
22+
private readonly _additionalResourceContext;
23+
24+
private static readonly _configSchema = {
2325
type: 'object',
2426
properties: {
2527
organizations: {
@@ -77,12 +79,12 @@ export class ResourceConfig extends Config {
7779
required: ['organizations'],
7880
};
7981

80-
constructor(octokit: ProbotOctokit, configPath: string) {
82+
constructor(octokit: ProbotOctokit, configPath: string, additionalResourceContext: boolean) {
8183
super('ResourceConfig');
82-
this.configData = ResourceConfig.readConfig(configPath);
83-
this.configSchema = ResourceConfig.configSchema;
84-
ResourceConfig.validateConfig(this.configData, this.configSchema);
85-
this.octokit = octokit;
84+
this._configData = ResourceConfig.readConfig(configPath);
85+
this._octokit = octokit;
86+
this._additionalResourceContext = additionalResourceContext;
87+
ResourceConfig.validateConfig(this.configData, ResourceConfig._configSchema);
8688
}
8789

8890
private async _initProjects(orgData: OrganizationData): Promise<Map<number, Project>> {
@@ -91,13 +93,13 @@ export class ResourceConfig extends Config {
9193
await Promise.all(
9294
orgData.projects.map(async (projData) => {
9395
const projObj = new Project(orgData.name, projData.number);
94-
await projObj.setContext(this.octokit);
96+
if (this._additionalResourceContext) await projObj.setContext(this._octokit);
9597

9698
if (projData.fields) {
9799
await Promise.all(
98100
projData.fields.map(async (projFieldData) => {
99101
const projFieldObj = new ProjectField(orgData.name, projData.number, projFieldData.name);
100-
await projFieldObj.setContext(this.octokit, projObj.getNodeId());
102+
if (this._additionalResourceContext) await projFieldObj.setContext(this._octokit, projObj.nodeId);
101103
projObj.addField(projFieldObj);
102104
}),
103105
);
@@ -115,7 +117,7 @@ export class ResourceConfig extends Config {
115117
await Promise.all(
116118
orgData.repositories.map(async (repoData) => {
117119
const repoObj = new Repository(orgData.name, repoData.name);
118-
await repoObj.setContext(this.octokit);
120+
if (this._additionalResourceContext) await repoObj.setContext(this._octokit);
119121
repoObjMap.set(repoData.name, repoObj);
120122
}),
121123
);
@@ -131,7 +133,7 @@ export class ResourceConfig extends Config {
131133
const repoObjMap = orgData.repositories ? await this._initRepositories(orgData) : new Map<string, Repository>();
132134

133135
const orgObj = new Organization(orgData.name, projObjMap, repoObjMap);
134-
await orgObj.setContext(this.octokit);
136+
if (this._additionalResourceContext) await orgObj.setContext(this._octokit);
135137

136138
orgObjMap.set(orgData.name, orgObj);
137139
}),

‎src/config/types.ts

+1-1
Original file line numberDiff line numberDiff line change
@@ -41,7 +41,7 @@ export interface OperationData {
4141
export interface TaskData {
4242
name?: string;
4343
call: string;
44-
args: TaskArgData;
44+
args?: TaskArgData;
4545
}
4646

4747
export interface TaskArgData {

‎src/service/operation/operation.ts

+12-12
Original file line numberDiff line numberDiff line change
@@ -10,27 +10,27 @@
1010
import { Task } from './task';
1111

1212
export class Operation {
13-
private name: string; // uid
13+
private readonly _name: string; // uid
1414

15-
private events: string[];
15+
private readonly _events: string[];
1616

17-
private tasks: Task[];
17+
private readonly _tasks: Task[];
1818

1919
constructor(name: string, events: string[], tasks: Task[]) {
20-
this.name = name;
21-
this.events = events;
22-
this.tasks = tasks;
20+
this._name = name;
21+
this._events = events;
22+
this._tasks = tasks;
2323
}
2424

25-
public getName(): string {
26-
return this.name;
25+
public get name(): string {
26+
return this._name;
2727
}
2828

29-
public getEvents(): string[] {
30-
return this.events;
29+
public get events(): string[] {
30+
return this._events;
3131
}
3232

33-
public getTasks(): Task[] {
34-
return this.tasks;
33+
public get tasks(): Task[] {
34+
return this._tasks;
3535
}
3636
}

0 commit comments

Comments
 (0)