diff --git a/README.md b/README.md index ec469f7c..2a453ecd 100644 --- a/README.md +++ b/README.md @@ -3,21 +3,21 @@ - [OpenSearch Continuous Integration](#opensearch-continuous-integration) - [Getting Started](#getting-started) - [Deployment](#deployment) - - [CI Deployment](#ci-deployment) - - [Dev Deployment](#dev-deployment) - - [Executing Optional Tasks](#executing-optional-tasks) - - [SSL Configuration](#ssl-configuration) - - [Setup OpenId Connect (OIDC) via Federate](#setup-openid-connect-oidc-via-federate) - - [Restricting Server Access](#restricting-server-access) - - [Data Retention](#data-retention) - - [Add environment variable](#add-environment-variables) - - [Assume role](#cross-account-assume-role) - - [Mac agents](#mac-agents) - - [Use Production Agents](#use-production-agents) - - [Troubleshooting](#troubleshooting) - - [Main Node](#main-node) - - [Useful commands](#useful-commands) - - [Architecture Overview](#architecture-overview) + - [CI Deployment](#ci-deployment) + - [Dev Deployment](#dev-deployment) + - [Executing Optional Tasks](#executing-optional-tasks) + - [SSL Configuration](#ssl-configuration) + - [Setup Authentication using OpenId Connect (OIDC) or GitHub Authentication](#setup-authentication-using-openid-connect-oidc-or-github-authentication) + - [Restricting Server Access](#restricting-server-access) + - [Data Retention](#data-retention) + - [Add environment variable](#add-environment-variables) + - [Assume role](#cross-account-assume-role) + - [Mac agents](#mac-agents) + - [Use Production Agents](#use-production-agents) + - [Troubleshooting](#troubleshooting) + - [Main Node](#main-node) + - [Useful commands](#useful-commands) + - [Architecture Overview](#architecture-overview) - [Contributing](#contributing) - [Getting Help](#getting-help) - [Code of Conduct](#code-of-conduct) @@ -51,16 +51,16 @@ OpenSearch Continuous Integration is an open source CI system for OpenSearch and 4. Update the `assetsSettings` according to the environment needs such as SSL or strict deployment, see [deployAwsAssetProps](./lib/ci-stack.ts) for details. 5. Deploy using the CI system of your choice. -### Dev Deployment +### Dev Deployment 1. Setup your local machine to credentials to deploy to the AWS Account 2. Deploy the bootstrap stack by running the following command that sets up required resources to create the stacks. [More info](https://docs.aws.amazon.com/cdk/latest/guide/bootstrapping.html) - + `npm run cdk bootstrap -- -c useSsl=false -c runWithOidc=false -c serverAccessType=ipv4 -c restrictServerAccessTo=10.10.10.10/32` - -3. Deploy the ci-config-stack using the following (takes ~1 minute to deploy) - - + +3. Deploy the ci-config-stack using the following (takes ~1 minute to deploy) - + `npm run cdk deploy OpenSearch-CI-Config-Dev -- -c useSsl=false -c runWithOidc=false -c serverAccessType=ipv4 -c restrictServerAccessTo=10.10.10.10/32` - + 4. Locate the secret manager arns in the ci-config-stack outputs for `CASC_RELOAD_TOKEN` and update the secret value ([see docs](https://docs.aws.amazon.com/cli/latest/reference/secretsmanager/put-secret-value.html)) with the password you want to use to reload jenkins configuration. _Do not enclose it in quotes_ ``` $aws secretsmanager put-secret-value \ @@ -71,9 +71,9 @@ $aws secretsmanager put-secret-value \ 5. [Optional](#ssl-configuration) Configure the elements of the config stack for SSL configuration 6. [Optional](#setup-openid-connect-oidc-via-federate) Configure the elements setting up oidc via federate 7. Deploy the ci-stack, takes ~10 minutes to deploy (parameter values depend on step 2 and step 3) - + `npm run cdk deploy OpenSearch-CI-Dev -- -c useSsl=false -c runWithOidc=false -c serverAccessType=ipv4 -c restrictServerAccessTo=10.10.10.10/32` -8. Fetch the key-pair id of `AgentNodeKeyPair` and locate actual value in SSM Parameter Store, it will of the format `/ec2/keypair/{key_pair_id}`. Add the actual value in Secrets Manager to secret named `jenkins-agent-node-key-pair`. This will allow jenkins manager node to be able to connect to agent nodes. +8. Fetch the key-pair id of `AgentNodeKeyPair` and locate actual value in SSM Parameter Store, it will of the format `/ec2/keypair/{key_pair_id}`. Add the actual value in Secrets Manager to secret named `jenkins-agent-node-key-pair`. This will allow jenkins manager node to be able to connect to agent nodes. 9. When OIDC is disabled, this set up will enforce the user to secure jenkins by adding first admin user on deployment. Create admin user and password, fill in all other details like name and email id to start using jenkins. 10. Go to the `OpenSearch-CI-Dev.JenkinsExternalLoadBalancerDns` url returned by CDK output to access the jenkins host. @@ -81,20 +81,20 @@ $aws secretsmanager put-secret-value \ ### Executing Optional Tasks #### Construct Props -| Name | Type | Description | -|------------------------------------------------------------------|:---------|:-----------------------------------------------------------------------------------------| -| [useSsl](#ssl-configuration) | boolean | Should the Jenkins use https | -| [runWithOidc](#setup-openid-connect-oidc-via-federate) | boolean | Should an OIDC provider be installed on Jenkins | -| [restrictServerAccessTo](#restricting-server-access) | Ipeer | Restrict jenkins server access | -| [ignoreResourcesFailures](#ignore-resources-failure) | boolean | Additional verification during deployment and resource startup | -| [adminUsers](#setup-openid-connect-oidc-via-federate) | string[] | List of users with admin access during initial deployment | -| [additionalCommands](#runnning-additional-commands) | string | Additional logic that needs to be run on Master Node. The value has to be path to a file | -| [dataRetention](#data-retention) | boolean | Do you want to retain jenkins jobs and build history | -| [agentAssumeRole](#assume-role) | string | IAM role ARN to be assumed by jenkins agent nodes | -| [envVarsFilePath](#add-environment-variables) | string | Path to file containing env variables in the form of key value pairs | -| [macAgent](#mac-agents) | boolean | Add mac agents to jenkins | -| [useProdAgents](#use-production-agents) | boolean | Should jenkins server use production agents | -| [enableViews](#enable-views) | boolean | Adds Build, Test, Release and Misc views to Jenkins Dashboard . Defaults to false | +| Name | Type | Description | +|-----------------------------------------------------------|:---------|:-----------------------------------------------------------------------------------------| +| [useSsl](#ssl-configuration) | boolean | Should the Jenkins use https | +| [restrictServerAccessTo](#restricting-server-access) | Ipeer | Restrict jenkins server access | +| [authType](#setup-authentication-using-openid-connect-oidc-or-github-authentication) | string | Authentication type for Jenkins login. Acceptable values: github, oidc, default | +| [ignoreResourcesFailures](#ignore-resources-failure) | boolean | Additional verification during deployment and resource startup | +| [adminUsers](#setup-openid-connect-oidc-via-federate) | string[] | List of users with admin access during initial deployment | +| [additionalCommands](#runnning-additional-commands) | string | Additional logic that needs to be run on Master Node. The value has to be path to a file | +| [dataRetention](#data-retention) | boolean | Do you want to retain jenkins jobs and build history | +| [agentAssumeRole](#assume-role) | string | IAM role ARN to be assumed by jenkins agent nodes | +| [envVarsFilePath](#add-environment-variables) | string | Path to file containing env variables in the form of key value pairs | +| [macAgent](#mac-agents) | boolean | Add mac agents to jenkins | +| [useProdAgents](#use-production-agents) | boolean | Should jenkins server use production agents | +| [enableViews](#enable-views) | boolean | Adds Build, Test, Release and Misc views to Jenkins Dashboard . Defaults to false | #### SSL Configuration 1. Locate the secret manager arns in the ci-config-stack outputs 1. Update the secret value ([see docs](https://docs.aws.amazon.com/cli/latest/reference/secretsmanager/put-secret-value.html)) for the `certContentsSecret` with the certificate contents @@ -108,14 +108,18 @@ $aws secretsmanager put-secret-value \ 1. Update the secret value for the `certificateArnSecret` with the certificate arn generated by IAM 1. Update the secret value for `redirectUrlSecret` with a dummy or valid redirect URL. eg: https://dummyJenkinsUrl.com 1. Run with parameter using one of the following (refer [this](#setup-openid-connect-oidc-via-federate) for value of `runWithOidc`) - 1. `npm run cdk deploy OpenSearch-CI-Dev -- -c useSsl=true -c runWithOidc=true` or, - 1. `cdk deploy OpenSearch-CI-Dev -c useSsl=true -c runWithOidc=true` + 1. `npm run cdk deploy OpenSearch-CI-Dev -- -c useSsl=true -c runWithOidc=true` or, + 1. `cdk deploy OpenSearch-CI-Dev -c useSsl=true -c runWithOidc=true` 1. Continue with [next steps](#dev-deployment) -#### Setup OpenId Connect (OIDC) via Federate -1. Locate the secret manager arns in the ci-config-stack outputs -1. Update the secret value ([see docs](https://docs.aws.amazon.com/cli/latest/reference/secretsmanager/put-secret-value.html)) for the `OIDCClientIdSecret` with the credentials as json as follows: - 1. JSON format +#### Setup Authentication using OpenId Connect (OIDC) or GitHub Authentication +There are 3 types of authentication that can be used with this setup. The code modifies the [securityRealm](https://www.jenkins.io/doc/book/security/managing-security/#enabling-security) setting of jenkins. +1. **Default**: Adopts whatever is mentioned in the [initial jenkins.yaml](resources/baseJenkins.yaml) file. Defaults to 'Jenkins Own User Database'. +1. **OpenID Connect**: User any OpenID Connect provider to the jenkins. + Steps: + 1. Locate the secret manager arns in the ci-config-stack outputs + 1. Update the secret value ([see docs](https://docs.aws.amazon.com/cli/latest/reference/secretsmanager/put-secret-value.html)) for the `authConfigValueSecret` with the credentials as json as follows: + JSON format ``` { "clientId": "example_id", @@ -126,16 +130,28 @@ $aws secretsmanager put-secret-value \ "userInfoServerUrl": "https://example.com/userinfo" } ``` - 1. Command Eg: [see docs](https://docs.aws.amazon.com/cli/latest/reference/secretsmanager/put-secret-value.html) +1. **GitHub Authentication**: Use GitHub as Authentication mechanism for jenkins. This set up uses [github-oauth](https://plugins.jenkins.io/github-oauth/) plugin. + Steps: + 1. Create a GitHub OAuth app by following the instructions mentioned on the [plugin info page](https://plugins.jenkins.io/github-oauth/). + 1. Locate the secret manager arns in the ci-config-stack outputs + 1. Update the secret value ([see docs](https://docs.aws.amazon.com/cli/latest/reference/secretsmanager/put-secret-value.html)) for the `authConfigValueSecret` with the credentials as json as follows: + JSON format + ``` + { + "clientID": "example_id", + "clientSecret": "example_password" + } + ``` +1. Command to update secrets Eg: [see docs](https://docs.aws.amazon.com/cli/latest/reference/secretsmanager/put-secret-value.html) ``` $aws secretsmanager put-secret-value \ --secret-id MyTestDatabaseSecret_or_ARN \ --secret-string file://mycreds.json_or_value - ``` + ``` 1. Add additional `adminUsers` for role based authentication according to your needs, see [CIStackProps](./lib/ci-stack.ts) for details. 1. Run with parameter with one of the following (refer [this](#ssl-configuration) for value of `useSsL`) - - 1. `npm run cdk deploy OpenSearch-CI-Dev -- -c runWithOidc=true -c useSsl=true` or, - 1. `cdk deploy OpenSearch-CI-Dev -c runWithOidc=true -c useSsl=true` + 1. `npm run cdk deploy OpenSearch-CI-Dev -- -c authType=oidc/github/default -c useSsl=true` or, + 1. `cdk deploy OpenSearch-CI-Dev -c authType=oidc/github/default -c useSsl=true` 1. Continue with [next steps](#dev-deployment) #### Restricting Server Access @@ -144,10 +160,10 @@ You need to restrict access to your jenkins endpoint (load balancer). Here's how 1. Using command line as below: ``` -npm run cdk synth OpenSearch-CI-Dev -- -c useSsl=false -c runWithOidc=false -c serverAccessType=ipv4 -c restrictServerAccessTo=10.10.10.10/32 +npm run cdk synth OpenSearch-CI-Dev -- -c useSsl=false -c serverAccessType=ipv4 -c restrictServerAccessTo=10.10.10.10/32 ``` Below values are allowed: -| serverAccessType| restrictServerAccessTo| +| serverAccessType| restrictServerAccessTo| |-----------------|:---------| | ipv4 | all (0.0.0.0/0) or any ipv4 CIDR (eg: 10.10.10.10/32) | | ipv6 | all (::/0) or any ipv6 CIDR (eg: 2001:0db8:85a3:0000:0000:8a2e:0370:7334) | @@ -156,7 +172,7 @@ Below values are allowed: 2. You can also update the `restrictServerAccessTo` property in `ciSettings` to your desired [Ipeer](https://docs.aws.amazon.com/cdk/api/v1/docs/@aws-cdk_aws-ec2.IPeer.html). -See [CIStackProps](./lib/ci-stack.ts) for details. + See [CIStackProps](./lib/ci-stack.ts) for details. Example: ``` @@ -171,9 +187,9 @@ This parameter is used to ignore resources failure if the option is available. R #### Data Retention Change in any EC2 config (specially init config) leads to replacement of EC2. The jenkins configuration is managed via code using configuration as code plugin. [More details](https://plugins.jenkins.io/configuration-as-code/). See inital [jenkins.yaml](./resources/baseJenkins.yaml) -If you want to retain all the jobs and its build history, +If you want to retain all the jobs and its build history, 1. Update the `dataRetention` property in `ciSettings` to true (defaults to false) see [CIStackProps](./lib/ci-stack.ts) for details. -This will create an EFS (Elastic File System) and mount it on `/var/lib/jenkins` which will retain all jobs and its build history. + This will create an EFS (Elastic File System) and mount it on `/var/lib/jenkins` which will retain all jobs and its build history. #### Add environment variables Users can add global level environment variables using configuration as code as follows: @@ -199,7 +215,7 @@ To deploy mac agents, as a prerequisites make sure the backend AWS account has d ##### Configuration To configure ec2 Mac agent setup run the stack with `-c macAgent=true`. -Example: +Example: ``` npm run cdk deploy OpenSearch-CI-Dev -- -c useSsl=false -c runWithOidc=false -c macAgent=true ``` @@ -210,7 +226,7 @@ Make sure there is an existing Windows AMI with necessary requirements, see [pac #### Runnning additional commands In cases where you need to run additional logic/commands, such as adding a cron to emit ssl cert expiry metric, you can pass the commands as a script using `additionalCommands` context parameter. -Below sample will write the python script to $HOME/hello-world path on jenkins master node and then execute it once the jenkins master node has been brought up. +Below sample will write the python script to $HOME/hello-world path on jenkins master node and then execute it once the jenkins master node has been brought up. ``` cat << EOF > $HOME/hello-world && chmod 755 $HOME/hello-world && $HOME/hello-world #!/usr/bin/env python3 @@ -229,7 +245,7 @@ npm run cdk deploy OpenSearch-CI-Dev -- -c useSsl=false -c runWithOidc=false -c ``` #### Use Production Agents -Please note that if you have decided to use the provided production jenkins agents then please make sure that you are +Please note that if you have decided to use the provided production jenkins agents then please make sure that you are deploying the stack in US-EAST-1 region as the AMIs used are only publicly available in US-EAST-1 region. If you want to deploy the stack in another region then please make sure you copy the public AMIs used from us-east-1 region to your region of choice and update the new ami-id in agent-nodes.ts file accordingly. @@ -240,7 +256,7 @@ If you do not specify this flag or use `false` then jenkins server will spin up they will be using the latest ami available. #### Enable Views -Views on Jenkins dashboard allows us to classify jobs into different sections. By enabling views, all the jobs will be classified into 4 categories namely Build, Test, Release and Misc. +Views on Jenkins dashboard allows us to classify jobs into different sections. By enabling views, all the jobs will be classified into 4 categories namely Build, Test, Release and Misc. * Build - Any job containing word `build` in it. * Test - Any job containing word `test` in it. * Release - Any job containing word `release` in it. @@ -272,7 +288,7 @@ Built using [AWS Cloud Development Kit](https://aws.amazon.com/cdk/) the configu ## Contributing -See [developer guide](DEVELOPER_GUIDE.md) and [how to contribute to this project](CONTRIBUTING.md). +See [developer guide](DEVELOPER_GUIDE.md) and [how to contribute to this project](CONTRIBUTING.md). ## Getting Help diff --git a/bin/ci-stack.ts b/bin/ci-stack.ts index cc3b4139..f6a67b76 100644 --- a/bin/ci-stack.ts +++ b/bin/ci-stack.ts @@ -25,3 +25,4 @@ const ciStack = new CIStack(app, `OpenSearch-CI-${defaultEnv}`, { const ciCdnStack = new CiCdnStack(app, `OpenSearch-CI-Cdn-${defaultEnv}`, {}); ciCdnStack.addDependency(ciStack); +ciStack.addDependency(ciConfigStack); diff --git a/lib/ci-config-stack.ts b/lib/ci-config-stack.ts index 90e1562c..a55fbf4a 100644 --- a/lib/ci-config-stack.ts +++ b/lib/ci-config-stack.ts @@ -21,7 +21,7 @@ export class CIConfigStack extends Stack { static readonly REDIRECT_URL_SECRET_EXPORT_VALUE: string = 'redirectUrlSecret'; - static readonly OIDC_CONFIGURATION_VALUE_SECRET_EXPORT_VALUE: string = 'OIDCConfigValueSecret'; + static readonly AUTH_CONFIGURATION_VALUE_SECRET_EXPORT_VALUE: string = 'authConfigValueSecret'; static readonly CASC_RELOAD_TOKEN_SECRET_EXPORT_VALUE: string = 'casc'; @@ -43,9 +43,10 @@ export class CIConfigStack extends Stack { const redirectUrlSecret = new Secret(this, 'redirectUrl', { description: 'Redirect url for Jenkins', }); - const OIDCConfigValuesSecret = new Secret(this, 'OIDCConfigValues', { - description: 'OIDC params in JSON format', + const authConfigValuesSecret = new Secret(this, 'authConfigValues', { + description: 'Auth credentials in JSON format', }); + const CascReloadTokenValuesSecret = new Secret(this, 'CascReloadTokenValue', { description: 'Reload token (password) required for configuration as code plugin', }); @@ -75,9 +76,9 @@ export class CIConfigStack extends Stack { exportName: CIConfigStack.REDIRECT_URL_SECRET_EXPORT_VALUE, }); - new CfnOutput(this, 'OIDCConfigValuesSecret', { - value: OIDCConfigValuesSecret.secretArn, - exportName: CIConfigStack.OIDC_CONFIGURATION_VALUE_SECRET_EXPORT_VALUE, + new CfnOutput(this, 'authConfigValuesSecret', { + value: authConfigValuesSecret.secretArn, + exportName: CIConfigStack.AUTH_CONFIGURATION_VALUE_SECRET_EXPORT_VALUE, }); new CfnOutput(this, 'cascSecretValue', { diff --git a/lib/ci-stack.ts b/lib/ci-stack.ts index 41dbb820..c098486c 100644 --- a/lib/ci-stack.ts +++ b/lib/ci-stack.ts @@ -30,8 +30,8 @@ import { JenkinsWAF } from './security/waf'; export interface CIStackProps extends StackProps { /** Should the Jenkins use https */ readonly useSsl?: boolean; - /** Should an OIDC provider be installed on Jenkins. */ - readonly runWithOidc?: boolean; + /** Type of login mechanism to adopt */ + readonly authType?:string; /** Restrict jenkins access to */ readonly restrictServerAccessTo?: IPeer; /** Additional verification during deployment and resource startup. */ @@ -54,7 +54,7 @@ export interface CIStackProps extends StackProps { readonly useProdAgents?: boolean; } -function getServerAccess(serverAccessType: string, restrictServerAccessTo: string) : IPeer { +function getServerAccess(serverAccessType: string, restrictServerAccessTo: string): IPeer { if (typeof restrictServerAccessTo === 'undefined') { throw new Error('restrictServerAccessTo should be specified'); } @@ -100,9 +100,12 @@ export class CIStack extends Stack { const useSsl = useSslParameter === 'true'; - const runWithOidcParameter = `${props?.runWithOidc ?? this.node.tryGetContext('runWithOidc')}`; - if (runWithOidcParameter !== 'true' && runWithOidcParameter !== 'false') { - throw new Error('runWithOidc parameter is required to be set as - true or false'); + let authType = `${props?.authType ?? this.node.tryGetContext('authType')}`; + if (authType.toString() === 'undefined') { + authType = 'default'; + } + if (authType !== 'default' && authType !== 'github' && authType !== 'oidc') { + throw new Error('authType parameter is required to be set as - default, github or oidc'); } let useProdAgents = `${props?.useProdAgents ?? this.node.tryGetContext('useProdAgents')}`; @@ -110,8 +113,6 @@ export class CIStack extends Stack { useProdAgents = 'false'; } - const runWithOidc = runWithOidcParameter === 'true'; - const serverAccessType = this.node.tryGetContext('serverAccessType'); const restrictServerAccessTo = this.node.tryGetContext('restrictServerAccessTo'); const serverAcess = props?.restrictServerAccessTo ?? getServerAccess(serverAccessType, restrictServerAccessTo); @@ -124,7 +125,7 @@ export class CIStack extends Stack { // Setting CfnParameters to record the value in cloudFormation new CfnParameter(this, 'runWithOidc', { description: 'If the jenkins instance should use OIDC + federate', - default: runWithOidc, + default: authType, }); // Setting CfnParameters to record the value in cloudFormation @@ -139,7 +140,7 @@ export class CIStack extends Stack { const importedCertSecretBucketValue = Fn.importValue(`${CIConfigStack.PRIVATE_KEY_SECRET_EXPORT_VALUE}`); const importedArnSecretBucketValue = Fn.importValue(`${CIConfigStack.CERTIFICATE_ARN_SECRET_EXPORT_VALUE}`); const importedRedirectUrlSecretBucketValue = Fn.importValue(`${CIConfigStack.REDIRECT_URL_SECRET_EXPORT_VALUE}`); - const importedOidcConfigValuesSecretBucketValue = Fn.importValue(`${CIConfigStack.OIDC_CONFIGURATION_VALUE_SECRET_EXPORT_VALUE}`); + const importedAuthConfigValuesSecretBucketValue = Fn.importValue(`${CIConfigStack.AUTH_CONFIGURATION_VALUE_SECRET_EXPORT_VALUE}`); const certificateArn = Secret.fromSecretCompleteArn(this, 'certificateArn', importedArnSecretBucketValue.toString()); const importedReloadPasswordSecretsArn = Fn.importValue(`${CIConfigStack.CASC_RELOAD_TOKEN_SECRET_EXPORT_VALUE}`); const listenerCertificate = ListenerCertificate.fromArn(certificateArn.secretValue.toString()); @@ -186,9 +187,9 @@ export class CIStack extends Stack { sslCertChainArn: importedContentsChainBucketValue.toString(), sslCertPrivateKeyContentsArn: importedCertSecretBucketValue.toString(), redirectUrlArn: importedRedirectUrlSecretBucketValue.toString(), - oidcCredArn: importedOidcConfigValuesSecretBucketValue.toString(), + authCredsSecretsArn: importedAuthConfigValuesSecretBucketValue.toString(), useSsl, - runWithOidc, + authType, failOnCloudInitError: props?.ignoreResourcesFailures, adminUsers: props?.adminUsers, agentNodeSecurityGroup: this.securityGroups.agentNodeSG.securityGroupId, diff --git a/lib/compute/auth-config.ts b/lib/compute/auth-config.ts new file mode 100644 index 00000000..fa4f578b --- /dev/null +++ b/lib/compute/auth-config.ts @@ -0,0 +1,120 @@ +/** + * SPDX-License-Identifier: Apache-2.0 + * + * The OpenSearch Contributors require contributions made to + * this file be licensed under the Apache-2.0 license or a + * compatible open source license. + */ + +export class AuthConfig { + private static readonly adminRolePermissions: string[] = [ + 'Overall/Administer', + 'Overall/Read', + 'Job/Move', + 'Job/Build', + 'Job/Read', + 'Job/Delete', + 'Job/Create', + 'Job/Discover', + 'Job/Cancel', + 'Job/Configure', + 'Job Config History/DeleteEntry', + 'Job/Workspace', + 'Credentials/Delete', + 'Credentials/ManageDomains', + 'Credentials/Update', + 'Credentials/View', + 'Credentials/Create', + 'Manage ownership/Nodes', + 'Manage ownership/Jobs', + 'Agent/Configure', + 'Agent/Create', + 'Agent/Build', + 'Agent/Provision', + 'Agent/Connect', + 'Agent/Delete', + 'Agent/Disconnect', + 'Run/Replay', + 'Run/Delete', + 'Run/Update', + 'View/Delete', + 'View/Read', + 'View/Create', + 'View/Configure', + 'SCM/Tag', + ]; + + private static readonly readOnlyRolePermissions: string[] = [ + 'Overall/Read', + 'Job/Read', + 'View/Read', + ]; + + public static addOidcConfigToJenkinsYaml(yamlObject: any, authType: string, admins?: string[]): any { + const jenkinsYaml: any = yamlObject; + let adminUsers: string[] = ['admin']; + const readOnlyUsers: string[] = ['anonymous']; + + if (admins) { + adminUsers = adminUsers.concat(admins); + } + + const oidcConfig: { [x: string]: any; } = { + oic: { + clientId: 'clientId', + clientSecret: 'clientSecret', + authorizationServerUrl: 'http://localhost', + wellKnownOpenIDConfigurationUrl: 'wellKnownOpenIDConfigurationUrl', + tokenServerUrl: 'tokenServerUrl', + userInfoServerUrl: 'userInfoServerUrl', + disableSslVerification: false, + userNameField: 'sub', + escapeHatchEnabled: false, + logoutFromOpenidProvider: true, + postLogoutRedirectUrl: '', + scopes: 'openid', + escapeHatchSecret: 'random', + }, + }; + + const githubAuthConfig: { [x: string]: any; } = { + github: { + githubWebUri: 'https://github.com', + githubApiUri: 'https://api.github.com', + clientID: 'clientID', + clientSecret: 'clientSecret', + oauthScopes: 'read:org,user:email', + }, + }; + + const rolesAndPermissions: { [x: string]: any; } = { + roleBased: { + roles: { + global: [{ + entries: adminUsers.map((user) => ({ user })), + name: 'admin', + pattern: '.*', + permissions: AuthConfig.adminRolePermissions + , + }, + { + entries: readOnlyUsers.map((user) => ({ user })), + name: 'read', + pattern: '.*', + permissions: AuthConfig.readOnlyRolePermissions, + }, + ], + }, + }, + }; + + jenkinsYaml.jenkins.authorizationStrategy = rolesAndPermissions; + + if (authType === 'github') { + jenkinsYaml.jenkins.securityRealm = githubAuthConfig; + } else { + jenkinsYaml.jenkins.securityRealm = oidcConfig; + } + return jenkinsYaml; + } +} diff --git a/lib/compute/jenkins-main-node.ts b/lib/compute/jenkins-main-node.ts index 8fe0d5ca..505b9672 100644 --- a/lib/compute/jenkins-main-node.ts +++ b/lib/compute/jenkins-main-node.ts @@ -36,7 +36,7 @@ import { join } from 'path'; import { CloudwatchAgent } from '../constructs/cloudwatch-agent'; import { AgentNodeConfig, AgentNodeNetworkProps, AgentNodeProps } from './agent-node-config'; import { EnvConfig } from './env-config'; -import { OidcConfig } from './oidc-config'; +import { AuthConfig } from './auth-config'; import { ViewsConfig } from './views'; interface HttpConfigProps { @@ -47,9 +47,9 @@ interface HttpConfigProps { readonly useSsl: boolean; } -interface OidcFederateProps { - readonly oidcCredArn: string; - readonly runWithOidc: boolean; +interface LoginAuthProps { + readonly authCredsSecretsArn: string; + readonly authType: string; readonly adminUsers?: string[]; } @@ -58,7 +58,7 @@ interface DataRetentionProps { readonly efsSG?: SecurityGroup; } -export interface JenkinsMainNodeProps extends HttpConfigProps, OidcFederateProps, AgentNodeNetworkProps, DataRetentionProps { +export interface JenkinsMainNodeProps extends HttpConfigProps, LoginAuthProps, AgentNodeNetworkProps, DataRetentionProps { readonly vpc: Vpc; readonly sg: SecurityGroup; readonly envVarsFilePath: string; @@ -78,8 +78,6 @@ export class JenkinsMainNode { static readonly PRIVATE_KEY_PATH: String = '/etc/ssl/private/test-jenkins.opensearch.org.key'; - static readonly JENKINS_DEFAULT_ID_PASS_PATH: String = '/var/lib/jenkins/secrets/myIdPassDefault'; - private readonly EFS_ID: string; private static ACCOUNT: string; @@ -231,8 +229,14 @@ export class JenkinsMainNode { } public static configElements(stackName: string, stackRegion: string, httpConfigProps: HttpConfigProps, - oidcFederateProps: OidcFederateProps, dataRetentionProps: DataRetentionProps, jenkinsyaml: string, + loginAuthProps: LoginAuthProps, dataRetentionProps: DataRetentionProps, jenkinsyaml: string, reloadPasswordSecretsArn: string, efsId?: string): InitElement[] { + let realm = ''; + if (loginAuthProps.authType === 'github') { + realm = 'github'; + } else if (loginAuthProps.authType === 'oidc') { + realm = 'oic'; + } return [ InitPackage.yum('wget'), InitPackage.yum('cronie'), @@ -272,8 +276,8 @@ export class JenkinsMainNode { // Change hop limit for IMDSv2 from 1 to 2 InitCommand.shellCommand('TOKEN=`curl -f -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 21600"` &&' - + ' instance_id=`curl -f -H "X-aws-ec2-metadata-token: $TOKEN" http://169.254.169.254/latest/meta-data/instance-id` && echo $ami_id &&' - + ` aws ec2 --region ${stackRegion} modify-instance-metadata-options --instance-id $instance_id --http-put-response-hop-limit 2`), + + ' instance_id=`curl -f -H "X-aws-ec2-metadata-token: $TOKEN" http://169.254.169.254/latest/meta-data/instance-id` && echo $ami_id &&' + + ` aws ec2 --region ${stackRegion} modify-instance-metadata-options --instance-id $instance_id --http-put-response-hop-limit 2`), // Jenkins CVE https://www.jenkins.io/security/advisory/2024-01-24/ mitigation InitCommand.shellCommand('mkdir -p /var/lib/jenkins/init.groovy.d'), @@ -427,12 +431,12 @@ export class JenkinsMainNode { InitFile.fromFileInline('/initial_jenkins.yaml', jenkinsyaml), // Make any changes to initial jenkins.yaml - InitCommand.shellCommand(oidcFederateProps.runWithOidc + InitCommand.shellCommand(loginAuthProps.authType !== 'default' // eslint-disable-next-line max-len - ? `var=\`aws --region ${stackRegion} secretsmanager get-secret-value --secret-id ${oidcFederateProps.oidcCredArn} --query SecretString --output text\` && ` + ? `var=\`aws --region ${stackRegion} secretsmanager get-secret-value --secret-id ${loginAuthProps.authCredsSecretsArn} --query SecretString --output text\` && ` + ' varkeys=`echo $var | yq \'keys\' | cut -d "-" -f2 | cut -d " " -f2` &&' // eslint-disable-next-line max-len - + ' for i in $varkeys; do newvalue=`echo $var | yq .$i` && myenv=$newvalue i=$i yq -i \'.jenkins.securityRealm.oic.[env(i)]=env(myenv)\' /initial_jenkins.yaml ; done' + + ` for i in $varkeys; do newvalue=\`echo $var | yq .$i\` && myenv=$newvalue i=$i yq -i '.jenkins.securityRealm.${realm}.[env(i)]=env(myenv)' /initial_jenkins.yaml ; done` : 'echo No changes made to initial_jenkins.yaml with respect to OIDC'), InitCommand.shellCommand('while [[ "$(curl -s -o /dev/null -w \'\'%{http_code}\'\' localhost:8080/api/json?pretty)" != "200" ]]; do sleep 5; done'), @@ -444,11 +448,11 @@ export class JenkinsMainNode { ]; } - public static addConfigtoJenkinsYaml(stack: Stack, jenkinsMainNodeProps: JenkinsMainNodeProps, oidcProps: OidcFederateProps, agentNodeObject: AgentNodeConfig, - props: AgentNodeNetworkProps, agentNode: AgentNodeProps[], macAgent: string): string { + public static addConfigtoJenkinsYaml(stack: Stack, jenkinsMainNodeProps: JenkinsMainNodeProps, loginAuthProps: LoginAuthProps, + agentNodeObject: AgentNodeConfig, props: AgentNodeNetworkProps, agentNode: AgentNodeProps[], macAgent: string): string { let updatedConfig = agentNodeObject.addAgentConfigToJenkinsYaml(stack, agentNode, props, macAgent); - if (oidcProps.runWithOidc) { - updatedConfig = OidcConfig.addOidcConfigToJenkinsYaml(updatedConfig, oidcProps.adminUsers); + if (loginAuthProps.authType !== 'default') { + updatedConfig = AuthConfig.addOidcConfigToJenkinsYaml(updatedConfig, loginAuthProps.authType, loginAuthProps.adminUsers); } if (jenkinsMainNodeProps.envVarsFilePath !== '' && jenkinsMainNodeProps.envVarsFilePath != null) { updatedConfig = EnvConfig.addEnvConfigToJenkinsYaml(updatedConfig, jenkinsMainNodeProps.envVarsFilePath); diff --git a/lib/compute/oidc-config.ts b/lib/compute/oidc-config.ts deleted file mode 100644 index 9bb7fc1b..00000000 --- a/lib/compute/oidc-config.ts +++ /dev/null @@ -1,104 +0,0 @@ -/** - * SPDX-License-Identifier: Apache-2.0 - * - * The OpenSearch Contributors require contributions made to - * this file be licensed under the Apache-2.0 license or a - * compatible open source license. - */ - -export class OidcConfig { - private static readonly adminRolePermissions: string[] = [ - 'Overall/Administer', - 'Overall/Read', - 'Job/Move', - 'Job/Build', - 'Job/Read', - 'Job/Delete', - 'Job/Create', - 'Job/Discover', - 'Job/Cancel', - 'Job/Configure', - 'Job Config History/DeleteEntry', - 'Job/Workspace', - 'Credentials/Delete', - 'Credentials/ManageDomains', - 'Credentials/Update', - 'Credentials/View', - 'Credentials/Create', - 'Manage ownership/Nodes', - 'Manage ownership/Jobs', - 'Agent/Configure', - 'Agent/Create', - 'Agent/Build', - 'Agent/Provision', - 'Agent/Connect', - 'Agent/Delete', - 'Agent/Disconnect', - 'Run/Replay', - 'Run/Delete', - 'Run/Update', - 'View/Delete', - 'View/Read', - 'View/Create', - 'View/Configure', - 'SCM/Tag', - ]; - - private static readonly readOnlyRolePermissions: string[] = [ - 'Overall/Read', - 'Job/Read', - ]; - - public static addOidcConfigToJenkinsYaml(yamlObject: any, admins?: string[]): any { - const jenkinsYaml: any = yamlObject; - let adminUsers: string[] = ['admin']; - const readOnlyUsers: string[] = ['anonymous']; - - if (admins) { - adminUsers = adminUsers.concat(admins); - } - - const oidcConfig: { [x: string]: any; } = { - oic: { - clientId: 'clientId', - clientSecret: 'clientSecret', - authorizationServerUrl: 'http://localhost', - wellKnownOpenIDConfigurationUrl: 'wellKnownOpenIDConfigurationUrl', - tokenServerUrl: 'tokenServerUrl', - userInfoServerUrl: 'userInfoServerUrl', - disableSslVerification: false, - userNameField: 'sub', - escapeHatchEnabled: false, - logoutFromOpenidProvider: true, - postLogoutRedirectUrl: '', - scopes: 'openid', - escapeHatchSecret: 'random', - }, - }; - const rolesAndPermissions: { [x: string]: any; } = { - roleBased: { - roles: { - global: [{ - entries: adminUsers.map((user) => ({ user })), - name: 'admin', - pattern: '.*', - permissions: OidcConfig.adminRolePermissions - , - }, - { - entries: readOnlyUsers.map((user) => ({ user })), - name: 'read', - pattern: '.*', - permissions: OidcConfig.readOnlyRolePermissions, - }, - - ], - }, - }, - }; - - jenkinsYaml.jenkins.authorizationStrategy = rolesAndPermissions; - jenkinsYaml.jenkins.securityRealm = oidcConfig; - return jenkinsYaml; - } -} diff --git a/package-lock.json b/package-lock.json index 3747ef6c..3554af78 100644 --- a/package-lock.json +++ b/package-lock.json @@ -1,12 +1,12 @@ { "name": "opensearch-ci-stack", - "version": "0.3.0", + "version": "1.0.0", "lockfileVersion": 2, "requires": true, "packages": { "": { "name": "opensearch-ci-stack", - "version": "0.3.0", + "version": "1.0.0", "dependencies": { "@typescript-eslint/eslint-plugin": "^4.31.1", "@typescript-eslint/parser": "^4.31.1", diff --git a/package.json b/package.json index 81224626..62bd63dc 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "opensearch-ci-stack", - "version": "0.3.0", + "version": "1.0.0", "bin": { "ci": "bin/ci-stack.js" }, diff --git a/test/compute/auth-config.test.ts b/test/compute/auth-config.test.ts new file mode 100644 index 00000000..754df0ed --- /dev/null +++ b/test/compute/auth-config.test.ts @@ -0,0 +1,96 @@ +/** + * SPDX-License-Identifier: Apache-2.0 + * + * The OpenSearch Contributors require contributions made to + * this file be licensed under the Apache-2.0 license or a + * compatible open source license. + */ + +import { readFileSync } from 'fs'; +import { load } from 'js-yaml'; +import { JenkinsMainNode } from '../../lib/compute/jenkins-main-node'; +import { AuthConfig } from '../../lib/compute/auth-config'; + +describe('Test authType OIDC', () => { + // WHEN + const testYaml = 'test/data/jenkins.yaml'; + + const oidcConfig = { + authorizationServerUrl: 'http://localhost', + clientId: 'clientId', + disableSslVerification: true, + emailFieldName: 'emailFieldName', + escapeHatchEnabled: true, + escapeHatchGroup: 'escapeHatchGroup', + escapeHatchUsername: 'escapeHatchUsername', + fullNameFieldName: 'fullNameFieldName', + groupsFieldName: 'groupsFieldName', + logoutFromOpenidProvider: true, + scopes: 'scopes', + tokenServerUrl: 'http://localhost', + userNameField: 'userNameField', + }; + const admins = ['admin1', 'admin2']; + const jenkinsYaml = load(readFileSync(JenkinsMainNode.BASE_JENKINS_YAML_PATH, 'utf-8')); + AuthConfig.addOidcConfigToJenkinsYaml(jenkinsYaml, 'oidc', admins); + const yml: any = load(readFileSync(testYaml, 'utf-8')); + + // THEN + test('Verify oidcConfig', async () => { + const addedOidcConfig = yml.jenkins.securityRealm.oic; + expect(addedOidcConfig).toEqual(oidcConfig); + }); + + test('Verify oidcConfig', async () => { + const adminRole = yml.jenkins.authorizationStrategy.roleBased.roles.global[0].assignments; + expect(adminRole).toEqual(['admin', 'admin1', 'admin2']); + }); +}); + +describe('Test authType github', () => { + // WHEN + const testYaml = 'test/data/github_auth.yaml'; + const admins = ['foo', 'bar']; + const jenkinsYaml = load(readFileSync(JenkinsMainNode.BASE_JENKINS_YAML_PATH, 'utf-8')); + AuthConfig.addOidcConfigToJenkinsYaml(jenkinsYaml, 'github', admins); + const yml: any = load(readFileSync(testYaml, 'utf-8')); + const globalRoles = yml.jenkins.authorizationStrategy.roleBased.roles.global; + + const githubAuthConfig: { [x: string]: any; } = { + githubWebUri: 'https://github.com', + githubApiUri: 'https://api.github.com', + clientID: 'clientID', + clientSecret: 'clientSecret', + oauthScopes: 'read:org,user:email', + }; + + // THEN + test('Verify github config', async () => { + const addedGithubConfig = yml.jenkins.securityRealm.github; + expect(addedGithubConfig).toEqual(githubAuthConfig); + }); + + test('Verify roles', async () => { + const roleNames = globalRoles.map((role: any) => role.name); + expect(roleNames).toEqual(['admin', 'read']); + }); + + test('Verify github admins', async () => { + // Find the admin role + const adminRole = globalRoles.find((role: any) => role.name === 'admin'); + + // Check admin users + const adminUsers = adminRole.entries.map((entry: any) => entry.user); + expect(adminUsers).toEqual(['bar', 'foo']); + }); + + test('Verify read only', async () => { + // Find the read role + const readRole = globalRoles.find((role: any) => role.name === 'read'); + expect(readRole).toBeTruthy(); + + // Check read users + const readUsers = readRole.entries.map((entry: any) => entry.user); + expect(readUsers).toEqual(['anonymous']); + }); +}); diff --git a/test/compute/jenkins-main-node.test.ts b/test/compute/jenkins-main-node.test.ts index 55913fd8..f1215e93 100644 --- a/test/compute/jenkins-main-node.test.ts +++ b/test/compute/jenkins-main-node.test.ts @@ -17,8 +17,8 @@ describe('JenkinsMainNode Config Elements', () => { sslCertChainArn: 'ARN:DEF', useSsl: true, }, { - oidcCredArn: 'ABC:EFG', - runWithOidc: true, + authCredsSecretsArn: 'ABC:EFG', + authType: 'oidc', }, { dataRetention: true, }, 'test/data/jenkins.yaml', diff --git a/test/compute/oidc-config.test.ts b/test/compute/oidc-config.test.ts deleted file mode 100644 index 80207d95..00000000 --- a/test/compute/oidc-config.test.ts +++ /dev/null @@ -1,48 +0,0 @@ -/** - * SPDX-License-Identifier: Apache-2.0 - * - * The OpenSearch Contributors require contributions made to - * this file be licensed under the Apache-2.0 license or a - * compatible open source license. - */ - -import { readFileSync } from 'fs'; -import { load } from 'js-yaml'; -import { JenkinsMainNode } from '../../lib/compute/jenkins-main-node'; -import { OidcConfig } from '../../lib/compute/oidc-config'; - -describe('JenkinsMainNode Config Elements', () => { - // WHEN - const testYaml = 'test/data/jenkins.yaml'; - - const oidcConfig = { - authorizationServerUrl: 'http://localhost', - clientId: 'clientId', - disableSslVerification: true, - emailFieldName: 'emailFieldName', - escapeHatchEnabled: true, - escapeHatchGroup: 'escapeHatchGroup', - escapeHatchUsername: 'escapeHatchUsername', - fullNameFieldName: 'fullNameFieldName', - groupsFieldName: 'groupsFieldName', - logoutFromOpenidProvider: true, - scopes: 'scopes', - tokenServerUrl: 'http://localhost', - userNameField: 'userNameField', - }; - const admins = ['admin1', 'admin2']; - const jenkinsYaml = load(readFileSync(JenkinsMainNode.BASE_JENKINS_YAML_PATH, 'utf-8')); - OidcConfig.addOidcConfigToJenkinsYaml(jenkinsYaml, admins); - const yml: any = load(readFileSync(testYaml, 'utf-8')); - - // THEN - test('Verify oidcConfig', async () => { - const addedOidcConfig = yml.jenkins.securityRealm.oic; - expect(addedOidcConfig).toEqual(oidcConfig); - }); - - test('Verify oidcConfig', async () => { - const adminRole = yml.jenkins.authorizationStrategy.roleBased.roles.global[0].assignments; - expect(adminRole).toEqual(['admin', 'admin1', 'admin2']); - }); -}); diff --git a/test/data/github_auth.yaml b/test/data/github_auth.yaml new file mode 100644 index 00000000..4682adb1 --- /dev/null +++ b/test/data/github_auth.yaml @@ -0,0 +1,572 @@ +jenkins: + agentProtocols: + - "JNLP4-connect" + - "Ping" + authorizationStrategy: + roleBased: + roles: + global: + - entries: + - user: "bar" + - user: "foo" + name: "admin" + pattern: ".*" + permissions: + - "Job/Move" + - "Job/Build" + - "Credentials/Delete" + - "Credentials/ManageDomains" + - "View/Create" + - "Agent/Configure" + - "Job/Read" + - "Credentials/Update" + - "Agent/Create" + - "Manage ownership/Nodes" + - "Job/Delete" + - "Agent/Build" + - "View/Configure" + - "Agent/Provision" + - "SCM/Tag" + - "Job/Create" + - "Job/Discover" + - "Manage ownership/Jobs" + - "Credentials/View" + - "Agent/Connect" + - "Agent/Delete" + - "Run/Replay" + - "Agent/Disconnect" + - "Run/Delete" + - "Job/Cancel" + - "Overall/Read" + - "Run/Update" + - "Credentials/Create" + - "Overall/Administer" + - "View/Delete" + - "Job/Configure" + - "Job Config History/DeleteEntry" + - "Job/Workspace" + - "View/Read" + - entries: + - user: "anonymous" + name: "read" + pattern: ".*" + permissions: + - "Overall/Read" + - "Job/Read" + clouds: + - amazonEC2: + name: "Amazon_ec2_cloud" + region: "us-west-2" + sshKeysCredentialsId: "jenkins-agent-node-key-pair" + templates: + - ami: "ami-046b5b8111c19b3ac" + amiType: + unixData: + sshPort: "22" + associatePublicIp: false + connectBySSHProcess: false + connectionStrategy: PRIVATE_IP + customDeviceMapping: "/dev/xvda=:300:true:::encrypted" + deleteRootOnTermination: true + description: "jenkinsAgentNode-Jenkins-Default-Agent-X64-C5xlarge-Single-Host" + ebsEncryptRootVolume: ENCRYPTED + ebsOptimized: false + hostKeyVerificationStrategy: 'OFF' + iamInstanceProfile: "arn:aws:iam::529213360297:instance-profile/OpenSearch-CI-lol-JenkinsAgentNodeInstanceProfile-ntMjuYvZaWaR" + idleTerminationMinutes: "60" + initScript: "sudo amazon-linux-extras install java-openjdk11 -y && sudo yum\ + \ install -y cmake python3 python3-pip && sudo yum groupinstall -y 'Development\ + \ Tools' && sudo ln -sfn `which pip3` /usr/bin/pip && pip3 install pipenv\ + \ && sudo ln -s ~/.local/bin/pipenv /usr/local/bin" + javaPath: "java" + labelString: "Jenkins-Default-Agent-X64-C5xlarge-Single-Host" + launchTimeoutStr: "300" + maxTotalUses: -1 + metadataEndpointEnabled: true + metadataHopsLimit: 2 + metadataSupported: true + metadataTokensRequired: true + minimumNumberOfInstances: 0 + minimumNumberOfSpareInstances: 1 + mode: EXCLUSIVE + monitoring: true + nodeProperties: + - envVars: + env: + - key: "JAVA11_HOME" + value: "/usr/lib/jvm/temurin-11-jdk-amd64" + - key: "JAVA14_HOME" + value: "/usr/lib/jvm/adoptopenjdk-14-amd64" + - key: "JAVA17_HOME" + value: "/usr/lib/jvm/temurin-17-jdk-amd64" + - key: "JAVA19_HOME" + value: "/usr/lib/jvm/temurin-19-jdk-amd64" + - key: "JAVA20_HOME" + value: "/usr/lib/jvm/temurin-20-jdk-amd64" + - key: "JAVA21_HOME" + value: "/usr/lib/jvm/temurin-21-jdk-amd64" + - key: "JAVA23_HOME" + value: "/usr/lib/jvm/temurin-23-jdk-amd64" + - key: "JAVA8_HOME" + value: "/usr/lib/jvm/temurin-8-jdk-amd64" + - key: "JENKINS_HOME_PATH" + value: "/home/ec2-user" + numExecutors: 1 + remoteAdmin: "ec2-user" + remoteFS: "/home/ec2-user" + securityGroups: "sg-0a7de99890e29657e" + stopOnTerminate: false + subnetId: "subnet-0fd55c2bd295b5216" + t2Unlimited: false + tags: + - name: "Name" + value: "OpenSearch-CI-lol/AgentNode/Jenkins-Default-Agent-X64-C5xlarge-Single-Host" + - name: "type" + value: "jenkinsAgentNode-Jenkins-Default-Agent-X64-C5xlarge-Single-Host" + tenancy: Default + type: C54xlarge + useEphemeralDevices: false + - ami: "ami-0334ab986e1816201" + amiType: + unixData: + sshPort: "22" + associatePublicIp: false + connectBySSHProcess: false + connectionStrategy: PRIVATE_IP + customDeviceMapping: "/dev/xvda=:300:true:::encrypted" + deleteRootOnTermination: true + description: "jenkinsAgentNode-Jenkins-Default-Agent-ARM64-C5xlarge-Single-Host" + ebsEncryptRootVolume: ENCRYPTED + ebsOptimized: false + hostKeyVerificationStrategy: 'OFF' + iamInstanceProfile: "arn:aws:iam::529213360297:instance-profile/OpenSearch-CI-lol-JenkinsAgentNodeInstanceProfile-ntMjuYvZaWaR" + idleTerminationMinutes: "60" + initScript: "sudo amazon-linux-extras install java-openjdk11 -y && sudo yum\ + \ install -y cmake python3 python3-pip && sudo yum groupinstall -y 'Development\ + \ Tools' && sudo ln -sfn `which pip3` /usr/bin/pip && pip3 install pipenv\ + \ && sudo ln -s ~/.local/bin/pipenv /usr/local/bin" + javaPath: "java" + labelString: "Jenkins-Default-Agent-ARM64-C5xlarge-Single-Host" + launchTimeoutStr: "300" + maxTotalUses: -1 + metadataEndpointEnabled: true + metadataHopsLimit: 2 + metadataSupported: true + metadataTokensRequired: true + minimumNumberOfInstances: 0 + minimumNumberOfSpareInstances: 1 + mode: EXCLUSIVE + monitoring: true + nodeProperties: + - envVars: + env: + - key: "JAVA11_HOME" + value: "/usr/lib/jvm/temurin-11-jdk-amd64" + - key: "JAVA14_HOME" + value: "/usr/lib/jvm/adoptopenjdk-14-amd64" + - key: "JAVA17_HOME" + value: "/usr/lib/jvm/temurin-17-jdk-amd64" + - key: "JAVA19_HOME" + value: "/usr/lib/jvm/temurin-19-jdk-amd64" + - key: "JAVA20_HOME" + value: "/usr/lib/jvm/temurin-20-jdk-amd64" + - key: "JAVA21_HOME" + value: "/usr/lib/jvm/temurin-21-jdk-amd64" + - key: "JAVA23_HOME" + value: "/usr/lib/jvm/temurin-23-jdk-amd64" + - key: "JAVA8_HOME" + value: "/usr/lib/jvm/temurin-8-jdk-amd64" + - key: "JENKINS_HOME_PATH" + value: "/home/ec2-user" + numExecutors: 1 + remoteAdmin: "ec2-user" + remoteFS: "/home/ec2-user" + securityGroups: "sg-0a7de99890e29657e" + stopOnTerminate: false + subnetId: "subnet-0fd55c2bd295b5216" + t2Unlimited: false + tags: + - name: "Name" + value: "OpenSearch-CI-lol/AgentNode/Jenkins-Default-Agent-ARM64-C5xlarge-Single-Host" + - name: "type" + value: "jenkinsAgentNode-Jenkins-Default-Agent-ARM64-C5xlarge-Single-Host" + tenancy: Default + type: C6g4xlarge + useEphemeralDevices: false + useInstanceProfileForCredentials: true + crumbIssuer: + standard: + excludeClientIPFromCrumb: false + disableRememberMe: false + labelAtoms: + - name: "built-in" + - name: "main-node" + labelString: "main-node" + log: + recorders: + - loggers: + - level: "FINE" + name: "org.jenkinsci.plugins.workflow.job.WorkflowRun" + name: "workflowRun" + markupFormatter: + rawHtml: + disableSyntaxHighlighting: true + mode: EXCLUSIVE + myViewsTabBar: "standard" + nodeMonitors: + - diskSpaceMonitor: + freeSpaceThreshold: "1GB" + - tmpSpace: + freeSpaceThreshold: "1GB" + numExecutors: 4 + primaryView: + all: + name: "all" + projectNamingStrategy: "standard" + quietPeriod: 5 + remotingSecurity: + enabled: true + scmCheckoutRetryCount: 0 + securityRealm: + github: + clientID: "clientID" + clientSecret: "clientSecret" + githubApiUri: "https://api.github.com" + githubWebUri: "https://github.com" + oauthScopes: "read:org,user:email" + slaveAgentPort: 50000 + updateCenter: + sites: + - id: "default" + url: "https://updates.jenkins.io/update-center.json" + views: + - all: + name: "all" + viewsTabBar: "standard" +support: + automatedBundleConfiguration: + componentIds: + - "AgentsConfigFile" + - "ConfigFileComponent" + - "OtherConfigFilesComponent" + - "AboutBrowser" + - "AboutJenkins" + - "AboutUser" + - "AdministrativeMonitors" + - "AgentProtocols" + - "BuildQueue" + - "CustomLogs" + - "DumpExportTable" + - "EnvironmentVariables" + - "FileDescriptorLimit" + - "GCLogs" + - "HeapUsageHistogram" + - "ItemsContent" + - "AgentsJVMProcessSystemMetricsContents" + - "MasterJVMProcessSystemMetricsContents" + - "JenkinsLogs" + - "LoadStats" + - "LoggerManager" + - "Metrics" + - "NetworkInterfaces" + - "NodeMonitors" + - "OtherLogs" + - "ReverseProxy" + - "RootCAs" + - "RunningBuilds" + - "SlaveCommandStatistics" + - "SlaveLaunchLogs" + - "SlaveLogs" + - "AgentsSystemConfiguration" + - "MasterSystemConfiguration" + - "SystemProperties" + - "TaskLogs" + - "ThreadDumps" + - "UpdateCenter" + - "UserCount" + - "SlowRequestComponent" + - "HighLoadComponent" + - "DeadlockRequestComponent" + - "PipelineTimings" + - "PipelineThreadDump" + enabled: true + period: 1 +globalCredentialsConfiguration: + configuration: + providerFilter: "none" + typeFilter: "none" +appearance: + loginTheme: + useDefaultTheme: true + prism: + theme: PRISM +security: + anonymizeSupportBundle: + enabled: false + apiToken: + creationOfLegacyTokenEnabled: false + tokenGenerationOnCreationEnabled: false + usageStatisticsEnabled: true + copyartifact: + mode: PRODUCTION + cps: + hideSandbox: false + gitHooks: + allowedOnAgents: false + allowedOnController: false + gitHostKeyVerificationConfiguration: + sshHostKeyVerificationStrategy: "knownHostsFileVerificationStrategy" + globalJobDslSecurityConfiguration: + useScriptSecurity: true + sSHD: + port: -1 + scriptApproval: + forceSandbox: false +unclassified: + ansiColorBuildWrapper: + colorMaps: + - black: "#000000" + blackB: "#4C4C4C" + blue: "#1E90FF" + blueB: "#4682B4" + cyan: "#00CDCD" + cyanB: "#00FFFF" + green: "#00CD00" + greenB: "#00FF00" + magenta: "#CD00CD" + magentaB: "#FF00FF" + name: "xterm" + red: "#CD0000" + redB: "#FF0000" + white: "#E5E5E5" + whiteB: "#FFFFFF" + yellow: "#CDCD00" + yellowB: "#FFFF00" + - black: "#000000" + blackB: "#555555" + blue: "#0000AA" + blueB: "#5555FF" + cyan: "#00AAAA" + cyanB: "#55FFFF" + defaultBackground: 0 + defaultForeground: 7 + green: "#00AA00" + greenB: "#55FF55" + magenta: "#AA00AA" + magentaB: "#FF55FF" + name: "vga" + red: "#AA0000" + redB: "#FF5555" + white: "#AAAAAA" + whiteB: "#FFFFFF" + yellow: "#AA5500" + yellowB: "#FFFF55" + - black: "black" + blackB: "black" + blue: "blue" + blueB: "blue" + cyan: "cyan" + cyanB: "cyan" + green: "green" + greenB: "green" + magenta: "magenta" + magentaB: "magenta" + name: "css" + red: "red" + redB: "red" + white: "white" + whiteB: "white" + yellow: "yellow" + yellowB: "yellow" + - black: "#2E3436" + blackB: "#2E3436" + blue: "#3465A4" + blueB: "#3465A4" + cyan: "#06989A" + cyanB: "#06989A" + defaultBackground: 0 + defaultForeground: 7 + green: "#4E9A06" + greenB: "#4E9A06" + magenta: "#75507B" + magentaB: "#75507B" + name: "gnome-terminal" + red: "#CC0000" + redB: "#CC0000" + white: "#D3D7CF" + whiteB: "#D3D7CF" + yellow: "#C4A000" + yellowB: "#C4A000" + audit-trail: + displayUserName: false + logBuildCause: true + logCredentialsUsage: true + pattern: ".*/(?:configSubmit|doDelete|postBuildResult|enable|disable|cancelQueue|stop|toggleLogKeep|doWipeOutWorkspace|createItem|createView|toggleOffline|cancelQuietDown|quietDown|restart|exit|safeExit)/?.*" + awsCredentialsProvider: + cache: false + client: + credentialsProvider: "default" + bitbucketEndpointConfiguration: + endpoints: + - bitbucketCloudEndpoint: + enableCache: false + manageHooks: false + repositoriesCacheDuration: 0 + teamCacheDuration: 0 + buildDiscarders: + configuredBuildDiscarders: + - "jobBuildDiscarder" + buildStepOperation: + enabled: false + buildTimestamp: + enableBuildTimestamp: true + pattern: "yyyy-MM-dd HH:mm:ss z" + timezone: "Etc/UTC" + descriptionSetterWrapper: + disableTokens: false + email-ext: + adminRequiredForTemplateTesting: false + allowUnregisteredEnabled: false + charset: "UTF-8" + debugMode: false + defaultBody: |- + $PROJECT_NAME - Build # $BUILD_NUMBER - $BUILD_STATUS: + + Check console output at $BUILD_URL to view the results. + defaultSubject: "$PROJECT_NAME - Build # $BUILD_NUMBER - $BUILD_STATUS!" + defaultTriggerIds: + - "hudson.plugins.emailext.plugins.trigger.FailureTrigger" + maxAttachmentSize: -1 + maxAttachmentSizeMb: -1 + precedenceBulk: false + watchingEnabled: false + enrichedSummaryConfig: + enrichedSummaryEnabled: false + httpClientDelayBetweenRetriesInSeconds: 1 + httpClientMaxRetries: 3 + httpClientTimeoutInSeconds: 1 + fingerprints: + fingerprintCleanupDisabled: false + storage: "file" + ghprbTrigger: + autoCloseFailedPullRequests: false + cron: "H/5 * * * *" + extensions: + - ghprbSimpleStatus: + addTestResults: false + showMatrixStatus: false + githubAuth: + - description: "Anonymous connection" + id: "d4456c70-9c5e-4b40-bee4-e1ebb693153b" + serverAPIUrl: "https://api.github.com" + manageWebhooks: true + okToTestPhrase: ".*ok\\W+to\\W+test.*" + retestPhrase: ".*test\\W+this\\W+please.*" + skipBuildPhrase: ".*\\[skip\\W+ci\\].*" + useComments: false + useDetailedComments: false + whitelistPhrase: ".*add\\W+to\\W+whitelist.*" + gitHubConfiguration: + apiRateLimitChecker: ThrottleForNormalize + gitHubPluginConfig: + hookUrl: "http://localhost:8080/github-webhook/" + globalTimeOutConfiguration: + operations: + - "abortOperation" + overwriteable: false + injectionConfig: + allowUntrusted: false + checkForBuildAgentErrors: false + enabled: false + enforceUrl: false + injectCcudExtension: false + injectMavenExtension: false + ivyBuildTrigger: + extendedVersionMatching: false + jobConfigHistory: + excludePattern: "queue\\.xml|nodeMonitors\\.xml|UpdateCenter\\.xml|global-build-stats|LockableResourcesManager\\\ + .xml|MilestoneStep\\.xml|cloudbees-disk-usage-simple\\.xml" + saveModuleConfiguration: false + showBuildBadges: "always" + showChangeReasonCommentWindow: true + skipDuplicateHistory: true + junitTestResultStorage: + storage: "file" + location: + adminAddress: "address not configured yet " + mailer: + charset: "UTF-8" + useSsl: false + useTls: false + pipelineStepsAWS: + enableCredentialsFromNode: false + pollSCM: + pollingThreadCount: 10 + scmGit: + addGitTagAction: false + allowSecondFetch: false + createAccountBasedOnEmail: false + disableGitToolChooser: false + hideCredentials: false + showEntireCommitSummaryInChanges: false + useExistingAccountWithSameEmail: false + timestamper: + allPipelines: false + elapsedTimeFormat: "''HH:mm:ss.S' '" + systemTimeFormat: "''HH:mm:ss' '" + upstream: + globalUpstreamFilterStrategy: UseOldest + whitelist: + enabled: false +tool: + git: + installations: + - home: "git" + name: "Default" + jdk: + installations: + - name: "openjdk-8" + properties: + - installSource: + installers: + - adoptOpenJdkInstaller: + id: "jdk8u332-b09" + - name: "openjdk-11" + properties: + - installSource: + installers: + - adoptOpenJdkInstaller: + id: "jdk-11.0.15+10" + - name: "openjdk-17" + properties: + - installSource: + installers: + - adoptOpenJdkInstaller: + id: "jdk-17.0.3+7" + - name: "openjdk-19" + properties: + - installSource: + installers: + - adoptOpenJdkInstaller: + id: "jdk-19.0.1+10" + - name: "openjdk-20" + properties: + - installSource: + installers: + - adoptOpenJdkInstaller: + id: "jdk-20.0.1+9" + - name: "openjdk-21" + properties: + - installSource: + installers: + - adoptOpenJdkInstaller: + id: "jdk-21.0.1+12" + mavenGlobalConfig: + globalSettingsProvider: "standard" + settingsProvider: "standard" + powerShellInstallation: + installations: + - home: "powershell.exe" + name: "DefaultWindows" + - home: "pwsh" + name: "DefaultLinux"