Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding Jenkins CI to GaeaC6 using role account #3389

Open
wants to merge 22 commits into
base: develop
Choose a base branch
from

Conversation

TerrenceMcGuinness-NOAA
Copy link
Collaborator

@TerrenceMcGuinness-NOAA TerrenceMcGuinness-NOAA commented Feb 25, 2025

Description

This PR has the necessary updates to the CI scripts in Jenkins for running on Gaea C6.
The GaeaC6-EMC Node is all ready configured and connected to the Gaea C6 Head Node gaea64

PyGithub is not in Spack Stack yet and is needed in support of the GitHub logging and gist creation python codes. This python module will be installed in $HOME/.local in the role accounts until the module is added to Spack Stack.

Same with GitHub CLI (which will be supported in Spack Stack 1.9) it will be installed in ${ROLE_HOME}/utils/bin which should be in role account's $PATH where ROLE_HOME is at the same level as fix and data.

This is setup to run in the role account with the following added directories and utils:

/gpfs/f6/drsa-precip3/proj-shared/global/CI

Location on disk where the CI test cases actually build and run specified in Jenkinsfile

└── GFS_CI_CD
    ├── global-workflow
    │   └── workflow
    ├── Bash
    ├── GitLab
    └── Jenkins
        ├── agent
        └── workspace

Top level CI directories (this will file structure be duplicated for each role account)

GFS_CI_CD

CI system directory for Jenkins launching scripts and in support of BASH CI

GFS_CI_CD/global-workflow

Directory where the JAVA agent is launched from

Jenkins/agent

Jenkins work space (system level workspace)

Jenkins/workspace

GitHub CLI (gh utility):

Location where gh was installed (pending when supporting Spack Stack 1.9)

gpfs/f6/drsa-precip3/world-shared/role.glopara/utils/bin/gh

soft link to gh utility for Jenkins to find

/ncrc/home1/role.glopara/bin

Resolves #3375

Type of change

  • Bug fix (fixes something broken)
  • New feature (adds functionality)
  • Maintenance (code refactor, clean-up, new CI test, etc.)

Change characteristics

  • Is this a breaking change (a change in existing functionality)? NO
  • Does this change require a documentation update? NO
  • Does this change require an update to any of the following submodules? NO (If YES, please add a link to any PRs that are pending.)
    • EMC verif-global
    • GDAS
    • GFS-utils
    • GSI
    • GSI-monitor
    • GSI-utils
    • UFS-utils
    • UFS-weather-model
    • wxflow

How has this been tested?

This PR can be tested in place after Dave's PR is merged for cycling on GaeaC6

@TerrenceMcGuinness-NOAA TerrenceMcGuinness-NOAA changed the title Ci jenkins role gaea6 Adding Jenkins CI to GaeaC6 using role account Feb 25, 2025
@TerrenceMcGuinness-NOAA TerrenceMcGuinness-NOAA added the CI/CD Issue related to CI/CD label Feb 25, 2025
@TerrenceMcGuinness-NOAA TerrenceMcGuinness-NOAA marked this pull request as draft February 25, 2025 19:51
…tarting with parameters in the Jenkins controler
@TerrenceMcGuinness-NOAA TerrenceMcGuinness-NOAA added the CI-Gaeac6-Ready **CM use only** PR is ready for CI testing on Gaea C6 label Feb 27, 2025
@emcbot emcbot added CI-Gaeac6-Building **Bot use only** CI testing is cloning/building on Gaea C6 CI-Gaeac6-Failed **Bot use only** CI testing on Gaea C6 for this PR has failed and removed CI-Gaeac6-Ready **CM use only** PR is ready for CI testing on Gaea C6 CI-Gaeac6-Building **Bot use only** CI testing is cloning/building on Gaea C6 labels Feb 27, 2025
@TerrenceMcGuinness-NOAA TerrenceMcGuinness-NOAA marked this pull request as ready for review February 27, 2025 20:14
@emcbot
Copy link

emcbot commented Feb 27, 2025

Build FAILED on Gaeac6 in Build# 2 with error logs:

/gpfs/f6/drsa-precip3/proj-shared/global/CI/3389/global-workflow/sorc/logs/gdas.log

@TerrenceMcGuinness-NOAA TerrenceMcGuinness-NOAA added CI-Gaeac6-Ready **CM use only** PR is ready for CI testing on Gaea C6 and removed CI-Gaeac6-Failed **Bot use only** CI testing on Gaea C6 for this PR has failed labels Feb 27, 2025
@emcbot emcbot added CI-Gaeac6-Building **Bot use only** CI testing is cloning/building on Gaea C6 and removed CI-Gaeac6-Ready **CM use only** PR is ready for CI testing on Gaea C6 labels Feb 27, 2025
@emcbot
Copy link

emcbot commented Feb 27, 2025

Build FAILED on Gaeac6 in Build# 3 with error logs:
```
/gpfs/f6/drsa-precip3/proj-shared/global/CI/3389/global-workflow/sorc/logs/gdas.log
```

Follow link here to view the contents of the above file(s): [(link)](2025-02-27 16:27:05,418 - INFO - root : Using TOKEN from gh CLI tool.
2025-02-27 16:27:05,420 - DEBUG - urllib3.connectionpool: Starting new HTTPS connection (1): api.github.com:443
2025-02-27 16:27:05,594 - DEBUG - urllib3.connectionpool: https://api.github.com:443 "GET /repos/emcbot/ci-global-workflows HTTP/1.1" 200 None
2025-02-27 16:27:05,846 - DEBUG - urllib3.connectionpool: Starting new HTTPS connection (1): api.github.com:443
2025-02-27 16:27:06,629 - DEBUG - urllib3.connectionpool: https://api.github.com:443 "POST /gists HTTP/1.1" 201 82085
https://gist.github.com/emcbot/6f3ce430128dbb7e73138c5bc23bf02a)

@emcbot emcbot added CI-Gaeac6-Failed **Bot use only** CI testing on Gaea C6 for this PR has failed and removed CI-Gaeac6-Building **Bot use only** CI testing is cloning/building on Gaea C6 labels Feb 27, 2025
@TerrenceMcGuinness-NOAA TerrenceMcGuinness-NOAA added CI-Gaeac6-Ready **CM use only** PR is ready for CI testing on Gaea C6 and removed CI-Gaeac6-Failed **Bot use only** CI testing on Gaea C6 for this PR has failed labels Feb 27, 2025
@emcbot emcbot added CI-Gaeac6-Building **Bot use only** CI testing is cloning/building on Gaea C6 and removed CI-Gaeac6-Ready **CM use only** PR is ready for CI testing on Gaea C6 labels Feb 27, 2025
@emcbot
Copy link

emcbot commented Feb 27, 2025

Build FAILED on Gaeac6 in Build# 4 with error logs:

/gpfs/f6/drsa-precip3/proj-shared/global/CI/3389/global-workflow/sorc/logs/gdas.log

Follow link here to view the contents of the above file(s): (link)

@emcbot emcbot added CI-Gaeac6-Failed **Bot use only** CI testing on Gaea C6 for this PR has failed and removed CI-Gaeac6-Building **Bot use only** CI testing is cloning/building on Gaea C6 labels Feb 27, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CI/CD Issue related to CI/CD CI-Gaeac6-Failed **Bot use only** CI testing on Gaea C6 for this PR has failed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Migrate Jenkins CI automation to Role Accounts on Gaea6
3 participants