Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[patch] added support for MariaDB and Minio - both dependencies are mandatory for AIBrooker FVT integration #1620

Open
wants to merge 20 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 17 commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
0501050
[patch] add install minio operator steps
karol-czarnecki Nov 18, 2024
1203ac0
[patch] add missing file for determine default storage class
karol-czarnecki Nov 18, 2024
d05a161
[patch] update source name
karol-czarnecki Nov 18, 2024
555e530
[patch] add empty line on minio main and operator
karol-czarnecki Nov 18, 2024
beff92d
[patch] add wait for CRD deployed
karol-czarnecki Nov 18, 2024
d5d53be
[patch] add simple role do deploy minio
karol-czarnecki Nov 19, 2024
757b9f2
[patch] add simple role to install mariadb
karol-czarnecki Nov 19, 2024
36af2e4
[patch] added missing database name for mariadb
karol-czarnecki Nov 19, 2024
f64d9b1
[patch] add readme for mariadb and minio
karol-czarnecki Nov 19, 2024
e630288
[patch] fix linter issues
karol-czarnecki Nov 19, 2024
0ff5094
Squashed commit of the following:
karol-czarnecki Nov 19, 2024
6214f29
[patch] fix copy paste errors in readme
karol-czarnecki Nov 19, 2024
9c7c5a7
Squashed commit of the following:
karol-czarnecki Dec 3, 2024
65337a1
[patch] update default storage classes location
karol-czarnecki Dec 4, 2024
e63e666
[patch] update storage classes task
karol-czarnecki Dec 4, 2024
ab71b0b
[patch] update
karol-czarnecki Dec 4, 2024
81cbce6
[patch] added var for parametrise mariadb and minio versions
karol-czarnecki Jan 8, 2025
f3befe5
Squashed commit of the following:
karol-czarnecki Jan 14, 2025
796aa46
[patch] update Minio and MariaDB readme.md
karol-czarnecki Jan 14, 2025
c2bbfb8
Merge branch 'master' into masaib425
karol-czarnecki Jan 14, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion build/bin/copy-role-docs.sh
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,6 @@ copyDoc gencfg_workspace
copyDoc grafana
copyDoc ibm_catalogs
copyDoc ibmcloud_resource_key
copyDoc install_operator
copyDoc kafka
copyDoc key_rotation
copyDoc kmodels
Expand Down
176 changes: 0 additions & 176 deletions docs/execution-environment.md
Original file line number Diff line number Diff line change
Expand Up @@ -245,148 +245,6 @@ If you need to set an environment variable then you can do this in the playbook
- ibm.mas_devops.ocp_efs
```

### backup/restore
The backup and restore tasks and playbooks provided in the `ibm.mas_devops` collection has a number of differences with other roles/playbooks that can make creating the playbook difficult. The following playbook is an example of how a backup for core playbook can be written to work with AAP, along with a rclone.conf.j2 file that should be stored in the same location as the playbook in your SCM. The playbook needs to set both the `environment` so that the `ansible_env` var is set, and for task vars to be set to override the common_vars that are looking for environment variables on the local controller (note that `environment` does not update the local controller environment so lookups using the ansible builtin `env` won't pick up those values).

This playbook requires the following survey questions to be setup:

```yaml
{
"aws_access_key_id": "$encrypted$",
"aws_secret_access_key": "$encrypted$",
"ocp_token": "$encrypted$",
"ocp_server": "https://api.your_ocp_cluster:6443"
}
```

```yaml
---
- name: "Backup/Restore MAS Core"
hosts: localhost
any_errors_fatal: true

vars:
# Define the target for backup/restore
mas_instance_id: aap1

# Define what action to perform
masbr_action: backup

# Define storage type
masbr_storage_type: cloud
masbr_storage_cloud_rclone_file: rclone.conf
masbr_storage_cloud_rclone_name: masbr
masbr_storage_cloud_bucket: mas-backup

# Define what to backup/restore
masbr_job_component:
name: "core"
instance: "{{ mas_instance_id }}"
namespace: "mas-{{ mas_instance_id }}-core"

# Configure path to backup_restore tasks that should be present in the execution environment
role_path: "/usr/share/ansible/collections/ansible_collections/ibm/mas_devops/roles/suite_backup_restore"

# Set here to get into ansible_env
environment:
MASBR_STORAGE_TYPE: cloud
MASBR_STORAGE_CLOUD_RCLONE_FILE: rclone.conf
MASBR_STORAGE_CLOUD_RCLONE_NAME: masbr
MASBR_STORAGE_CLOUD_BUCKET: mas-backup

pre_tasks:
- name: "Asserts aws_access_key_id secret defined"
assert:
that: aws_access_key_id is defined
fail_msg: "aws_access_key_id not defined"

- name: "Asserts aws_secret_access_key secret defined"
assert:
that: aws_secret_access_key is defined
fail_msg: "aws_secret_access_key not defined"

# Template out the rclone.conf so no credentials are stored in git
- name: "Create rclone.conf"
template:
src: "{{ masbr_storage_cloud_rclone_file }}.j2"
dest: "{{ masbr_storage_cloud_rclone_file }}"

roles:
- ibm.mas_devops.ocp_login

tasks:

# Common checks before run tasks
# -------------------------------------------------------------------------
- name: "Before run tasks"
include_tasks:
file: "{{ role_path }}/../../common_tasks/backup_restore/before_run_tasks.yml"
vars:
_job_type: "{{ masbr_action }}"
masbr_storage_type: cloud
masbr_storage_cloud_rclone_file: rclone.conf
masbr_storage_cloud_rclone_name: masbr
masbr_storage_cloud_bucket: mas-backup

# Create k8s Job to run backup/restore tasks
# -------------------------------------------------------------------------
- name: "Create k8s Job to run {{ masbr_action }} tasks"
when: masbr_create_task_job
include_tasks:
file: "{{ role_path }}/../../common_tasks/backup_restore/create_run_tasks_job.yml"
vars:
_rt_playbook_name: "br_core"
_rt_env:
- name: "MASBR_ACTION"
value: "{{ masbr_action }}"
- name: "MASBR_JOB_VERSION"
value: "{{ masbr_job_version }}"
- name: "MAS_INSTANCE_ID"
value: "{{ mas_instance_id }}"
masbr_storage_type: cloud
masbr_storage_cloud_rclone_file: rclone.conf
masbr_storage_cloud_rclone_name: masbr
masbr_storage_cloud_bucket: mas-backup

# Run backup/restore tasks locally
# -------------------------------------------------------------------------
- name: "Run {{ masbr_action }} tasks"
when: not masbr_create_task_job
block:
- name: "MongoDB: {{ masbr_action }}"
include_role:
name: ibm.mas_devops.mongodb
vars:
mongodb_action: "{{ masbr_action }}"
mas_app_id: "core"
masbr_storage_type: cloud
masbr_storage_cloud_rclone_file: rclone.conf
masbr_storage_cloud_rclone_name: masbr
masbr_storage_cloud_bucket: mas-backup

- name: "MAS Core namespace: {{ masbr_action }}"
include_role:
name: ibm.mas_devops.suite_backup_restore
vars:
masbr_storage_type: cloud
masbr_storage_cloud_rclone_file: rclone.conf
masbr_storage_cloud_rclone_name: masbr
masbr_storage_cloud_bucket: mas-backup
```

The following file is stored in your SCM and is a template to be injected with the credentials needed during the play. This allows the file to be stored in SCM without exposing credentials.

rclone.conf.j2
```
[masbr]
type = s3
provider = Minio
endpoint = http://minio-api.apps.mydomain.com
access_key_id = "{{ aws_access_key_id }}"
secret_access_key = "{{ aws_secret_access_key }}"
region = minio
```

Troubleshooting
-------------------------------------------------------------------------------

Expand Down Expand Up @@ -459,37 +317,3 @@ Example setting role vars:
vars:
kafka_version: 3.7.0
```

One caveat to the above is the `backup/restore` tasks as these use a combination of ansible vars and expecting environment variables to be set in the `ansible_env` variable. In order to work with `the backup/restore` tasks you need to set both `environment` and role/task vars (not play vars).

Example, showing `environment` and task vars set. See [backup exmaple](#examples-of-playbooks) for more complete play:
```yaml
- name: "Backup/Restore MAS Core"
hosts: localhost
any_errors_fatal: true

vars:
# Configure path to backup_restore tasks
role_path: "/usr/share/ansible/collections/ansible_collections/ibm/mas_devops/roles/suite_backup_restore"

# Set here to get into ansible_env
environment:
MASBR_STORAGE_TYPE: cloud
MASBR_STORAGE_CLOUD_RCLONE_FILE: rclone.conf
MASBR_STORAGE_CLOUD_RCLONE_NAME: masbr
MASBR_STORAGE_CLOUD_BUCKET: mas-backup

tasks:

# Common checks before run tasks
# -------------------------------------------------------------------------
- name: "Before run tasks"
include_tasks:
file: "{{ role_path }}/../../common_tasks/backup_restore/before_run_tasks.yml"
vars:
_job_type: "{{ masbr_action }}"
masbr_storage_type: cloud
masbr_storage_cloud_rclone_file: rclone.conf
masbr_storage_cloud_rclone_name: masbr
masbr_storage_cloud_bucket: mas-backup
```
Loading
Loading