Merge pull request #81 from EOEPCA/openeo-deployment-guide

Improve configuration for OpenEO deployments

docs/building-blocks/openeo-engine.md

@@ -2,22 +2,14 @@
 
 The **OpenEO Engine** building block provides early access to a unified processing platform for Earth observation data. It brings together two key components: **openEO Geotrellis** and **openEO Aggregator**. These components work together to offer a standards-based API for connecting to diverse EO cloud back-ends and to federate multiple openEO services into a cohesive processing platform.
 
-> **Note:** Integration of the openEO Engine is still in early access. The steps provided here are work-in-progress and may evolve in future releases.
+> **Note:** You must have a valid OIDC Provider to submit jobs to the OpenEO Engine. If you do not have one, refer to the [IAM Deployment Guide](./iam/main-iam.md) to set up an OIDC Provider.
 
 ---
 
-## Introduction
+## Components
 
-- **openEO Geotrellis:** Provides an API that simplifies connecting to EO cloud back-ends, running on Apache Spark in a Kubernetes environment.
-- **openEO Aggregator:** Groups multiple openEO back-ends into a unified, federated processing platform.
-
-### Key Features
-
-- **Unified API Access:** Standardized endpoints allow clients to connect easily with multiple EO data services.
-- **Federated Processing:** Seamlessly aggregates back-ends, enabling flexible data processing.
-- **Standards Compliance:** Follows openEO specifications to maintain broad interoperability.
-- **Scalability:** Uses Kubernetes and Helm for robust, scalable deployments.
-- **Early Access Deployment:** Offers raw steps to deploy and experiment with the openEO Engine components.
+- **openEO Geotrellis:** Provides an API that simplifies connecting to EO cloud back-ends, running on Apache Spark.
+- **openEO Aggregator:** Groups multiple openEO back-ends into a unified processing platform.
 
 ---
 
@@ -32,6 +24,8 @@ Before deploying, ensure your environment meets the following requirements:
 |kubectl|Configured for cluster access|[Installation Guide](https://kubernetes.io/docs/tasks/tools/)|
 |Ingress|Properly installed|[Installation Guide](../prerequisites/ingress-controller.md)|
 |Cert Manager|Properly installed|[Installation Guide](../prerequisites/tls.md)|
+|OIDC Provider|Required to submit jobs|[Installation Guide](./iam/main-iam.md)|
+
 
 **Clone the Deployment Guide Repository:**
 
@@ -55,7 +49,7 @@ bash check-prerequisites.sh
 ### 1. Run the Configuration Script
 
 ```bash
-bash configure-openeo-geotrellis.sh
+bash configure-openeo.sh
 ```
 
 During this process, you will be prompted for:
@@ -117,19 +111,6 @@ Deploy ingress
 kubectl apply -f openeo-geotrellis/generated-ingress.yaml
 ```
 
-#### Step 4: Deploy openEO Aggregator using Helm
-
-The openEO Aggregator federates multiple openEO back-ends into a unified processing platform.
-
-
-```bash
-helm upgrade -i openeofed \
-  https://artifactory.vgt.vito.be/artifactory/helm-charts/openeo-aggregator-2025.01.10-14.tgz \
-  --namespace openeo-geotrellis \
-  --create-namespace \
-  --values openeo-aggregator/generated-values.yaml
-```
-
 ---
 
 ## Validation
@@ -184,64 +165,100 @@ curl -L https://openeo.${INGRESS_HOST}/openeo/1.2/processes | jq .
 
 _Expected output:_ A JSON object with an array of processes. Use your terminal’s scroll or `jq` to inspect the output.
 
-#### Validate Aggregator Response
+### 3. Usage
+
+Before running any jobs, you must obtain an access token from your OIDC Provider. Use the following command to get an access token if you followed our [IAM Deployment Guide](./iam/main-iam.md).
+
+#### Create a Keycloak Client
+
+Use the `create-client.sh` script in the `/scripts/utils/` directory. This script prompts you for basic details and automatically creates a Keycloak client in your chosen realm:
 
 ```bash
-curl -L https://openeofed.${INGRESS_HOST}/openeo/ | jq .
+bash ../../utils/create-client.sh
 ```
 
-_Expected output:_ A JSON response including federation details and links, confirming that the aggregator is aware of multiple back-ends.
+When prompted:
 
-### 3. Usage
+- **Keycloak Admin Username and Password**: Enter the credentials of your Keycloak admin user (these are also in `~/.eoepca/state` if you have them set).
+- **Keycloak base domain**: e.g. `auth.example.com`
+- **Realm**: Typically `eoepca`.
+
+- **Client ID**: Use `demo-openeo-eoepca`.
+- **Client name** and **description**: Provide any helpful text (e.g., "OpenEO Client").
+- **Client secret**: Leave blank and let Keycloak generate one.
+- **Subdomain**: Use `openeo`.
+- **Additional Subdomains**: Leave blank.
 
-If your deployment includes sample processes and supports job submissions, you can test job execution as follows:
+After it completes, you should see a JSON snippet confirming the newly created client.
 
-#### 1. Submit a Job Using the "add" Process
 
-Submit a job that adds 5 and 2.5 by sending a process graph to the `/jobs` endpoint:
+#### Create a Keycloak User
+
+If you have not already created a user, you can do so using the helper scripts or through the Keycloak UI. To create a user using the helper script, run
 
 ```bash
-curl -X POST "https://openeofed.${INGRESS_HOST}/openeo/1.2/jobs" \
-  -H "Content-Type: application/json" \
-  -d '{
-        "process_graph": {
-          "sum": {
-            "process_id": "add",
-            "arguments": {
-              "x": 5,
-              "y": 2.5
-            },
-            "result": true
-          }
-        }
-      }' | jq .
+bash ../../utils/create-user.sh
 ```
 
-The response should include a `job_id` (e.g., `"job_id": "12345"`) along with other job details.
-
-#### 2. Monitor the Job Status
 
-Replace `<JOB_ID>` with the actual job ID from the previous step and run:
+#### Get an Accces Token
 
 ```bash
-curl -X GET "https://openeo.${INGRESS_HOST}/openeo/1.2/jobs/<JOB_ID>" | jq .
+source ~/.eoepca/state
+
+ACCESS_TOKEN=$(
+    curl --silent --show-error \
+         -X POST \
+         -d "username=<YOUR-USERNAME>" \
+         --data-urlencode "password=<YOUR-PASSWORD>" \
+         -d "grant_type=password" \
+         -d "client_id=demo-openeo-eoepca" \
+         -d "client_secret=<CLIENT-SECRET>" \
+         -d "scope=openid profile email" \
+         "https://${KEYCLOAK_HOST}/realms/${REALM}/protocol/openid-connect/token" |
+         jq -r '.access_token'
+)
+
+AUTH_TOKEN="oidc/eoepca/${ACCESS_TOKEN}"
 ```
 
-Check that the job’s status changes from `submitted` to `running` and eventually to `successful`.
+Make sure that you replace the following values:
+- <YOUR-USERNAME> with your username
+- <YOUR-PASSWORD> with your password
+- <CLIENT-SECRET> with your client secret
 
-#### 3. Retrieve the Job Result
+We need to format the token as `oidc/eoepca/${ACCESS_TOKEN}` to comply with the `oidc_providers` variable seen in the Helm values.
 
-Once the job has completed, retrieve the output:
+
+#### Submit a Job Using the "add" Process
+
+Submit a job that adds 5 and 6.5 by sending a process graph to the `/jobs` endpoint:
 
 ```bash
-curl -X GET "https://openeo.${INGRESS_HOST}/openeo/1.2/jobs/<JOB_ID>/results" | jq .
+curl -X POST "https://openeo.${INGRESS_HOST}/openeo/1.2/result" \
+  -H "Content-Type: application/json" \
+  -H "Authorization: Bearer ${AUTH_TOKEN}" \
+  -d '{
+        "process": {
+          "process_graph": {
+            "arccos_node": {
+              "process_id": "sum",
+              "arguments": {
+                "data": [5,6.5]
+              },
+              "result": true
+            }
+          }
+        }
+      }'
 ```
 
+
 **Expected output:**  
 A simple numeric result:
 
 ```json
-7.5
+11.5
 ```
 
 This confirms that the "add" process is operational and returning the correct computed sum.
@@ -253,4 +270,3 @@ This confirms that the "add" process is operational and returning the correct co
 - [openEO Documentation](https://open-eo.github.io/openeo-api/)
 - [openEO Geotrellis GitHub Repository](https://github.com/Open-EO/openeo-geotrellis-kubernetes)
 - [openEO Aggregator Documentation](https://open-eo.github.io/openeo-aggregator/)
-- [EOEPCA+ Documentation](https://eoepca.readthedocs.io/)

scripts/processing/openeo/check-prerequisites.sh

@@ -0,0 +1,17 @@
+#!/bin/bash
+
+source ../common/utils.sh
+source ../common/prerequisite-utils.sh
+echo "🔍 Checking prerequisites for OpenEO Deployment..."
+
+declare -a checks=(
+    "check_kubernetes_access"
+    "check_kubectl_installed"
+    "check_helm_installed"
+    "check_git_installed"
+    "check_cert_manager_installed"
+    "check_ingress_controller_installed"
+    "check_oidc_provider_accessible"
+)
+
+run_validation "${checks[@]}"

scripts/openeo/configure-openeo.sh → scripts/processing/openeo/configure-openeo.sh

@@ -4,9 +4,9 @@ source ../common/utils.sh
 echo "Configuring openEO..."
 
 ask "INGRESS_HOST" "Enter the base domain name" "example.com" is_valid_domain
+ask "STORAGE_CLASS" "Enter the storage class name" "standard" is_not_empty
 configure_cert
 
-envsubst <"openeo-aggregator/values-template.yaml" >"openeo-aggregator/generated-values.yaml"
 envsubst <"openeo-geotrellis/values-template.yaml" >"openeo-geotrellis/generated-values.yaml"
 envsubst <"sparkoperator/values-template.yaml" >"sparkoperator/generated-values.yaml"
 envsubst <"zookeeper/values-template.yaml" >"zookeeper/generated-values.yaml"

scripts/openeo/openeo-geotrellis/values-template.yaml → scripts/processing/openeo/openeo-geotrellis/values-template.yaml

@@ -28,35 +28,19 @@ configMaps:
 
     oidc_providers = [
         OidcProvider(
-            id="egi",
-            title="EGI Check-in",
-            issuer="https://aai.egi.eu/auth/realms/egi/",
-            scopes=["openid", "email"],
+            id="eoepca",
+            title="EOEPCA",
+            issuer="${OIDC_ISSUER_URL}",
             default_clients=[
                 {
-                    "id": "vito-default-client",
+                    "id": "demo-openeo-eoepca",
                     "grant_types": [
+                        "password",
                         "authorization_code+pkce",
                         "urn:ietf:params:oauth:grant-type:device_code+pkce",
                         "refresh_token",
                     ],
-                    "redirect_urls": ["https://editor.openeo.org"],
-                }
-            ],
-        ),
-        OidcProvider(
-            id="egi-dev",
-            title="EGI Check-in (dev)",
-            issuer="https://aai-dev.egi.eu/auth/realms/egi/",
-            default_clients=[
-                {
-                    "id": "openeo-eoepca-demo",
-                    "grant_types": [
-                        "authorization_code+pkce",
-                        "urn:ietf:params:oauth:grant-type:device_code+pkce",
-                        "refresh_token",
-                    ],
-                    "redirect_urls": ["https://editor.openeo.org"],
+                    "redirect_urls": ["https://openeo.$INGRESS_HOST"],
                 }
             ],
         ),