Union Helm Charts

Dataplane

Assumptions

• You have a union organization that has already been created.
• You have a running kubernetes cluster and object storage provided by a vendor or an S3 compatible platform (such as Minio. Some sample terraform configurations are available in the providers directory.

Installation

• Install Helm 3
• Install union and uctl.
• Install the Union.ai helm charts.

helm repo add unionai https://unionai.github.io/helm-charts/
helm repo update

• Generate a new client and client secret to communicate with your Union control plane by creating a new AppSpec configuration and using the create app command from uctl.

cat > dataplane-operator.yaml << EOF
clientId: dataplane-operator
clientName: dataplane-operator
grantTypes:
- AUTHORIZATION_CODE
- CLIENT_CREDENTIALS
redirectUris:
- http://localhost:8080/authorization-code/callback
responseTypes:
- CODE
tokenEndpointAuthMethod: CLIENT_SECRET_BASIC
EOF
uctl config init --host=<cloud.host>
uctl create app --appSpecFile dataplane-operator.yaml

• The output will emit the ID, name, and a secret that will be used by the union services to communicate with your control plane.

Initializing app config from file dataplane-operator.yaml
 -------------------- -------------------- ------------------------------------------------------------------ ---------
| CLIENT ID          | CLIENT NAME        | SECRET                                                           | CREATED |
 -------------------- -------------------- ------------------------------------------------------------------ ---------
| dataplane-operator | dataplane-operator | secretxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx |         |
 -------------------- -------------------- ------------------------------------------------------------------ ---------
1 rows

• Save the secret that is displayed. Union does not store the credentials and it cannot be retrieved later.
• Optionally configure any values that are relevant to your installation. A full list of values can be found here: charts/dataplane/README.md. At a minimum you will need to provide the following values in a file or as an argument on the command line:
  • host: The admin host used to communicate with your Union.ai organization's control plane. You will have been provided this value when union has created your control plane.
  • clusterName: An arbitrary and unique identifier for your cluster.
  • orgName: The name of your Union.ai organization.
  • provider: The cloud provider your cluster is running in. Acceptable values include aws, gcp, azure, oci, and metal (for self-managed or on-prem clusters).
  • storage: Several storage parameters exist for configuration. By default the authType of accesskey is used and requires storage.accessKey and storage.secretKey to be set. A full list of storage parameters can be found here: charts/dataplane/README.md
  • secrets: You have the option of creating the admin client secrets by setting secrets.admin.create to true and providing both the clientId and the clientSecret either in the values file or as command line arguments.
  • Optionally configure the resource limits and requests for the different services. By default these will be set minimally, will vary depending on usage, and follow the kubernetes ResourceRequirements specification.
    • clusterresourcesync.resources
    • flytepropeller.resources
    • flytepropellerwebhook.resources
    • operator.resources
    • proxy.resources
• Create a values file.

# union-dataplane.yaml
host: <host>
clusterName: <cluster_name>
orgName: <org>
provider: <provider>

secrets:
  admin:
    create: true
    clientId: <client_id>
    clientSecret: "<client_secret>"
storage:
  endpoint: <endpoint>
  bucketName: union-dp-bucket
  fastRegistrationBucketName: union-dp-bucket
  accessKey: "<access_key>"
  secretKey: "<secret_key>"
  region: "<region>" 

config:
  logger:
    level: 4

• Install the dataplane CRDS and dataplane.

helm upgrade --install unionai-dataplane-crds unionai/dataplane-crds
helm upgrade --install unionai-dataplane unionai/dataplane \
    --create-namespace \
    --namespace union \
    --values union-dataplane.yaml

Once deployed you can check to see if the cluster has been successfully registered to the control plane by running the get cluster command in uctl

uctl get cluster
 ----------- ------- --------------- -----------
| NAME      | ORG   | STATE         | HEALTH    |
 ----------- ------- --------------- -----------
| <cluster> | <org> | STATE_ENABLED | HEALTHY   |
 ----------- ------- --------------- -----------
1 rows