Skip to content

Processing Building Block Deployment Guide⚓︎

The Processing Building Block provides a platform-hosted execution engine through which users can deploy, manage, and execute Earth Observation applications using the OGC API Processes standards. The Processing Building Block aligns with the Zoo Project which is a WPS (Web Processing Service) implementation written in C, Python and JavaScript. It is an open source platform which implements the WPS 1.0.0 and WPS 2.0.0 standards edited by the Open Geospatial Consortium (OGC).

Table of Contents⚓︎

  1. Introduction
  2. Architecture Overview
  3. Prerequisites
  4. Deployment
  5. Validation and Operation
  6. Uninstallation
  7. Further Reading
  8. Feedback

Introduction⚓︎

The Processing Building Block is made up of the following components:

  • ZOO-Kernel: A WPS compliant implementation written in C offering a powerful WPS server able to manage and chain WPS services. by loading dynamic libraries and code written in different languages.
  • ZOO-Services: A growing collection of ready to use Web Processing Services built on top of reliable open source libraries such as GDAL, GRASS GIS, OrfeoToolbox, CGAL and SAGA GIS.
  • ZOO-API: A server-side JavaScript API for creating, chaining and orchestrating the available WPS Services.
  • ZOO-Client: A client side JavaScript API for interacting with WPS servers and executing standard requests from web applications.

Architecture Overview⚓︎

  • Standardised Interfaces: Implements OGC API Processes standards for interoperability.
  • Application Deployment: Supports deployment, replacement, and undeployment of applications.
  • Execution Engine: Executes applications using Kubernetes and Calrissian for CWL workflows.
  • Integration: Works seamlessly with other EOEPCA+building blocks like Identity Management.

![[processing-ADES-execute.png]]

Prerequisites⚓︎

Before deploying the Processing Building Block, ensure you have the following:

Component Requirement Documentation Link
Kubernetes Cluster (tested on v1.28) Installation Guide
Helm Version 3.5 or newer Installation Guide
kubectl Configured for cluster access Installation Guide
Ingress Properly installed Installation Guide
TLS Certificates Managed via cert-manager or manually TLS Certificate Management Guide
Stage-In S3 Accessible MinIO Deployment Guide
Stage-Out S3 Accessible MinIO Deployment Guide

Clone the Deployment Guide Repository:

git clone -b 2.0-beta https://github.com/EOEPCA/deployment-guide
cd deployment-guide/scripts/processing

Validate your environment:

Run the validation script to ensure all prerequisites are met:

bash check-prerequisites.sh

Deployment Steps⚓︎

  1. Run the Configuration Script
bash configure-processing.sh

Configuration Parameters - INGRESS_HOST: Base domain for ingress hosts. - Example: example.com - CLUSTER_ISSUER (if using cert-manager): Name of the ClusterIssuer. - Example: letsencrypt-prod - STORAGE_CLASS: Storage class for persistent volumes. - Example: default

Stage-Out S3 Configuration: Before proceeding, ensure you have an existing S3 object store. If you need to set one up, refer to the MinIO Deployment Guide(). These values get automatically set in the EOEPCA+state if you followed the Deployment Steps.

  • S3_ENDPOINT: S3 Endpoint URL.
    • Example: minio.example.com
  • S3_ACCESS_KEY: S3 Access Key.
  • S3_SECRET_KEY: S3 Secret Key.
  • S3_REGION: S3 Region.
    • Example: us-west-1

Stage-In S3 Configuration: This is where you will get the incoming data. - This can be set to the same as the Stage-Out configuration if you are storing the data in the same S3. - Alternatively if your stage-in is in a different location, for example, you are using the data hosted by CloudFerro, then update these.

  • STAGEIN_S3_ENDPOINT: Stage-In S3 Endpoint URL.
    • Example: stage-in-s3.example.com
  • STAGEIN_S3_ACCESS_KEY: Stage-In S3 Access Key.
  • STAGEIN_S3_SECRET_KEY: Stage-In S3 Secret Key.
  • STAGEIN_S3_REGION: Stage-In S3 Region.
    • Example: eu-west-2

Important Notes:

  • If you choose not to use cert-manager, you will need to create the TLS secrets manually before deploying.
  • The required TLS secret names are:
    • zoo-open-tls
  • For instructions on creating TLS secrets manually, please refer to the Manual TLS Certificate Management section in the TLS Certificate Management Guide.
  1. Deploy the Processing BB Using Helm
helm install zoo-project-dru zoo-project-dru \
  --version 0.2.6 \
  --values generated-values.yaml \
  --repo https://zoo-project.github.io/charts/ \
  --namespace processing \
  --create-namespace

Validation and Operation⚓︎

Automated Validation:

bash validation.sh

Manual Validation:

  1. Check Kubernetes Resources:
kubectl get all -l app.kubernetes.io/name=zoo-project-dru-kubeproxy --all-namespaces
kubectl get all -l app.kubernetes.io/instance=zoo-project-dru --all-namespaces
  1. Access Domain:
https://zoo-open.<your-domain>
  1. Test the OGC API Processes Endpoint:
curl -X GET 'https://zoo-open.<your-domain>/ogc-api/processes' \
  -H 'Accept: application/json'
  1. Access the Swagger page and test the endpoints:
https://zoo-open.<your-domain>/swagger-ui/oapip

Uninstallation⚓︎

To remove the Processing Building Block from your cluster:

helm uninstall zoo-project-dru

Additional Cleanup⚓︎

  • Delete Persistent Volume Claims (PVCs):
kubectl delete pvc -l app.kubernetes.io/instance=zoo-project-dru

Further Reading⚓︎

Feedback⚓︎

If you have any issues or suggestions, please open an issue on the EOEPCA+Deployment Guide GitHub Repository.