Changelog⚓︎
Release 2.0⚓︎
Release 2.0 is the first formal release of EOEPCA+, since the last EOEPCA release 1.4 - representing a full refresh of the EOEPCA+ building blocks. EOEPCA+ 2.0 provides many new capabilities, with improved stability - ready for production deployment.
The release is additionally accompanied by supporting materials - including, for each Building Block:
- Notebooks - that provide a quick demonstration of core capabilities
- Tutorials - that showcase the building block and its basic installation in a generic reproducible environment
The following table summarises the evolution that EOEPCA+ Release 2.0 represents compared to EOEPCA 1.4.
Note that all EOEPCA+ 2.0 building blocks are integrated with the authentication and authorization approach of the IAM building block
| Building Block | EOEPCA 1.4 | EOEPCA+ 2.0 | Migration Strategy |
|---|---|---|---|
| Resource Discovery | pycsw (all resource types) | pycsw (resource discovery) Catalogue for all resource types supporting Open Science Key enhancements: - Latest OGC API Records, incl. transactions - Latest STAC API, incl. transactions, CEOS best practice eoAPI (data discovery) Data discovery, access and visualisation (see also Data Access) Key capabilities: - Latest STAC API, incl. transactions, CEOS best practice |
pycsw Upgrade eoAPI New installation |
| Data Access | EOxServer | eoAPI Data discovery, access and visualisation (see also Data Discovery) Key capabilities: - Latest STAC API - TiTiler-PgSTAC raster tiling service (XYZ/OGC-WMTS, OGC API Tiles, COG dynamic tiling, Virtual mosaics) - TiPg vector tiling service (OGC Features and Tiles) - MultiDimensional dataset support via Xarray - Scalable in production Additional - STAC Browser for data discovery and visualisation - STAC Manager for metadata management |
New installation |
| Resource Registration | Registration API (bespoke) | Registration API API for registration of resources into discovery and access services. - API via pygeoapi (OGC API Processes) for resource register and deregister - Integrates with Resource Discovery and Data Access - Resource types for reproducible science Registration Harvester Automated harvesting of datasets (and other resources) from external providers into discovery and access services. - Flowable BPMN Engine - extensible workflows for harvesting Example workflows… - Harvest Landsat data from USGS - Harvest Sentinel data from CDSE Generic workflows… - Harvest from static STAC catalog - Harvest from STAC API |
New installation |
| Workspace | Workspace API (bespoke) Workspace Provisioning Helm templates for workspace definitions |
Isolated environments for users/teams providing storage/compute for data management, processing and collaboration. Key capabilities: - Workspace API preserved for workspace management - Alternative declarative approach using Kubernetes custom resources - Workspace provisioning via declarative Crossplane pipelines - Object storage provisioning (declarative and via UI), with multiple buckets per workspace - Workspace UI providing users an interactive web interface with: Terminal, Code Editor, Storage Browser - Workspace UI provides user-controlled management of permissions and sharing, with RBAC approach - Automated rollover of bucket credentials - vCluster providing dedicated, isolated Kubernetes cluster for workspace user service provisioning Workspace hibernate and auto-suspend |
New installation |
| Datacube Access | n/a | Ensuring support for datacube semantics in EOEPCA+ BBs. Key Capabilities: - Filtered discovery and access to multi-dimensional datasets - STAC Best Practice for Datacubes |
New installation |
| Processing OGC API Processes |
ZOO-Project with Calrissian - Kubernetes Runner | User-defined processing following OGC API Processes standard ZOO-Project (common) Key enhancements: - Improved job log management - OGC types in CWL for inputs/outputs (cwl2ogc) - Zoo pod auto-scaling - New Zoo web UI - Many stability, efficiency and maintainability improvements - Unit tests for OGC-AP Patterns (see below) Kubernetes Runner (Calrissian) Key enhancements: - GPU resource requirements in CWL and node selectors - Jobs can use existing namespace - Fully reworked Calrissian integration (new eoap-cwlwrap) HPC Runner (Toil) - Zoo integration with Toil for HPC CWL execution Additional - Resources & Guides for Application Packages - Documented Application Package patterns, including datacubes - OGC API Processes python client |
Zoo-Calrissian Upgrade Zoo-Toil New Installation |
| Processing openEO |
n/a | API providing a simple/unified approach for connection to Earth Observation platforms openeo-geotrellis Key capabilities: - openEO API - Predefined processing specifications for portable applications - Dataset load via STAC - with many enhancements for compatibility and datacube support - Support for running CWL/OGC-App-Package - Many process improvements: geojson, spatial aggregation, bbox filter, … - Improvements for federation openEO Client / Web Editor Key enhancements: - Support for running CWL/OGC-App-Package |
New Installation |
| Application Hub | JupyterHub-based | Web-based tooling for interactive analysis and development JupyterHub-based Key enhancements: - Support for init containers, and user image pull secrets - Configuration generator tool - to tailor available applications - Dask Gateway support - Optional harbor for image proxy cache - CSI annotations for persistent volume claims |
Upgrade |
| IAM | Keycloak with gogatekeeper and Identity API | Authenticated user identity, and authorisation of access to platform resources Keycloak with APISIX, Open Policy Agent and Crossplane Key enhancements: - Keycloak as for EOEPCA 1.4, with upgraded version - Identity API deprecated in favour of Crossplane CRDs - Crossplane Keycloak Provider for declarative configuration of Keycloak - Open Policy Agent as policy engine - APISIX (ingress) for policy enforcement - with plugins for Authn (OIDC) and Authz (UMA and OPA) - Integration with other EOEPCA+ BBs for policy enforcement and resource sharing |
New Installation |
| Data Gateway | n/a | EODAG Capability for accessing the data offering of an extensible set of data providers and datasets Key enhancements: - STAC-formatted metadata representation - Integrate FedEO CEOS STAC for CCI data - Support STAC serialise/deserialise - STAC metadata augmentation via xarray |
n/a |
| Application Quality | n/a | Application Quality supports the transition of scientific algorithms to production-grade workflows. This is achieved through automated pipelines that execute tooling which verifies workflow quality against industry best practices for coding and security. - Python code quality checks - Best practices for Jupyter Notebooks - OGC Application Package validation - Trivy vulerability scanning - Grafan dashboards and reports for quality outcomes |
New Installation |
For more details please refer to the documentation for each component, or ask the EOEPCA team for support - in particular if you are migrating an existing deployment to EOEPCA+ 2.0.