Resource Discovery Deployment Guide⚓︎
The Resource Discovery building block provides a catalogue for Earth Observation (EO) metadata using open standards. It supports common standards such as OGC CSW, OGC API Records, STAC, and OpenSearch. Previously known as the “Resource Catalogue”, it has now expanded to handle more types of resources and better integrates with tools like eoAPI. Internally, it uses pycsw to ensure OGC compatibility and can also work with pgSTAC via eoAPI to handle large amounts of STAC metadata.
This guide shows you step-by-step how to set up Resource Discovery in your Kubernetes cluster.
Introduction⚓︎
Resource Discovery is an important component of the EOEPCA ecosystem. It helps users easily manage and search EO metadata from various sources. By using open standards, it makes data easier to find and integrate with other systems.
Key Features⚓︎
- Easy Metadata Management: Collect and search EO metadata efficiently.
- Uses Open Standards: Supports OGC CSW, OGC API Records, STAC, and OpenSearch.
- Advanced Search: Search by area (bounding boxes), time intervals, text, and more.
- Federation: Can connect and share metadata with other EOEPCA platforms.
- Transactional Updates: Allows creating, updating, and deleting records when enabled.
Interfaces⚓︎
Resource Discovery includes the following APIs:
- OGC CSW (versions 2.0.2 and 3.0)
- OGC API - Records (Core)
- STAC API 1.0.0
- OpenSearch (with support for EO, Geo, and Time queries)
- Optional STAC Dataset Catalogue via eoAPI
Prerequisites⚓︎
Component | Requirement | Documentation Link |
---|---|---|
Kubernetes | Cluster (tested on v1.28) | Installation Guide |
Helm | Version 3.5 or newer | Installation Guide |
kubectl | Configured for cluster access | Installation Guide |
Ingress | Properly installed | Installation Guide |
Cert Manager | Properly installed | Installation Guide |
Clone the Deployment Guide Repository:
Validate your environment:
Deployment Steps⚓︎
- Run the Configuration Script
Configuration Parameters
During the script execution, you will be prompted to provide:
INGRESS_HOST
: Base domain for ingress hosts.- Example:
example.com
- Example:
CLUSTER_ISSUER
: Cert-manager Cluster Issuer for TLS certificates.- Example:
letsencrypt-http01-apisix
- Example:
STORAGE_CLASS
: Storage class for persistent volumes.- Example:
standard
- Example:
- Deploy Resource Discovery Using Helm
helm repo add eoepca-dev https://eoepca.github.io/helm-charts-dev
helm repo update eoepca-dev
helm upgrade -i resource-discovery eoepca-dev/rm-resource-catalogue \
--values generated-values.yaml \
--version 2.0.0-rc1 \
--namespace resource-discovery \
--create-namespace
Deploy the ingress for the Resource Discovery service:
Validation and Operation⚓︎
1. Automated Validation (Optional Script)⚓︎
2. Manual Validation via Web Browser⚓︎
Most Resource Discovery endpoints can be accessed directly in a browser:
- Landing/Home Page
- Swagger UI (OpenAPI)
- OGC API - Records / STAC Collections
- Conformance
If these return meaningful responses (especially HTTP 200 with JSON or HTML data), it indicates that your Resource Discovery instance is operational.
3. Manual Validation via cURL / Command Line⚓︎
Using the command line can be a quick way to check endpoints and see raw responses. Below are some example commands.
We recommend executing source ~/.eoepca/state
to load the environment variables, or manually set the INGRESS_HOST
variable.
3.1. Basic Liveness Check⚓︎
Returns response headers only…
3.2. Testing OGC CSW⚓︎
curl "https://resource-catalogue.${INGRESS_HOST}/csw?service=CSW&version=2.0.2&request=GetCapabilities"
- A successful response should be an XML Capabilities document containing service metadata.
3.3. Testing STAC API⚓︎
- You should see a JSON object containing STAC-related metadata, including a list of links to collections and search endpoints.
3.4. Searching STAC Items⚓︎
curl -X POST "https://resource-catalogue.${INGRESS_HOST}/stac/search" \
--silent --show-error \
-H "Content-Type: application/json" \
-d '{
"bbox": [-180, -90, 180, 90],
"datetime": "2010-01-01T00:00:00Z/2025-12-31T23:59:59Z",
"limit": 5
}'
You should receive a JSON response listing zero or more STAC items that match the query. If you have not yet ingested any items, you may get an empty result array ("features": []
).
4. Ingesting Sample Records⚓︎
Find the Resource Catalogue Pod:
Look for a pod name similar to resource-catalogue-service-abcd1234-efgh5678
.
Copy the Sample File (sample_record.xml
) to the pod.
The sample record is provided in the
scripts/resource-discovery
directory of the deployment guide repository.
kubectl cp sample_record.xml \
resource-discovery/<YOUR-RESOURCE-CATALOGUE-POD>:/tmp/sample_record.xml
- Access the Pod:
- Load the Sample Record using pycsw
From within the container, run:
- Verify the Ingested Record
-
Via Web Browser / UI
Navigate to:Confirm that the newly ingested record (titled
EOEPCA Sample Record
) appears in the search results. -
Via Command Line
You can also usecurl
or other OGC-compliant requests to verify that the sample record is now discoverable.
Validating Kubernetes Resources⚓︎
Ensure all Kubernetes resources are running correctly:
- All pods should be in
Running
state. - No pods should be stuck in
CrashLoopBackOff
orError
.
Further Reading & Official Docs⚓︎
- EOEPCA Resource Discovery Documentation
- pycsw Official Documentation
- pycsw GitHub Repository
- eoAPI-k8s Documentation (if using eoAPI for dataset-level STAC ingestion)