The European Space Agency
esa-logo
esa-logo
Navigation

Datasets Integration 

This page describes the required process for showcasing representative datasets from the Sentinel Users Preparation (SUP) activities in the APEx Geospatial Explorer (GE) and ingesting them into the ESA Project Results Repository (PRR) for long-term preservation. 

 

Visualizing data on the APEx Geospatial Explorer

ESA's Application Propagation Environments (APEx) initiative provides easy access to ESA's Earth observation application outcomes for the EO community, with a range of services and tools to build upon EO projects and sharing results. 

The Geospatial Explorer (GE) provides an interactive web front end that can be used for the display and visualisation of geospatial and tabular data ingested in the PRR or fetched from web services following common interoperable protocols (e.g. OGC Standards, STAC). 

Datasets intended for visualisation in the GE must comply with the APEx Data Provider Guidelines.                                                                                                                                                             

In particular, they should follow Scenario 3, which includes: 

  • the baseline requirements defined in Scenario 1; 
  • the general visualization recommendations described in Scenario 2; 
  • additional formatting requirements for optimal rendering in GE. 

The following sections describe the key requirements that must be met: 

  • To support the smooth integration of datasets in the GE, it is essential that all datasets adhere to the general file format recommendations. Following these guidelines, which mandate cloud-optimised formats and metadata, ensures compatibility and ease of integration within the APEx GE. 
  • When referencing data through a STAC catalogue, it is further advised to comply with the STAC metadata recommendations. Adhering to these metadata standards will improve dataset discoverability, interoperability, and enable seamless integration with the GE. 
  • For raster data, cloud-optimised GeoTiff (COG) settings such as tile size and band interleave significantly affect rendering performance in the GE. APEx provides recommended settings per specific use cases examples - including for hyperspectral imagery with 200+ bands - and recommends testing sample outputs in the GE before processing full operational pipelines. 
  • Datasets should be delivered in a single, standardised projection covering the full spatial extent of the data. All GE instances support a defined set of projections, and any other CRS supported by the PROJ4JS library can be added through the GE configuration. Data in a CRS other than the GE display CRS is reprojected on the fly, at a performance cost depending on the transformation. Widely adopted CRS are therefore preferable: less common national projections can also complicate the downstream integration of the data with other EO datasets. 

Additional Resources

Typical GE functions include the ability to visualize EO data, derived products and associated vector layers with control over layer ordering, styling, product comparisons, dynamic statistics visualisation aggregated over administrative units (e.g. NUTS regions), constraint sliders filtering a primary layer by multiple criteria (e.g. wind power density map constrained by elevation, slope and land cover) and interactive execution of algorithms. 

  • This is a demo of the GE features. 
  • The GE application is configured through a hosted JSON object with the following structure and properties options. 
  • The GE Configuration Builder utility can be used to build GE configurations along with the JSON config file. Furthermore, this workshop covers how to create new configuration from scratch, showing how to connect to various data sources (e.g. COG, FlatGeobuf) and services (WMS / WMTS / STAC). The current SUP configuration is available on GitHub. It can be directly imported into the Configuration Builder to validate the configuration and preview the dataset visualization in the SUP GE demonstration environment.

 

Ingesting new datasets in the ESA PRR

The ESA Project Results Repository (PRR) is an ESA managed, public data repository for EO value-added and geospatial products created within projects initiated by the EOP-S department (Climate Action, Sustainability and Science Department). The repository is publicly accessible and catalogued using STAC. It is based on cloud-native geospatial formats that support data streaming rather than downloading of entire files. 

  • The APEx interoperability guidelines and requirements for EO projects and data providers intending to deliver datasets to the PRR are available here. These guidelines define the requirements for long-term preservation and subsequent downstream utilization of the datasets.  
  • For dataset visualization in the APEx Geospatial Explorer, ensure the datasets meet the additional compliance requirements outlined in the previous section Visualizing data on the APEx Geospatial Explorer.  

Additional Resources

  • The ESA PRR Frequently Asked Questions section offers additional details on datasets contribution and integration procedures. 
  • The reg-api-client includes a client application utilized to ingest data into the PRR.  
  • PRR compliant STAC catalogues should shall follow a specified STAC profile and can be created and validated using this PRR STAC Collection editor.  
  • The EarthCODE project provides practical tutorials on how to generate STAC-compliant data collections directly from files and to data ingestion operations. 

 

 

User Forum

 

Visit the APEx user forum to ask questions, share insights, and connect with others using or contributing to APEx services. It offers support on the usage of APEx services and tools, including sharing feedback, requesting features and learning from others experience. 

 

APEx Solutions

 

Discover the APEx solutions across different projects by browsing this gallery. 

Our partners