¶ FedEarthData: Getting started for Service Providers
These pages are targeted to Service Providers who wish to be onboarded onto the FedEarthData Service. If you want to join, email us at contact<at>c-scale.eu.
¶ Join C-SCALE
New providers are welcome to join the FedEarthData Service following one or more of the following options (see Figure below):
- As Infrastructure Providers (Cloud, HTC/HPC)
- As Service Providers (JupyterHub, openEO)
- As Data Providers.
¶ Infrastructure Providers
Infrastructure providers are those operating computing resources in the form of Cloud and/or HTC/HPC clusters.
Cloud providers wanting to join FedEarthData are expected to run OpenStack as the Cloud Management Framework of choice. Cloud providers need to configure OpenStack to connect to EGI Check-in, the AAI underpinning the cloud federation. The steps for cloud providers to be onboarded in C-SCALE are documented in D3.1 Initial Design of the Compute Federation, Section 3.4.
A common denominator for HTC/HPC providers in FedEarthData is to operate a local LDAP directory for user management that will be automatically synchronised with SURF Research Access Management (SRAM), the AAI underpinning the HTC/HPC federation. The steps for HTC/HPC providers to be onboarded are documented in D3.1 Initial Design of the Compute Federation, Section 4.3.
¶ Service Providers
Service providers are expected to contribute either with a JupyterHub instance and/or with an openEO back-end deployment.
Providers running an instance of JupyterHub optimised for Copernicus data analytics are welcome to join FedEarthData. A mandatory step is to federate JupyterHub with EGI Check-in following these instructions.
Some providers in the C-SCALE project have deployed their own back-end of openEO to better serve certain use cases. In particular, INCD has shared instructions on how to deploy openEO via their gitlab repository. Additionally, INFN has built a TOSCA template to automate the deployment of openEO backend using the PaaS Orchestrator. Detailed instructions are provided in D3.2 Compute federation optimised for Copernicus data analytics, Section 2.3, for those new providers that would like to deploy their own openEO back-end and join FedEarthData.
¶ Data Providers
Copernicus data providers operating a STAC API endpoint may be onboarded in two complementary ways:
- Metadata access and/or
- Data access.
Metadata access refers to the ability to show information about what data is hosted at the provider. Data access refers to the ability to download the data by external parties. If your are interested, check out this page describing the steps to join.
¶ Appendix: SRAM/Check-in integration
Traditionally in C-SCALE SRAM has been used as the AAI system for the HTC/HPC federation and EGI Check-in for the Cloud federation. After long discussions on how to integrate both systems, and thanks to fruitful collaboration between SRAM and Check-in teams, on 11th May 2023 we successfully tested a proof-of-concept with SRAM acting as a community AAI and Check-in as infrastructure proxy. We used the demo instance of Check-in and a test development of OpenStack for the tests.
As a reference, below is the mapping used:
[
{
"local": [
{
"user": {
"name": "{0}"
},
"group": {
"id": "4657ac74bc3544d3ae088a8aad7030b9"
}
}
],
"remote": [
{
"type": "HTTP_OIDC_SUB"
},
{
"type": "HTTP_OIDC_ISS",
"any_one_of": [
"https://aai-demo.egi.eu/auth/realms/egi"
]
},
{
"type": "OIDC-eduperson_entitlement",
"regex": true,
"any_one_of": [
"^urn:mace:surf.nl:sram:group:egi:checkin_sram$"
]
}
]
}
]
User successfully created:
cloudadm@vnode-0:~$ openstack user show 2c096b11a1410d44e3936fa40479ad26eaa649cfd6887f06b3c6669e5d6c03d0
+---------------------+------------------------------------------------------------------------------------------------------------------------------------------+
| Field | Value |
+---------------------+------------------------------------------------------------------------------------------------------------------------------------------+
| domain_id | 0125ed0ebc8045a49ed8c34c2a78740d |
| enabled | True |
| federated | [{'idp_id': 'egi.eu', 'protocols': [{'protocol_id': 'openid', 'unique_id': 'efb8534478028e92f5fd5d658c7efeed9766fafc%40sram.surf.nl'}]}] |
| id | 2c096b11a1410d44e3936fa40479ad26eaa649cfd6887f06b3c6669e5d6c03d0 |
| name | efb8534478028e92f5fd5d658c7efeed9766fafc@sram.surf.nl |
| options | {} |
| password_expires_at | None |
+---------------------+------------------------------------------------------------------------------------------------------------------------------------------+
From now on new users with an account on SRAM are able to access both Cloud and HTC/HPC systems in the C-SCALE federation.
¶ How to get support
Please use the C-SCALE Helpdesk.
There is also a forum for C-SCALE on GitHub.