The eXtreme-DataCloud project is pleased to announce the general availability of its first public software release, codenamed Pulsar.

The release notes can be found here. This release comes after an initial phase of requirement gathering which involved several European scientific collaborations in areas as diverse as life sciences, astrophysics, high-energy physics, photon science and clinical research.

This resulted in the development of new or improved functionalities of a set of software components, some of them already available in the European Open Science Cloud ecosystem, addressing important topics like federation of storage resources, smart caching solutions, policy driven data management based on Quality of Service, data lifecycle management, metadata handling and manipulation, optimized data management based on access patterns.

Some of the building blocks of the eXtreme-DataCloud architecture come from the INDIGO-DataCloud project while others are developed and maintained by partners of the Consortium and are available as widely used open source software. Work has been carried out in this period to implement also numerous enhancements targeting tighter integration between the participating components in order to enable workflows which exploit the whole data platform as a coherent infrastructure rather than a set of disparate services:

  • Orchestrator – Dynafed integration for data location

  • XCache – EOS integration for efficient data access

  • EOS integration with S3 or WebDAV (dCache) storage for flexible data management.

  • Orchestrator – FTS integration for data movement

  • Progress on a WebDAV interface for OneData which would permit data movement between it and dCache.

  • Implementation of OIDC support in FTS and Dynafed, that offers the possibility of a full token-based authentication path through the system

  • QoS support in FTS, in order to allow managed QoS transitions on dCache storage systems, intended ultimately to be driven by the Orchestrator

  • Notification support in dCache, that will serve as the foundation for tighter integration with caches or adopted storage.

More details are available here. These components are now released into a consistent and modular suite, under the Apache License Version 2.0 (approved by the Open Source Initiative), except for the modifications contributed to existing projects, where the corresponding open source license has been used. The services can be deployed on both public and private cloud infrastructures. Installation, configuration guides and documentation can be consulted here.

All the eXtreme-DataCloud components are integrated into a comprehensive and flexible architecture, whose Authentication and Authorization approach follows the AARC blueprint, with support for user authentication through multiple methods thanks to the INDIGO IAM user authentication (SAML, OpenID Connect and X.509), support for distributed authorization policies and a Token Translation Service, creating credentials for services that do not natively support OpenID Connect.