Onedata | XDC - eXtreme DataCloud

Onedata – Global Data Access is a data management system providing easy, scalable and high-performance access and management of distributed storage resources. It was developed with the contribution of the INDIGO-DataCloud project to support a wide range of use cases. It can access, store, process and publish data using global data storage backed by computing centers and storage providers worldwide.

Onedata focuses on instant, transparent access to distributed data sets, without unnecessary staging and migration, allowing access to the data directly from a local computer or worker node and features intuitive web based user interfaces for administrators and users as well as a flexible set of APIs.

Onedata federations

Onedata will be at the heart of the project development for use cases concerning direct access to remote data based on POSIX, metadata management and encryption services as well as dynamic extension of sites. Most of the Onedata functionalities are or will be at TRL8 by the end of INDIGO-DataCloud project, while the cross-federation capabilities will be released at TRL6.

It will be used to implement the following functionalities:

Smart caching mechanisms
Metadata management

Provide smart caching mechanisms to support the remote extension of a site to remote locations and to provide alternative models for large data centers. Data stored in the original site should be accessible in a transparent way from the remote location.

Caching mechanism should guarantee that data are accessed transparently from any location without the need of explicitly copying them on the client location.

The user should be able to associate metadata in a flexible way and without a predefined format to the data that are uploaded.

The user should be able to search the data exploiting the metadata service. This should be possible for very large datasets