HPFELD LogoThe Hosted Processing Facility for the Exploitation of Large Datasets (HPFELD) project aims to demonstrate a means to make the processing of very large datasets much easier for users.  

HPFELD is a collaborative project, involving STFC, Magellium and Terradue. There are several barriers to the expolitation of larger datasets held at data centres. There are many large science and commercial data archives, but members of the user community may be  unable to access this data easily. Even if they can, there is the problem of transferring (and storing) the huge quantities of data involved. This confinement of data is a major obstacle to the progress of both  research and business. A far more sensible and forward looking approach is to couple the archives to processing capacity, made available over the network. Remote users upload their processing algorithms for local execution, and only the data-reduced result is then downloaded. This generates large savings in time and effort for the data users. The key innovation being developed within the project is the generalisation of such a  service to work with all types of data centre.

Magellium logoTerradue Logo

This website and others run by CEDA uses cookies. By continuing to use this website you are agreeing to our use of cookies.

OK | Find out more