Sentinel data weighs in at a 'horn'-ormous 2PB
Posted on July 20, 2017 (Last modified on October 19, 2023) • 3 min read • 476 wordsThe CEDA archive now contains more than 2 Petabytes of Sentinel data comprised of a whopping ~1.6 million individual Sentinel data products. If 2PB’s were represented by weight (when 1gram = 1GB), the archived Sentinel data would ‘weigh’ the equivalent of a Rhino - absolutely ‘horn’-ormous! In contrast, your average smartphone data capacity would weigh the equivalent of a mouse. See infographic below for more details.
Each Sentinel mission collects various measurements and images for land, ocean and atmospheric monitoring. There will be a total of 7 Sentinel missions, some of which will be comprised of up to 4 satellites. This maximises global coverage. Some Sentinel missions already have 2 satellites in orbit retrieving data - such as Sentinel1A and Sentinel1B.
The Sentinel family (Credits: ESA)
CEDA currently have over 100 users downloading Sentinel 1 data and nearly 70 using Sentinel 2 data. CEDA users have downloaded over 500TB via the web. However, this doesn’t take into account the data accessed via JASMIN and group workspaces. As this data is accessed directly via the file system, it is a difficult statistic to produce accurately per user.
These ever increasing data volumes have challenged the CEDA archive. To cater for this data deluge, CEDA has developed the Near Line Archive (NLA) system. The NLA is used to archive older data onto a state of the art tape library in the JASMIN infrastructure. This means that the newest data can be made available via instant access. Older data can still be requested and will take no longer than a few days to be made available. The retrieved data is restored to it’s original location in the archive and can be accessed in the same way for a limited period.
The CEDA Satellite Data Finder is another recent development. This provides a map application for locating scenes and scans of several datasets held in the CEDA archive. More information can be found here. Behind the scenes is the CEDA OpenSearch service - this provides a simple way to query the catalogue using scripts via html. These are vital tools to allow users to easily find and access data from the huge Sentinel mirror archive.
The huge data volumes produced by the Sentinel missions will not slow down with several more satellites due to be launched in the next few years. This is a big data issue that will continue far into the future and CEDA will continue to innovate and apply technologies to allow users to easily find and use this data.
Extra info:
Sentinel data is publicly available, all you have to do is register for a CEDA account.
CEDA currently archive the following Sentinel products:
- Sentinel 1A and 1B- IW SLC, IW GRDH, EW GRDM, SM SLC
- Sentinel 2A- L1 MSI
- Sentinel 3A- L1 SLSTR, L1 OLCI EFR+ERR and L1 SRAL