Supporting COP26 hackathons

Posted on July 1, 2021 (Last modified on October 19, 2023) • 8 min read • 1,584 words

Researchers across the UK have been hacking their way through Terabytes of climate data ahead of the 26th UN Climate Change Conference of the Parties (COP26) in Glasgow on 1st-12th November 2021. Three virtual hackathon events were run, during May and June 2021, by a range of UK Universities and the Met Office, covering various topics:

• Africa hackathon
• Aim: to generate usable and impactful climate narratives for decision makers in East and West Africa
• CMIP6 data hackathon
• Aim: to produce cutting-edge research using Climate Model Intercomparison Project 6 (CMIP6) data, via a variety of projects
• Energy hackathon
• Aim: to produce novel solutions for understanding and quantifying climate risks in future energy systems

Image: some of the hackathon teams explored ways to produce novel solutions for understanding and quantifying climate risks in future energy systems. Photo by Alexandru Boicu on Unsplash

All three hackathons conducted their data storage and analysis on JASMIN - the UK’s data analysis facility for environmental science. Access to climate data (such as CMIP6) held in the CEDA Archive, alongside the processing capabilities of JASMIN, were essential for these hackathons. The hackathon organisers hope to present outputs at COP26 in November and through peer-reviewed papers.

The hackathons had over 150 attendees in various teams, investigating topics ranging from climate change to oceanography, biogeochemistry, and more.

Julia Crook, organiser of the Africa hackathon, said “During the hackathon, we had two groups looking at the effects of climate change, one focused on how it will affect hydroelectric power in Kenya, the other on agriculture in Ghana. Having a shared workspace, alongside analysis tools such as the Notebook Service, with direct access to CMIP6 data was essential for the success of our hackathon.”

Attendees made use of a variety of JASMIN services, depending on their experience and research needs. James Thomas, organiser of the CMIP6 Data hackathon, said “The ability to give participants access to the JASMIN infrastructure in such a variety of ways meant we were able to cater for a wider range of users at the hackathon, and allow researchers to concentrate on the science as much as possible. Whether this be Jupyter Notebooks for newer users, command line access, or even batch processing systems for our more advanced users, there was something for everyone”.

A five Terabyte shared workspace allowed teams to collaboratively share and analyse data across the many different academic organisations.

The JASMIN Notebook Service provided a web interface where the attendees could interactively analyse data using Python. The Notebook Service is perfect for non-experts - allowing code, documentation and visualisations to be edited and shared. At one point, over 70 notebooks were running simultaneously, the largest number to date!

Training accounts were provided to some attendees, allowing short-term access to data and services for those who may not usually be able to access JASMIN. James Thomas explains how these accounts helped “The ability to provide short-term access to researchers without accounts definitely removed a barrier to getting involved. In fact I’m happy to say that based on their experiences during the hackathon, some have now gone on to request full JASMIN accounts and intend to utilise the platform in the future.”

Energy hackathon organiser, Hannah Bloomfield, said “I don’t think there is anything the JASMIN team could have done to make this easier for us. The online help is really good - once users got into the swing of things they were able to answer their own questions by searching the help pages. The JASMIN team also helped us to get some specialist software, called Gurobi, set up at very short notice - this software turned out to be key to a couple of projects’ success. We couldn’t have done it without the staff expertise at JASMIN.”

If you are interested in using JASMIN for a training event or hackathon, please get in touch with the JASMIN helpdesk.

The CMIP6 hackathon has created a template virtual hackathon website for future events. This is freely available with instructions on how to copy and adapt for your specific event.

Image: One hackathon team investigated how future weather would impact agriculture in Ghana. In particular they looked at how drought intensity and duration changed under future climate predictions. Photo by Anaya Katlego on Unsplash

---

Footnotes:

Africa hackathon

• Purpose: to generate usable and impactful climate narratives for decision makers in East and West Africa

• Organised by: University of Leeds, Met Office

• For further details about the event take a look at the webpage

• Contact: Julia Crook

• Some details about the two projects are below, further details will be published in a future paper:

• The KenGen team looked at the impact of climate change on hydroelectric power in Kenya and built a dam level model to estimate shutdown and overflow events from precipitation and evaporation data; the model was parameterised against observed weather and dam response. Future CMIP6 scenarios were used to feed the model to estimate future responses.

• The Agriculture team looked at how future weather would impact agriculture in Ghana. In particular they looked at how drought intensity and duration changed under future climate predictions. They found droughts of a similar magnitude to the very rare 1983 drought occur a bit more frequently under SSP119 but much more frequently under SSP585.

CMIP6 Data hackathon

• Purpose: to produce cutting-edge research using Climate Model Intercomparison Project 6 (CMIP6) data, via a variety of projects.
• Organised by: University of Bristol’s Cabot Institute and Jean Golding Institute, in association with the Met Office and universities of Exeter, Leeds and UCL.
• For further information take a look at the website or these blogs
• Contact: James Thomas
• The projects covered in this hackathon are described here, with two specific examples below:
• Projecting of human heat stress with 3-hourly data from CMIP6 models, using the Universal Thermal Climate Index and nighttime minimum temperatures. JASMIN were very helpful in making as much 3-hourly data from CMIP6 models available as possible at short notice. Our group had diverse experience and interests and we investigated a number of different aspects. The main data extraction was a 2-step process: first using the LOTUS batch compute nodes for intensive processing of the data and then the Notebooks server was used for analysis. One striking result was that under an emissions pathway broadly similar to that if all countries met their current Nationally Determined Contributions, 4% of the Earth’s land surface will exceed the threshold for human habitability by 2100 compared to very little today. These results still need to be bias-corrected, and will be included in a planned paper.
• Impact of climate change on the annual bipolar movements of the arctic tern, a seabird that annually migrates from the Arctic to Antarctic in the longest recorded migration of any animal, integrating CMIP6 modelled climate data with an extraordinary biological dataset. The group of seven PhDs and Post-docs investigated (1) changes to wind support during their migration and the non-breeding season, (2) changes in ocean productivity at key areas where migrating arctic terns refuel, and (3) how observed changes in past sea ice extent may have altered the locations that arctic terns overwinter. The group are continuing this work to investigate how different climate scenarios may impact arctic terns in the future.

Energy hackathon

• Purpose: to produce novel solutions for understanding and quantifying climate risks in future energy systems

• Organised by: University of Oxford and University of Reading

• For further details take a look at the website or watch the teams final presentations during the hackathon

• Contact: Sarah Sparrow and Hannah Bloomfield

• During the hackathon, groups looked into the following topics related to energy systems:

• Building a forecasting model to improve predictions of solar power generation over a network of sites in Reading using smart meter data.

• Exploring the adverse weather scenarios for renewable energy systems using a dataset from the Met Office with the aim of finding “How extreme is too extreme?”. Team members focussed on visualisation, comparisons, resilience and how the data can be used as an input into a power system model.

• Supporting power system modelers in choosing appropriate meteorological data through climate input sensitivity experiments using a European scale power system model (PyPSAEUR)

• Building a workflow to quantify climate risk in power system design using distributed computing. The developed workflow converts hourly climate data to dependent energy variables which can then be used as inputs for a power system model (Calliope). Performing the computation in the distributed framework removes the need to store gridded hourly climate output.

• Downscaling daily reanalysis/climate model output to daily resolution as an input for power system models using a variety of machine learning techniques.

CEDA and JASMIN

The Centre for Environmental Data Analysis (CEDA) provides services on behalf of the Natural Environment Research Council (NERC) via the National Centre for Atmospheric Science (NCAS) and the National Centre for Earth Observation (NCEO). CEDA is based within RAL Space, a department of the Science and Technology Facilities Council (STFC).

This work used the JASMIN infrastructure. JASMIN is hosted and managed jointly between STFC’s Scientific Computing Department and CEDA.

For more information about using JASMIN or data held in the CEDA Archive, please contact our helpdesk.