Chasing Storms through Terabytes of Data

March 14, 2018

You are here

Chasing Storms through Terabytes of Data
The Toolkit for Extreme Climate Analysis (TECA) detects and tracks extreme weather events in large climate datasets. This visualization depicts tropical cyclone tracks overlaid on atmospheric flow patterns.

The Science

Scientists wanted to detect and track tropical cyclones and other extreme weather events in terabyte-sized computer simulations. In response, a team built TECA. It stands for the Toolkit for Extreme Climate Analysis.

The Impact

Tropical storms are one of the most damaging climate events in terms of monetary destruction and loss of human life. By using TECA on supercomputers, scientists can analyze computational models and changes in tropical storm system frequency in a matter of hours.

Summary

Climate simulations create massive amounts of data ("big data"), requiring sophisticated pattern recognition. Simulations that today produce tens of gigabytes of data will soon be modified to produce tens of terabytes per time step. This dramatic increase in data size would present a significant challenge to the climate community. When coupled with increased complexity of the models, timely analysis of the data could become insurmountable. A team from Lawrence Berkeley and Argonne National Laboratories used TECA on the National Energy Research Scientific Computing Center's (NERSC) Hopper and Argonne Leadership Computing Facility's Mira systems to analyze 56 terabytes of climate data on nearly 1,000,000 supercomputer cores. The data was from the fifth phase of the Coupled Model Intercomparison Project. They identified three classes of storms: tropical cyclones, atmospheric rivers, and extra-tropical cyclones. They preprocessed the data on the Hopper system at NERSC. Preprocessing took about two weeks and resulted in a final 15-terabyte dataset. At the 16th International Conference on Computer Analysis of Images and Patterns, held in September 2015, TECA was honored with the Juelich Supercomputing Center prize. The prize honors the best application of high-performance computing technology in solving a pattern recognition problem.

Contact

Prabhat
Lawrence Berkeley National Laboratory
prabhat@lbl.gov

Funding

This material is based on work supported by the Department of Energy, Office of Science, Office of Biological and Environmental Research, Climate and Environmental Sciences Division, Regional & Global Climate Modeling Program. The computational resources of the National Energy Research Scientific Computing Center (NERSC, Lawrence Berkeley National Laboratory) and the Argonne Leadership Computing Facility (ALCF, Argonne National Laboratory), both of which are Office of Science scientific user facilities, were used.

Publications

Prabhat, S. Byna, V. Vishwanath, E. Dart, M. Wehner, and W.D. Collins, "TECA: Petascale Pattern Recognition for Climate ScienceExternal link." In: G. Azzopardi and N. Petkov (eds), Computer Analysis of Images and Patterns. Lecture Notes in Computer Science, 9257, 426 (2015). [DOI: 10.1007/978-3-319-23117-4_37]

Related Links

National Energy Research Scientific Computing Center news article: Berkeley Lab Climate Software Honored for Pattern Recognition AdvancesExternal link

Highlight Categories

Program: ASCR, BER, CESD

Performer/Facility: DOE Laboratory, SC User Facilities, ASCR User Facilities, NERSC, ALCF