Scalability

A major programme to ensure NWP exploits the potential of future energy-efficient computing systems

Progress in numerical weather prediction is intimately connected with progress in supercomputing. Over the years, more computing power has enabled us to increase the skill and detail of our forecasts. This has brought huge value to society, not least through early warnings of severe weather. But as the forecasting system becomes more complex, with current computing architectures it will soon be impossible to issue forecasts within schedule and at a reasonable cost.

Supercomputer energy consumption at ECMWF would have to increase unviably if the more complex forecasting systems of the future were to be run on the current architecture.

Scalability efficiency gains graphic

Simulation experiments. The need for greater computing performance is driven by advances in modelling the Earth’s physical processes, the use of more observational data and finer model grid resolutions. The graph shows the results of simulations carried out by ECMWF at a range of hypothetical model grid resolutions. They indicate that ECMWF’s ability to continue to increase the resolution at which it runs its forecasts, while keeping power consumption within reasonable limits, depends very much on the success of the Scalability Programme. The number of required compute cores, and hence the amount of power consumed, rises rapidly as the resolution of single ‘deterministic’ forecasts on the one hand and ensemble forecasts on the other increases. Efficiency gains in all parts of the forecasting system are required in order to make a goal such as a 5 km horizontal resolution for ECMWF’s ensemble forecasts by 2025 realisable.

A new generation of computing systems with exascale capabilities promise much greater energy efficiency – but they will rely on parallel processing at levels to which current NWP codes are not adapted. Changes are needed throughout the entire NWP processing chain if we are to exploit these new opportunities for energy efficiency. ECMWF’s Scalability Programme, launched at the end of 2013, brings together meteorological modellers, computer scientists and hardware providers from around the world for a coordinated approach to hardware and software development.

Programme objectives

  • An integrated forecasting system combining a flexible framework for scientific choices to be made with maximum achievable parallelism
  • Portable code structures ensuring efficiency and code readability, and exploiting a range of expected future technologies
  • Metrics and framework for code testing, allowing a quantitative assessment of scalability

Programme scope

This ten-year programme encompasses the entire NWP processing chain, from processing and assimilating observational data to delivering forecasts to Member States. Projects will cover six main areas:

  • Observational data processing
  • Data assimilation
  • Numerical methods
  • Data processing
  • IFS code adaptation
  • Computer architecture support

Related projects

  • CRESTA  – Collaborative Research into Exascale Systemware, Tools & Applications
  • ESiWACE – Centre of Excellence in Simulation of Weather and Climate in Europe
  • Earthserver-2 – Agile Analytics on Big Data Cubes
  • ESCAPE – Energy-efficient Scalable Algorithms for Weather Prediction at Exascale
  • ESCAPE-2 – Energy-efficient Scalable Algorithms for Weather Prediction at Exascale-2
  • NEXTGenIO – Next Generation I/O for the Exascale
  • PantaRhei – developing an interdisciplinary forecasting system for simulating multi-scale fluid flows
  • PRIMAVERA – Process-based climate simulation: advances in high-resolution modelling and European climate risk assessment

A co-ordinated approach

The programme promotes an integrated approach to code development with active participation from ECMWF Member States – through membership in programme and project boards, and direct partnership in internal and externally funded projects. Consultation and collaboration with Member States and partners from the public and commercial sectors are vital, from workshops to define the programme scope through to development and testing.

More broadly, ECMWF has joined the ETP4HPC, an industry-led forum helping to define European HPC technology research priorities and action plans.

ECMWF Scalability Infographic

PDF iconView larger ECMWF Scalability Infographic

More information