New study: Artificial (aka stolen) intelligence fails to predict unprecedented weather events, a worrying finding as human pollution escalates unimaginable extremes. How can AI be taught to predict what humans can’t?

In Test, A.I. Weather Model Fails to Predict Freak Storm by D360 Digest, May 22, 2025, Yale School of the Environment

Artificial intelligence is powering weather forecasts that are generally more accurate than conventional forecasts and are faster and cheaper to produce. But new research shows A.I. may fail to predict unprecedented weather events, a troubling finding as warming fuels new extremes.

Weather prediction relies on neural networks, a form of A.I. that can learn to make predictions by identifying patterns in vast amounts of data.

The problem, the study finds, is that neural networks may not be able to predict events that lie outside their training data.

For A.I. weather models, that means failing to forecast droughts, storms, and heat waves that have little or no precedent in the weather record.

For the new study, scientists trained an A.I. model on decades of weather data, but omitted any hurricane stronger than Category 2. When the A.I. was given the conditions that would lead to a Category 5 hurricane and asked to make a forecast, it consistently came up short.

“It always underestimated the event,” said lead author Yongqiang Sun, of the University of Chicago. “The model knows something is coming, but it always predicts it’ll only be a Category 2 hurricane.” The findings were published in the Proceedings of the National Academy of Sciences.

Recent research shows how A.I. can outperform conventional weather models, producing more detailed forecasts that look further into the future, but concerns about extreme weather persist. To address this issue, researchers plan to use conventional models to generate examples of extreme events for the purposes of training A.I.

A.I. weather models are “remarkable, but not magical,” said study coauthor Pedram Hassanzadeh, of the University of Chicago. “We’ve only had these models for a few years, so there’s a lot of room for innovation.”And one hell of a lot of room for deadly mistakes and costly KkaKka.

There have been nearly 400 Tornado Warnings, and counting, across 22 US states over the past 5 days.

An insane run of severe weather. pic.twitter.com/xZZLxaLgjp
— Colin McCarthy (@US_Stormwatch) May 21, 2025

In 2025, Tornado Alley has become almost everything east of the Rockies – and it’s been a violent year by Daniel Chavas, May 23, 2025, The Conversation

…

“What stands out about 2025 so far isn’t just the number of tornadoes, but how Tornado Alley has encompassed just about everything east of the Rockies, and how tornado season is becoming all year.”

…

… Over the past few decades, the U.S. has seen a broad shift in tornadoes in three ways: to the east, earlier in the year and clustered into larger outbreaks. …

Deadlier tornadoes

This eastward shift is likely making tornadoes deadlier.

Tornadoes in the Southeastern U.S. are more likely to strike overnight, when people are asleep and cannot quickly protect themselves, which makes these events dramatically more dangerous. The tornado that hit London, Kentucky, struck after 11 p.m. Many of the victims were over age 65. …

Can AI weather models predict out-of-distribution gray swan tropical cyclones? by Y. Qiang Sun, Pedram Hassanzadeh, Mohsen Zand, +2 , and Dorian S. Abbot, May 20, 2025, Earth, Atmospheric, and Planetary Sciences Vol. 122 | No. 21 PNAS

Edited by Isaac Held, Geophysical Fluid Dynamics Laboratory/NOAA, Princeton, NJ; received October 18, 2024; accepted April 8, 2025

May 20, 2025

https://doi.org/10.1073/pnas.2420914122

Significance

AI models produce skillful weather forecasts, including for some extreme events. However, forecasting the strongest events that are so rare they did not exist in the training set (the so-called gray swans) remains a major concern for these models’ operational use, especially as climate change introduces unprecedented conditions. Here, we train an AI weather model after removing Category 3–5 tropical cyclones from its training set and test it on Category 5 storms. The model could not accurately forecast these unseen cyclones. However, the model shows promise in learning from strong storms in one region and forecasting them in another region. Our work highlights the need for better understanding the limitations of AI weather models and innovations to improve them.

Abstract

Predicting gray swan weather extremes, which are possible but so rare that they are absent from the training dataset, is a major concern for AI weather models and long-term climate emulators. An important open question is whether AI models can extrapolate from weaker weather events present in the training set to stronger, unseen weather extremes. To test this, we train independent versions of the AI weather model FourCastNet on the 1979–2015 ERA5 dataset with all data, or with Category 3–5 tropical cyclones (TCs) removed, either globally or only over the North Atlantic or Western Pacific basin. We then test these versions of FourCastNet on 2018–2023 Category 5 TCs (gray swans). All versions yield similar accuracy for global weather, but the one trained without Category 3–5 TCs cannot accurately forecast Category 5 TCs, indicating that these models cannot extrapolate from weaker storms. The versions trained without Category 3–5 TCs in one basin show some skill forecasting Category 5 TCs in that basin, suggesting that FourCastNet can generalize across tropical basins. This is encouraging and surprising because regional information is implicitly encoded in inputs. Given that current state-of-the-art AI weather and climate models have similar learning strategies, we expect our findings to apply to other models. Other types of weather extremes need to be similarly investigated. Our work demonstrates that novel learning strategies are needed for AI models to reliably provide early warning or estimated statistics for the rarest, most impactful TCs, and, possibly, other weather extremes.

Supporting Information

Appendix 01 (PDF)

Download 2.89 MB

Data, Materials, and Software Availability

Some study data are available. Due to size limitations, some data cannot be uploaded to the repository. But we will share all our codes and provide all the details required for the readers to reproduce all the data used in this study themselves. Here are more details: We use the original FourCastNet with modifications for our customized training sets. These codes are publicly available at https://github.com/envfluids/FourCastNet (68). The necessary data to reproduce the results, including the weights of the 25 trained models and indices of dates that are removed in each training dataset, can be found on Zenodo at https://zenodo.org/uploads/13835657 (69) and https://zenodo.org/uploads/13834149 (70).

***

https://twitter.com/ryankatzrosene/status/1926093773131436076

Ryan Katz-Rosene, PhD @ryankatzrosene May 23, 2025:

Japan has set up a domestic weather attribution centre to help conduct rapid analysis of extreme weather events to see role of anthropogenic climate change. For instance, it found the severe heat wave that hit Japan last July was made about 21% more probable in a warming world.