New Wisdom Report
Editor: Mindy
Advertisement
[Introduction to New Wisdom]Weather is inherently stochastic, and in order to quantify uncertainty, traditional approaches often require the high cost of physically simulating large numbers of predictions. Google's latest research, published in the Science sub-journal, uses generative AI models to efficiently generate large-scale weather forecast collections, opening up new opportunities for weather and climate science.
In December 1972, at the annual meeting of the American Association for the Advancement of Science in Washington, D.C., Ed Lorenz, a professor of meteorology at the Massachusetts Institute of Technology, gave a speech entitled “Can the instigation of a butterfly in Brazil trigger a tornado in Texas?” ” speech, which coined the term “butterfly effect.”
In his 1963 paper, he found that in time integration and numerical weather prediction models, even small starting condition errors would rapidly expand in the numerical model, leading to a rapid increase in the uncertainty of the prediction results. It is called chaos phenomenon.
As a result, the reliability of weather forecasts is limited, especially in predicting extreme weather events, such as hurricanes, heat waves or floods.
Advertisement
For the above reasons, current weather forecasts are actually probabilistic forecasts.
These forecasts use a method called ensemble forecasting, by introducing some randomness into the initial conditions and then running several different models to generate a variety of possible outcomes.
Error growth is reduced by averaging all forecasts in the ensemble, and the amount of variation in the forecasts in the ensemble quantifies the uncertainty in weather conditions.
Although effective, generating these probabilistic forecasts is computationally expensive. They require multiple runs of highly complex numerical weather models on large-scale supercomputers.
Extreme weather, in particular, often requires larger collections to assess. For example, to predict the likelihood of an event with a 1% probability of occurring, a set of 10,000 members is required with a relative error of less than 10%.
But accurate and timely weather forecast is necessary for people's daily life. From what you need to take out to prepare for the day's activities, to what preparations you need to make in advance when facing dangerous weather.
Google's latest research has brought a new development to weather forecasting: Scalable Ensemble Envelope Diffusion Sampler (SEEDS) is a generative AI model that can efficiently generate large-scale weather forecast collections at a cost only that of traditional physics-based prediction models. a small part of.
This technique opens up new opportunities for weather and climate science and represents the first application of probabilistic diffusion models in weather and climate predictions.
Generative AI accurately generates weather forecast collections
In the paper mentioned above, Google proposes Scalable Ensemble Envelope Diffusion Samplers (SEEDS), a generative artificial intelligence technology for weather forecast ensemble generation.
SEEDS is based on the denoising diffusion probabilistic model, a state-of-the-art generative artificial intelligence approach pioneered in part by Google Research.
Generative AI is known for producing highly detailed images and videos, a feature that is particularly useful for generating ensemble forecasts that are consistent with real weather patterns.
SEEDS can generate large ensembles based on one or two forecasts from operational numerical weather prediction systems.
The generated ensembles match or exceed physically based ensembles on skill metrics such as Root Mean Square Error (RMSE) and Continuous Rank Probability Score (CRPS); not only that, but also for the tails of the forecast distributions such as ±2σ and ±2σ. 3σ weather events) are assigned more accurate probabilities.
Best of all, the computational cost of the model is almost negligible compared to supercomputers that require hours of computation time. On Google Cloud TPUv3-32 instances, its throughput is 256 collective members per 3 minutes (2° resolution) and can be easily scaled to higher throughput by deploying more accelerators.
The chart below compares SEEDS and operational US weather forecast system (Global Ensemble Forecast System, GEFS) forecasts for a specific date during the 2022 European heat wave.
where A is a proxy for real observations, (Ca-Ch) are the 8 samples simulated by SEEDS, and (Da-Dh) is the forecast from GEFS. Although it may be difficult to see obvious differences directly with the naked eye, SEEDS is better able to capture cross fields and spatial correlations, which will be closer to real weather.
This is because SEEDS directly simulates the joint distribution of atmospheric states, which actually captures the spatial covariance of atmospheric states and the correlation between midtropospheric height and mean sea level pressure, both of which are often used by meteorologists for forecast evaluation. And verification. (Some technical explanation: gradients in mean sea-level pressure drive winds at the surface, while gradients in midtropospheric height create upper-level winds that move large-scale weather patterns.)
More accurate coverage of extreme events
The SEEDS ensemble is able to extrapolate from one or two seed forecasts, showing the range of possible weather states, with better statistical coverage of events.
That is, the highly scalable generation method is able to create very large collections of weather samples that can describe very rare events with any different settings of state thresholds.
For example, the figure below shows the joint distribution of 2-meter temperature and total column water vapor for an extreme high temperature event that occurred near Lisbon at 1:00 local time on July 14, 2022.
For each plot, the Google research team used SEEDS to generate an ensemble of 16,384 members based on seed predictions from 2 physical models, shown as green points.
Correct weather events are from ERA5 and are indicated by an asterisk.
Forecasts for the conventional method-generated ensemble are represented by squares, and remaining conventional method ensemble members are represented by triangles.
It can be seen that the blue squares and yellow triangles are not close to the star at all, that is, it is simply impossible to observe this extreme situation in advance by traditional means, and none of its 31 members predicted a warm near surface as observed temperature.
In fact, the event probability calculated from the Gaussian kernel density estimate is less than 1%, which means that ensembles with less than 100 members are unlikely to contain forecasts as extreme as this event.
The green points generated by SEEDS can provide better statistical coverage based on its accurate generation capability and efficient generation speed.
A new model for weather forecasting?
The experiments described above demonstrate a hybrid forecast system where SEEDS utilizes a few weather tracks calculated based on physical models to more efficiently generate a dispersion model for more forecasts.
This approach offers an alternative to the current paradigm of operational weather forecasting.
At the same time, the computing resources saved by statistical simulators can be used to improve the resolution of physically based models or to issue forecasts more frequently.
This may be the future of weather forecasting, or it may be that SEEDS represents just one of many ways in which AI will accelerate progress in operational numerical weather forecasting in the coming years.
But what Google's research shows is the practicality of generative artificial intelligence in weather forecast simulation and post-processing, providing a new direction for accurately quantifying uncertainty in future climate and climate risk assessment.
References:
https://blog.research.google/2024/03/generative-ai-to-quantify-uncertainty.html