Can General Circulation Toy Models run with lossy-compressed states?
Date:
Tyree, J. & Ekblom, M. (15.05.2024) Can General Circulation Toy Models run with lossy-compressed states? 2024 OpenIFS User Meeting. Available from: doi:10.5281/zenodo.14095029.
Abstract
The output volumes from high-resolution general circulation models are increasing exponentially but data storage, access, and analysis methods have not kept up. There is thus an increasing demand for data compression. Lossy compression methods like ZFP, SZ3, or BitRound promise higher compression ratios than bit-reproducible lossless methods. Since they lose some information (e.g. precision or resolution), care must be taken to only discard noise but no real information that impacts the scientific research performed on the data. What level of lossy compression is safe to use when the application of the model outputs is not known in advance?
Model restart files can be seen as a form of lossless compression, as they allow the full model outputs to be reproduced by resuming the model from a particular system state. By using lossy compression on the restart model states, we investigate what level of lossy compression can be safely applied without affecting the model dynamics, which may provide a safe lower bound on the compression that is possible.
We present preliminary results from our ongoing research with several toy models. Taking the concept of model restarts to its extreme, we explore the effect of lossy-compressing the model’s inner state on every timestep and analyse the effect of compressing less frequently. In particular, we investigate the impact of numerical stability in shallow water models. We also explore the relationship between compression errors and initial state perturbations in Lorenz’63 or stochastic physics in Lorenz’96 models, and the effect of compressing members of an ensemble individually.
We plan to extend our research beyond toy- and towards operational models like OpenIFS in future work.