AtmoDist: Self-supervised Representation Learning for Atmospheric
Dynamics
S. Hoffmann, и C. Lessig. (2022)cite arxiv:2202.01897Comment: Submitted to "Environmental Data Science", Cambridge University Press. Revised version. Journal-version of "Towards Representation Learning for Atmospheric Dynamics. arXiv:2109.09076".
Аннотация
Representation learning has proven to be a powerful methodology in a wide
variety of machine learning applications. For atmospheric dynamics, however, it
has so far not been considered, arguably due to the lack of large-scale,
labeled datasets that could be used for training. In this work, we show that
the difficulty is benign and introduce a self-supervised learning task that
defines a categorical loss for a wide variety of unlabeled atmospheric
datasets. Specifically, we train a neural network on the simple yet intricate
task of predicting the temporal distance between atmospheric fields from
distinct but nearby times. We demonstrate that training with this task on ERA5
reanalysis leads to internal representations capturing intrinsic aspects of
atmospheric dynamics. We do so by introducing a data-driven distance metric for
atmospheric states. When employed as a loss function in other machine learning
applications, this Atmodist distance leads to improved results compared to the
classical $\ell_2$-loss. For example, for downscaling one obtains higher
resolution fields that match the true statistics more closely than previous
approaches and for the interpolation of missing or occluded data the AtmoDist
distance leads to results that contain more realistic fine scale features.
Since it is derived from observational data, AtmoDist also provides a novel
perspective on atmospheric predictability.
Описание
[2202.01897] AtmoDist: Self-supervised Representation Learning for Atmospheric Dynamics
cite arxiv:2202.01897Comment: Submitted to "Environmental Data Science", Cambridge University Press. Revised version. Journal-version of "Towards Representation Learning for Atmospheric Dynamics. arXiv:2109.09076"
%0 Generic
%1 hoffmann2022atmodist
%A Hoffmann, Sebastian
%A Lessig, Christian
%D 2022
%K climate climatemodeling dynamics idea:bee_audio_llm idea:big_data_geo_2 representationlearning
%T AtmoDist: Self-supervised Representation Learning for Atmospheric
Dynamics
%U http://arxiv.org/abs/2202.01897
%X Representation learning has proven to be a powerful methodology in a wide
variety of machine learning applications. For atmospheric dynamics, however, it
has so far not been considered, arguably due to the lack of large-scale,
labeled datasets that could be used for training. In this work, we show that
the difficulty is benign and introduce a self-supervised learning task that
defines a categorical loss for a wide variety of unlabeled atmospheric
datasets. Specifically, we train a neural network on the simple yet intricate
task of predicting the temporal distance between atmospheric fields from
distinct but nearby times. We demonstrate that training with this task on ERA5
reanalysis leads to internal representations capturing intrinsic aspects of
atmospheric dynamics. We do so by introducing a data-driven distance metric for
atmospheric states. When employed as a loss function in other machine learning
applications, this Atmodist distance leads to improved results compared to the
classical $\ell_2$-loss. For example, for downscaling one obtains higher
resolution fields that match the true statistics more closely than previous
approaches and for the interpolation of missing or occluded data the AtmoDist
distance leads to results that contain more realistic fine scale features.
Since it is derived from observational data, AtmoDist also provides a novel
perspective on atmospheric predictability.
@misc{hoffmann2022atmodist,
abstract = {Representation learning has proven to be a powerful methodology in a wide
variety of machine learning applications. For atmospheric dynamics, however, it
has so far not been considered, arguably due to the lack of large-scale,
labeled datasets that could be used for training. In this work, we show that
the difficulty is benign and introduce a self-supervised learning task that
defines a categorical loss for a wide variety of unlabeled atmospheric
datasets. Specifically, we train a neural network on the simple yet intricate
task of predicting the temporal distance between atmospheric fields from
distinct but nearby times. We demonstrate that training with this task on ERA5
reanalysis leads to internal representations capturing intrinsic aspects of
atmospheric dynamics. We do so by introducing a data-driven distance metric for
atmospheric states. When employed as a loss function in other machine learning
applications, this Atmodist distance leads to improved results compared to the
classical $\ell_2$-loss. For example, for downscaling one obtains higher
resolution fields that match the true statistics more closely than previous
approaches and for the interpolation of missing or occluded data the AtmoDist
distance leads to results that contain more realistic fine scale features.
Since it is derived from observational data, AtmoDist also provides a novel
perspective on atmospheric predictability.},
added-at = {2023-04-20T16:04:13.000+0200},
author = {Hoffmann, Sebastian and Lessig, Christian},
biburl = {https://www.bibsonomy.org/bibtex/20fd9302a4e0892b640742538e32444d3/annakrause},
description = {[2202.01897] AtmoDist: Self-supervised Representation Learning for Atmospheric Dynamics},
interhash = {53ee6971a4207f62e79047d07ecb940f},
intrahash = {0fd9302a4e0892b640742538e32444d3},
keywords = {climate climatemodeling dynamics idea:bee_audio_llm idea:big_data_geo_2 representationlearning},
note = {cite arxiv:2202.01897Comment: Submitted to "Environmental Data Science", Cambridge University Press. Revised version. Journal-version of "Towards Representation Learning for Atmospheric Dynamics. arXiv:2109.09076"},
timestamp = {2023-04-20T16:04:30.000+0200},
title = {AtmoDist: Self-supervised Representation Learning for Atmospheric
Dynamics},
url = {http://arxiv.org/abs/2202.01897},
year = 2022
}