A dataset plugin for climetlab for the dataset maelstrom-radiation.
Project description
maelstrom-radiation
A dataset plugin for climetlab (https://climetlab.readthedocs.io/en/latest/) for the dataset maelstrom-radiation.
Datasets description
This data is for learning the emulation of the ECMWF radiation scheme, using the McICA solver, found in the ecRad package (https://github.com/ecmwf/ecrad). Building an accurate emulator of radiative heating could accelerate weather and climate models partially by enabling the use of GPU hardware within our models.
There are two datasets, allowing different views on the same data:
1 : maelstom-radiation
Supports the to_xarray method and allows users to explore the data with all structure kept intact.
2 : maelstrom-radiation-tf
Loads the same data but from a shuffled and repacked into the TFRecord format. This dataset supports
to_tfdataset which uses Tensorflow's dataset object.
Using climetlab to access the data
Both datasets and downloaded and explained in the demo notebook here https://git.ecmwf.int/projects/MLFET/repos/maelstrom-radiation/browse/notebooks/demo_radiation.ipynb
The climetlab python package allows easy access to the data with a few lines of code such as:
!pip install climetlab climetlab_maelstrom_radiation
import climetlab as cml
cml_ds = cml.load_dataset("maelstrom-radiation", subset="tier-1")
ds = cml_ds.to_xarray()
!or for the TFdataset version
cml_ds = cml.load_dataset("maelstrom-radiation-tf", subset="tier-1")
ds = cml_ds.to_tfdataset(batch_size=256)
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for climetlab_maelstrom_radiation-0.7.5.tar.gz
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 32aeff7e009625fe3dbf6eeb2020c3a77d81f01ba84ca68d27bca3ce22b0cdc1 |
|
| MD5 | 6f66519be0f9c22c31e6a7c8973dba85 |
|
| BLAKE2b-256 | 768b0509e939007a7900c81082401db150a808a71c917c29edf573be2023485b |
Hashes for climetlab_maelstrom_radiation-0.7.5-py3-none-any.whl
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | c3be2dc5257879a8df3822f5a1e2b5b52a180829b1fdf5f6f5acc0c3b20afc8e |
|
| MD5 | a12c4faa4dab7e47e0fc915b71382eb4 |
|
| BLAKE2b-256 | f853a6fea4f5e7fe679dbdcd47ef6d1fed2c8f83709e4f25092a07858c7e9369 |
