EO-LINCS project: Cube generation for Scientific Case Study (SCS) 3¶

Model-Data Fusion for Understanding Carbon State-Flux Relationships Across Space¶

Objective: SCS3 aims to improve the accuracy of ecosystem carbon cycle models by integrating in-situ observation networks with complementary EO products as inputs. The data cubes will be generated in a manner that allows seamless integration within the SINDBAD ecosystem modelling framework.

Outcomes: Use a terrestrial carbon model structure that can deliver a process understanding of carbon stateflux relationships across space by leveraging and cross-comparing EO data of biomass and vegetation states (fAPAR, LAI, vegetation fraction, etc.) together with ecosystem carbon flux measurements and estimates. Additionally, provide open-source novel MDI tools and workflows for community use paving the way for adoption by other data-assimilation frameworks.

Required datasets:

• Sentinel-2 L2A from CDSE
• ERA-5 land
• ESA CCI Biomass

---

The following notebook shows how the users can load data from various sources defined in scs3_config.yml using the MultiSourceDataStore tool.

What You Can Do with This Notebook¶

• Generate a configuration file based on fluxtower locations
• Load datasets from various sources as defined in the generated scs3_config.yml
• View the progress of each data request to the MultiSourceDataStore
• Quickly preview the datasets by plotting them.

Requirements¶

Before you begin, follow these steps:

• Install xcube-multistore via conda-forge by running: conda install --channel conda-forge xcube-multistore
• To access EO data via S3 from CDSE, generate your S3 credentials and add them to the data_stores section in the scs3_config.yml file.
• To access ERA5-Land data from the Copernicus Climate Data Store, obtain a CDS Personal Access Token by creating an account on the CDS Website. After logging in, navigate to your user page to find your Personal Access Token. Add this token to the data_stores section in the scs3_config.yml file.

Once you have it installed, you are ready to proceed.

This Multistore mainly works with a file called scs3_config.yml which is at the same file level as this notebook. To understand what goes into the schema, you can read more here.

---

Let's import the MultiSourceDataStore

In [1]:

import yaml

import pandas as pd
from xcube_multistore import MultiSourceDataStore
from xcube_multistore.utils import get_bbox

import yaml import pandas as pd from xcube_multistore import MultiSourceDataStore from xcube_multistore.utils import get_bbox

You can find out how to fill out the config file by also using this super helpful function get_config_schema(). Run it and try expand the fields to learn more about the possible properties that the configuration file accepts along with the Configuration Guide.

In [2]:

MultiSourceDataStore.get_config_schema()

MultiSourceDataStore.get_config_schema()

Out[2]:

<xcube.util.jsonschema.JsonObjectSchema at 0x78fb45c7d1d0>

This science case requires data from several flux sites. So we define them in a scs3_sites.csv file for easier management and access. For the purpose of this example, we will focus on the first 3 sites

In [3]:

sites = pd.read_csv("scs3_sites.csv")
sites = sites.iloc[:3]
sites

sites = pd.read_csv("scs3_sites.csv") sites = sites.iloc[:3] sites

Out[3]:

	Site ID	latitude	longitude	IGBP
0	AU-Dry	-15.2588	132.3706	SAV
1	AU-How	-12.4943	131.1523	WSA
2	BE-Lon	50.5516	4.7462	CRO

In the following cell, we will create the config object which will then be saved as scs3_config.yml for persistance and ready to be read by MultiSourceDataStore.

To read more about how this config file is structured, you can find the Configuration Guide here.

Specifically, we are using the single dataset object and data stores schemas here.

In [4]:

config = dict(datasets=[])
for index, site in sites.iterrows():
    bbox_final, crs_final = get_bbox(
        site["latitude"], site["longitude"], cube_width=4000, crs_final="utm"
    )

    # append config for Sentinel-2
    config_ds = dict(
        identifier=f"{site['Site ID']}_sen2",
        store="stac-cdse",
        data_id="sentinel-2-l2a",
        open_params=dict(
            time_range=["2019-03-01", "2019-03-15"],
            bbox=bbox_final,
            spatial_res=10,
            crs=f"EPSG:{crs_final.to_epsg()}",
            asset_names=[
                "B01",
                "B02",
                "B03",
                "B04",
                "B05",
                "B06",
                "B07",
                "B08",
                "B8A",
                "B09",
                "B11",
                "B12",
                "SCL",
            ],
        ),
    )
    config["datasets"].append(config_ds)

    # append config for ERA5
    config_ds = dict(
        identifier=f"{site['Site ID']}_era5land",
        store="cds",
        data_id="reanalysis-era5-land",
        open_params=dict(
            variable_names=["2m_temperature", "total_precipitation"],
            time_range=["2019-03-01", "2019-03-15"],
            point=[site["latitude"], site["longitude"]],
            spatial_res=0.1,
        ),
    )
    config["datasets"].append(config_ds)

    # append config for ESA CCI
    config_ds = dict(
        identifier=f"{site['Site ID']}_ccibiomass",
        store="esa_cci",
        grid_mapping=f"{site['Site ID']}_sen2",
        data_id="esacci.BIOMASS.yr.L4.AGB.multi-sensor.multi-platform.MERGED.5-0.100m",
        open_params=dict(
            time_range=["2019-01-01", "2019-12-31"],
        ),
    )
    config["datasets"].append(config_ds)

# define stores
config["data_stores"] = []
# add storage data store
config_store = dict(
    identifier="storage",
    store_id="file",
    store_params=dict(root="data"),
)
config["data_stores"].append(config_store)
# add ESA CCI data store
config_store = dict(
    identifier="esa_cci",
    store_id="cciodp",
)
config["data_stores"].append(config_store)
# add STAC data store
config_store = dict(
    identifier="stac-cdse",
    store_id="stac-cdse-ardc",
    store_params=dict(
        key="<CDSE_S3_key>",
        secret="<CDSE_S3_secret>",
    ),
)
config["data_stores"].append(config_store)
# add CDS data store
config_store = dict(
    identifier="cds",
    store_id="cds",
    store_params=dict(
        endpoint_url="https://cds.climate.copernicus.eu/api",
        cds_api_key="<cds_personal_access_token>",,
        normalize_names=True,
    ),
)
config["data_stores"].append(config_store)

config = dict(datasets=[]) for index, site in sites.iterrows(): bbox_final, crs_final = get_bbox( site["latitude"], site["longitude"], cube_width=4000, crs_final="utm" ) # append config for Sentinel-2 config_ds = dict( identifier=f"{site['Site ID']}_sen2", store="stac-cdse", data_id="sentinel-2-l2a", open_params=dict( time_range=["2019-03-01", "2019-03-15"], bbox=bbox_final, spatial_res=10, crs=f"EPSG:{crs_final.to_epsg()}", asset_names=[ "B01", "B02", "B03", "B04", "B05", "B06", "B07", "B08", "B8A", "B09", "B11", "B12", "SCL", ], ), ) config["datasets"].append(config_ds) # append config for ERA5 config_ds = dict( identifier=f"{site['Site ID']}_era5land", store="cds", data_id="reanalysis-era5-land", open_params=dict( variable_names=["2m_temperature", "total_precipitation"], time_range=["2019-03-01", "2019-03-15"], point=[site["latitude"], site["longitude"]], spatial_res=0.1, ), ) config["datasets"].append(config_ds) # append config for ESA CCI config_ds = dict( identifier=f"{site['Site ID']}_ccibiomass", store="esa_cci", grid_mapping=f"{site['Site ID']}_sen2", data_id="esacci.BIOMASS.yr.L4.AGB.multi-sensor.multi-platform.MERGED.5-0.100m", open_params=dict( time_range=["2019-01-01", "2019-12-31"], ), ) config["datasets"].append(config_ds) # define stores config["data_stores"] = [] # add storage data store config_store = dict( identifier="storage", store_id="file", store_params=dict(root="data"), ) config["data_stores"].append(config_store) # add ESA CCI data store config_store = dict( identifier="esa_cci", store_id="cciodp", ) config["data_stores"].append(config_store) # add STAC data store config_store = dict( identifier="stac-cdse", store_id="stac-cdse-ardc", store_params=dict( key="", secret="", ), ) config["data_stores"].append(config_store) # add CDS data store config_store = dict( identifier="cds", store_id="cds", store_params=dict( endpoint_url="https://cds.climate.copernicus.eu/api", cds_api_key="",, normalize_names=True, ), ) config["data_stores"].append(config_store)

In [5]:

with open("scs3_config.yml", "w") as file:
    yaml.dump(config, file, sort_keys=False)

with open("scs3_config.yml", "w") as file: yaml.dump(config, file, sort_keys=False)

We can view the configuration using the method display_config, executing the follwoing cell.

In [2]:

MultiSourceDataStore.display_config("scs3_config.yml")

MultiSourceDataStore.display_config("scs3_config.yml")

Configuration

User-defined ID	Data Store ID	Data Store Params	Data ID	Open Data Params	Grid-Mapping	Format
AU-Dry_sen2	stac-cdse-ardc	key: <CDSE_S3_key>; secret: <CDSE_S3_secret>	sentinel-2-l2a	time_range: ['2019-03-01', '2019-03-15']; bbox: [215577, 8309343, 219577, 8313343]; spatial_res: 10; crs: EPSG:32753; asset_names: ['B01', 'B02', 'B03', 'B04', 'B05', 'B06', 'B07', 'B08', 'B8A', 'B09', 'B11', 'B12', 'SCL']	-	Zarr
AU-Dry_era5land	cds	endpoint_url: https://cds.climate.copernicus.eu/api; cds_api_key: <cds_personal_access_token>; normalize_names: True	reanalysis-era5-land	variable_names: ['2m_temperature', 'total_precipitation']; time_range: ['2019-03-01', '2019-03-15']; point: [-15.2588, 132.3706]; spatial_res: 0.1	-	Zarr
AU-Dry_ccibiomass	cciodp	-	esacci.BIOMASS.yr.L4.AGB.multi-sensor.multi-platform.MERGED.5-0.100m	time_range: ['2019-01-01', '2019-12-31']	Like 'AU-Dry_sen2'	Zarr
AU-How_sen2	stac-cdse-ardc	key: <CDSE_S3_key>; secret: <CDSE_S3_secret>	sentinel-2-l2a	time_range: ['2019-03-01', '2019-03-15']; bbox: [731912, 8615836, 735912, 8619836]; spatial_res: 10; crs: EPSG:32752; asset_names: ['B01', 'B02', 'B03', 'B04', 'B05', 'B06', 'B07', 'B08', 'B8A', 'B09', 'B11', 'B12', 'SCL']	-	Zarr
AU-How_era5land	cds	endpoint_url: https://cds.climate.copernicus.eu/api; cds_api_key: <cds_personal_access_token>; normalize_names: True	reanalysis-era5-land	variable_names: ['2m_temperature', 'total_precipitation']; time_range: ['2019-03-01', '2019-03-15']; point: [-12.4943, 131.1523]; spatial_res: 0.1	-	Zarr
AU-How_ccibiomass	cciodp	-	esacci.BIOMASS.yr.L4.AGB.multi-sensor.multi-platform.MERGED.5-0.100m	time_range: ['2019-01-01', '2019-12-31']	Like 'AU-How_sen2'	Zarr
BE-Lon_sen2	stac-cdse-ardc	key: <CDSE_S3_key>; secret: <CDSE_S3_secret>	sentinel-2-l2a	time_range: ['2019-03-01', '2019-03-15']; bbox: [621704, 5599419, 625704, 5603419]; spatial_res: 10; crs: EPSG:32631; asset_names: ['B01', 'B02', 'B03', 'B04', 'B05', 'B06', 'B07', 'B08', 'B8A', 'B09', 'B11', 'B12', 'SCL']	-	Zarr
BE-Lon_era5land	cds	endpoint_url: https://cds.climate.copernicus.eu/api; cds_api_key: <cds_personal_access_token>; normalize_names: True	reanalysis-era5-land	variable_names: ['2m_temperature', 'total_precipitation']; time_range: ['2019-03-01', '2019-03-15']; point: [50.5516, 4.7462]; spatial_res: 0.1	-	Zarr
BE-Lon_ccibiomass	cciodp	-	esacci.BIOMASS.yr.L4.AGB.multi-sensor.multi-platform.MERGED.5-0.100m	time_range: ['2019-01-01', '2019-12-31']	Like 'BE-Lon_sen2'	Zarr

Now, we can initialize the MultiSourceDataStore by passing the path to the scs3_config.yml` which currently is on the same level as this notebook.

By running the cell below, you would start seeing progress tables for each data that you requested in the scs3_config.yml.

In [6]:

msds = MultiSourceDataStore("scs3_config.yml")

msds = MultiSourceDataStore("scs3_config.yml")

Cube Generation

Dataset identifier	Status	Message	Exception
AU-Dry_sen2	COMPLETED	Dataset 'AU-Dry_sen2' already generated.	-
AU-Dry_era5land	COMPLETED	Dataset 'AU-Dry_era5land' already generated.	-
AU-Dry_ccibiomass	COMPLETED	Dataset 'AU-Dry_ccibiomass' already generated.	-
AU-How_sen2	COMPLETED	Dataset 'AU-How_sen2' already generated.	-
AU-How_era5land	COMPLETED	Dataset 'AU-How_era5land' already generated.	-
AU-How_ccibiomass	COMPLETED	Dataset 'AU-How_ccibiomass' already generated.	-
BE-Lon_sen2	COMPLETED	Dataset 'BE-Lon_sen2' already generated.	-
BE-Lon_era5land	COMPLETED	Dataset 'BE-Lon_era5land' already generated.	-
BE-Lon_ccibiomass	COMPLETED	Dataset 'BE-Lon_ccibiomass' already generated.	-

We can now open the data using the xcube datastore framework API as usual. Note that the multi-source data store requires a data store called storage, which is configured in our config.yml under the data_stores section.

In [7]:

ds = msds.stores.storage.open_data("AU-Dry_sen2.zarr")
ds

ds = msds.stores.storage.open_data("AU-Dry_sen2.zarr") ds

Out[7]:

<xarray.Dataset> Size: 25MB
Dimensions:      (time: 3, y: 401, x: 401)
Coordinates:
    spatial_ref  int64 8B ...
  * time         (time) datetime64[ns] 24B 2019-03-03T01:27:11.024000 ... 201...
  * x            (x) float64 3kB 2.156e+05 2.156e+05 ... 2.196e+05 2.196e+05
  * y            (y) float64 3kB 8.313e+06 8.313e+06 ... 8.309e+06 8.309e+06
Data variables: (12/13)
    B01          (time, y, x) float32 2MB dask.array<chunksize=(1, 401, 401), meta=np.ndarray>
    B02          (time, y, x) float32 2MB dask.array<chunksize=(1, 401, 401), meta=np.ndarray>
    B03          (time, y, x) float32 2MB dask.array<chunksize=(1, 401, 401), meta=np.ndarray>
    B04          (time, y, x) float32 2MB dask.array<chunksize=(1, 401, 401), meta=np.ndarray>
    B05          (time, y, x) float32 2MB dask.array<chunksize=(1, 401, 401), meta=np.ndarray>
    B06          (time, y, x) float32 2MB dask.array<chunksize=(1, 401, 401), meta=np.ndarray>
    ...           ...
    B08          (time, y, x) float32 2MB dask.array<chunksize=(1, 401, 401), meta=np.ndarray>
    B09          (time, y, x) float32 2MB dask.array<chunksize=(1, 401, 401), meta=np.ndarray>
    B11          (time, y, x) float32 2MB dask.array<chunksize=(1, 401, 401), meta=np.ndarray>
    B12          (time, y, x) float32 2MB dask.array<chunksize=(1, 401, 401), meta=np.ndarray>
    B8A          (time, y, x) float32 2MB dask.array<chunksize=(1, 401, 401), meta=np.ndarray>
    SCL          (time, y, x) float32 2MB dask.array<chunksize=(1, 401, 401), meta=np.ndarray>
Attributes:
    stac_catalog_url:    https://stac.dataspace.copernicus.eu/v1
    stac_item_ids:       {'2019-03-03T01:27:11.024000': ['S2A_MSIL2A_20190303...
    xcube_stac_version:  1.1.0

xarray.Dataset

• Dimensions:
  • time: 3
  • y: 401
  • x: 401
• Coordinates: (4)
  • spatial_ref
    ()
    int64
    ...

    crs_wkt :

    PROJCRS["WGS 84 / UTM zone 53S",BASEGEOGCRS["WGS 84",DATUM["World Geodetic System 1984",ELLIPSOID["WGS 84",6378137,298.257223563,LENGTHUNIT["metre",1]]],PRIMEM["Greenwich",0,ANGLEUNIT["degree",0.0174532925199433]],ID["EPSG",4326]],CONVERSION["UTM zone 53S",METHOD["Transverse Mercator",ID["EPSG",9807]],PARAMETER["Latitude of natural origin",0,ANGLEUNIT["degree",0.0174532925199433],ID["EPSG",8801]],PARAMETER["Longitude of natural origin",135,ANGLEUNIT["degree",0.0174532925199433],ID["EPSG",8802]],PARAMETER["Scale factor at natural origin",0.9996,SCALEUNIT["unity",1],ID["EPSG",8805]],PARAMETER["False easting",500000,LENGTHUNIT["metre",1],ID["EPSG",8806]],PARAMETER["False northing",10000000,LENGTHUNIT["metre",1],ID["EPSG",8807]]],CS[Cartesian,2],AXIS["easting",east,ORDER[1],LENGTHUNIT["metre",1]],AXIS["northing",north,ORDER[2],LENGTHUNIT["metre",1]],ID["EPSG",32753]]

    false_easting :

    500000.0

    false_northing :

    10000000.0

    geographic_crs_name :

    WGS 84

    grid_mapping_name :

    transverse_mercator

    horizontal_datum_name :

    World Geodetic System 1984

    inverse_flattening :

    298.257223563

    latitude_of_projection_origin :

    0.0

    longitude_of_central_meridian :

    135.0

    longitude_of_prime_meridian :

    0.0

    prime_meridian_name :

    Greenwich

    projected_crs_name :

    WGS 84 / UTM zone 53S

    reference_ellipsoid_name :

    WGS 84

    scale_factor_at_central_meridian :

    0.9996

    semi_major_axis :

    6378137.0

    semi_minor_axis :

    6356752.314245179

    [1 values with dtype=int64]

  • time
    (time)
    datetime64[ns]
    2019-03-03T01:27:11.024000 ... 2...

    array(['2019-03-03T01:27:11.024000000', '2019-03-08T01:27:19.024000000',
           '2019-03-13T01:27:11.024000000'], dtype='datetime64[ns]')

  • x
    (x)
    float64
    2.156e+05 2.156e+05 ... 2.196e+05

    array([215575., 215585., 215595., ..., 219555., 219565., 219575.], shape=(401,))

  • y
    (y)
    float64
    8.313e+06 8.313e+06 ... 8.309e+06

    array([8313345., 8313335., 8313325., ..., 8309365., 8309355., 8309345.],
          shape=(401,))

• Data variables: (13)
  • B01
    (time, y, x)
    float32
    dask.array<chunksize=(1, 401, 401), meta=np.ndarray>

    grid_mapping :

    spatial_ref

    Array Chunk Bytes 1.84 MiB 628.13 kiB Shape (3, 401, 401) (1, 401, 401) Dask graph 3 chunks in 2 graph layers Data type float32 numpy.ndarray

  • B02
    (time, y, x)
    float32
    dask.array<chunksize=(1, 401, 401), meta=np.ndarray>

    grid_mapping :

    spatial_ref

    Array Chunk Bytes 1.84 MiB 628.13 kiB Shape (3, 401, 401) (1, 401, 401) Dask graph 3 chunks in 2 graph layers Data type float32 numpy.ndarray

  • B03
    (time, y, x)
    float32
    dask.array<chunksize=(1, 401, 401), meta=np.ndarray>

    grid_mapping :

    spatial_ref

    Array Chunk Bytes 1.84 MiB 628.13 kiB Shape (3, 401, 401) (1, 401, 401) Dask graph 3 chunks in 2 graph layers Data type float32 numpy.ndarray

  • B04
    (time, y, x)
    float32
    dask.array<chunksize=(1, 401, 401), meta=np.ndarray>

    grid_mapping :

    spatial_ref

    Array Chunk Bytes 1.84 MiB 628.13 kiB Shape (3, 401, 401) (1, 401, 401) Dask graph 3 chunks in 2 graph layers Data type float32 numpy.ndarray

  • B05
    (time, y, x)
    float32
    dask.array<chunksize=(1, 401, 401), meta=np.ndarray>

    grid_mapping :

    spatial_ref

    Array Chunk Bytes 1.84 MiB 628.13 kiB Shape (3, 401, 401) (1, 401, 401) Dask graph 3 chunks in 2 graph layers Data type float32 numpy.ndarray

  • B06
    (time, y, x)
    float32
    dask.array<chunksize=(1, 401, 401), meta=np.ndarray>

    grid_mapping :

    spatial_ref

    Array Chunk Bytes 1.84 MiB 628.13 kiB Shape (3, 401, 401) (1, 401, 401) Dask graph 3 chunks in 2 graph layers Data type float32 numpy.ndarray

  • B07
    (time, y, x)
    float32
    dask.array<chunksize=(1, 401, 401), meta=np.ndarray>

    grid_mapping :

    spatial_ref

    Array Chunk Bytes 1.84 MiB 628.13 kiB Shape (3, 401, 401) (1, 401, 401) Dask graph 3 chunks in 2 graph layers Data type float32 numpy.ndarray

  • B08
    (time, y, x)
    float32
    dask.array<chunksize=(1, 401, 401), meta=np.ndarray>

    grid_mapping :

    spatial_ref

    Array Chunk Bytes 1.84 MiB 628.13 kiB Shape (3, 401, 401) (1, 401, 401) Dask graph 3 chunks in 2 graph layers Data type float32 numpy.ndarray

  • B09
    (time, y, x)
    float32
    dask.array<chunksize=(1, 401, 401), meta=np.ndarray>

    grid_mapping :

    spatial_ref

    Array Chunk Bytes 1.84 MiB 628.13 kiB Shape (3, 401, 401) (1, 401, 401) Dask graph 3 chunks in 2 graph layers Data type float32 numpy.ndarray

  • B11
    (time, y, x)
    float32
    dask.array<chunksize=(1, 401, 401), meta=np.ndarray>

    grid_mapping :

    spatial_ref

    Array Chunk Bytes 1.84 MiB 628.13 kiB Shape (3, 401, 401) (1, 401, 401) Dask graph 3 chunks in 2 graph layers Data type float32 numpy.ndarray

  • B12
    (time, y, x)
    float32
    dask.array<chunksize=(1, 401, 401), meta=np.ndarray>

    grid_mapping :

    spatial_ref

    Array Chunk Bytes 1.84 MiB 628.13 kiB Shape (3, 401, 401) (1, 401, 401) Dask graph 3 chunks in 2 graph layers Data type float32 numpy.ndarray

  • B8A
    (time, y, x)
    float32
    dask.array<chunksize=(1, 401, 401), meta=np.ndarray>

    grid_mapping :

    spatial_ref

    Array Chunk Bytes 1.84 MiB 628.13 kiB Shape (3, 401, 401) (1, 401, 401) Dask graph 3 chunks in 2 graph layers Data type float32 numpy.ndarray

  • SCL
    (time, y, x)
    float32
    dask.array<chunksize=(1, 401, 401), meta=np.ndarray>

    grid_mapping :

    spatial_ref

    Array Chunk Bytes 1.84 MiB 628.13 kiB Shape (3, 401, 401) (1, 401, 401) Dask graph 3 chunks in 2 graph layers Data type float32 numpy.ndarray

• Indexes: (3)
  • time
    PandasIndex

    PandasIndex(DatetimeIndex(['2019-03-03 01:27:11.024000', '2019-03-08 01:27:19.024000',
                   '2019-03-13 01:27:11.024000'],
                  dtype='datetime64[ns]', name='time', freq=None))

  • x
    PandasIndex

    PandasIndex(Index([215575.0, 215585.0, 215595.0, 215605.0, 215615.0, 215625.0, 215635.0,
           215645.0, 215655.0, 215665.0,
           ...
           219485.0, 219495.0, 219505.0, 219515.0, 219525.0, 219535.0, 219545.0,
           219555.0, 219565.0, 219575.0],
          dtype='float64', name='x', length=401))

  • y
    PandasIndex

    PandasIndex(Index([8313345.0, 8313335.0, 8313325.0, 8313315.0, 8313305.0, 8313295.0,
           8313285.0, 8313275.0, 8313265.0, 8313255.0,
           ...
           8309435.0, 8309425.0, 8309415.0, 8309405.0, 8309395.0, 8309385.0,
           8309375.0, 8309365.0, 8309355.0, 8309345.0],
          dtype='float64', name='y', length=401))

• Attributes: (3)

  stac_catalog_url :

  https://stac.dataspace.copernicus.eu/v1

  stac_item_ids :

  {'2019-03-03T01:27:11.024000': ['S2A_MSIL2A_20190303T012711_N0500_R131_T52LHJ_20221209T122536', 'S2A_MSIL2A_20190303T012711_N0500_R131_T53LKD_20221209T122536'], '2019-03-08T01:27:19.024000': ['S2B_MSIL2A_20190308T012719_N0500_R131_T52LHJ_20221120T105602', 'S2B_MSIL2A_20190308T012719_N0500_R131_T53LKD_20221120T105602'], '2019-03-13T01:27:11.024000': ['S2A_MSIL2A_20190313T012711_N0500_R131_T52LHJ_20221118T074550', 'S2A_MSIL2A_20190313T012711_N0500_R131_T53LKD_20221118T074550']}

  xcube_stac_version :

  1.1.0

We can now select a variable for one timestep and plot it for a quick preview of the data

In [8]:

ds.B04.isel(time=0).plot(vmin=0., vmax=0.2)

ds.B04.isel(time=0).plot(vmin=0., vmax=0.2)

Out[8]:

<matplotlib.collections.QuadMesh at 0x78fb4181f770>

In [9]:

ds = msds.stores.storage.open_data("AU-Dry_era5land.zarr")
ds

ds = msds.stores.storage.open_data("AU-Dry_era5land.zarr") ds

Out[9]:

<xarray.Dataset> Size: 12kB
Dimensions:  (time: 360)
Coordinates:
    expver   (time) <U4 6kB dask.array<chunksize=(360,), meta=np.ndarray>
    lat      float64 8B ...
    lon      float64 8B ...
    number   int64 8B ...
  * time     (time) datetime64[ns] 3kB 2019-03-01 ... 2019-03-15T23:00:00
Data variables:
    t2m      (time) float32 1kB dask.array<chunksize=(360,), meta=np.ndarray>
    tp       (time) float32 1kB dask.array<chunksize=(360,), meta=np.ndarray>
Attributes:
    Conventions:             CF-1.7
    GRIB_centre:             ecmf
    GRIB_centreDescription:  European Centre for Medium-Range Weather Forecasts
    GRIB_subCentre:          0
    history:                 2025-09-11T11:57 GRIB to CDM+CF via cfgrib-0.9.1...
    institution:             European Centre for Medium-Range Weather Forecasts

xarray.Dataset

• Dimensions:
  • time: 360
• Coordinates: (5)
  • expver
    (time)
    <U4
    dask.array<chunksize=(360,), meta=np.ndarray>

    Array Chunk Bytes 5.62 kiB 5.62 kiB Shape (360,) (360,) Dask graph 1 chunks in 2 graph layers Data type

  • lat
    ()
    float64
    ...

    long_name :

    latitude

    standard_name :

    latitude

    stored_direction :

    decreasing

    units :

    degrees_north

    [1 values with dtype=float64]

  • lon
    ()
    float64
    ...

    long_name :

    longitude

    standard_name :

    longitude

    units :

    degrees_east

    [1 values with dtype=float64]

  • number
    ()
    int64
    ...

    long_name :

    ensemble member numerical id

    standard_name :

    realization

    units :

    1

    [1 values with dtype=int64]

  • time
    (time)
    datetime64[ns]
    2019-03-01 ... 2019-03-15T23:00:00

    long_name :

    time

    standard_name :

    time

    array(['2019-03-01T00:00:00.000000000', '2019-03-01T01:00:00.000000000',
           '2019-03-01T02:00:00.000000000', ..., '2019-03-15T21:00:00.000000000',
           '2019-03-15T22:00:00.000000000', '2019-03-15T23:00:00.000000000'],
          shape=(360,), dtype='datetime64[ns]')

• Data variables: (2)
  • t2m
    (time)
    float32
    dask.array<chunksize=(360,), meta=np.ndarray>

    GRIB_NV :

    0

    GRIB_Nx :

    4

    GRIB_Ny :

    4

    GRIB_cfName :

    unknown

    GRIB_cfVarName :

    t2m

    GRIB_dataType :

    fc

    GRIB_gridDefinitionDescription :

    Latitude/Longitude Grid

    GRIB_gridType :

    regular_ll

    GRIB_iDirectionIncrementInDegrees :

    0.1

    GRIB_iScansNegatively :

    0

    GRIB_jDirectionIncrementInDegrees :

    0.1

    GRIB_jPointsAreConsecutive :

    0

    GRIB_jScansPositively :

    0

    GRIB_latitudeOfFirstGridPointInDegrees :

    -15.108

    GRIB_latitudeOfLastGridPointInDegrees :

    -15.409

    GRIB_longitudeOfFirstGridPointInDegrees :

    132.22

    GRIB_longitudeOfLastGridPointInDegrees :

    132.521

    GRIB_missingValue :

    3.4028234663852886e+38

    GRIB_name :

    2 metre temperature

    GRIB_numberOfPoints :

    16

    GRIB_paramId :

    167

    GRIB_shortName :

    2t

    GRIB_stepType :

    instant

    GRIB_stepUnits :

    1

    GRIB_surface :

    0.0

    GRIB_totalNumber :

    0

    GRIB_typeOfLevel :

    surface

    GRIB_units :

    K

    GRIB_uvRelativeToGrid :

    0

    long_name :

    2 metre temperature

    standard_name :

    unknown

    units :

    K

    Array Chunk Bytes 1.41 kiB 1.41 kiB Shape (360,) (360,) Dask graph 1 chunks in 2 graph layers Data type float32 numpy.ndarray

  • tp
    (time)
    float32
    dask.array<chunksize=(360,), meta=np.ndarray>

    GRIB_NV :

    0

    GRIB_Nx :

    4

    GRIB_Ny :

    4

    GRIB_cfName :

    unknown

    GRIB_cfVarName :

    tp

    GRIB_dataType :

    fc

    GRIB_gridDefinitionDescription :

    Latitude/Longitude Grid

    GRIB_gridType :

    regular_ll

    GRIB_iDirectionIncrementInDegrees :

    0.1

    GRIB_iScansNegatively :

    0

    GRIB_jDirectionIncrementInDegrees :

    0.1

    GRIB_jPointsAreConsecutive :

    0

    GRIB_jScansPositively :

    0

    GRIB_latitudeOfFirstGridPointInDegrees :

    -15.108

    GRIB_latitudeOfLastGridPointInDegrees :

    -15.409

    GRIB_longitudeOfFirstGridPointInDegrees :

    132.22

    GRIB_longitudeOfLastGridPointInDegrees :

    132.521

    GRIB_missingValue :

    3.4028234663852886e+38

    GRIB_name :

    Total precipitation

    GRIB_numberOfPoints :

    16

    GRIB_paramId :

    228

    GRIB_shortName :

    tp

    GRIB_stepType :

    accum

    GRIB_stepUnits :

    1

    GRIB_surface :

    0.0

    GRIB_totalNumber :

    0

    GRIB_typeOfLevel :

    surface

    GRIB_units :

    m

    GRIB_uvRelativeToGrid :

    0

    long_name :

    Total precipitation

    standard_name :

    unknown

    units :

    m

    Array Chunk Bytes 1.41 kiB 1.41 kiB Shape (360,) (360,) Dask graph 1 chunks in 2 graph layers Data type float32 numpy.ndarray

• Indexes: (1)
  • time
    PandasIndex

    PandasIndex(DatetimeIndex(['2019-03-01 00:00:00', '2019-03-01 01:00:00',
                   '2019-03-01 02:00:00', '2019-03-01 03:00:00',
                   '2019-03-01 04:00:00', '2019-03-01 05:00:00',
                   '2019-03-01 06:00:00', '2019-03-01 07:00:00',
                   '2019-03-01 08:00:00', '2019-03-01 09:00:00',
                   ...
                   '2019-03-15 14:00:00', '2019-03-15 15:00:00',
                   '2019-03-15 16:00:00', '2019-03-15 17:00:00',
                   '2019-03-15 18:00:00', '2019-03-15 19:00:00',
                   '2019-03-15 20:00:00', '2019-03-15 21:00:00',
                   '2019-03-15 22:00:00', '2019-03-15 23:00:00'],
                  dtype='datetime64[ns]', name='time', length=360, freq=None))

• Attributes: (6)

  Conventions :

  CF-1.7

  GRIB_centre :

  ecmf

  GRIB_centreDescription :

  European Centre for Medium-Range Weather Forecasts

  GRIB_subCentre :

  0

  history :

  2025-09-11T11:57 GRIB to CDM+CF via cfgrib-0.9.15.0/ecCodes-2.42.0 with {"source": "data.grib", "filter_by_keys": {}, "encode_cf": ["parameter", "time", "geography", "vertical"]}

  institution :

  European Centre for Medium-Range Weather Forecasts

In [10]:

ds.t2m.plot()

ds.t2m.plot()

Out[10]:

[<matplotlib.lines.Line2D at 0x78fb4109ed50>]

In [11]:

ds = msds.stores.storage.open_data("AU-Dry_ccibiomass.zarr")
ds

ds = msds.stores.storage.open_data("AU-Dry_ccibiomass.zarr") ds

Out[11]:

<xarray.Dataset> Size: 3MB
Dimensions:      (time: 1, y: 401, x: 401)
Coordinates:
    spatial_ref  int64 8B ...
  * time         (time) datetime64[ns] 8B 2019-07-02T11:59:59
  * x            (x) float64 3kB 2.156e+05 2.156e+05 ... 2.196e+05 2.196e+05
  * y            (y) float64 3kB 8.313e+06 8.313e+06 ... 8.309e+06 8.309e+06
Data variables:
    agb          (time, y, x) float64 1MB dask.array<chunksize=(1, 401, 401), meta=np.ndarray>
    agb_sd       (time, y, x) float64 1MB dask.array<chunksize=(1, 401, 401), meta=np.ndarray>
Attributes: (12/19)
    Conventions:                CF-1.7
    date_created:               2025-09-11T14:05:16.016542
    date_modified:              2025-09-11T14:05:22.454435
    geospatial_bounds:          POLYGON((132.34933332965429 -15.2782222218250...
    geospatial_bounds_crs:      CRS84
    geospatial_lat_max:         -15.237333331825397
    ...                         ...
    history:                    [{'cube_params': {'time_range': ['2019-01-01T...
    processing_level:           L4
    time_coverage_duration:     P364DT23H59M59S
    time_coverage_end:          2019-12-31T23:59:59
    time_coverage_start:        2019-01-01T00:00:00
    title:                      esacci.BIOMASS.yr.L4.AGB.multi-sensor.multi-p...

xarray.Dataset

• Dimensions:
  • time: 1
  • y: 401
  • x: 401
• Coordinates: (4)
  • spatial_ref
    ()
    int64
    ...

    crs_wkt :

    PROJCRS["WGS 84 / UTM zone 53S",BASEGEOGCRS["WGS 84",DATUM["World Geodetic System 1984",ELLIPSOID["WGS 84",6378137,298.257223563,LENGTHUNIT["metre",1]]],PRIMEM["Greenwich",0,ANGLEUNIT["degree",0.0174532925199433]],ID["EPSG",4326]],CONVERSION["UTM zone 53S",METHOD["Transverse Mercator",ID["EPSG",9807]],PARAMETER["Latitude of natural origin",0,ANGLEUNIT["degree",0.0174532925199433],ID["EPSG",8801]],PARAMETER["Longitude of natural origin",135,ANGLEUNIT["degree",0.0174532925199433],ID["EPSG",8802]],PARAMETER["Scale factor at natural origin",0.9996,SCALEUNIT["unity",1],ID["EPSG",8805]],PARAMETER["False easting",500000,LENGTHUNIT["metre",1],ID["EPSG",8806]],PARAMETER["False northing",10000000,LENGTHUNIT["metre",1],ID["EPSG",8807]]],CS[Cartesian,2],AXIS["easting",east,ORDER[1],LENGTHUNIT["metre",1]],AXIS["northing",north,ORDER[2],LENGTHUNIT["metre",1]],ID["EPSG",32753]]

    false_easting :

    500000.0

    false_northing :

    10000000.0

    geographic_crs_name :

    WGS 84

    grid_mapping_name :

    transverse_mercator

    horizontal_datum_name :

    World Geodetic System 1984

    inverse_flattening :

    298.257223563

    latitude_of_projection_origin :

    0.0

    longitude_of_central_meridian :

    135.0

    longitude_of_prime_meridian :

    0.0

    prime_meridian_name :

    Greenwich

    projected_crs_name :

    WGS 84 / UTM zone 53S

    reference_ellipsoid_name :

    WGS 84

    scale_factor_at_central_meridian :

    0.9996

    semi_major_axis :

    6378137.0

    semi_minor_axis :

    6356752.314245179

    [1 values with dtype=int64]

  • time
    (time)
    datetime64[ns]
    2019-07-02T11:59:59

    bounds :

    time_bnds

    standard_name :

    time

    array(['2019-07-02T11:59:59.000000000'], dtype='datetime64[ns]')

  • x
    (x)
    float64
    2.156e+05 2.156e+05 ... 2.196e+05

    bounds :

    x_bnds

    long_name :

    x coordinate of projection

    standard_name :

    projection_x_coordinate

    array([215575., 215585., 215595., ..., 219555., 219565., 219575.], shape=(401,))

  • y
    (y)
    float64
    8.313e+06 8.313e+06 ... 8.309e+06

    bounds :

    y_bnds

    long_name :

    y coordinate of projection

    standard_name :

    projection_y_coordinate

    array([8313345., 8313335., 8313325., ..., 8309365., 8309355., 8309345.],
          shape=(401,))

• Data variables: (2)
  • agb
    (time, y, x)
    float64
    dask.array<chunksize=(1, 401, 401), meta=np.ndarray>

    ancillary_variables :

    agb_se

    chunk_sizes :

    [1, 1875, 1875]

    data_type :

    int16

    dimensions :

    ['time', 'lat', 'lon']

    file_chunk_sizes :

    [1, 1875, 1875]

    file_dimensions :

    ['time', 'lat', 'lon']

    fill_value :

    -32768

    grid_mapping :

    spatial_ref

    long_name :

    Above-ground biomass

    orig_data_type :

    int16

    shape :

    [1, 157500, 405000]

    size :

    63787500000

    units :

    Mg/ha

    valid_max :

    10000.0

    valid_min :

    0.0

    Array Chunk Bytes 1.23 MiB 1.23 MiB Shape (1, 401, 401) (1, 401, 401) Dask graph 1 chunks in 2 graph layers Data type float64 numpy.ndarray

  • agb_sd
    (time, y, x)
    float64
    dask.array<chunksize=(1, 401, 401), meta=np.ndarray>

    chunk_sizes :

    [1, 1875, 1875]

    data_type :

    int16

    dimensions :

    ['time', 'lat', 'lon']

    file_chunk_sizes :

    [1, 1875, 1875]

    file_dimensions :

    ['time', 'lat', 'lon']

    fill_value :

    -32768

    grid_mapping :

    spatial_ref

    long_name :

    Above-ground biomass standard_deviation

    orig_data_type :

    int16

    shape :

    [1, 157500, 405000]

    size :

    63787500000

    units :

    Mg/ha

    valid_max :

    10000.0

    valid_min :

    0.0

    Array Chunk Bytes 1.23 MiB 1.23 MiB Shape (1, 401, 401) (1, 401, 401) Dask graph 1 chunks in 2 graph layers Data type float64 numpy.ndarray

• Indexes: (3)
  • time
    PandasIndex

    PandasIndex(DatetimeIndex(['2019-07-02 11:59:59'], dtype='datetime64[ns]', name='time', freq=None))

  • x
    PandasIndex

    PandasIndex(Index([215575.0, 215585.0, 215595.0, 215605.0, 215615.0, 215625.0, 215635.0,
           215645.0, 215655.0, 215665.0,
           ...
           219485.0, 219495.0, 219505.0, 219515.0, 219525.0, 219535.0, 219545.0,
           219555.0, 219565.0, 219575.0],
          dtype='float64', name='x', length=401))

  • y
    PandasIndex

    PandasIndex(Index([8313345.0, 8313335.0, 8313325.0, 8313315.0, 8313305.0, 8313295.0,
           8313285.0, 8313275.0, 8313265.0, 8313255.0,
           ...
           8309435.0, 8309425.0, 8309415.0, 8309405.0, 8309395.0, 8309385.0,
           8309375.0, 8309365.0, 8309355.0, 8309345.0],
          dtype='float64', name='y', length=401))

• Attributes: (19)

  Conventions :

  CF-1.7

  date_created :

  2025-09-11T14:05:16.016542

  date_modified :

  2025-09-11T14:05:22.454435

  geospatial_bounds :

  POLYGON((132.34933332965429 -15.278222221825004, 132.34933332965429 -15.237333331825397, 132.3919999974316 -15.237333331825397, 132.3919999974316 -15.278222221825004, 132.34933332965429 -15.278222221825004))

  geospatial_bounds_crs :

  CRS84

  geospatial_lat_max :

  -15.237333331825397

  geospatial_lat_min :

  -15.278222221825004

  geospatial_lat_resolution :

  0.00088889

  geospatial_lat_units :

  degrees_north

  geospatial_lon_max :

  132.3919999974316

  geospatial_lon_min :

  132.34933332965429

  geospatial_lon_resolution :

  0.00088889

  geospatial_lon_units :

  degrees_east

  history :

  [{'cube_params': {'time_range': ['2019-01-01T00:00:00', '2019-12-31T00:00:00'], 'variable_names': ['agb', 'agb_sd', 'crs']}, 'program': 'xcube_cci.chunkstore.CciChunkStore'}]

  processing_level :

  L4

  time_coverage_duration :

  P364DT23H59M59S

  time_coverage_end :

  2019-12-31T23:59:59

  time_coverage_start :

  2019-01-01T00:00:00

  title :

  esacci.BIOMASS.yr.L4.AGB.multi-sensor.multi-platform.MERGED.5-0.100m

In [12]:

ds["agb"].plot()

ds["agb"].plot()

Out[12]:

<matplotlib.collections.QuadMesh at 0x78fb41522990>