EO-LINCS project: Cube generation for Scientific Case Study (SCS) 3¶

Model-Data Fusion for Understanding Carbon State-Flux Relationships Across Space¶

Objective: SCS3 aims to improve the accuracy of ecosystem carbon cycle models by integrating in-situ observation networks with complementary EO products as inputs. The data cubes will be generated in a manner that allows seamless integration within the SINDBAD ecosystem modelling framework.

Outcomes: Use a terrestrial carbon model structure that can deliver a process understanding of carbon stateflux relationships across space by leveraging and cross-comparing EO data of biomass and vegetation states (fAPAR, LAI, vegetation fraction, etc.) together with ecosystem carbon flux measurements and estimates. Additionally, provide open-source novel MDI tools and workflows for community use paving the way for adoption by other data-assimilation frameworks.

Required datasets:

• Sentinel-2 L2A from CDSE
• ERA-5 land
• ESA CCI Biomass

---

The following notebook shows how the users can load data from various sources defined in scs3_config.yml using the MultiSourceDataStore tool.

What You Can Do with This Notebook¶

• Generate a configuration file based on fluxtower locations
• Load datasets from various sources as defined in the generated scs3_config.yml
• View the progress of each data request to the MultiSourceDataStore
• Quickly preview the datasets by plotting them.

Requirements¶

Before you begin, follow these steps:

• Install xcube-multistore via conda-forge by running: conda install --channel conda-forge xcube-multistore
• To access EO data via S3 from CDSE, generate your S3 credentials and add them to the data_stores section in the scs3_config.yml file.
• To access ERA5-Land data from the Copernicus Climate Data Store, obtain a CDS Personal Access Token by creating an account on the CDS Website. After logging in, navigate to your user page to find your Personal Access Token. Add this token to the data_stores section in the scs3_config.yml file.

Once you have it installed, you are ready to proceed.

This Multistore mainly works with a file called scs3_config.yml which is at the same file level as this notebook. To understand what goes into the schema, you can read more here.

---

Let's import the MultiSourceDataStore

In [1]:

import yaml

import pandas as pd
from xcube_multistore.multistore import MultiSourceDataStore
from xcube_multistore.utils import get_bbox

import yaml import pandas as pd from xcube_multistore.multistore import MultiSourceDataStore from xcube_multistore.utils import get_bbox

You can find out how to fill out the config file by also using this super helpful function get_config_schema(). Run it and try expand the fields to learn more about the possible properties that the configuration file accepts along with the Configuration Guide.

In [2]:

MultiSourceDataStore.get_config_schema()

MultiSourceDataStore.get_config_schema()

Out[2]:

<xcube.util.jsonschema.JsonObjectSchema at 0x743b6a5cda60>

This science case requires data from several flux sites. So we define them in a scs3_sites.csv file for easier management and access. For the purpose of this example, we will focus on the first 3 sites

In [3]:

sites = pd.read_csv("scs3_sites.csv")
sites = sites.iloc[:3]
sites

sites = pd.read_csv("scs3_sites.csv") sites = sites.iloc[:3] sites

Out[3]:

	Site ID	latitude	longitude	IGBP
0	AU-Dry	-15.2588	132.3706	SAV
1	AU-How	-12.4943	131.1523	WSA
2	BE-Lon	50.5516	4.7462	CRO

In the following cell, we will create the config object which will then be saved as scs3_config.yml for persistance and ready to be read by MultiSourceDataStore.

To read more about how this config file is structured, you can find the Configuration Guide here.

Specifically, we are using the single dataset object and data stores schemas here.

In [4]:

config = dict(datasets=[])
for index, site in sites.iterrows():
    bbox_final, crs_final = get_bbox(
        site["latitude"], site["longitude"], cube_width=4000, crs_final="utm"
    )

    # append config for Sentinel-2
    config_ds = dict(
        identifier=f"{site['Site ID']}_sen2",
        store="stac-cdse",
        data_id="sentinel-2-l2a",
        open_params=dict(
            time_range=["2019-03-01", "2019-03-15"],
            bbox=bbox_final,
            spatial_res=10,
            crs=f"EPSG:{crs_final.to_epsg()}",
            apply_scaling=True,
            asset_names=[
                "B01",
                "B02",
                "B03",
                "B04",
                "B05",
                "B06",
                "B07",
                "B08",
                "B8A",
                "B09",
                "B11",
                "B12",
                "SCL",
            ],
        ),
    )
    config["datasets"].append(config_ds)

    # append config for ERA5
    config_ds = dict(
        identifier=f"{site['Site ID']}_era5land",
        store="cds",
        data_id="reanalysis-era5-land",
        open_params=dict(
            variable_names=["2m_temperature", "total_precipitation"],
            time_range=["2019-03-01", "2019-03-15"],
            point=[site["latitude"], site["longitude"]],
            spatial_res=0.1,
        ),
    )
    config["datasets"].append(config_ds)

    # append config for ESA CCI
    config_ds = dict(
        identifier=f"{site['Site ID']}_ccibiomass",
        store="esa_cci",
        grid_mapping=f"{site['Site ID']}_sen2",
        data_id="esacci.BIOMASS.yr.L4.AGB.multi-sensor.multi-platform.MERGED.5-0.100m",
        open_params=dict(
            time_range=["2019-01-01", "2019-12-31"],
        ),
    )
    config["datasets"].append(config_ds)

# define stores
config["data_stores"] = []
# add storage data store
config_store = dict(
    identifier="storage",
    store_id="file",
    store_params=dict(root="data"),
)
config["data_stores"].append(config_store)
# add ESA CCI data store
config_store = dict(
    identifier="esa_cci",
    store_id="cciodp",
)
config["data_stores"].append(config_store)
# add STAC data store
config_store = dict(
    identifier="stac-cdse",
    store_id="stac-cdse",
    store_params=dict(
        key="<CDSE_S3_key>",
        secret="<CDSE_S3_secret>",
        stack_mode=True,
    ),
)
config["data_stores"].append(config_store)
# add CDS data store
config_store = dict(
    identifier="cds",
    store_id="cds",
    store_params=dict(
        endpoint_url="https://cds.climate.copernicus.eu/api",
        cds_api_key="<cds_personal_access_token>",
        normalize_names=True,
    ),
)
config["data_stores"].append(config_store)

config = dict(datasets=[]) for index, site in sites.iterrows(): bbox_final, crs_final = get_bbox( site["latitude"], site["longitude"], cube_width=4000, crs_final="utm" ) # append config for Sentinel-2 config_ds = dict( identifier=f"{site['Site ID']}_sen2", store="stac-cdse", data_id="sentinel-2-l2a", open_params=dict( time_range=["2019-03-01", "2019-03-15"], bbox=bbox_final, spatial_res=10, crs=f"EPSG:{crs_final.to_epsg()}", apply_scaling=True, asset_names=[ "B01", "B02", "B03", "B04", "B05", "B06", "B07", "B08", "B8A", "B09", "B11", "B12", "SCL", ], ), ) config["datasets"].append(config_ds) # append config for ERA5 config_ds = dict( identifier=f"{site['Site ID']}_era5land", store="cds", data_id="reanalysis-era5-land", open_params=dict( variable_names=["2m_temperature", "total_precipitation"], time_range=["2019-03-01", "2019-03-15"], point=[site["latitude"], site["longitude"]], spatial_res=0.1, ), ) config["datasets"].append(config_ds) # append config for ESA CCI config_ds = dict( identifier=f"{site['Site ID']}_ccibiomass", store="esa_cci", grid_mapping=f"{site['Site ID']}_sen2", data_id="esacci.BIOMASS.yr.L4.AGB.multi-sensor.multi-platform.MERGED.5-0.100m", open_params=dict( time_range=["2019-01-01", "2019-12-31"], ), ) config["datasets"].append(config_ds) # define stores config["data_stores"] = [] # add storage data store config_store = dict( identifier="storage", store_id="file", store_params=dict(root="data"), ) config["data_stores"].append(config_store) # add ESA CCI data store config_store = dict( identifier="esa_cci", store_id="cciodp", ) config["data_stores"].append(config_store) # add STAC data store config_store = dict( identifier="stac-cdse", store_id="stac-cdse", store_params=dict( key="", secret="", stack_mode=True, ), ) config["data_stores"].append(config_store) # add CDS data store config_store = dict( identifier="cds", store_id="cds", store_params=dict( endpoint_url="https://cds.climate.copernicus.eu/api", cds_api_key="", normalize_names=True, ), ) config["data_stores"].append(config_store)

In [5]:

with open("scs3_config.yml", "w") as file:
    yaml.dump(config, file, sort_keys=False)

with open("scs3_config.yml", "w") as file: yaml.dump(config, file, sort_keys=False)

Now, we can initialize the MultiSourceDataStore by passing the path to the scs3_config.yml` which currently is on the same level as this notebook.

By running the cell below, you would start seeing progress tables for each data that you requested in the scs3_config.yml.

In [6]:

msds = MultiSourceDataStore("scs3_config.yml")

msds = MultiSourceDataStore("scs3_config.yml")

<frozen abc>:106: FutureWarning: xarray subclass VectorDataCube should explicitly define __slots__

Cube Generation

Dataset identifier	Status	Message	Exception
AU-Dry_sen2	STOPPED	Dataset 'AU-Dry_sen2' already generated.	-
AU-Dry_era5land	STOPPED	Dataset 'AU-Dry_era5land' finished.	-
AU-Dry_ccibiomass	STOPPED	Dataset 'AU-Dry_ccibiomass' already generated.	-
AU-How_sen2	STOPPED	Dataset 'AU-How_sen2' already generated.	-
AU-How_era5land	STOPPED	Dataset 'AU-How_era5land' finished.	-
AU-How_ccibiomass	STOPPED	Dataset 'AU-How_ccibiomass' already generated.	-
BE-Lon_sen2	STOPPED	Dataset 'BE-Lon_sen2' already generated.	-
BE-Lon_era5land	STOPPED	Dataset 'BE-Lon_era5land' finished.	-
BE-Lon_ccibiomass	STOPPED	Dataset 'BE-Lon_ccibiomass' already generated.	-

xcube-cds version 1.0.0
2025-03-24 16:32:25,427 INFO [2024-09-26T00:00:00] Watch our [Forum](https://forum.ecmwf.int/) for Announcements, news and other discussed topics.
2025-03-24 16:32:25,427 WARNING [2024-06-16T00:00:00] CDS API syntax is changed and some keys or parameter names may have also changed. To avoid requests failing, please use the "Show API request code" tool on the dataset Download Form to check you are using the correct syntax for your API request.
2025-03-24 16:32:25,637 INFO Request ID is 357bb882-f0c3-4252-a121-39d69f053ef5
2025-03-24 16:32:25,690 INFO status has been updated to accepted
2025-03-24 16:32:46,996 INFO status has been updated to running
2025-03-24 16:35:17,657 INFO status has been updated to successful

6e4e6922fc045e51339bf07d11581386.zip:   0%|          | 0.00/76.1k [00:00<?, ?B/s]

xcube-cds version 1.0.0
2025-03-24 16:35:21,192 INFO [2024-09-26T00:00:00] Watch our [Forum](https://forum.ecmwf.int/) for Announcements, news and other discussed topics.
2025-03-24 16:35:21,194 WARNING [2024-06-16T00:00:00] CDS API syntax is changed and some keys or parameter names may have also changed. To avoid requests failing, please use the "Show API request code" tool on the dataset Download Form to check you are using the correct syntax for your API request.
2025-03-24 16:35:25,256 INFO Request ID is 1145534d-8f8c-42ce-a3a0-b547d6c6d166
2025-03-24 16:35:25,316 INFO status has been updated to accepted
2025-03-24 16:35:33,690 INFO status has been updated to running
2025-03-24 16:38:17,038 INFO status has been updated to successful

48c8dd07f6fd4495b30b58fefeb9258c.zip:   0%|          | 0.00/85.8k [00:00<?, ?B/s]

xcube-cds version 1.0.0
2025-03-24 16:38:19,227 INFO [2024-09-26T00:00:00] Watch our [Forum](https://forum.ecmwf.int/) for Announcements, news and other discussed topics.
2025-03-24 16:38:19,231 WARNING [2024-06-16T00:00:00] CDS API syntax is changed and some keys or parameter names may have also changed. To avoid requests failing, please use the "Show API request code" tool on the dataset Download Form to check you are using the correct syntax for your API request.
2025-03-24 16:38:19,407 INFO Request ID is 4586d0a8-10ea-4286-b5b7-534800619034
2025-03-24 16:38:19,465 INFO status has been updated to accepted
2025-03-24 16:38:24,380 INFO status has been updated to running
2025-03-24 16:42:43,269 INFO status has been updated to successful

b38822f9185a261a9d382c21af8e023c.zip:   0%|          | 0.00/85.7k [00:00<?, ?B/s]

We can now open the data using the xcube datastore framework API as usual. Note that the multi-source data store requires a data store called storage, which is configured in our config.yml under the data_stores section.

In [7]:

ds = msds.stores.storage.open_data("AU-Dry_sen2.zarr")
ds

ds = msds.stores.storage.open_data("AU-Dry_sen2.zarr") ds

Out[7]:

<xarray.Dataset> Size: 24MB
Dimensions:      (time: 3, y: 401, x: 401)
Coordinates:
    spatial_ref  int64 8B ...
  * time         (time) datetime64[ns] 24B 2019-03-03T09:27:11.024000 ... 201...
  * x            (x) float64 3kB 2.156e+05 2.156e+05 ... 2.196e+05 2.196e+05
  * y            (y) float64 3kB 8.313e+06 8.313e+06 ... 8.309e+06 8.309e+06
Data variables: (12/13)
    B01          (time, y, x) float32 2MB dask.array<chunksize=(1, 401, 401), meta=np.ndarray>
    B02          (time, y, x) float32 2MB dask.array<chunksize=(1, 401, 401), meta=np.ndarray>
    B03          (time, y, x) float32 2MB dask.array<chunksize=(1, 401, 401), meta=np.ndarray>
    B04          (time, y, x) float32 2MB dask.array<chunksize=(1, 401, 401), meta=np.ndarray>
    B05          (time, y, x) float32 2MB dask.array<chunksize=(1, 401, 401), meta=np.ndarray>
    B06          (time, y, x) float32 2MB dask.array<chunksize=(1, 401, 401), meta=np.ndarray>
    ...           ...
    B08          (time, y, x) float32 2MB dask.array<chunksize=(1, 401, 401), meta=np.ndarray>
    B09          (time, y, x) float32 2MB dask.array<chunksize=(1, 401, 401), meta=np.ndarray>
    B11          (time, y, x) float32 2MB dask.array<chunksize=(1, 401, 401), meta=np.ndarray>
    B12          (time, y, x) float32 2MB dask.array<chunksize=(1, 401, 401), meta=np.ndarray>
    B8A          (time, y, x) float32 2MB dask.array<chunksize=(1, 401, 401), meta=np.ndarray>
    SCL          (time, y, x) uint8 482kB dask.array<chunksize=(1, 401, 401), meta=np.ndarray>
Attributes:
    stac_catalog_url:  https://stac.dataspace.copernicus.eu/v1
    stac_item_ids:     {'2019-03-03T09:27:11.024000': ['S2A_MSIL2A_20190303T0...

xarray.Dataset

• Dimensions:
  • time: 3
  • y: 401
  • x: 401
• Coordinates: (4)
  • spatial_ref
    ()
    int64
    ...

    crs_wkt :

    PROJCRS["WGS 84 / UTM zone 53S",BASEGEOGCRS["WGS 84",ENSEMBLE["World Geodetic System 1984 ensemble",MEMBER["World Geodetic System 1984 (Transit)"],MEMBER["World Geodetic System 1984 (G730)"],MEMBER["World Geodetic System 1984 (G873)"],MEMBER["World Geodetic System 1984 (G1150)"],MEMBER["World Geodetic System 1984 (G1674)"],MEMBER["World Geodetic System 1984 (G1762)"],MEMBER["World Geodetic System 1984 (G2139)"],MEMBER["World Geodetic System 1984 (G2296)"],ELLIPSOID["WGS 84",6378137,298.257223563,LENGTHUNIT["metre",1]],ENSEMBLEACCURACY[2.0]],PRIMEM["Greenwich",0,ANGLEUNIT["degree",0.0174532925199433]],ID["EPSG",4326]],CONVERSION["UTM zone 53S",METHOD["Transverse Mercator",ID["EPSG",9807]],PARAMETER["Latitude of natural origin",0,ANGLEUNIT["degree",0.0174532925199433],ID["EPSG",8801]],PARAMETER["Longitude of natural origin",135,ANGLEUNIT["degree",0.0174532925199433],ID["EPSG",8802]],PARAMETER["Scale factor at natural origin",0.9996,SCALEUNIT["unity",1],ID["EPSG",8805]],PARAMETER["False easting",500000,LENGTHUNIT["metre",1],ID["EPSG",8806]],PARAMETER["False northing",10000000,LENGTHUNIT["metre",1],ID["EPSG",8807]]],CS[Cartesian,2],AXIS["(E)",east,ORDER[1],LENGTHUNIT["metre",1]],AXIS["(N)",north,ORDER[2],LENGTHUNIT["metre",1]],USAGE[SCOPE["Navigation and medium accuracy spatial referencing."],AREA["Between 132°E and 138°E, southern hemisphere between 80°S and equator, onshore and offshore. Australia. Indonesia."],BBOX[-80,132,0,138]],ID["EPSG",32753]]

    false_easting :

    500000.0

    false_northing :

    10000000.0

    geographic_crs_name :

    WGS 84

    grid_mapping_name :

    transverse_mercator

    horizontal_datum_name :

    World Geodetic System 1984 ensemble

    inverse_flattening :

    298.257223563

    latitude_of_projection_origin :

    0.0

    longitude_of_central_meridian :

    135.0

    longitude_of_prime_meridian :

    0.0

    prime_meridian_name :

    Greenwich

    projected_crs_name :

    WGS 84 / UTM zone 53S

    reference_ellipsoid_name :

    WGS 84

    scale_factor_at_central_meridian :

    0.9996

    semi_major_axis :

    6378137.0

    semi_minor_axis :

    6356752.314245179

    [1 values with dtype=int64]

  • time
    (time)
    datetime64[ns]
    2019-03-03T09:27:11.024000 ... 2...

    array(['2019-03-03T09:27:11.024000000', '2019-03-08T09:27:19.024000000',
           '2019-03-13T09:27:11.024000000'], dtype='datetime64[ns]')

  • x
    (x)
    float64
    2.156e+05 2.156e+05 ... 2.196e+05

    bounds :

    x_bnds

    long_name :

    x coordinate of projection

    standard_name :

    projection_x_coordinate

    array([215577., 215587., 215597., ..., 219557., 219567., 219577.])

  • y
    (y)
    float64
    8.313e+06 8.313e+06 ... 8.309e+06

    bounds :

    y_bnds

    long_name :

    y coordinate of projection

    standard_name :

    projection_y_coordinate

    array([8313343., 8313333., 8313323., ..., 8309363., 8309353., 8309343.])

• Data variables: (13)
  • B01
    (time, y, x)
    float32
    dask.array<chunksize=(1, 401, 401), meta=np.ndarray>

    grid_mapping :

    spatial_ref

    Array Chunk Bytes 1.84 MiB 628.13 kiB Shape (3, 401, 401) (1, 401, 401) Dask graph 3 chunks in 2 graph layers Data type float32 numpy.ndarray

  • B02
    (time, y, x)
    float32
    dask.array<chunksize=(1, 401, 401), meta=np.ndarray>

    grid_mapping :

    spatial_ref

    Array Chunk Bytes 1.84 MiB 628.13 kiB Shape (3, 401, 401) (1, 401, 401) Dask graph 3 chunks in 2 graph layers Data type float32 numpy.ndarray

  • B03
    (time, y, x)
    float32
    dask.array<chunksize=(1, 401, 401), meta=np.ndarray>

    grid_mapping :

    spatial_ref

    Array Chunk Bytes 1.84 MiB 628.13 kiB Shape (3, 401, 401) (1, 401, 401) Dask graph 3 chunks in 2 graph layers Data type float32 numpy.ndarray

  • B04
    (time, y, x)
    float32
    dask.array<chunksize=(1, 401, 401), meta=np.ndarray>

    grid_mapping :

    spatial_ref

    Array Chunk Bytes 1.84 MiB 628.13 kiB Shape (3, 401, 401) (1, 401, 401) Dask graph 3 chunks in 2 graph layers Data type float32 numpy.ndarray

  • B05
    (time, y, x)
    float32
    dask.array<chunksize=(1, 401, 401), meta=np.ndarray>

    grid_mapping :

    spatial_ref

    Array Chunk Bytes 1.84 MiB 628.13 kiB Shape (3, 401, 401) (1, 401, 401) Dask graph 3 chunks in 2 graph layers Data type float32 numpy.ndarray

  • B06
    (time, y, x)
    float32
    dask.array<chunksize=(1, 401, 401), meta=np.ndarray>

    grid_mapping :

    spatial_ref

    Array Chunk Bytes 1.84 MiB 628.13 kiB Shape (3, 401, 401) (1, 401, 401) Dask graph 3 chunks in 2 graph layers Data type float32 numpy.ndarray

  • B07
    (time, y, x)
    float32
    dask.array<chunksize=(1, 401, 401), meta=np.ndarray>

    grid_mapping :

    spatial_ref

    Array Chunk Bytes 1.84 MiB 628.13 kiB Shape (3, 401, 401) (1, 401, 401) Dask graph 3 chunks in 2 graph layers Data type float32 numpy.ndarray

  • B08
    (time, y, x)
    float32
    dask.array<chunksize=(1, 401, 401), meta=np.ndarray>

    grid_mapping :

    spatial_ref

    Array Chunk Bytes 1.84 MiB 628.13 kiB Shape (3, 401, 401) (1, 401, 401) Dask graph 3 chunks in 2 graph layers Data type float32 numpy.ndarray

  • B09
    (time, y, x)
    float32
    dask.array<chunksize=(1, 401, 401), meta=np.ndarray>

    grid_mapping :

    spatial_ref

    Array Chunk Bytes 1.84 MiB 628.13 kiB Shape (3, 401, 401) (1, 401, 401) Dask graph 3 chunks in 2 graph layers Data type float32 numpy.ndarray

  • B11
    (time, y, x)
    float32
    dask.array<chunksize=(1, 401, 401), meta=np.ndarray>

    grid_mapping :

    spatial_ref

    Array Chunk Bytes 1.84 MiB 628.13 kiB Shape (3, 401, 401) (1, 401, 401) Dask graph 3 chunks in 2 graph layers Data type float32 numpy.ndarray

  • B12
    (time, y, x)
    float32
    dask.array<chunksize=(1, 401, 401), meta=np.ndarray>

    grid_mapping :

    spatial_ref

    Array Chunk Bytes 1.84 MiB 628.13 kiB Shape (3, 401, 401) (1, 401, 401) Dask graph 3 chunks in 2 graph layers Data type float32 numpy.ndarray

  • B8A
    (time, y, x)
    float32
    dask.array<chunksize=(1, 401, 401), meta=np.ndarray>

    grid_mapping :

    spatial_ref

    Array Chunk Bytes 1.84 MiB 628.13 kiB Shape (3, 401, 401) (1, 401, 401) Dask graph 3 chunks in 2 graph layers Data type float32 numpy.ndarray

  • SCL
    (time, y, x)
    uint8
    dask.array<chunksize=(1, 401, 401), meta=np.ndarray>

    grid_mapping :

    spatial_ref

    Array Chunk Bytes 471.10 kiB 157.03 kiB Shape (3, 401, 401) (1, 401, 401) Dask graph 3 chunks in 2 graph layers Data type uint8 numpy.ndarray

• Indexes: (3)
  • time
    PandasIndex

    PandasIndex(DatetimeIndex(['2019-03-03 09:27:11.024000', '2019-03-08 09:27:19.024000',
                   '2019-03-13 09:27:11.024000'],
                  dtype='datetime64[ns]', name='time', freq=None))

  • x
    PandasIndex

    PandasIndex(Index([215577.0, 215587.0, 215597.0, 215607.0, 215617.0, 215627.0, 215637.0,
           215647.0, 215657.0, 215667.0,
           ...
           219487.0, 219497.0, 219507.0, 219517.0, 219527.0, 219537.0, 219547.0,
           219557.0, 219567.0, 219577.0],
          dtype='float64', name='x', length=401))

  • y
    PandasIndex

    PandasIndex(Index([8313343.0, 8313333.0, 8313323.0, 8313313.0, 8313303.0, 8313293.0,
           8313283.0, 8313273.0, 8313263.0, 8313253.0,
           ...
           8309433.0, 8309423.0, 8309413.0, 8309403.0, 8309393.0, 8309383.0,
           8309373.0, 8309363.0, 8309353.0, 8309343.0],
          dtype='float64', name='y', length=401))

• Attributes: (2)

  stac_catalog_url :

  https://stac.dataspace.copernicus.eu/v1

  stac_item_ids :

  {'2019-03-03T09:27:11.024000': ['S2A_MSIL2A_20190303T012711_N0500_R131_T52LHJ_20221209T122536', 'S2A_MSIL2A_20190303T012711_N0500_R131_T53LKD_20221209T122536'], '2019-03-08T09:27:19.024000': ['S2B_MSIL2A_20190308T012719_N0500_R131_T52LHJ_20221120T105602', 'S2B_MSIL2A_20190308T012719_N0500_R131_T53LKD_20221120T105602'], '2019-03-13T09:27:11.024000': ['S2A_MSIL2A_20190313T012711_N0500_R131_T52LHJ_20221118T074550', 'S2A_MSIL2A_20190313T012711_N0500_R131_T53LKD_20221118T074550']}

We can now select a variable for one timestep and plot it for a quick preview of the data

In [8]:

ds.B04.isel(time=0).plot(vmin=0., vmax=0.2)

ds.B04.isel(time=0).plot(vmin=0., vmax=0.2)

Out[8]:

<matplotlib.collections.QuadMesh at 0x743b5d7052e0>

In [9]:

ds = msds.stores.storage.open_data("AU-Dry_era5land.zarr")
ds

ds = msds.stores.storage.open_data("AU-Dry_era5land.zarr") ds

Out[9]:

<xarray.Dataset> Size: 12kB
Dimensions:  (time: 360)
Coordinates:
    expver   (time) <U4 6kB dask.array<chunksize=(360,), meta=np.ndarray>
    lat      float64 8B ...
    lon      float64 8B ...
    number   int64 8B ...
  * time     (time) datetime64[ns] 3kB 2019-03-01 ... 2019-03-15T23:00:00
Data variables:
    t2m      (time) float32 1kB dask.array<chunksize=(360,), meta=np.ndarray>
    tp       (time) float32 1kB dask.array<chunksize=(360,), meta=np.ndarray>
Attributes:
    Conventions:             CF-1.7
    GRIB_centre:             ecmf
    GRIB_centreDescription:  European Centre for Medium-Range Weather Forecasts
    GRIB_subCentre:          0
    history:                 2025-03-24T15:34 GRIB to CDM+CF via cfgrib-0.9.1...
    institution:             European Centre for Medium-Range Weather Forecasts

xarray.Dataset

• Dimensions:
  • time: 360
• Coordinates: (5)
  • expver
    (time)
    <U4
    dask.array<chunksize=(360,), meta=np.ndarray>

    Array Chunk Bytes 5.62 kiB 5.62 kiB Shape (360,) (360,) Dask graph 1 chunks in 2 graph layers Data type

  • lat
    ()
    float64
    ...

    long_name :

    latitude

    standard_name :

    latitude

    stored_direction :

    decreasing

    units :

    degrees_north

    [1 values with dtype=float64]

  • lon
    ()
    float64
    ...

    long_name :

    longitude

    standard_name :

    longitude

    units :

    degrees_east

    [1 values with dtype=float64]

  • number
    ()
    int64
    ...

    long_name :

    ensemble member numerical id

    standard_name :

    realization

    units :

    1

    [1 values with dtype=int64]

  • time
    (time)
    datetime64[ns]
    2019-03-01 ... 2019-03-15T23:00:00

    long_name :

    time

    standard_name :

    time

    array(['2019-03-01T00:00:00.000000000', '2019-03-01T01:00:00.000000000',
           '2019-03-01T02:00:00.000000000', ..., '2019-03-15T21:00:00.000000000',
           '2019-03-15T22:00:00.000000000', '2019-03-15T23:00:00.000000000'],
          dtype='datetime64[ns]')

• Data variables: (2)
  • t2m
    (time)
    float32
    dask.array<chunksize=(360,), meta=np.ndarray>

    GRIB_NV :

    0

    GRIB_Nx :

    4

    GRIB_Ny :

    4

    GRIB_cfName :

    unknown

    GRIB_cfVarName :

    t2m

    GRIB_dataType :

    fc

    GRIB_gridDefinitionDescription :

    Latitude/Longitude Grid

    GRIB_gridType :

    regular_ll

    GRIB_iDirectionIncrementInDegrees :

    0.1

    GRIB_iScansNegatively :

    0

    GRIB_jDirectionIncrementInDegrees :

    0.1

    GRIB_jPointsAreConsecutive :

    0

    GRIB_jScansPositively :

    0

    GRIB_latitudeOfFirstGridPointInDegrees :

    -15.108

    GRIB_latitudeOfLastGridPointInDegrees :

    -15.409

    GRIB_longitudeOfFirstGridPointInDegrees :

    132.22

    GRIB_longitudeOfLastGridPointInDegrees :

    132.521

    GRIB_missingValue :

    3.4028234663852886e+38

    GRIB_name :

    2 metre temperature

    GRIB_numberOfPoints :

    16

    GRIB_paramId :

    167

    GRIB_shortName :

    2t

    GRIB_stepType :

    instant

    GRIB_stepUnits :

    1

    GRIB_surface :

    0.0

    GRIB_totalNumber :

    0

    GRIB_typeOfLevel :

    surface

    GRIB_units :

    K

    GRIB_uvRelativeToGrid :

    0

    long_name :

    2 metre temperature

    standard_name :

    unknown

    units :

    K

    Array Chunk Bytes 1.41 kiB 1.41 kiB Shape (360,) (360,) Dask graph 1 chunks in 2 graph layers Data type float32 numpy.ndarray

  • tp
    (time)
    float32
    dask.array<chunksize=(360,), meta=np.ndarray>

    GRIB_NV :

    0

    GRIB_Nx :

    4

    GRIB_Ny :

    4

    GRIB_cfName :

    unknown

    GRIB_cfVarName :

    tp

    GRIB_dataType :

    fc

    GRIB_gridDefinitionDescription :

    Latitude/Longitude Grid

    GRIB_gridType :

    regular_ll

    GRIB_iDirectionIncrementInDegrees :

    0.1

    GRIB_iScansNegatively :

    0

    GRIB_jDirectionIncrementInDegrees :

    0.1

    GRIB_jPointsAreConsecutive :

    0

    GRIB_jScansPositively :

    0

    GRIB_latitudeOfFirstGridPointInDegrees :

    -15.108

    GRIB_latitudeOfLastGridPointInDegrees :

    -15.409

    GRIB_longitudeOfFirstGridPointInDegrees :

    132.22

    GRIB_longitudeOfLastGridPointInDegrees :

    132.521

    GRIB_missingValue :

    3.4028234663852886e+38

    GRIB_name :

    Total precipitation

    GRIB_numberOfPoints :

    16

    GRIB_paramId :

    228

    GRIB_shortName :

    tp

    GRIB_stepType :

    accum

    GRIB_stepUnits :

    1

    GRIB_surface :

    0.0

    GRIB_totalNumber :

    0

    GRIB_typeOfLevel :

    surface

    GRIB_units :

    m

    GRIB_uvRelativeToGrid :

    0

    long_name :

    Total precipitation

    standard_name :

    unknown

    units :

    m

    Array Chunk Bytes 1.41 kiB 1.41 kiB Shape (360,) (360,) Dask graph 1 chunks in 2 graph layers Data type float32 numpy.ndarray

• Indexes: (1)
  • time
    PandasIndex

    PandasIndex(DatetimeIndex(['2019-03-01 00:00:00', '2019-03-01 01:00:00',
                   '2019-03-01 02:00:00', '2019-03-01 03:00:00',
                   '2019-03-01 04:00:00', '2019-03-01 05:00:00',
                   '2019-03-01 06:00:00', '2019-03-01 07:00:00',
                   '2019-03-01 08:00:00', '2019-03-01 09:00:00',
                   ...
                   '2019-03-15 14:00:00', '2019-03-15 15:00:00',
                   '2019-03-15 16:00:00', '2019-03-15 17:00:00',
                   '2019-03-15 18:00:00', '2019-03-15 19:00:00',
                   '2019-03-15 20:00:00', '2019-03-15 21:00:00',
                   '2019-03-15 22:00:00', '2019-03-15 23:00:00'],
                  dtype='datetime64[ns]', name='time', length=360, freq=None))

• Attributes: (6)

  Conventions :

  CF-1.7

  GRIB_centre :

  ecmf

  GRIB_centreDescription :

  European Centre for Medium-Range Weather Forecasts

  GRIB_subCentre :

  0

  history :

  2025-03-24T15:34 GRIB to CDM+CF via cfgrib-0.9.15.0/ecCodes-2.39.0 with {"source": "data.grib", "filter_by_keys": {}, "encode_cf": ["parameter", "time", "geography", "vertical"]}

  institution :

  European Centre for Medium-Range Weather Forecasts

In [10]:

ds.t2m.plot()

ds.t2m.plot()

Out[10]:

[<matplotlib.lines.Line2D at 0x743b5d78bb00>]

In [11]:

ds = msds.stores.storage.open_data("AU-Dry_ccibiomass.zarr")
ds

ds = msds.stores.storage.open_data("AU-Dry_ccibiomass.zarr") ds

Out[11]:

<xarray.Dataset> Size: 3MB
Dimensions:      (time: 1, y: 401, x: 401)
Coordinates:
    spatial_ref  int64 8B ...
  * time         (time) datetime64[ns] 8B 2019-07-02T11:59:59
  * x            (x) float64 3kB 2.156e+05 2.156e+05 ... 2.196e+05 2.196e+05
  * y            (y) float64 3kB 8.313e+06 8.313e+06 ... 8.309e+06 8.309e+06
Data variables:
    agb          (time, y, x) float64 1MB dask.array<chunksize=(1, 401, 401), meta=np.ndarray>
    agb_sd       (time, y, x) float64 1MB dask.array<chunksize=(1, 401, 401), meta=np.ndarray>
Attributes: (12/19)
    Conventions:                CF-1.7
    date_created:               2025-03-24T13:07:10.913757
    date_modified:              2025-03-24T13:07:23.730961
    geospatial_bounds:          POLYGON((132.34933332965429 -15.2782222218250...
    geospatial_bounds_crs:      CRS84
    geospatial_lat_max:         -15.237333331825397
    ...                         ...
    history:                    [{'cube_params': {'time_range': ['2019-01-01T...
    processing_level:           L4
    time_coverage_duration:     P364DT23H59M59S
    time_coverage_end:          2019-12-31T23:59:59
    time_coverage_start:        2019-01-01T00:00:00
    title:                      esacci.BIOMASS.yr.L4.AGB.multi-sensor.multi-p...

xarray.Dataset

• Dimensions:
  • time: 1
  • y: 401
  • x: 401
• Coordinates: (4)
  • spatial_ref
    ()
    int64
    ...

    crs_wkt :

    PROJCRS["WGS 84 / UTM zone 53S",BASEGEOGCRS["WGS 84",ENSEMBLE["World Geodetic System 1984 ensemble",MEMBER["World Geodetic System 1984 (Transit)"],MEMBER["World Geodetic System 1984 (G730)"],MEMBER["World Geodetic System 1984 (G873)"],MEMBER["World Geodetic System 1984 (G1150)"],MEMBER["World Geodetic System 1984 (G1674)"],MEMBER["World Geodetic System 1984 (G1762)"],MEMBER["World Geodetic System 1984 (G2139)"],MEMBER["World Geodetic System 1984 (G2296)"],ELLIPSOID["WGS 84",6378137,298.257223563,LENGTHUNIT["metre",1]],ENSEMBLEACCURACY[2.0]],PRIMEM["Greenwich",0,ANGLEUNIT["degree",0.0174532925199433]],ID["EPSG",4326]],CONVERSION["UTM zone 53S",METHOD["Transverse Mercator",ID["EPSG",9807]],PARAMETER["Latitude of natural origin",0,ANGLEUNIT["degree",0.0174532925199433],ID["EPSG",8801]],PARAMETER["Longitude of natural origin",135,ANGLEUNIT["degree",0.0174532925199433],ID["EPSG",8802]],PARAMETER["Scale factor at natural origin",0.9996,SCALEUNIT["unity",1],ID["EPSG",8805]],PARAMETER["False easting",500000,LENGTHUNIT["metre",1],ID["EPSG",8806]],PARAMETER["False northing",10000000,LENGTHUNIT["metre",1],ID["EPSG",8807]]],CS[Cartesian,2],AXIS["(E)",east,ORDER[1],LENGTHUNIT["metre",1]],AXIS["(N)",north,ORDER[2],LENGTHUNIT["metre",1]],USAGE[SCOPE["Navigation and medium accuracy spatial referencing."],AREA["Between 132°E and 138°E, southern hemisphere between 80°S and equator, onshore and offshore. Australia. Indonesia."],BBOX[-80,132,0,138]],ID["EPSG",32753]]

    false_easting :

    500000.0

    false_northing :

    10000000.0

    geographic_crs_name :

    WGS 84

    grid_mapping_name :

    transverse_mercator

    horizontal_datum_name :

    World Geodetic System 1984 ensemble

    inverse_flattening :

    298.257223563

    latitude_of_projection_origin :

    0.0

    longitude_of_central_meridian :

    135.0

    longitude_of_prime_meridian :

    0.0

    prime_meridian_name :

    Greenwich

    projected_crs_name :

    WGS 84 / UTM zone 53S

    reference_ellipsoid_name :

    WGS 84

    scale_factor_at_central_meridian :

    0.9996

    semi_major_axis :

    6378137.0

    semi_minor_axis :

    6356752.314245179

    [1 values with dtype=int64]

  • time
    (time)
    datetime64[ns]
    2019-07-02T11:59:59

    bounds :

    time_bnds

    standard_name :

    time

    array(['2019-07-02T11:59:59.000000000'], dtype='datetime64[ns]')

  • x
    (x)
    float64
    2.156e+05 2.156e+05 ... 2.196e+05

    bounds :

    x_bnds

    long_name :

    x coordinate of projection

    standard_name :

    projection_x_coordinate

    array([215577., 215587., 215597., ..., 219557., 219567., 219577.])

  • y
    (y)
    float64
    8.313e+06 8.313e+06 ... 8.309e+06

    bounds :

    y_bnds

    long_name :

    y coordinate of projection

    standard_name :

    projection_y_coordinate

    array([8313343., 8313333., 8313323., ..., 8309363., 8309353., 8309343.])

• Data variables: (2)
  • agb
    (time, y, x)
    float64
    dask.array<chunksize=(1, 401, 401), meta=np.ndarray>

    ancillary_variables :

    agb_se

    chunk_sizes :

    [1, 1875, 1875]

    data_type :

    int16

    dimensions :

    ['time', 'lat', 'lon']

    file_chunk_sizes :

    [1, 1875, 1875]

    file_dimensions :

    ['time', 'lat', 'lon']

    fill_value :

    -32768

    grid_mapping :

    spatial_ref

    long_name :

    Above-ground biomass

    orig_data_type :

    int16

    shape :

    [1, 157500, 405000]

    size :

    63787500000

    units :

    Mg/ha

    valid_max :

    10000.0

    valid_min :

    0.0

    Array Chunk Bytes 1.23 MiB 1.23 MiB Shape (1, 401, 401) (1, 401, 401) Dask graph 1 chunks in 2 graph layers Data type float64 numpy.ndarray

  • agb_sd
    (time, y, x)
    float64
    dask.array<chunksize=(1, 401, 401), meta=np.ndarray>

    chunk_sizes :

    [1, 1875, 1875]

    data_type :

    int16

    dimensions :

    ['time', 'lat', 'lon']

    file_chunk_sizes :

    [1, 1875, 1875]

    file_dimensions :

    ['time', 'lat', 'lon']

    fill_value :

    -32768

    grid_mapping :

    spatial_ref

    long_name :

    Above-ground biomass standard_deviation

    orig_data_type :

    int16

    shape :

    [1, 157500, 405000]

    size :

    63787500000

    units :

    Mg/ha

    valid_max :

    10000.0

    valid_min :

    0.0

    Array Chunk Bytes 1.23 MiB 1.23 MiB Shape (1, 401, 401) (1, 401, 401) Dask graph 1 chunks in 2 graph layers Data type float64 numpy.ndarray

• Indexes: (3)
  • time
    PandasIndex

    PandasIndex(DatetimeIndex(['2019-07-02 11:59:59'], dtype='datetime64[ns]', name='time', freq=None))

  • x
    PandasIndex

    PandasIndex(Index([215577.0, 215587.0, 215597.0, 215607.0, 215617.0, 215627.0, 215637.0,
           215647.0, 215657.0, 215667.0,
           ...
           219487.0, 219497.0, 219507.0, 219517.0, 219527.0, 219537.0, 219547.0,
           219557.0, 219567.0, 219577.0],
          dtype='float64', name='x', length=401))

  • y
    PandasIndex

    PandasIndex(Index([8313343.0, 8313333.0, 8313323.0, 8313313.0, 8313303.0, 8313293.0,
           8313283.0, 8313273.0, 8313263.0, 8313253.0,
           ...
           8309433.0, 8309423.0, 8309413.0, 8309403.0, 8309393.0, 8309383.0,
           8309373.0, 8309363.0, 8309353.0, 8309343.0],
          dtype='float64', name='y', length=401))

• Attributes: (19)

  Conventions :

  CF-1.7

  date_created :

  2025-03-24T13:07:10.913757

  date_modified :

  2025-03-24T13:07:23.730961

  geospatial_bounds :

  POLYGON((132.34933332965429 -15.278222221825004, 132.34933332965429 -15.237333331825397, 132.3919999974316 -15.237333331825397, 132.3919999974316 -15.278222221825004, 132.34933332965429 -15.278222221825004))

  geospatial_bounds_crs :

  CRS84

  geospatial_lat_max :

  -15.237333331825397

  geospatial_lat_min :

  -15.278222221825004

  geospatial_lat_resolution :

  0.00088889

  geospatial_lat_units :

  degrees_north

  geospatial_lon_max :

  132.3919999974316

  geospatial_lon_min :

  132.34933332965429

  geospatial_lon_resolution :

  0.00088889

  geospatial_lon_units :

  degrees_east

  history :

  [{'cube_params': {'time_range': ['2019-01-01T00:00:00', '2019-12-31T00:00:00'], 'variable_names': ['agb', 'agb_sd', 'crs']}, 'program': 'xcube_cci.chunkstore.CciChunkStore'}]

  processing_level :

  L4

  time_coverage_duration :

  P364DT23H59M59S

  time_coverage_end :

  2019-12-31T23:59:59

  time_coverage_start :

  2019-01-01T00:00:00

  title :

  esacci.BIOMASS.yr.L4.AGB.multi-sensor.multi-platform.MERGED.5-0.100m

In [12]:

ds["agb"].plot()

ds["agb"].plot()

Out[12]:

<matplotlib.collections.QuadMesh at 0x743b5c3186b0>