EO-LINCS project: Cube generation for Scientific Case Study (SCS) 4¶

EO enhanced benchmarking of GCB DGVMs¶

Objective: SCS4 aims to deepen the understanding of the processes that drive the European land carbon sink, with a focus on productivity, turnover, and the impacts of disturbances and land management. Leveraging new EO data and the International Land Model Benchmarking (ILAMB) system, it will assess Dynamic Global Vegetation Models (DGVMs) that contribute to the Global Carbon Budget (GCB) reports. The project will result in an enhanced ILAMB tool, offering insights into carbon dynamics and DGVM performance, and providing a roadmap for future model improvements.

Outcomes: An enhanced ILAMB evaluation tool with a focus on internal carbon dynamics and temporal change able to provide novel insights into DGVM capabilities to simulate the European land carbon sink and identify its main drivers. The spatiotemporal analysis will enable us to produce a roadmap for model improvements, in particular regarding forest management

Requried datasets:

• ESA-CCI Biomass
• Changes in Global Terrestrial Live Biomass over the 21st Century

---

The following notebook shows how the users can load data from various sources defined in scs4_config.yml using the MultiSourceDataStore tool.¶

What You Can Do with This Notebook¶

• Load datasets from various sources as defined in the scs4_config.yml
• View the progress of each data request to the MultiSourceDataStore
• Quickly preview the datasets by plotting them.

Requirements¶

Before proceeding, ensure you have the necessary dependencies installed:

• install xcube-multistore by executing: conda install --channel conda-forge xcube-multistore

Once you have it installed, you are ready to proceed.

This Multistore mainly works with a file called scs4_config.yml which is at the same file level as this notebook. To understand what goes into the schema, you can read more here.

---

Let's import the MultiSourceDataStore

In [1]:

from xcube_multistore import MultiSourceDataStore

from xcube_multistore import MultiSourceDataStore

You can find out how to fill out the config file by also using this super helpful function get_config_schema(). Run it and try expand the fields to learn more about the possible properties that the configuration file accepts along with the Configuration Guide.

In [2]:

MultiSourceDataStore.get_config_schema()

MultiSourceDataStore.get_config_schema()

Out[2]:

<xcube.util.jsonschema.JsonObjectSchema at 0x7120577c6c10>

To set up the config file, please refer to the notebook setup_config.ipynb or the Configuration Guide in the documentation.

Now, we can initialize the MultiSourceDataStore by passing the path to the scs4_config.yml which currently is on the same level as this notebook.

NOTE: In the scs4_config.yml we are also using the custom_processing feature of this tool that allows us to run a function for processing each dataset separately. In this example, we have defined a module called modify_dataset that does some custom processing which takes a xarray.Dataset argument as input and returns a new xarray.Dataset object. To read more about this custom_processing function, you can see more here.

In [3]:

msds = MultiSourceDataStore("scs4_config.yml")

msds = MultiSourceDataStore("scs4_config.yml")

And we can display the overview of the configuration file for each dataset.

In [4]:

msds.display_config()

msds.display_config()

Configuration

User-defined ID	Data Store ID	Data Store Params	Data ID	Open Data Params	Grid-Mapping	Format
biomass_xu	zenodo	root: 4161694	test10a_cd_ab_pred_corr_2000_2019_v2.tif	band_as_variable: False	bbox: [-180, -90, 180, 90]; spatial_res: 0.1; crs: EPSG:4326; tile_size: [1800, 1800]	netcdf
esa_cci_biomass	cciodp	-	esacci.BIOMASS.yr.L4.AGB.multi-sensor.multi-platform.MERGED.5-0.10000m	-	bbox: [-180, -90, 180, 90]; spatial_res: 0.1; crs: EPSG:4326; tile_size: [1800, 1800]	netcdf

We can display the selected bounding box as shown in the following cell.

In [5]:

msds.display_geolocations()

msds.display_geolocations()

Out[5]:

Make this Notebook Trusted to load map: File -> Trust Notebook

We can now generate the datacubes:

In [6]:

msds.generate()

msds.generate()

Cube Generation

Dataset identifier	Status	Message	Exception
biomass_xu	COMPLETED	Dataset 'biomass_xu' finished: 0:00:28	-
esa_cci_biomass	COMPLETED	Dataset 'esa_cci_biomass' finished: 0:00:58	-

We can now open the data using the xcube datastore framework API as usual. Note that the multi-source data store requires a data store called storage, which is configured in our scs4_config.yml under the data_stores section.

In [7]:

ds = msds.stores.storage.open_data("biomass_xu.nc", chunks="auto")
ds

ds = msds.stores.storage.open_data("biomass_xu.nc", chunks="auto") ds

Out[7]:

<xarray.Dataset> Size: 518MB
Dimensions:         (time: 20, lat: 1800, lon: 3600)
Coordinates:
  * time            (time) datetime64[ns] 160B 2000-01-01 ... 2019-01-01
  * lat             (lat) float64 14kB 89.95 89.85 89.75 ... -89.85 -89.95
  * lon             (lon) float64 29kB -179.9 -179.8 -179.8 ... 179.8 179.9
    spatial_ref     int64 8B ...
Data variables:
    carbon_density  (time, lat, lon) float32 518MB dask.array<chunksize=(12, 1181, 2365), meta=np.ndarray>

xarray.Dataset

• Dimensions:
  • time: 20
  • lat: 1800
  • lon: 3600
• Coordinates: (4)
  • time
    (time)
    datetime64[ns]
    2000-01-01 ... 2019-01-01

    array(['2000-01-01T00:00:00.000000000', '2001-01-01T00:00:00.000000000',
           '2002-01-01T00:00:00.000000000', '2003-01-01T00:00:00.000000000',
           '2004-01-01T00:00:00.000000000', '2005-01-01T00:00:00.000000000',
           '2006-01-01T00:00:00.000000000', '2007-01-01T00:00:00.000000000',
           '2008-01-01T00:00:00.000000000', '2009-01-01T00:00:00.000000000',
           '2010-01-01T00:00:00.000000000', '2011-01-01T00:00:00.000000000',
           '2012-01-01T00:00:00.000000000', '2013-01-01T00:00:00.000000000',
           '2014-01-01T00:00:00.000000000', '2015-01-01T00:00:00.000000000',
           '2016-01-01T00:00:00.000000000', '2017-01-01T00:00:00.000000000',
           '2018-01-01T00:00:00.000000000', '2019-01-01T00:00:00.000000000'],
          dtype='datetime64[ns]')

  • lat
    (lat)
    float64
    89.95 89.85 89.75 ... -89.85 -89.95

    array([ 89.95,  89.85,  89.75, ..., -89.75, -89.85, -89.95], shape=(1800,))

  • lon
    (lon)
    float64
    -179.9 -179.8 ... 179.8 179.9

    array([-179.95, -179.85, -179.75, ...,  179.75,  179.85,  179.95],
          shape=(3600,))

  • spatial_ref
    ()
    int64
    ...

    crs_wkt :

    GEOGCRS["WGS 84",DATUM["World Geodetic System 1984",ELLIPSOID["WGS 84",6378137,298.257223563,LENGTHUNIT["metre",1]]],PRIMEM["Greenwich",0,ANGLEUNIT["degree",0.0174532925199433]],CS[ellipsoidal,2],AXIS["geodetic latitude (Lat)",north,ORDER[1],ANGLEUNIT["degree",0.0174532925199433]],AXIS["geodetic longitude (Lon)",east,ORDER[2],ANGLEUNIT["degree",0.0174532925199433]],ID["EPSG",4326]]

    semi_major_axis :

    6378137.0

    semi_minor_axis :

    6356752.314245179

    inverse_flattening :

    298.257223563

    reference_ellipsoid_name :

    WGS 84

    longitude_of_prime_meridian :

    0.0

    prime_meridian_name :

    Greenwich

    geographic_crs_name :

    WGS 84

    horizontal_datum_name :

    World Geodetic System 1984

    grid_mapping_name :

    latitude_longitude

    [1 values with dtype=int64]

• Data variables: (1)
  • carbon_density
    (time, lat, lon)
    float32
    dask.array<chunksize=(12, 1181, 2365), meta=np.ndarray>

    NETCDF_DIM_EXTRA :

    {time}

    NETCDF_DIM_time_DEF :

    [20. 10.]

    NETCDF_DIM_time_VALUES :

    [ 0. 366. 731. 1096. 1461. 1827. 2192. 2557. 2922. 3288. 3653. 4018. 4383. 4749. 5114. 5479. 5844. 6210. 6575. 6940.]

    AREA_OR_POINT :

    Area

    grid_mapping :

    spatial_ref

    Array Chunk Bytes 494.38 MiB 127.86 MiB Shape (20, 1800, 3600) (12, 1181, 2365) Dask graph 8 chunks in 2 graph layers Data type float32 numpy.ndarray

We can now select a variable for one timestep and plot it for a quick preview of the data

In [8]:

ds.carbon_density.isel(time=1).plot()

ds.carbon_density.isel(time=1).plot()

Out[8]:

<matplotlib.collections.QuadMesh at 0x712054139550>

In [9]:

ds = msds.stores.storage.open_data("esa_cci_biomass.nc", chunks="auto")
ds

ds = msds.stores.storage.open_data("esa_cci_biomass.nc", chunks="auto") ds

Out[9]:

<xarray.Dataset> Size: 415MB
Dimensions:      (time: 8, lat: 1800, lon: 3600)
Coordinates:
  * time         (time) datetime64[ns] 64B 2010-01-01 2015-01-01 ... 2021-01-01
  * lat          (lat) float64 14kB 89.95 89.85 89.75 ... -89.75 -89.85 -89.95
  * lon          (lon) float64 29kB -179.9 -179.8 -179.8 ... 179.7 179.8 179.9
    spatial_ref  int64 8B ...
Data variables:
    agb          (time, lat, lon) float32 207MB dask.array<chunksize=(6, 1671, 3344), meta=np.ndarray>
    agb_sd       (time, lat, lon) float32 207MB dask.array<chunksize=(6, 1671, 3344), meta=np.ndarray>
Attributes:
    Conventions:             CF-1.7
    title:                   esacci.BIOMASS.yr.L4.AGB.multi-sensor.multi-plat...
    date_created:            2025-12-18T15:19:51.388367
    processing_level:        L4
    time_coverage_start:     2010-01-01T00:00:00
    time_coverage_end:       2021-01-01T00:00:00
    time_coverage_duration:  P4018DT0H0M0S
    history:                 [{'program': 'xcube_cci.chunkstore.CciChunkStore...

xarray.Dataset

• Dimensions:
  • time: 8
  • lat: 1800
  • lon: 3600
• Coordinates: (4)
  • time
    (time)
    datetime64[ns]
    2010-01-01 ... 2021-01-01

    standard_name :

    time

    bounds :

    time_bnds

    array(['2010-01-01T00:00:00.000000000', '2015-01-01T00:00:00.000000000',
           '2016-01-01T00:00:00.000000000', '2017-01-01T00:00:00.000000000',
           '2018-01-01T00:00:00.000000000', '2019-01-01T00:00:00.000000000',
           '2020-01-01T00:00:00.000000000', '2021-01-01T00:00:00.000000000'],
          dtype='datetime64[ns]')

  • lat
    (lat)
    float64
    89.95 89.85 89.75 ... -89.85 -89.95

    array([ 89.95,  89.85,  89.75, ..., -89.75, -89.85, -89.95], shape=(1800,))

  • lon
    (lon)
    float64
    -179.9 -179.8 ... 179.8 179.9

    array([-179.95, -179.85, -179.75, ...,  179.75,  179.85,  179.95],
          shape=(3600,))

  • spatial_ref
    ()
    int64
    ...

    crs_wkt :

    GEOGCRS["\""WGS 84\""",DATUM["\""WGS_1984\""",ELLIPSOID["\""WGS 84\""",6378137,298.257223563,LENGTHUNIT["metre",1]]],PRIMEM["\""Greenwich\""",0,ANGLEUNIT["\""degree\""",0.0174532925199433]],CS[ellipsoidal,2],AXIS["longitude",east,ORDER[1],ANGLEUNIT["\""degree\""",0.0174532925199433]],AXIS["latitude",north,ORDER[2],ANGLEUNIT["\""degree\""",0.0174532925199433]],ID["\""EPSG\""","\""4326\"""]]

    semi_major_axis :

    6378137.0

    semi_minor_axis :

    6356752.314245179

    inverse_flattening :

    298.257223563

    reference_ellipsoid_name :

    \"WGS 84\"

    longitude_of_prime_meridian :

    0.0

    prime_meridian_name :

    \"Greenwich\"

    geographic_crs_name :

    \"WGS 84\"

    horizontal_datum_name :

    \"WGS_1984\"

    grid_mapping_name :

    latitude_longitude

    [1 values with dtype=int64]

• Data variables: (2)
  • agb
    (time, lat, lon)
    float32
    dask.array<chunksize=(6, 1671, 3344), meta=np.ndarray>

    long_name :

    Above-ground biomass

    grid_mapping :

    spatial_ref

    units :

    Mg/ha

    valid_min :

    0.0

    valid_max :

    10000.0

    orig_data_type :

    int16

    fill_value :

    -32768

    size :

    51840000

    shape :

    [ 8 1800 3600]

    chunk_sizes :

    [ 8 600 600]

    file_chunk_sizes :

    [ 8 600 600]

    data_type :

    int16

    dimensions :

    ['time', 'lat', 'lon']

    file_dimensions :

    ['time', 'lat', 'lon']

    Array Chunk Bytes 197.75 MiB 127.90 MiB Shape (8, 1800, 3600) (6, 1671, 3344) Dask graph 8 chunks in 2 graph layers Data type float32 numpy.ndarray

  • agb_sd
    (time, lat, lon)
    float32
    dask.array<chunksize=(6, 1671, 3344), meta=np.ndarray>

    long_name :

    Above-ground biomass standard_deviation

    grid_mapping :

    spatial_ref

    units :

    Mg/ha

    valid_min :

    0.0

    valid_max :

    10000.0

    orig_data_type :

    int16

    fill_value :

    -32768

    size :

    51840000

    shape :

    [ 8 1800 3600]

    chunk_sizes :

    [ 8 600 600]

    file_chunk_sizes :

    [ 8 600 600]

    data_type :

    int16

    dimensions :

    ['time', 'lat', 'lon']

    file_dimensions :

    ['time', 'lat', 'lon']

    Array Chunk Bytes 197.75 MiB 127.90 MiB Shape (8, 1800, 3600) (6, 1671, 3344) Dask graph 8 chunks in 2 graph layers Data type float32 numpy.ndarray

• Attributes: (8)

  Conventions :

  CF-1.7

  title :

  esacci.BIOMASS.yr.L4.AGB.multi-sensor.multi-platform.MERGED.5-0.10000m

  date_created :

  2025-12-18T15:19:51.388367

  processing_level :

  L4

  time_coverage_start :

  2010-01-01T00:00:00

  time_coverage_end :

  2021-01-01T00:00:00

  time_coverage_duration :

  P4018DT0H0M0S

  history :

  [{'program': 'xcube_cci.chunkstore.CciChunkStore', 'cube_params': {'time_range': ['2010-01-01T00:00:00', '2021-12-31T23:59:59'], 'variable_names': ['agb', 'agb_sd', 'crs']}}]

In [10]:

ds.agb.isel(time=0).plot()

ds.agb.isel(time=0).plot()

Out[10]:

<matplotlib.collections.QuadMesh at 0x712053ce0190>

In [11]:

ds.agb_sd.isel(time=0).plot()

ds.agb_sd.isel(time=0).plot()

Out[11]:

<matplotlib.collections.QuadMesh at 0x712053e2e990>