qubed/docs/quickstart.md

jupytext

jupytext
text_representation extension format_name format_version jupytext_version .md myst 0.13 1.16.4

Quickstart

Installation

pip install qubed

Usage

Make an uncompressed qube:

from qubed import Qube

q = Qube.from_dict({
    "class=od" : {
        "expver=0001": {"param=1":{}, "param=2":{}},
        "expver=0002": {"param=1":{}, "param=2":{}},
    },
    "class=rd" : {
        "expver=0001": {"param=1":{}, "param=2":{}, "param=3":{}},
        "expver=0002": {"param=1":{}, "param=2":{}},
    },
})
print(f"{q.n_leaves = }, {q.n_nodes = }")
q

Compress it:

cq = q.compress()
assert cq.n_leaves == q.n_leaves
print(f"{cq.n_leaves = }, {cq.n_nodes = }")
cq

Quick Tree Construction

One of the quickest ways to construct non-trivial trees is to use the Qube.from_datacube method to construct dense trees and then use the set operations to combine or intersect them:

q = Qube.from_datacube({
    "class": "d1",
    "dataset": ["climate-dt", "another-value"],
    'generation': ['1', "2", "3"],
})

r  = Qube.from_datacube({
    "class": "d1",
    "dataset": ["weather-dt", "climate-dt"],
    'generation': ['1', "2", "3", "4"],
})

q | r

Iteration / Flattening

Iterate over the leaves:

for i, identifier in enumerate(cq.leaves()):
    print(identifier)
    if i > 10:
        print("...")
        break

Iterate over the datacubes:

for i, datacube in enumerate(cq.datacubes()):
    print(datacube)
    if i > 10:
        print("...")
        break

A Real World Example

Load a larger example qube:

import requests
qube_json = requests.get("https://github.com/ecmwf/qubed/raw/refs/heads/main/tests/example_qubes/climate_dt.json").json()
climate_dt = Qube.from_json(qube_json)

# Using the html or print methods is optional but lets you specify things like the depth of the tree to display.
print(f"{climate_dt.n_leaves = }, {climate_dt.n_nodes = }")
climate_dt.html(depth=1) # Limit how much is open initially, click leave to see more.

Select a subset of the tree:

climate_dt.select({
    "activity": "scenariomip"
}).html(depth=1)

Use .span("key") to get the set of possibles values for a key, note this includes anywhere this key appears in the tree.

climate_dt.span("activity")

Use .axes() to get the span of every key in one go.

axes = climate_dt.axes()
for key, values in axes.items():
    print(f"{key} : {list(values)[:10]}")

Set Operations

The union/intersection/difference of two dense datacubes is not itself dense.

A = Qube.from_dict({"a=1/2/3" : {"b=i/j/k" : {}},})
B = Qube.from_dict({"a=2/3/4" : {"b=j/k/l" : {}},})

A.print(), B.print();

Union:

(A | B).print();

Intersection:

(A & B).print();

Difference:

(A - B).print();

Symmetric Difference:

(A ^ B).print();