qubed/ROADMAP.md

Initial Python Implementation [x] Basic Qube datastructure [x] Compression [x] Set Operations (Union, Difference, Intersection...) [x] Query with request [x] Iteration over leaves [x] Iteration over datacubes [x] Command line creation from fdb list --compact [ ] Set up periodic updates to climate-dt/extremes-dt again [ ] Maybe also do production db? [ ] Do mars list to contraints conversion [ ] protobuf serialization

Rust port [ ] Initial object [ ] Sort out ownership issues, (one arena owned by python object) [ ] Compression [ ] Set Operations [ ] Query with request [ ] Iteration over leaves [ ] Iteration over datacubes [ ] Set up periodic updates to climate-dt/extremes-dt again

API

Qubed will provide a core compressed tree data structure called a Qube with:

Methods to convert to and from:

• A human readable representation like those seen above.
• An HTML version where subtrees can be collapsed.
• An compact protobuf-based binary format
• Nested python dictionaries or JSON
• [/] The output of fdb list
• mars list
• constraints.json

Useful algorithms:

• Compression
• [/] Union/Intersection/Difference

Performant Membership Queries

• Identifier membership
• Datacube query (selection)

Metadata Storage