Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Encoding for object/array subtypes (e.g raster-cube)? #299

Open
m-mohr opened this issue Nov 10, 2021 · 5 comments
Open

Encoding for object/array subtypes (e.g raster-cube)? #299

m-mohr opened this issue Nov 10, 2021 · 5 comments
Labels
Milestone

Comments

@m-mohr
Copy link
Member

m-mohr commented Nov 10, 2021

We have some subtypes such as raster-cube, vector-cube and labeled-array that don't have an encoding in JSON and with the subtypes we work around that. For examples and tests (see #204) it would be good to have a JSON encoding defined for them.

For example, raster-cube could be encoded as follows:

{
  "cube:dimensions": {...}, // see STAC spec for details
  "cube:data": [...] // multi dimensional array
}

Nevertheless, in raster-cube you'd have the issue that some values can't be represented (i.e. nan and +/-inf)

For example, labeled-array could be encoded as follows:

[
 {
   label: '2010-01',
   value: 123
  },
 {
   label: '2010-02',
   value: 321
  },
  ...
]
@soxofaan
Copy link
Member

soxofaan commented Nov 18, 2021

"cube:data": [...] // multi dimensional array
Nevertheless, in raster-cube you'd have the issue that some values can't be represented (i.e. nan and +/-inf)

Working with multidimension JSON arrays is probably fine for small examples and tests, but for larger cases (and to also cover nan/inf/...) you might want the option to have the cube's data as a binary blob, encoded as a string in hex or base64 for example.

Related: some notion of the data type (signed/unsigned bool/byte/int8/int/float/double/ ...) should probably be considered too

@m-mohr
Copy link
Member Author

m-mohr commented Nov 18, 2021

Yes, this is mostly meant for small chunks and tests. Nevertheless, the nan and +/-inf issue is somewhat a blocker for me, but we may still allow the limited JSON as a simple way of doing things and find an alternative for these special cases. Binary is also not straightforward though as you've mentioned with e.g. data types etc. If we need the data type details in the metadata though, we can just re-use what has been defined in STAC.

@soxofaan
Copy link
Member

just a quick note (and I haven't checked if this is some kind of standardized extension) but the standard json module in python does support encoding NaN and infinity in JSON (by default):

>>> import json
>>> json.dumps([1,2,3, float("nan"), float("inf")])
'[1, 2, 3, NaN, Infinity]'

@m-mohr
Copy link
Member Author

m-mohr commented Nov 23, 2021

That's unfortunately a Python-specific thing and not supported in other environments. In JS (and most other languages) this will lead to an error.

@m-mohr m-mohr added this to the 1.3.0 milestone Nov 29, 2021
@m-mohr
Copy link
Member Author

m-mohr commented Dec 1, 2021

Once this is done, update the implementation guide (chapter 'log process') correspondingly.

@m-mohr m-mohr modified the milestones: 1.3.0, 2.0.0, future Feb 1, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants