Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

load_collection and load_stac: Clarify the dimension names and labels #491

Open
wants to merge 3 commits into
base: draft
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- `eq` and `neq`: Explicitly set the minimum value for the `delta` parameter.
- `filter_bbox`, `load_collection`, `load_stac`: Clarified that the bounding box is reprojected to the CRS of the spatial data cube dimensions if required.
- `filter_spatial`: Clarified that masking is applied using the given geometries. [#469](https://github.com/Open-EO/openeo-processes/issues/469)
- `load_collection` and `load_stac`: Clarify the dimension naming and the order of dimension labels for nominal labels. [#488](https://github.com/Open-EO/openeo-processes/issues/488), [#489](https://github.com/Open-EO/openeo-processes/issues/489)
- `mod`: Clarified behavior for y = 0
- `sqrt`: Clarified that NaN is returned for negative numbers.

Expand Down
4 changes: 2 additions & 2 deletions load_collection.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"id": "load_collection",
"summary": "Load a collection",
"description": "Loads a collection from the current back-end by its id and returns it as a processable data cube. The data that is added to the data cube can be restricted with the parameters `spatial_extent`, `temporal_extent`, `bands` and `properties`. If no data is available for the given extents, a `NoDataAvailable` exception is thrown.\n\n**Remarks:**\n\n* The bands (and all dimensions that specify nominal dimension labels) are expected to be ordered as specified in the metadata if the `bands` parameter is set to `null`.\n* If no additional parameter is specified this would imply that the whole data set is expected to be loaded. Due to the large size of many data sets, this is not recommended and may be optimized by back-ends to only load the data that is actually required after evaluating subsequent processes such as filters. This means that the values in the data cube should be processed only after the data has been limited to the required extent and as a consequence also to a manageable size.",
"description": "Loads a collection from the current back-end by its id and returns it as a processable data cube. The data that is added to the data cube can be restricted with the parameters `spatial_extent`, `temporal_extent`, `bands` and `properties`. If no data is available for the given extents, a `NoDataAvailable` exception is thrown.\n\n**Remarks:**\n\n* All dimensions that specify nominal dimension labels (e.g. bands) are ordered as specified in the data cube metadata (`cube:dimensions`) unless otherwise specified in a corresponding parameter (e.g. `bands`).\n* If no additional parameter is specified this would imply that the whole data set is expected to be loaded. Due to the large size of many data sets, this is not recommended and may be optimized by back-ends to only load the data that is actually required after evaluating subsequent processes such as filters. This means that the values in the data cube should be processed only after the data has been limited to the required extent and as a consequence also to a manageable size.",
"categories": [
"cubes",
"import"
Expand Down Expand Up @@ -159,7 +159,7 @@
},
{
"name": "bands",
"description": "Only adds the specified bands into the data cube so that bands that don't match the list of band names are not available. Applies to all dimensions of type `bands`.\n\nEither the unique band name (metadata field `name` in bands) or one of the common band names (metadata field `common_name` in bands) can be specified. If the unique band name and the common name conflict, the unique band name has a higher priority.\n\nThe order of the specified array defines the order of the bands in the data cube. If multiple bands match a common name, all matched bands are included in the original order.\n\nIt is recommended to use this parameter instead of using ``filter_bands()`` directly after loading unbounded data.",
"description": "Only adds the specified bands into the data cube so that bands that don't match the list of band names are not available. Applies to all dimensions of type `bands`.\n\nEither the unique band name (metadata field `name` in bands) or one of the common band names (metadata field `common_name` in bands) can be specified. If the unique band name and the common name conflict, the unique band name has a higher priority.\n\nThe order of the specified array defines the order of the bands in the data cube. If multiple bands match a common name, all matched bands are included in the original order. If the parameter is set to `null`, the order of the bands is as defined for the bands dimension in the data cube metadata (metadata field `cube:dimensions`).\n\nIt is recommended to use this parameter instead of using ``filter_bands()`` directly after loading unbounded data.",
"schema": [
{
"type": "array",
Expand Down
4 changes: 2 additions & 2 deletions proposals/load_stac.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"id": "load_stac",
"summary": "Loads data from STAC",
"description": "Loads data from a static STAC catalog or a STAC API Collection and returns the data as a processable data cube. A batch job result can be loaded by providing a reference to it.\n\nIf supported by the underlying metadata and file format, the data that is added to the data cube can be restricted with the parameters `spatial_extent`, `temporal_extent` and `bands`. If no data is available for the given extents, a `NoDataAvailable` exception is thrown.\n\n**Remarks:**\n\n* The bands (and all dimensions that specify nominal dimension labels) are expected to be ordered as specified in the metadata if the `bands` parameter is set to `null`.\n* If no additional parameter is specified this would imply that the whole data set is expected to be loaded. Due to the large size of many data sets, this is not recommended and may be optimized by back-ends to only load the data that is actually required after evaluating subsequent processes such as filters. This means that the values should be processed only after the data has been limited to the required extent and as a consequence also to a manageable size.",
"description": "Loads data from a static STAC catalog or a STAC API Collection and returns the data as a processable data cube. A batch job result can be loaded by providing a reference to it.\n\nIf supported by the underlying metadata and file format, the data that is added to the data cube can be restricted with the parameters `spatial_extent`, `temporal_extent` and `bands`. If no data is available for the given extents, a `NoDataAvailable` exception is thrown.\n\n**Remarks:**\n\n* The dimensions (e.g. names) follow the data cube metadata (`cube:dimensions`), if present. Otherwise, it tries to preserve any dimension names available in the files. Otherwise, it falls back to the recommended dimension names as specified in the openEO API (`x`, `y`, `z`, `t`, `bands`, `geometry`).\n* All dimensions that specify nominal dimension labels (e.g. bands) are ordered as specified in the data cube metadata (`cube:dimensions`) unless otherwise specified in a corresponding parameter (e.g. `bands`). If no data cube metadata is present, the order of the dimension labels will reflect the structure in the files.\n* If no additional parameter is specified this would imply that the whole data set is expected to be loaded. Due to the large size of many data sets, this is not recommended and may be optimized by back-ends to only load the data that is actually required after evaluating subsequent processes such as filters. This means that the values should be processed only after the data has been limited to the required extent and as a consequence also to a manageable size.",
"categories": [
"cubes",
"import"
Expand Down Expand Up @@ -165,7 +165,7 @@
},
{
"name": "bands",
"description": "Only adds the specified bands into the data cube so that bands that don't match the list of band names are not available. Applies to all dimensions of type `bands`.\n\nEither the unique band name (metadata field `name` in bands) or one of the common band names (metadata field `common_name` in bands) can be specified. If the unique band name and the common name conflict, the unique band name has a higher priority.\n\nThe order of the specified array defines the order of the bands in the data cube. If multiple bands match a common name, all matched bands are included in the original order.\n\nIt is recommended to use this parameter instead of using ``filter_bands()`` directly after loading unbounded data.",
"description": "Only adds the specified bands into the data cube so that bands that don't match the list of band names are not available. Applies to all dimensions of type `bands`.\n\nEither the unique band name (metadata field `name` in bands) or one of the common band names (metadata field `common_name` in bands) can be specified. If the unique band name and the common name conflict, the unique band name has a higher priority.\n\nThe order of the specified array defines the order of the bands in the data cube. If multiple bands match a common name, all matched bands are included in the original order. If the parameter is set to `null`, the order of the bands is as defined for the bands dimension in the data cube metadata (metadata field `cube:dimensions`).\n\nIt is recommended to use this parameter instead of using ``filter_bands()`` directly after loading unbounded data.",
"schema": [
{
"type": "array",
Expand Down
4 changes: 2 additions & 2 deletions rename_labels.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"id": "rename_labels",
"summary": "Rename dimension labels",
"description": "Renames the labels of the specified dimension in the data cube from `source` to `target`.\n\nIf the array for the source labels is empty (the default), the dimension labels are expected to be enumerated with zero-based numbering (0,1,2,3,...) so that the dimension labels directly map to the indices of the array specified for the parameter `target`. Otherwise, the number of the source and target labels must be equal. If none of these requirements is fulfilled, the `LabelMismatch` exception is thrown.\n\nThis process doesn't change the order of the labels and their corresponding data.",
"description": "Renames the labels of the specified dimension in the data cube from `source` to `target`.\n\nIf the array for the source labels is empty (the default), the dimension labels are enumerated with zero-based numbering (0,1,2,3,...) so that the dimension labels directly map to the indices of the array specified for the parameter `target`. Otherwise, the number of the source and target labels must be equal. If none of these requirements is fulfilled, the `LabelMismatch` exception is thrown.\n\nThis process doesn't change the order of the labels and their corresponding data.",
"categories": [
"cubes"
],
Expand Down Expand Up @@ -37,7 +37,7 @@
},
{
"name": "source",
"description": "The original names of the labels to be renamed to corresponding array elements in the parameter `target`. It is allowed to only specify a subset of labels to rename, as long as the `target` and `source` parameter have the same length. The order of the labels doesn't need to match the order of the dimension labels in the data cube. By default, the array is empty so that the dimension labels in the data cube are expected to be enumerated.\n\nIf the dimension labels are not enumerated and the given array is empty, the `LabelsNotEnumerated` exception is thrown. If one of the source dimension labels doesn't exist, the `LabelNotAvailable` exception is thrown.",
"description": "The original names of the labels to be renamed to corresponding array elements in the parameter `target`. It is allowed to only specify a subset of labels to rename, as long as the `target` and `source` parameter have the same length. The order of the labels doesn't need to match the order of the dimension labels in the data cube. By default, the array is empty so that the dimension labels in the data cube are enumerated.\n\nIf the dimension labels are not enumerated and the given array is empty, the `LabelsNotEnumerated` exception is thrown. If one of the source dimension labels doesn't exist, the `LabelNotAvailable` exception is thrown.",
"schema": {
"type": "array",
"uniqueItems": true,
Expand Down