reset using ml branch

CHANGELOG.md

@@ -6,24 +6,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## Unreleased / Draft
 
-### Changed
-
-- `clip`: Throw an exception if min > max [#472](https://github.com/Open-EO/openeo-processes/issues/472)
-
-### Fixed
-
-- `between`: Clarify that `null` is passed through.
-- `eq` and `neq`: Explicitly set the minimum value for the `delta` parameter.
-- `filter_bbox`, `load_collection`, `load_stac`: Clarified that the bounding box is reprojected to the CRS of the spatial data cube dimensions if required.
-- `filter_spatial`: Clarified that masking is applied using the given geometries. [#469](https://github.com/Open-EO/openeo-processes/issues/469)
-- `sqrt`: Clarified that NaN is returned for negative numbers.
-
 ## [2.0.0-rc.1] - 2023-05-25
 
-### Fixed
-
-- `array_append`: Added `number` type for labels to be consistent with other processes. Default to numerical index instead of string. Clarify that the `label` parameter only applies to labeled arrays.
-
 ### Added
 
 - New processes in proposal state:
@@ -32,7 +16,13 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
     - `filter_vector`
     - `flatten_dimensions`
     - `load_geojson`
+    - `load_ml_model`
     - `load_url`
+    - `ml_fit_class_random_forest`
+    - `ml_fit_regr_random_forest`
+    - `ml_fit_class-xgboost`
+    - `ml_predict`
+    - `save_ml_model`
     - `unflatten_dimension`
     - `vector_buffer`
     - `vector_reproject`

README.md

@@ -35,7 +35,7 @@ This repository contains a set of files formally describing the openEO Processes
 * [implementation.md](meta/implementation.md) in the `meta` folder provide some additional implementation details for back-ends. For back-end implementors, it's highly recommended to read them.
 * [subtype-schemas.json](meta/subtype-schemas.json) in the `meta` folder defines common data types (`subtype`s) for JSON Schema used in openEO processes.
 * Previously, an `examples` folder contained examples of user-defined processes. These have been migrated to the [openEO Community Examples](https://github.com/Open-EO/openeo-community-examples/tree/main/processes) repository.
-* The [`dev`](dev/) folder can be used to test the process specification for validity and consistent "style". It also allows rendering the processes in a web browser. Check the [development documentation](dev/README.md) for details.
+* The [`tests`](tests/) folder can be used to test the process specification for validity and consistent "style". It also allows rendering the processes in a web browser. Check the [tests documentation](tests/README.md) for details.
 
 ## Process
 

and.json

@@ -1,7 +1,7 @@
 {
     "id": "and",
     "summary": "Logical AND",
-    "description": "Checks if **both** values are true.\n\nEvaluates parameter `x` before `y` and stops once the outcome is unambiguous. If any argument is `null`, the result will be `null` if the outcome is ambiguous.\n\n**Truth table:**\n\n```\nx \\ y || null  | false | true\n----- || ----- | ----- | -----\nnull  || null  | false | null\nfalse || false | false | false\ntrue  || null  | false | true\n```",
+    "description": "Checks if **both** values are true.\n\nEvaluates parameter `x` before `y` and stops once the outcome is unambiguous. If any argument is `null`, the result will be `null` if the outcome is ambiguous.\n\n**Truth table:**\n\n```\na \\ b || null  | false | true\n----- || ----- | ----- | -----\nnull  || null  | false | null\nfalse || false | false | false\ntrue  || null  | false | true\n```",
     "categories": [
         "logic"
     ],
@@ -90,4 +90,4 @@
             "result": true
         }
     }
-}
+}

array_append.json

@@ -25,20 +25,15 @@
         },
         {
             "name": "label",
-            "description": "Provides a label for the new value. If not given or `null`, the natural next array index as number is used as the label. If in any case the label exists, a `LabelExists` exception is thrown.\n\nThis parameter only applies if the given array is a labeled array. If a non-null values is provided and the array is not labeled, an `ArrayNotLabeled` exception is thrown.",
+            "description": "If the given array is a labeled array, a new label for the new value should be given. If not given or `null`, the array index as string is used as the label. If in any case the label exists, a `LabelExists` exception is thrown.",
             "optional": true,
             "default": null,
-            "schema": [
-                {
-                    "type": "number"
-                },
-                {
-                    "type": "string"
-                },
-                {
-                    "type": "null"
-                }
-            ]
+            "schema": {
+                "type": [
+                    "string",
+                    "null"
+                ]
+            }
         }
     ],
     "returns": {
@@ -53,9 +48,6 @@
     "exceptions": {
         "LabelExists": {
             "message": "An array element with the specified label already exists."
-        },
-        "ArrayNotLabeled": {
-            "message": "A label can't be provided as the given array is not labeled."
         }
     },
     "examples": [

between.json

@@ -8,7 +8,7 @@
     "parameters": [
         {
             "name": "x",
-            "description": "The value to check.\n\nThe no-data value `null` is passed through and therefore gets propagated.",
+            "description": "The value to check.",
             "schema": {
                 "description": "Any data type is allowed."
             }
@@ -38,7 +38,7 @@
         }
     ],
     "returns": {
-        "description": "`true` if `x` is between the specified bounds, `null` if `x` is a no-data value, `false` otherwise.",
+        "description": "`true` if `x` is between the specified bounds, otherwise `false`.",
         "schema": {
             "type": [
                 "boolean",

clip.json

@@ -1,7 +1,7 @@
 {
     "id": "clip",
     "summary": "Clip a value between a minimum and a maximum",
-    "description": "Clips a number between specified minimum and maximum values. A value larger than the maximum value is set to the maximum value, a value lower than the minimum value is set to the minimum value. If the maximum value is smaller than the minimum number, the process throws a `MinMaxSwapped` exception.\n\nThe no-data value `null` is passed through and therefore gets propagated.",
+    "description": "Clips a number between specified minimum and maximum values. A value larger than the maximum value is set to the maximum value, a value lower than the minimum value is set to the minimum value.\n\nThe no-data value `null` is passed through and therefore gets propagated.",
     "categories": [
         "math"
     ],
@@ -40,11 +40,6 @@
             ]
         }
     },
-    "exceptions": {
-        "MinMaxSwapped": {
-            "message": "The minimum value should be lower than or equal to the maximum value."
-        }
-    },
     "examples": [
         {
             "arguments": {
@@ -78,5 +73,34 @@
             },
             "returns": null
         }
-    ]
-}
+    ],
+    "process_graph": {
+        "min": {
+            "process_id": "min",
+            "arguments": {
+                "data": [
+                    {
+                        "from_parameter": "max"
+                    },
+                    {
+                        "from_parameter": "x"
+                    }
+                ]
+            }
+        },
+        "max": {
+            "process_id": "max",
+            "arguments": {
+                "data": [
+                    {
+                        "from_parameter": "min"
+                    },
+                    {
+                        "from_node": "min"
+                    }
+                ]
+            },
+            "result": true
+        }
+    }
+}

eq.json

@@ -38,8 +38,7 @@
                 "type": [
                     "number",
                     "null"
-                ],
-                "minimumExclusive": 0
+                ]
             },
             "default": null,
             "optional": true

filter_bbox.json

@@ -1,7 +1,7 @@
 {
     "id": "filter_bbox",
     "summary": "Spatial filter using a bounding box",
-    "description": "Limits the data cube to the specified bounding box.\n\n* For raster data cubes, the filter retains a pixel in the data cube if the point at the pixel center intersects with the bounding box (as defined in the Simple Features standard by the OGC). Alternatively, ``filter_spatial()`` can be used to filter by geometry.\n* For vector data cubes, the filter retains the geometry in the data cube if the geometry is fully within the bounding box (as defined in the Simple Features standard by the OGC). All geometries that were empty or not contained fully within the bounding box will be removed from the data cube.\n\nAlternatively, filter spatially with geometries using ``filter_spatial()`` (on a raster data cube) or ``filter_vector()`` (on a vector data cube).",
+    "description": "Limits the data cube to the specified bounding box.\n\n* For raster data cubes, the filter retains a pixel in the data cube if the point at the pixel center intersects with the bounding box (as defined in the Simple Features standard by the OGC). Alternatively, ``filter_spatial()`` can be used to filter by geometry.\n* For vector data cubes, the filter retains the geometry in the data cube if the geometry is fully within the bounding box (as defined in the Simple Features standard by the OGC). All geometries that were empty or not contained fully within the bounding box will be removed from the data cube.\n\nAlternatively, ``filter_vector()`` can be used to filter by geometry.",
     "categories": [
         "cubes",
         "filter"
@@ -39,7 +39,7 @@
         },
         {
             "name": "extent",
-            "description": "A bounding box, which may include a vertical axis (see `base` and `height`).\n\nIf the bounding box is not provided in the coordinate reference system (CRS) of the data cube, the bounding box is reprojected to the CRS of the spatial data cube dimensions.",
+            "description": "A bounding box, which may include a vertical axis (see `base` and `height`).",
             "schema": {
                 "type": "object",
                 "subtype": "bounding-box",

filter_spatial.json

@@ -1,7 +1,7 @@
 {
     "id": "filter_spatial",
     "summary": "Spatial filter raster data cubes using geometries",
-    "description": "Limits the raster data cube over the spatial dimensions to the specified geometries.\n\n- For **polygons**, the filter retains a pixel in the data cube if the point at the pixel center intersects with at least one of the polygons (as defined in the Simple Features standard by the OGC).\n- For **points**, the process considers the closest pixel center.\n- For **lines** (line strings), the process considers all the pixels whose centers are closest to at least one point on the line.\n\nMore specifically, pixels outside of the bounding box of the given geometries will not be available after filtering. All pixels inside the bounding box that are not retained will be set to `null` (no data).\n\n Alternatively, use ``filter_bbox()`` to filter with a bounding box or ``filter_vector()`` to filter a vector data cube based on geometries. Use ``mask_polygon()`` to mask without changing the spatial extent of your data cube.",
+    "description": "Limits the raster data cube over the spatial dimensions to the specified geometries.\n\n- For **polygons**, the filter retains a pixel in the data cube if the point at the pixel center intersects with at least one of the polygons (as defined in the Simple Features standard by the OGC).\n- For **points**, the process considers the closest pixel center.\n- For **lines** (line strings), the process considers all the pixels whose centers are closest to at least one point on the line.\n\nMore specifically, pixels outside of the bounding box of the given geometry will not be available after filtering. All pixels inside the bounding box that are not retained will be set to `null` (no data).\n\n Alternatively, use ``filter_bbox()`` to filter by bounding box.",
     "categories": [
         "cubes",
         "filter"
@@ -26,7 +26,7 @@
         },
         {
             "name": "geometries",
-            "description": "One or more geometries used for spatial filtering and masking, given as GeoJSON or vector data cube.",
+            "description": "One or more geometries used for filtering, given as GeoJSON or vector data cube. If multiple geometries are provided, the union of them is used. Empty geometries are ignored.\n\nLimits the data cube to the bounding box of the given geometries. No implicit masking gets applied. To mask the pixels of the data cube use ``mask_polygon()``.",
             "schema": [
                 {
                     "title": "Vector Data Cube",

linear_scale_range.json

@@ -1,7 +1,7 @@
 {
     "id": "linear_scale_range",
     "summary": "Linear transformation between two ranges",
-    "description": "Performs a linear transformation between the input and output range.\n\nThe given number in `x` is clipped to the bounds specified in `inputMin` and `inputMax` so that the underlying formula *`((x - inputMin) / (inputMax - inputMin)) * (outputMax - outputMin) + outputMin`* never returns a value outside of the range defined by `outputMin` and `outputMax`.\n\nPotential use case include\n\n* scaling values to the 8-bit range (0 - 255) often used for numeric representation of values in one of the channels of the [RGB colour model](https://en.wikipedia.org/wiki/RGB_color_model#Numeric_representations) or\n* calculating percentages (0 - 100).\n\nThe no-data value `null` is passed through and therefore gets propagated.",
+    "description": "Performs a linear transformation between the input and output range.\n\nThe given number in `x` is clipped to the bounds specified in `inputMin` and `inputMax` so that the underlying formula *`((x - inputMin) / (inputMax - inputMin)) * (outputMax - outputMin) + outputMin`* never returns any value lower than `outputMin` or greater than `outputMax`.\n\nPotential use case include\n\n* scaling values to the 8-bit range (0 - 255) often used for numeric representation of values in one of the channels of the [RGB colour model](https://en.wikipedia.org/wiki/RGB_color_model#Numeric_representations) or\n* calculating percentages (0 - 100).\n\nThe no-data value `null` is passed through and therefore gets propagated.",
     "categories": [
         "math"
     ],
@@ -166,4 +166,4 @@
             "result": true
         }
     }
-}
+}

load_collection.json

@@ -64,7 +64,7 @@
                             "default": null
                         },
                         "crs": {
-                            "description": "Coordinate reference system of the extent, specified as as [EPSG code](http://www.epsg-registry.org/) or [WKT2 CRS string](http://docs.opengeospatial.org/is/18-010r7/18-010r7.html). Defaults to `4326` (EPSG code 4326) unless the client explicitly requests a different coordinate reference system. If the bounding box is not provided in the coordinate reference system (CRS) of the data cube, the bounding box is reprojected to the CRS of the spatial data cube dimensions.",
+                            "description": "Coordinate reference system of the extent, specified as as [EPSG code](http://www.epsg-registry.org/) or [WKT2 CRS string](http://docs.opengeospatial.org/is/18-010r7/18-010r7.html). Defaults to `4326` (EPSG code 4326) unless the client explicitly requests a different coordinate reference system.",
                             "anyOf": [
                                 {
                                     "title": "EPSG Code",

meta/subtype-schemas.json

@@ -232,6 +232,12 @@
                 }
             }
         },
+        "ml-model": {
+            "type": "object",
+            "subtype": "ml-model",
+            "title": "Machine Learning Model",
+            "description": "A machine learning model, accompanied with STAC metadata that implements the the STAC ml-model extension."
+        },
         "output-format": {
             "type": "string",
             "subtype": "output-format",

neq.json

@@ -38,8 +38,7 @@
                 "type": [
                     "number",
                     "null"
-                ],
-                "minimumExclusive": 0
+                ]
             },
             "default": null,
             "optional": true

or.json

@@ -1,7 +1,7 @@
 {
     "id": "or",
     "summary": "Logical OR",
-    "description": "Checks if **at least one** of the values is true. Evaluates parameter `x` before `y` and stops once the outcome is unambiguous. If a component is `null`, the result will be `null` if the outcome is ambiguous.\n\n**Truth table:**\n\n```\nx \\ y || null | false | true\n----- || ---- | ----- | ----\nnull  || null | null  | true\nfalse || null | false | true\ntrue  || true | true  | true\n```",
+    "description": "Checks if **at least one** of the values is true. Evaluates parameter `x` before `y` and stops once the outcome is unambiguous. If a component is `null`, the result will be `null` if the outcome is ambiguous.\n\n**Truth table:**\n\n```\na \\ b || null | false | true\n----- || ---- | ----- | ----\nnull  || null | null  | true\nfalse || null | false | true\ntrue  || true | true  | true\n```",
     "categories": [
         "logic"
     ],
@@ -90,4 +90,4 @@
             "result": true
         }
     }
-}
+}

proposals/filter_vector.json

@@ -1,7 +1,7 @@
 {
     "id": "filter_vector",
     "summary": "Spatial vector filter using geometries",
-    "description": "Limits the vector data cube to the specified geometries. The process works on geometries as defined in the Simple Features standard by the OGC. All geometries that were empty or become empty will be removed from the data cube. Alternatively, use ``filter_bbox()`` to filter with a bounding box or ``filter_spatial()`` to filter a raster data cube based on geometries.",
+    "description": "Limits the vector data cube to the specified geometries. The process works on geometries as defined in the Simple Features standard by the OGC. All geometries that were empty or become empty will be removed from the data cube. Alternatively, use ``filter_bbox()`` to filter by bounding box.",
     "categories": [
         "cubes",
         "filter",

proposals/load_ml_model.json

@@ -0,0 +1,46 @@
+{
+    "id": "load_ml_model",
+    "summary": "Load a ML model",
+    "description": "Loads a machine learning model from a STAC Item.\n\nSuch a model could be trained and saved as part of a previous batch job with processes such as  ``ml_fit_regr_random_forest()`` and ``save_ml_model()``.",
+    "categories": [
+        "machine learning",
+        "import"
+    ],
+    "experimental": true,
+    "parameters": [
+        {
+            "name": "uri",
+            "description": "The STAC Item to load the machine learning model from. The STAC Item must implement the `ml-model` extension.",
+            "schema": [
+                {
+                    "title": "URL",
+                    "type": "string",
+                    "format": "uri",
+                    "subtype": "uri",
+                    "pattern": "^https?://"
+                },
+                {
+                    "title": "User-uploaded File",
+                    "type": "string",
+                    "subtype": "file-path",
+                    "pattern": "^[^\r\n\\:'\"]+$"
+                }
+            ]
+        }
+    ],
+    "returns": {
+        "description": "A machine learning model to be used with machine learning processes such as ``ml_predict()``.",
+        "schema": {
+            "type": "object",
+            "subtype": "ml-model"
+        }
+    },
+    "links": [
+        {
+            "href": "https://github.com/stac-extensions/ml-model",
+            "title": "STAC ml-model extension",
+            "type": "text/html",
+            "rel": "about"
+        }
+    ]
+}