[v2 / v3 compat] add Group.array and data kwarg to array creation

[d-v-b]

• add a data keyword argument for array creation, matching the behavior from v2. data is npt.ArrayLike | None, if None (the default) an array is initialized as normal (i.e., only metadata is written). Otherwise, after creating array metadata, we will write the contents of data to the array.
• add an array method to Group, matching behavior from v2. this is basically an alias for Group.create_array, and has a deprecation warning
• adds a bunch of tests to the group tests.
• fixes a bug in MemoryStore.list_dir that was revealed by the aforementioned tests
• Adds contains_array and contains_group functions that report on whether a StorePath contains an array or group, respectively.
• Adds errors.py, a top-level module that contains ContainsGroupError and ContainsArrayError , which are much like the identically named classes from v2, but these ones also report the path to the container root in the error message which is much better than the v2 behavior of only reporting the name of the array / group.
• Allows users to leave the chunks unspecified, in which case a chunk size will be guessed using logic re-used from zarr-python 2.x

fixes #2028

cc @tomwhite

TODO:

• Add unit tests and/or doctests in docstrings
• Add docstrings and API docs for any new/modified user-facing classes and functions
• New/modified features documented in docs/tutorial.rst
• Changes documented in docs/release.rst
• GitHub Actions have all passed
• Test coverage is 100% (Codecov passes)

[d-v-b]

another change worth noting: in main we have some very thin array creation routines that just pass **kwargs down to another function. I filled out a complete function signature and docstring for these.

[d-v-b]

docs are failing but I don't think that's on me.

[dstansby]

I think the doc fails are on you. They are:

docstring of zarr.AsyncGroup.create_array:22: WARNING: Line block ends without a blank line.
docstring of zarr.Group.array:22: WARNING: Line block ends without a blank line.
docstring of zarr.Group.create_array:22: WARNING: Line block ends without a blank line.

At a guess, it might be because you've used | to separate types in the docstring, whereas it should be a , used to separate types.

[d-v-b]

Are warnings failures?

[dstansby]

Yes

[dstansby]

(because warnings are caused by things that mess up formatting in the docs - sphinx can still generate docs if warnings are emitted, but they might not be formatted correctly, hence it is sensible to error the doc build on warnings)

[d-v-b]

the doc failures were due to breaking a long type annotation into multiple lines. it should be fixed now.

On the subect of docs, I don't think the sphinx error message was very clear, and I also don't think sphinx is doing a good job. See the below example. The function signature is very hard to read.

For a different project I'm using mkdocs-material, and I think the way it renders docstrings is much nicer.

Is there some way we can get sphinx to display API docs like this?

[normanrz]

We probably shouldn't import from zarr.v2 in v3 code.

[d-v-b]

generally I agree, but the explicit goal here is v2 compatibility. we could re-implement the exact same code in the v3 codebase, but I think that has greater drawbacks.

[normanrz]

Didn't we decide to remove the zarr.v2 modules before releasing 3.0?

[d-v-b]

on the other hand, it would be good to get this routine in v3 so that we can make improvements to it. So I'm +1 on removing the v2 import and porting the logic. I will make that change.

[d-v-b]

Didn't we decide to remove the zarr.v2 modules before releasing 3.0?

yes, I think we did. that's another good reason to port this routine.

[d-v-b]

5abaab6 brings guess_chunks over to v3, and adds tests.

[normanrz]

Looks good. See my one comment on importing from zarr.v2.

[normanrz]

Excellent!

[d-v-b]

this is not quite ready -- I realized that we are checking for pre-existing arrays, but not for pre-existing groups. fix + tests incoming.

[d-v-b]

in main, when creating an array or group with exists_ok = False, we check for metadata documents and raise an AssertionError if the metadata documents are there. Notably, for v2, when creating an array, we only check for .zarray, which means that an existing .zgroup would be ignored, potentially resulting in an invalid hierarchy!

In this PR, I added checks for groups or arrays (or both, in the case of v2), and created specific exceptions for both. I tried to implement the array / group checks in a way that's somewhat respectful of IO budget, which means that the functions for checking for groups / arrays returns Literal["group", "array", "nothing"] instead of a simple boolean. Until python adds a trinary boolean type I think this will suffice, but I'm happy to revise it.