Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Array datasets are always written as chunked. #24

Closed
orenbenkiki opened this issue Mar 2, 2024 · 6 comments
Closed

Array datasets are always written as chunked. #24

orenbenkiki opened this issue Mar 2, 2024 · 6 comments

Comments

@orenbenkiki
Copy link

orenbenkiki commented Mar 2, 2024

It seem by default the Python anndata package writes contiguous datasets.

Perhaps an option to control this?

@orenbenkiki
Copy link
Author

Thanks!

@orenbenkiki
Copy link
Author

BTW, compress=9 is deprecated (according to the latest HDF5 package I'm using, anyway). The warning I get says:

┌ Warning: `compress=9` keyword option is deprecated, use `deflate=9` instead
│   caller = setproperty!(p::HDF5.DatasetCreateProperties, name::Symbol, val::UInt8) at properties.jl:59
└ @ HDF5 ~/.julia/packages/HDF5/Ws1wH/src/properties.jl:59

@ilia-kats
Copy link
Collaborator

I'm not sure where this is comping from, to be honest. There are exactly two places in the Muon code that request compression from HDF5, and both are using deflate=....

@orenbenkiki
Copy link
Author

orenbenkiki commented Mar 11, 2024

In the Muon code I see a lot of compress=..., mainly passed to write and write_metadata... In HDF5 the warning occurs whenever trying to set a compress property of an object. Perhaps because I'm using the latest release, which is old?

@ilia-kats
Copy link
Collaborator

Yes, but those are our functions that have nothing to do with HDF5.jl. Interaction with HDF5 happens here and here.

@orenbenkiki
Copy link
Author

Using the head revision, I get several WARNING: Wrapping Vararg directly in UnionAll is deprecated when compiling Muon, but no warnings when actually using it. So the compress warning was due to using the ancient released version (which is what I got when using pkg> add Muon). I've switched the project to using the head revision (which also allows me to use the newly added compress flag when writing to a file - nice). I'm also voting for #25 though :-)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants