You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I'm not sure what would be required to do to get this to work with h5py but zarr and Nvidia have a nice little library kvikio which can be used to load files directly from zarr into a GPU. The major benefit of this is that you can reduce the overhead of loading data using a CPU and then transferring to a GPU.
Describe the context
I see this being used in two places:
Writing directly to zarr object from a GPU which might be much more efficient
Fast pipelines for data processing.
Additional information
In general I think that GPU processing will only continue to improve a start to outperform CPU processing. It might be good to get an idea of how difficult it is to do something like this.
@ericpre I think that is the case as well. We have one computer in our lab which I should be able to get this to work on. The documentation for kvikio leaves a bit to be desired and installing it required a couple of specific cuda etc. versions.
Next time I have a microscope session I'll try to see if I can get this to work and we might be able to just add this to the documentation!
Describe the functionality you would like to see.
I'm not sure what would be required to do to get this to work with h5py but
zarr
and Nvidia have a nice little library kvikio which can be used to load files directly from zarr into a GPU. The major benefit of this is that you can reduce the overhead of loading data using a CPU and then transferring to a GPU.Describe the context
I see this being used in two places:
Additional information
In general I think that GPU processing will only continue to improve a start to outperform CPU processing. It might be good to get an idea of how difficult it is to do something like this.
https://xarray.dev/blog/xarray-kvikio
The text was updated successfully, but these errors were encountered: