Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GDAL Example #328

Open
anton-seaice opened this issue Apr 12, 2024 · 5 comments
Open

GDAL Example #328

anton-seaice opened this issue Apr 12, 2024 · 5 comments

Comments

@anton-seaice
Copy link
Collaborator

@mdsumner suggest this morning maybe we could have an example to demonstrate using ACCESS-OM output with geo-spatial analysis tools (i.e GDAL).

The goals would be:

  • to encourage us to get the coordinate reference system (CRS) and metadata correct in ACCESS models, so that GDAL can auto-detect this, and
  • to demonstrate functionality of GDAL to assist in comparing remote sensing and modeled output.

One possibility would be to compare Sea-Ice Concentration modeled vs observations (similar to https://cosima-recipes.readthedocs.io/en/latest/DocumentedExamples/SeaIce_Obs_Model_Compare.html#Sea-Ice-Concentration-Anomalies). But maybe its a good excuse to look at a different observational dataset. This could be satellite altimetry for sea ice freeboard, although maybe there are other examples (sea surface height?, chlorophyll?, a non-gridded product?, buoy/animal tracks? ).

We would need to investigate the best package to use for this ( GDAL wrappers like rasterio or rioxarray or the gdal python binding seem like the likely candidates). We think rasterio might not support 2D geo-location arrays (as we have for the tripolar grid) but GDAL should. (see discussion)

In ACCESS-OM2 output, its likely some metadata would need to be added manually, but hopefully the notebook would encourage us to get the metadata up to date for OM3.

@anton-seaice
Copy link
Collaborator Author

This example might overlap similar ground to this PROJ example by Aidan

@mdsumner
Copy link

mdsumner commented Apr 15, 2024

I will work through that example as a starting point. I'm still navigating my way on gadi but making progress. I see the temp copy of BedMachine is not in

/g/data/hh5/tmp/aph502/BedMachineAntarctica_2020-07-15_v02.nc

so I'm downloading my own copy (using earthdata creds). But, maybe it's on the thredds? At any rate we can hit it up directly by putting the earthdata creds in GDAL_HTTP_HEADER_FILE, or in GDAL_HTTP_HEADERS - but better to use a shared local copy first so we don't load too much up in the draft example. (I'm not familiar with network access on gadi, I assume we can't just blithely hit up URLs as I've become accustomed to).

@mdsumner
Copy link

mdsumner commented Apr 15, 2024

I got it going enough to the point I could warp cosima output to BedMachine (I realize what I said above was not viable, the other way is not possible, you can't warp to an un-regular grid - the workflow as-is makes sense).

I need some help finding NetCDFs that are relevant, but quick test shows that blithely warping to the Bedmachine grid is fine, and similarly we could use the NSIDC sea ice products in the same way. (I'm interested to flesh out python and R pathways that achieve this, though I've only done cli for now).

@anton-seaice
Copy link
Collaborator Author

I will work through that example as a starting point. I'm still navigating my way on gadi but making progress. I see the temp copy of BedMachine is not in

/g/data/hh5/tmp/aph502/BedMachineAntarctica_2020-07-15_v02.nc

so I'm downloading my own copy (using earthdata creds). But, maybe it's on the thredds? At any rate we can hit it up directly by putting the earthdata creds in GDAL_HTTP_HEADER_FILE, or in GDAL_HTTP_HEADERS - but better to use a shared local copy first so we don't load too much up in the draft example. (I'm not familiar with network access on gadi, I assume we can't just blithely hit up URLs as I've become accustomed to).

urls through the login node or ARE are ok, not through jobs in the the queue though. If they require credentials its probably better to add it to ik11 either way.

@mdsumner
Copy link

so I said this up there

way is not possible, you can't warp to an un-regular grid -

it totally is possible, I had no idea. First example here just uses URLs, for the second I downloaded the BedMachine.

https://gist.github.com/mdsumner/380fcc579e7c1736de189fa4e4fa67e9

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants