Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replace the guts of gplately.download.DataServer with plate-model-manager #141

Draft
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

brmather
Copy link
Collaborator

plate-model-manager is the new way (and the best way) to download plate models, age grids, and spreading rate grids. However, a lot of legacy workflows rely on the DataServer object in gplately. Currently, there are some advantages of retaining DataServer that perhaps will be propagated upstream:

  • pygplates.RotationModel is returned instead of a list of rotation filenames.
  • pygplates.FeatureCollection is returned instead of a list of topology filenames. Same applies for static polygons, coastlines, COBs, etc.
  • gplately.Raster is returned instead of a list of age grid / spreading rate grid filenames.
  • Missing features return an empty list and a DownloadWarning.

This wrapping of plate-model-manager implements lazy loading, where files are only downloaded when requested. These rotation models, topologies, etc. are accessed like attributes (using the @property decorator).

@brmather brmather self-assigned this Nov 14, 2023
@brmather brmather added the enhancement New feature or request label Nov 14, 2023
@brmather brmather linked an issue Nov 14, 2023 that may be closed by this pull request
@brmather
Copy link
Collaborator Author

IMPORTANT: I had to run gplately.download.clear_cache() to flush the cache of previously downloaded plate models, otherwise it would complain the folder (e.g. "Muller2019") already exists in the cache.

@michaelchin michaelchin added this to the release 2.0.0 milestone Jun 25, 2024
@brmather
Copy link
Collaborator Author

brmather commented Aug 5, 2024

@michaelchin is nearly ready to merge. The only problem I have is that gplately.download.clear_cache() needs to be run on the first run. Any idea how to do that or to get gplately.download.DataServer to overwrite the old plate models?

@michaelchin
Copy link
Contributor

clear_cache

Maybe replace _pooch.utils.make_local_storage(str(cache_path)) with Path(cache_path).mkdir(parents=True, exist_ok=True)???

@brmather
Copy link
Collaborator Author

brmather commented Aug 5, 2024

clear_cache

Maybe replace _pooch.utils.make_local_storage(str(cache_path)) with Path(cache_path).mkdir(parents=True, exist_ok=True)???

This is inside the clear_cache function, but is there a way to overwrite plate model files without clearing the cache?

@brmather
Copy link
Collaborator Author

Hi @michaelchin - do you have any suggestions about the above issue? I think this could be a blocker for the next gplately release until we sort it out. Ideally it would be best if existing plate models downloaded by the old DataServer object could be overwritten...

@michaelchin
Copy link
Contributor

Hi @michaelchin - do you have any suggestions about the above issue? I think this could be a blocker for the next gplately release until we sort it out. Ideally it would be best if existing plate models downloaded by the old DataServer object could be overwritten...

Yes, if the new and old files have the same paths and names, I guess it would be possible to overwrite the old ones with the new ones. Otherwise, we may need to track the old files somehow...

@brmather
Copy link
Collaborator Author

@michaelchin - Can you take over this PR? I don't have capacity any more and I am lost on how to get the new DataServer (powered by plate-model-manager) to overwrite old cached plate reconstructions.

@brmather brmather added the help wanted Extra attention is needed label Sep 12, 2024
@michaelchin
Copy link
Contributor

@michaelchin - Can you take over this PR? I don't have capacity any more and I am lost on how to get the new DataServer (powered by plate-model-manager) to overwrite old cached plate reconstructions.

Sure, no problem.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request help wanted Extra attention is needed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Modify DataServer to use plate-model-manager
2 participants