New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Issues with Bionet deletion and EMR doesn't terminate #1104

Open

patkyn opened this issue Sep 3, 2024 · 0 comments

Contributor

patkyn commented Sep 3, 2024

2 issues:

A need to implement a 'Timedout' in the request if the server is not responding with the response.
Currently, the request request.get seems to be stucked forever causing the cluster to hang.
https://data.bionet.nsw.gov.au/biosvcapp/odata/SpeciesSightings_CoreData?%24filter=dcterms_modified+ge+2024-08-12+and+dcterms_modified+lt+2024-08-19&%24top=50000&%24count=true&%24skip=0

Refer to EMR cluster: https://ap-southeast-2.console.aws.amazon.com/emr/home?region=ap-southeast-2#/clusterDetails/j-QK821F6BZCHM

A bug in the deletion, possibly from the latest release of dwcahandler v0.2.0

https://ap-southeast-2.console.aws.amazon.com/emr/home?region=ap-southeast-2#/clusterDetails/j-2CCVVO9JJKP6L

patkyn changed the title ~~Bionet fetcher~~ Issues with Bionet deletion and EMR doesn't terminate

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment