Skip to content

Commit

Permalink
Close processes in Work Queue and Task Vine shutdown
Browse files Browse the repository at this point in the history
This releases 2 file descriptors with work queue (from 21 to 19 at the end
of CI Work Queue test) and 4 file descriptors with Task Vine (from 19 to 15
at the end of CI Task Vine test)

This is part of work being merged from draft PR #3397 to shut down components
more cleanly, rather than relying on process exit.
  • Loading branch information
benclifford committed Aug 8, 2024
1 parent 1c7a0e4 commit 1b0b933
Show file tree
Hide file tree
Showing 2 changed files with 4 additions and 0 deletions.
2 changes: 2 additions & 0 deletions parsl/executors/taskvine/executor.py
Original file line number Diff line number Diff line change
Expand Up @@ -589,11 +589,13 @@ def shutdown(self, *args, **kwargs):
# Join all processes before exiting
logger.debug("Joining on submit process")
self._submit_process.join()
self._submit_process.close()
logger.debug("Joining on collector thread")
self._collector_thread.join()
if self.worker_launch_method == 'factory':
logger.debug("Joining on factory process")
self._factory_process.join()
self._factory_process.close()

# Shutdown multiprocessing queues
self._ready_task_queue.close()
Expand Down
2 changes: 2 additions & 0 deletions parsl/executors/workqueue/executor.py
Original file line number Diff line number Diff line change
Expand Up @@ -704,6 +704,8 @@ def shutdown(self, *args, **kwargs):

logger.debug("Joining on submit process")
self.submit_process.join()
self.submit_process.close()

logger.debug("Joining on collector thread")
self.collector_thread.join()

Expand Down

0 comments on commit 1b0b933

Please sign in to comment.