bash runner reports that "jobs were successfully submitted" but does not submit anything

[bhaddow]

For the bash runner, it doesn't really make sense to talk about submitting jobs. Also jobs running detached are harder to track. What should be the behaviour? I would be happy for the bash runner to block whilst jobs are executed.

[varisd]

The idea is that with bash runner, we don't have any scheduler process (if we had, there would be a risk of it being killed by the HPC job timeout, for example). For this reason, each submitted job process (running as detached) is in charge of tracking whether their dependencies are still being executed or finished and then running its own code. OpusPocus only takes care of submitting (i.e running) the job processes.

We can rework bash runner to be similar to the DebugRunner, specifically, the job processes will be submitted (in parallel, if feasible) whenever possible and then OpusPocus will be waiting until the processes finished and continue submitting processes which have their dependencies satisfied (successfully finished executing).

[bhaddow]

"Rework" sounds like it's a significant change. Maybe the primary issue is one of documentation/communication. At the moment, when I launch bashrunner, it is not clear what has happened, where my processes are. There's just a mysterious message about jobs being "submitted" and then it exits.

Maybe if the message says "jobs have been launched in the background". Using ./go.py status to check their progress." it would be more useful?

[varisd]

Maybe if the message says "jobs have been launched in the background". Using ./go.py status to check their progress." it would be more useful?
I think we can add at least report of under which jobid (process id) the job was submitted. I will check whether we can add the job/process info to the "status" subcommand that is, if the step is SUBMITTED/RUNNING it will also list pid/jid info.