[Dev Team] Upcoming Changes in 3.35.0

[robfalck]

OpenMDAO outputs a variety of files: optimizer output files, reports, derivative coloring files, data recordings, etc.

To date, these have been sort of distributed in the current working directory, and if two scripts or problems are run in the same working directory, it can be difficult to discern which runs generated the data. Starting with 3.35.0 we're going to be putting these into a single output directory, per problem. The user can set the name of this directory by specifying a name for their problem.

Dealing with directory clutter

That is, a problem named 'my_prob' will have its outputs sent to a directory under the current working directory named 'my_prob_out'. This directory will contain an empty file named .openmdao_out that will help OpenMDAO identify directories that it has created.

One benefit is that we're not going to be stomping on the same recording file names or coloring files over and over again. On the flip side, having each problem generate a unique output directory can start to clutter out your file system with these output directories.

To help with this, a new command line action openmdao clean will recurse through the directory structure and remove any directory that it finds matching our output names and containing the .openmdao_out file. This command prompts by default, and has a dry-run option to let you see what would be deleted when running it.

Impact on documentation

Some OpenMDAO-dependent packages have copied our documentation structure using jupyter-book. As part of this, we had been copying all output files to the build output directory so that plots and figures could be included. Doing this with the new file output structure will significantly balloon the size of documentation. We're currently addressing this on our dymos documentation and will do so soon for OpenMDAO, but in short we found that doing this copying of build artifacts is unnecessary for newer versions of Jupyterbook.

I wanted to put this out there because this is one of the more signficant user-facing changes that we've introduced in recent years, and while it will make dealing with OpenMDAO outputs easier overall, there are going to be some changes to get used to.