Changes to how we pick which Python to use.

[t-kalinowski]

This PR adjusts the logic that reticulate uses to determine which python installation to load. The changes are mostly related to the choices reticulate makes in the absence of any concrete preference expressed by the user via use_python(), RETICULATE_PYTHON, or similar.

Previously, reticulate would prompt the user to install miniconda, bootstrap a "r-reticulate" conda environment, or go searching through all known venvs and conda envs for the presence of a requested module.

New behavior highlights:

• No longer prompt to install miniconda. Instead, at a similar point in the order of discovery, we prompt the user to instead create a default "r-reticulate" venv.

• We give higher precedence now to explicitly provided hints of the form:

  • import("bar", delay_load = list(environment = "r-barlyr")
  • use_python("/path/to/bin/python", required = FALSE)

• We give higher precedence to module derived env names when we're initializing because of a call to import that module. In other words, in the absence of more a more concrete expression of preference (RETICULATE_PYTHON env var, use_python(required = TRUE), etc) if the user calls import("scipy"), and there exists an environment "r-scipy", we use it.

• If there is no suitable python installation to create a venv from, we ask the user to install python manually, suggesting options:

  • on all platforms, reticulate::install_python(),
  • on macOS and Windows, www.python.org/downloads/,
  • on Linux, https://github.com/rstudio/python-builds.
  • There is also a new environment variable RETICULATE_VIRTUALENV_STARTER and option("reticulate.virtualenv.starter") which users can use for pointing reticulate at suitable venv starters, encoded in the function virtualenv_starter(), though this should be not needed by most users.

• We no longer scan all known pythons. Absent any expression of preference for a particular python from the user, we fall back to a r-reticulate venv, or if user explicitly opts out of that, to the python on the PATH.

• The previous approach went to great lengths to succeed in finding a Python that satisfied a particular import() call, and often did. However, the search behavior made predicting which python installation would be found very difficult, and subject to change as the environments on the system changed. Now we no longer use the presence of an installed module as criteria for selecting a particular python installation.

This is the new order of precedence for selecting a python:

1. RETICULATE_PYTHON env var (path to python binary)

2. RETICULATE_PYTHON_ENV env var (path to venv or condaenv directory, or bare name of venv or conda env) (previously this only accepted a full path)

3. use_python(required = TRUE) call (and related, use_virtualenv(), use_condaenv())

4. Presence of a pyproject.toml file from a poetry environment in the current working directory

5. Presence of a pipfile associated with a pipenv in the current working directory

6. If there was a call import("bar", delay_load = list(environment = "r-barlyr"), and there exists an environment named "r-barlyr", use it.

7. use_python(required = FALSE)

8. If there was a call import("bar"), and there exists an environment named r-bar , use that.

9. If we're running under an activated virtualenv (i.e., the python on the PATH is a virtualenv, use that).

10. env var RETICULATE_PYTHON_FALLBACK

11. In the absence of any expression of preference from the user, (via one of the ways outlined above) we fallback to using a r-reticulate virtual environment. If one does not exist, we prompt the user to create one.

12. If the user declines to use the "r-reticulate" environment, then we fall back to using the python on the PATH.

[kevinushey]

No longer prompt to install miniconda. Instead, at a similar point in the search, we prompt the user to instead create a default "r-reticulate" venv.

I very much approve!