Why are Pyquil examples stored as .ipynb instead of .md files?

[jordandsullivan]

Nearly all our examples are stored in Github as .md (Markdown) files, but I noticed that these two Pyquil examples are stored as .ipynb (Python notebooks):
https://github.com/unitaryfund/mitiq/blob/main/docs/source/examples/pyquil_demo.ipynb
https://github.com/unitaryfund/mitiq/blob/main/docs/source/examples/vqe-pyquil-demo.ipynb

I saw that pyquil notebooks are excluded from the docs build in this conf file:

mitiq/docs/source/conf.py

Line 133 in 37a202b

nb_execution_excludepatterns = ["*pyquil*.ipynb"]

.

[natestemen]

This is a really good question that's not very obvious from looking at our documentation build systems/processes. I'll try my best to give an overview.

Our documentation is written in a flavor of markdown called MyST which is purpose-built for scientific and computational documents. In particular, it is a great format for pages which generate content (i.e. plots, figures, tables, etc) but that content should not be checked in to version control. This allows us to keep notebooks in git while not having to deal with the messiness that comes with diff-ing ipynb files.¹

So the question remains: why are these two pyquil demos in ipynb format? In order to run these notebooks, you need to have two docker containers spun up in order to run them. Getting them spun up is partially documented at the bottom of the running tests section of our contributing docs.² Getting the docker containers set up is generally a more complicated process which we don't expect everyone to know how to do, or need to do, and hence these tutorials have been left as pre-executed notebooks which do not need to be re-executed for local builds.

That said, you may have noticed that when you run make docs or make docs-clean there will still be errors from these two notebooks. That's because sphinx tries to build everything unless you tell it not to. You correctly found that there is an exclusion that should match these two notebooks, but it's hidden behind a conditional that checks for the existence of an environment variable.

mitiq/docs/source/conf.py

Lines 131 to 133 in 37a202b

	if os.environ.get("SKIP_PYQUIL"):
	print("Skipping PyQuil notebooks execution since SKIP_PYQUIL is True")
	nb_execution_excludepatterns = ["*pyquil*.ipynb"]

As far as I can tell, setting the environment variable is not currently documented.³

The "proper" documentation build workflow is laid out in code in the github actions which is run on the main branch, and on PRs:

mitiq/.github/workflows/build.yml

Lines 52 to 67 in 37a202b

	- name: Run the quilc & qvm Docker images
	run: |
	docker run --rm -idt -p 5000:5000 rigetti/qvm -S
	docker run --rm -idt -p 5555:5555 rigetti/quilc -R
	- name: Fetch Jupyter cache
	uses: actions/cache@v4
	with:
	path: ./docs/build/.jupyter_cache
	key: jupyter-cache-${{ github.run_id }}
	- name: Build and test Sphinx docs
	run: |
	export BQSKIT_DOC_CHECK_OVERRIDE=1
	export PYDEVD_DISABLE_FILE_VALIDATION=1
	make docs

Hopefully this is helpful. Happy to answer any questions. The documentation build system is complicated.

Footnotes

1. There are increasingly many tools to help make this easier, but in general I like our approach. ↩

2. We should really have this documented in the CONTRIBUTING_DOCS.md file as well. ↩

3. Again, probably should be. ↩

[cosenal]

Not sure I understand. Why couldn't we leave the pyquil notebook as .md for coherence with the rest and have an exclude pattern ["*pyquil*.md "] instead of ["*pyquil*.ipynb"]?

[natestemen]

From what I understand, there is not a way using a markdown file are able to both

1. store pre-executed content and use that when not executing the notebook, and
2. overwrite the pre-executed content when you are executing the notebook.