Migration to sphinx-gallery #8247
Conversation
NelleV
added the
Documentation
label
Mar 9, 2017
NelleV
referenced
this pull request
Mar 11, 2017
Merged
DOC moved spines examples sphinx-gallery #8268
NelleV
added the
MEP12
label
Mar 11, 2017
NelleV
changed the title from
[WIP] Migration to sphinx-gallery to [MRG] Migration to sphinx-gallery
Mar 23, 2017
tacaswell
changed the title from
[MRG] Migration to sphinx-gallery to [MRG+1] Migration to sphinx-gallery
Mar 23, 2017
tacaswell
approved these changes
Mar 23, 2017
View changes
I still have reservations about needing to re-name all of the examples, but the benefits out weigh the cost.
| @@ -12,3 +12,4 @@ ipython | ||
| mock | ||
| colorspacious | ||
| pillow | ||
| +sphinx-gallery |
NelleV
Mar 24, 2017
Contributor
unfortunately, we can't as this file is meant to be used by pip, and graphviz is not pip installable.
QuLogic
added this to the
2.1 (next point release)
milestone
Mar 24, 2017
| @@ -30,7 +30,7 @@ if [[ $BUILD_DOCS == false ]]; then | ||
| fi | ||
| else | ||
| cd doc | ||
| - python make.py html -n 2 | ||
| + python make.py html |
NelleV
Mar 25, 2017
Contributor
No, but ironically, the parallel build doesn't change the run time :D
We can easily activate the flags in sphinx-gallery to make it parallel-safe, but considering there is no gain at all, I haven't created the PR yet.
Actually making the code parallel requires more work, but should be doable.
tacaswell
Apr 9, 2017
Owner
I I have definitely seen good speed up on building the docs (as they are now) but have found you need to use ~2x the number of workers compared to the number of core.
| @@ -54,6 +55,12 @@ | ||
| "numpydoc to build the documentation.") | ||
| try: | ||
| + import sphinx_gallery | ||
| +except ImportError: | ||
| + raise ImportError("No module named sphinx-gallery - you need to install " |
| + import sphinx_gallery | ||
| +except ImportError: | ||
| + raise ImportError("No module named sphinx-gallery - you need to install " | ||
| + "numpydoc to build the documentation.") |
| +# Sphinx gallery configuration | ||
| +sphinx_gallery_conf = { | ||
| + # path to your examples scripts | ||
| + 'examples_dirs' : '../examples', |
NelleV
Mar 25, 2017
Contributor
Not yet. Our plot directive writes into mpl_examples, and unfortunately we have a bit more work to do before getting rid of this (including more examples to migrate). This will be my next step.
| + # path to your examples scripts | ||
| + 'examples_dirs' : '../examples', | ||
| + # path where to save gallery generated examples | ||
| + 'gallery_dirs' : 'auto_examples'} |
QuLogic
Mar 24, 2017
Member
Can we not put it in gallery instead? Or if it were mpl_examples, then some of the lines that were changed would not have been necessary.
Whatever it becomes, I don't like the auto_ prefix; it's an internal concern and doesn't really make any sense to present that in the URL to a user from a semantic point of view.
NelleV
Mar 25, 2017
Contributor
It's the default and is a convention followed by many (if not all) the projects. I'd rather stick with it for consistency.
We can't rename this to mpl_examples as mpl_examples still exists.
QuLogic
Mar 26, 2017
Member
I'm not sure those are compelling reasons. I went back through the blame 7 years and couldn't find any reasoning behind it (that's not to say that there wasn't one.) My reasoning here is that URLs should be discoverable, simple, and at least try to give some semantic meaning.
A user might go to matplotlib.org/gallery or matplotlib.org/examples directly (yes, there's Google, but let's assume they're not using it for some reason), but there's a 99% chance that no-one will spontaneously go to matplotlib.org/auto_examples.
And what does the "auto_" prefix mean? It means that the rst was automatically generated from the example Python code. This is a meaningless distinction to a user. When they see "auto", they see "automatically generated" aka "not hand-written" aka "not very good". I'm not going to claim that all of our examples are 100% clear, but they've all been hand-written in some fashion, and some of them are even quite amazing.
In fact, for that same reasoning, I've just talked myself into retracting the "mpl_examples" suggestion, since the "mpl_" prefix is redundant and meaningless in this context. Also, our existing site uses matplotlib.org/examples/ and matplotlib.org/gallery.html; I don't think we should break that.
NelleV
Mar 26, 2017
Contributor
For the time being, the folder cannot be neither "examples" nor "mpl_examples", and this will not change in the foreseeable future considering the state of our examples (unless we remove all examples that are not SG compatible).
dopplershift
Mar 27, 2017
Contributor
It's not possible to rename the current examples/mpl_examples to make room for the gallery-based examples? I strongly dislike the auto_examples name as well--I've overridden that name for sphinx-gallery in my own projects.
NelleV
Mar 28, 2017
Contributor
It is possible, but (1) so far there has been no compelling reason to do it; (2) the amount of work required to change this is sufficiently large that I am not doing it.
To extend on point 1, our naming patterns for example plots and links is obscure, inconsistent and so poorly descriptive of what is in the examples that I really don't think the name of the folder is ever going to help or prevent our users from finding anything (we have about 5 examples named "demo bar_X", with X an incremental number). We also need to be realistic: no one is ever going to guess our url or even attempt to guess our url, and possibly not even look at the url.
Second, it is a convention that is followed by most projects and following conventions have many benefits for easing contribution, maintenance, etc.
Third, taking personal preferences into account is not for me a compelling reason (I personally prefer auto_examples over both examples and mpl_examples, as it makes it to a certain extent more clear that this folder is auto-generated and thus should not be modified by our contributors, and this is typically the kind of questions where if you have N developers in the room, you'll have N opinions).
tacaswell
Apr 9, 2017
Owner
Will this end up exposed in the URL?
If so, I think we should pick something either a) shorter (ex sg or ae) or b) something more meaningful.
choldgraf
Apr 9, 2017
Contributor
I does show up in the URL, for example this is our gallery at MNE-python. It does have auto-examples in the page, though that's sorta become the convention with sphinx-gallery (same goes for scikit-learn: http://scikit-learn.org/stable/auto_examples/index.html).
I seem to remember conversations with folks that this was a non-trivial thing to change but I could be wrong...it's been working out well enough for us. If you want you could use an .rst file that contains the gallery itself, then the objects that the gallery links to would be in the auto_examples folder.
dopplershift
Apr 9, 2017
Contributor
Nope, it's completely trivial to change: https://unidata.github.io/MetPy/examples/index.html
I'm going to argue one more time NOT to use auto_examples. Whether they're generated automatically or not is immaterial to the user, so we should NOT be bleeding that implementation detail of our documentation build procedure to the outside world.
NelleV
Apr 10, 2017
Contributor
It's not that trivial, because matplotlib somehow does I don't know what in the plot directive that makes this a pain in the butt to change. Which is why I asked for feedback a month ago.
choldgraf
Apr 10, 2017
Contributor
FWIW I'll give this a shot to change the name to examples_gallery. It doesn't look completely trivial in the sense that there are links hard-coded all over the codebase. There are also some weird errors happening inside the plot directive that I can try to debug. I'll open a separate PR then if I can figure it out.
| + # path where to save gallery generated examples | ||
| + 'gallery_dirs' : 'auto_examples'} | ||
| + | ||
| +plot_gallery = True |
|
Is there a way to make sphinx-gallery less verbose? It prints out that it's going to run an example, then that it ran the example and how long it took plus a blank line, then at the end, it prints out how long it took to run all the examples again. That's 4 lines for each example. |
|
I don't think there is a way to make sphinx-gallery less verbose. It also doesn't bother me: it's actually great for debugging. |
This was referenced Mar 25, 2017
NelleV
added
some commits
Mar 8, 2017
|
Looking forward to seeing this get merged into the website! Once it's merged I can spend a little time trying to create more narrative-driven examples out of the ones that are there and generally improving them. Is there anything holding this back right now? |
| @@ -75,7 +75,8 @@ | ||
| <p>Matplotlib tries to make easy things easy and hard things possible. | ||
| You can generate plots, histograms, power spectra, bar charts, | ||
| errorcharts, scatterplots, etc., with just a few lines of code. | ||
| - For a sampling, see the <a href="{{ pathto('users/screenshots') }}">screenshots</a>, <a href="{{ pathto('gallery') }}">thumbnail</a> gallery, and | ||
| + For a sampling, see the <a href="{{ pathto('users/screenshots') | ||
| + }}">screenshots</a>, <a href="{{ pathto('auto_examples/index') }}">thumbnail</a> gallery, and |
NelleV
closed this
Apr 10, 2017
|
Chris is taking over this work. I don't have time and the energy to continue working on matplotlib's migration to sphinx-gallery. |
NelleV commentedMar 9, 2017
•
edited
This work is still in progress.
Right now, the documentation builds two gallery. The old one, which stays identical as on current master and a sphinx-gallery one. The sphinx-gallery one needs specific examples formatting, hence only includes so far a subset of our example.
I'll slowly update this PR with progress and build this on a local website to show how it looks. Once we decide it is "good enough", the switch can easily be done through a URL switch.
Here is the sphinx-gallery build