toctree contains reference to nonexisting document

[amueller]

I'm still trying to migrate scikit-learn to numpydoc master.
I'm having some trouble with this issue:
http://stackoverflow.com/questions/12206334/sphinx-autosummary-toctree-contains-reference-to-nonexisting-document-warnings

Apparently something goes wrong with when autosummary is called?
That didn't use to be the case.
I can't really do numpydoc_show_class_members = False because then there is no method table any more :(

[amueller]

so setting class_members_toctree = False solves my problem. But there still seems to be a bug...

[jorisvandenbossche]

For pandas we used the same 'hack' as in the numpy docs: https://raw.githubusercontent.com/numpy/numpy/master/doc/source/_templates/autosummary/class.rst, but I am not sure anymore if this was to solve those error messages (long time ago I did this for the pandas docs)

[amueller]

huh, we didn't need this hack, the numpydoc_show_class_members = True combine with class_members_toctree = False seems to produce the right results. It just seems kinda odd if there is only one specific configuration that is needed to make it actually work... well better than a hack.

[stefanv]

That is weird; if you figure out what's going on, let's put a note in the README at least.

[amueller]

I'm kinda punting on it for now. This configuration Works For Me™.

[jnothman]

So is this suggesting that the .. autosummary in numpydoc-mangled docstrings simply isn't working for autogeneration, hence the need for the above hacks?

This is very believable to me: I can't see anywhere in autosummary where it sees the mangled docs. It's either processing through the autosummary_generate option, at a builder-inited hook, which operates over paths, or autogen is run manually again on paths.

Now we no longer use .. autosummary for display of attributes, only methods, so we should rid of the hack use of autosummary from attribute lists, then. I'm not sure if class_members_toctree is having useful effect for method listing? (Or is it possible someone's been using numpydoc directly as a pre-process rather than as an autodoc-process-docstring hook?)

[jnothman]

So I think we basically need to recommend a hack at the template level, and that class_members_toctree should then match what the hack does :\

[mattip]

I am willing to work on this, but as I am new to the codebase would need some help parsing @jnothman 's comment above. Is there a general design document that describes what autosummary does and when? It seems there are three options for generating summaries:

• the autosummary_generate option (I guess this is what numpy does since we have autosummary_generate = glob.glob("reference/*.rst") in conf.py)
• a builder-initiated hook which operates over paths
• autogen is run manually again on paths

What do the second and third one mean? Should one use be preferred over the others?

Then there is the sentence below, what is meant by "the hack use" and what should replace it?

Now we no longer use .. autosummary for display of attributes, only methods, so we should rid of the hack use of autosummary from attribute lists

[mattip]

FWIW, adding these lines to the conf.py does not silence the warnings in numpy

numpydoc_show_class_members = True
class_members_toctree = False

[jnothman]

I'm not really sure what to say here, and my recall is weak. The different ways to run autosummary are not really relevant as long as they only collect names to autodoc from reST files, rather than from autodoc-generated reST. I suppose you could consider fixing that in autodoc. Otherwise I'm not sure how to go about fixing this.

[mattip]

TL;DR - Removing the .. HACK directive which wraps the .. autosummary directive for class attributes in the autosummary class template causes class attributes to be displayed and the warnings to disappear, with no changes needed to NumPy's conf.py

Here is what I have so far based on observations and trying out different alternatives. NumPy has a template in doc/source/_templates/autosummary/class.rst with nested directives. This template is filled in by the autosummary generator with class methods and attributes, and documentation of those methods and attributes is generated, one file for each method and attribute. Then an additional pass over the documentation is done to fill in TocTree contents. Class methods are properly added to the class documentation, but class attributes are not. If I remove the nested nature of the directive for the attributes, the class attributes now appear in the class summary and the warning is not generated.

So apparently at some point in time class methods and attributes were treated the same, and then something changed and class methods are now still added to the class summary but class attributes are not. See for instance how numpy.broadcast has a "Methods" section, but no "Attributes".

Beyond changing the template to properly index class attributes, is there something more that should be done in numpydocs to document or change this behavior?