DOC: Split api.rst in sections

[datapythonista]

The pandas API reference page is huge. Which makes it difficult for users to find information, and for us to edit.

Given the size of the page, I think it'd make more sense to split the current page api.rst into different pages, one per section:

• api/series.rst
• api/functions.rst
• ...

The document is already divided into sections, I think the current division would make sense with few changes:

• Input/Output
• General functions
• Series
• DataFrame
• Panel
• Index (will include all the Index classes/section in the top level of api.rst)
• Scalars
• Date Offsets and Frequencies
• Window
• GroupBy
• Resampling
• Style
• Plotting
• General utility functions
• Extensions

Does anybody think a single page is better? Any other division you think of?

[benjaminr]

I agree, think this would definitely more approachable split up into pages; it's far too monolithic as it stands. Makes sense to follow the sections already defined.

[mroeschke]

Makes sense to me. One small nit is that I would combine Date Offsets with Frequencies since they are essentially the same thing and Frequencies only has the to_offset function under that section.

[jorisvandenbossche]

@datapythonista I agree with your rationale ... but, this completely breaks all existing links to the api docs.

There might be some easy fixes (adding a (temporary) redirect, or keeping the generated pages in /generated/.., or ...). But at least we need to do something about it for 0.24.0 I think.

This will be in general a problem when reorganizing the docs, so it would be good to find some remedies for this.

[jorisvandenbossche]

So there are two issues:

• The main page /api.html no longer exists, as it is now /api/index.html (on stable docs: http://pandas.pydata.org/pandas-docs/stable/api.html, check http://pandas-docs.github.io/pandas-docs-travis/api.html vs http://pandas-docs.github.io/pandas-docs-travis/api/index.html)
  • I am wondering if there is no way for sphinx to keep api.html working if /api/ is a directory instead of a file?
  • Otherwise a redirect from api.html to api/index.html would not be hard to add
• The generated pages changed url from /generated/method_name.html to /api/generated/method_name.html (eg http://pandas-docs.github.io/pandas-docs-travis/generated/pandas.read_csv.html -> http://pandas-docs.github.io/pandas-docs-travis/api/generated/pandas.read_csv.html)
  • Is it needed to make this change? @datapythonista was there a technical reason that it needs to be in /api/generated, and not in /generated. In the PR you mentioned some problems with autosummary related to this?

The pandas API reference page is huge. Which makes it difficult for users to find information

This depends a bit on how the page is used. I personally use it for finding (the docstring page of) a certain function/method typically with ctrl-f. Having it split over many pages makes this actually harder .. (although I am probably an atypical user of the page).

[datapythonista]

the path of the generated files is decided by sphinx based on where the autosummary directives are. Before they were in the root, in api.rst, so they were generated in the generated/ directory in the root, and not they are generated in api, as the autosummary directives are in files there. We could have :toctree: ../generated/ I guess, but I don't think we want that.

Regarding the redirects it was briefly discussed in #23708 (comment)

And as said there, in my opinion, redirecting everything (also consider what I'm proposing in #24499) is not worth the effort and complexity. It'd be surely nice to have a better 404 that let the users continue the navigation and find the new page, but I wouldn't build a system to add redirects to possibly every single page of the docs.

I see the convinience of a ctrl+f in a single page, but I'd leave that for the pdf version of the docs, and move to the opposite direction in the html, and not have to ctrl+f anything because everything is easy to navigate.

[jorisvandenbossche]

We could have :toctree: ../generated/ I guess, but I don't think we want that.

Why wouldn't we want that?

And as said there, in my opinion, redirecting everything (also consider what I'm proposing in #24499) is not worth the effort and complexity

I certainly agree that at some point when properly reorganising the full doc structure, it is going to be impossible (and not worth the effort) to keep urls working.
But, here (speaking about the generated pages for a moment) it is about 1) some of the most visited pages, and 2) a case where it is relatively easy to do redirects or to avoid the need for redirects (by keeping the original url).

[datapythonista]

I'm waiting to see if anyone else wants to give feedback in #24499 before working on it. But I don't see the reorganisation of the full docs happening at "some point", but in the near future. :)

I think not much later after that is merged, it would be a good time to change the sphinx style, and we could switch to pandas.io (and dev.pandas.io) at the same time.

While I totally understand your point, my opinion is that it's better to forget about redirects (including deleting the redirects we already have). Search engines will take some time until they have everything indexed again. But I'd just make sure that any url under pandas.pydata.org explains that we moved and link the new page, and IMO that should be enough for the time search engines take to be updated.

An advantage I see besides the simplicity, is that if we redirect users to the new home pages, they'll see the value in the new navigation and get used to it (instead of using search engines).

[jorisvandenbossche]

It's not only about search engines, but also all the links to docstring pages in blogs, StackOverflow answers, ...

When we would move to pandas.io, I assume we will also have redirects from pandas.pydata.org/pandas-docs/ over there no?

An advantage I see besides the simplicity, is that if we redirect users to the new home pages, they'll see the value in the new navigation and get used to it (instead of using search engines).

Certainly true for all the user guide docs. But even with a fantastic navigation, I will probably still keep using google to go to a specific docstring page :)

Anyway, I still feel it is not needed to break all the API pages (if we would change the url, I think I would actually rather go with /reference or simply api/ instead of /generated or /api/generated. I don't think generated adds much to the long url)