Add type annotation stubs

[malmaud]

Hi,
I was wondering if you would be interested in PRs that add type annotation stubs for the plotly.py API. That would enable static type checkers like mypy and Pyre to statically type check Python projects using plotly, and as well as enhancing the IDE experience for editors able to offer richer experiences from type annotations (just as VSCode and PyCharM).

[jonmmease]

Hi @malmaud, I would love to do this... but we still support Python 2.7.

That said, in all of our new generated code we use numpydoc docstrings with return types (see, for example, plotly/graph_objs/_bar.py). This is enough for PyCharm to figure things out (I haven't really tested other editors very much) and it does provide a nice experience when working with objects in the graph_objs hierarchy.

I would be interested in gaining a better understanding of the most broadly compatible way to specify type information for a mixed Python 2/3 project!

[malmaud]

Great to hear that. The stubs can live in separate files that sit parallel to the original Python source files, which preserves Python 2 compatibility. I recently went through this with the Visdom project. See the `__init__.pyi` file in https://github.com/facebookresearch/visdom/tree/17f28cb258ad874d6fc5fcce5a4479abe4377738/py/visdom, which lives alongside the `__init__.py` that it annotates. This is the same system as Typescript's `.d.ts` files, if you're familiar.

…

On Thu, Aug 9, 2018 at 11:47 AM Jon Mease ***@***.***> wrote: Hi @malmaud <https://github.com/malmaud>, I would love to do this... but we still support Python 2.7. That said, in all of our new generated code we use numpydoc docstrings with return types (see, for example, plotly/graph_objs/_bar.py). This is enough for PyCharm to figure things out (I haven't really tested other editors very much) and it does provide a nice experience when working with objects in the graph_objs hierarchy. I would be interested in gaining a better understanding of the most broadly compatible way to specify type information for a mixed Python 2/3 project! — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1103 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AA8SvX25sAyyerFo8sTPtvFuVv0xX2JQks5uPFmAgaJpZM4V169w> .

[jonmmease]

That looks really interesting. I don't see any downside to adding these to at least the code generation output, because that would automatically stay up to date (and this covers the majority of the public API).

What has your experience been like keeping these up to date as a codebase evolves (for the human generated code 🙂)? What kind of tooling/testing would you recommend to that help flag inconsistencies?

[malmaud]

Well, I won't exaggerate - I became excited about static type checking for Python and started using it for my own research code, but I don't have have direct experience applying it to large multi-developer codebases. I'd look forward to trying and learning, however.

In terms of help from tooling, the only thing I know of is https://github.com/Instagram/MonkeyType. It attempts to produce stubs automatically based on the types it observes variables take during runtime, but I haven't tried it.

The good news is the worse-case scenario for an incorrect stub is a user gets an incorrect warning (or lack of warning) from their type checker and reports it as an issue. Actual runtime behavior can never be affected by incorrect stubs.

[jonmmease]

If you're interested in digging in, I'd like to start by adding stub generation to the graph_obj code generation logic. The current logic is in codegen/datatypes.py. As I said, this is the majority of the public API, and we can keep it correct for free.

Once we have this, I'd like to do a bit of an editor survey to get a full list of the Python editors that support completion across the full nested graph_objs hierarchy because of the addition of the stubs (as I mentioned, PyCharm does fine with just the types in the numpydoc strings).

Then I think we'd have enough information to make an informed decision about how much benefit users would get from the slight increase in maintenance burden.

How does that sound?

[malmaud]

Sounds great.

I should mention that there's a formalized generalization of PyCharm's docstring-aware approach whereby type annotations can be encoded into strings and put as comments inline with a source file, instead of as a separate stub (https://www.python.org/dev/peps/pep-0484/#suggested-syntax-for-python-2-7-and-straddling-code). I personally feel this is less elegant than separate stubs that get to use the nice new Python 3 syntax for type annotations and I'm probably less motivated personally to contribute annotations in that format, but it's a possibility.

[malmaud]

I'm going to link to pytorch/pytorch#12500 as an example of a big project that went through the process of adding stubs and is discussing testing strategies etc - might be a useful reference for me in the future or whoever else wants to take a stab at this.

[ldorigo]

Hey, I'm wondering if there is any news on this? With powerful static typecheckers such as Pylance (the new language server in VSCode) becoming widely available and adopted, it would be very nice to have decent typing information available for plotly objects.

[nicolaskruchten]

There's no news at this time, but if someone wants to pick up the torch and work on a PR or proposal for how to get this done, we'd happily consider it and help you out :)

[jonmmease]

Also, depending on how large they are (we have a lot or graph object classes), we may need to ship them in a separate package.

[harahu]

Since python 2 has reached EOL now, I guess maintaining support for it should not be part of the consideration any more?

[JP-Ellis]

As pointed out by @technic in (this comment)[https://github.com//issues/1682#issuecomment-922361704] from #1682, it looks like a simple workaround is to make use to the TYPE_CHECKING which is specifically designed to handle lazy imports.