Python API documentation

[sydduckworth]

ella's Python bindings are generated from Rust using pyo3/maturin. Unfortunately it seems that it's difficult to effectively create API documentation and type definitions for generated code.

Requirements

• API documentation is visible from IDEs and language servers like PyLance.
• Type information is available to type checkers and language servers like mypy or PyLance.
• API documentation is available to documentation renderers like sphinx, mkdocs, or pdoc.
• Type information is available to documentation renderers.
• Parameter and return types are hyperlinked correctly in documentation renderers.

Implementation

So far I haven't been able to find a single approach that satisfies all of these requirements. To meet all of them it seems like information would need to be duplicated or even triplicated.

For the actual documentation, the three options would be to put the documentation:

• In the Rust code (the best option since it also documents the Rust code)
• In .pyi type stub files
• In dedicated documentation markdown or RST files (the worst option since it would also require duplicating the module structure)

Method	pyright	sphinx	mkdocs	pdoc
Rust docstrings	❌	✅	✅	✅
Stub docstrings	✅	❌	✅	❌
Dedicated files	❌	✅	✅	✅

For type hints, the two options would be to put the type information:

• In .pyi type stub files
• In the docstrings as argument annotations

Method	pyright	sphinx	mkdocs	pdoc
stubs	✅	❌	✅*	✅
docstring args	❌	✅	✅*	❌

* the free version of mkdocstrings (the API generator tool for mkdocs) doesn't support hyperlinks/cross-references in the function signature, although it does in parameter blocks.

Conclusions

There isn't a single implementation that meets all use cases. I think the best option for now is to use pdoc as the documentation generator with Rust docstrings and type stub files. This doesn't get us docstrings in Pylance but it does get us type information at least. It may not be too difficult to write a script that copies the Rust docstrings into the stub files.

[pawamoy]

A few corrections/details: mkdocstrings does support getting docstrings from stubs. And the public version also supports cross-references (what you call hyperlinks here I assume), but in parameters sections (and the like), not directly in the rendered signature.

[sydduckworth]

@pawamoy thanks for the info! Clearly I didn't spend enough time exploring mkdocs.