Tutorial proposal: Best practices for being an ecosystem package

[adamgayoso]

There are often very non-obvious, but simple things that sometimes prevent ecosystem packages from "going mainstream". We should write a tutorial/guide on do's and don't's for building ecosystem packages.

I'm thinking here we can promote our template, but also make recommendations about requirements (don't pin versions, generally don't add unnecessary dependencies, etc etc), and some commentary on APIs.

[ivirshup]

Just seeing this now, but I'd broadly agree with this.

Related areas of developer docs

During the hackathon I had a good conversation with @emdann around our docs for package development. I think the perspective of coming from R to python was really good to get, since it pointed out many things I take for granted. A lot of the topics covered were more on the more practical side (e.g. what file do dependencies go in, how do you add docs for an argument) vs. abstract (e.g. how do you choose dependencies, how do you design an api). I've included my notes below, but the action items focussed on how the docs for working on a package could be improved.

The feedback was on the template documentation and scanpy developer guide.

I think better docs here also addresses the "never going mainstream" issue, but more on the "if the tests aren't running regularly, they're probably gonna fail" side of things.

During the conversation, I believe the bioconductor developer documentation was spoken highly of.

Some issues spawned from this discussion include:

• Document the pyproject.toml cookiecutter-scverse#133
• Documentation of test CI cookiecutter-scverse#132

My notes from the discussion

• Update how to open a PR to use gh
• More accesible titles for git stuff (e.g. "forking" - cloning)
• Mention CI under pull requests
• What tests are
• Developer docs for template
  • More information (very visible) on how to turn off optional parts (e.g. pre-commit)
• Explanation of docstrings/ sphinx
• Order of docs on the template repo, something more logical
  • Move docs above template/ release
• Probably move "setting up repo" to something attached to the template
• Guidelines for tutorials (e.g. make sure data is accesible)
  • "what should documentation look like"
• Function level API design?
• CI for template repo
  • Explain this a bit. At say which file does what
  • Link to actions docs?
  • Point to https://github.com/r-lib/actions for people with R dependencies?
• Template repo has no docs on pyproject.toml

An idea for organizing/ presenting developer docs

Thinking of how we could have a collection of "docs for developers" and how that would be organized (maybe with Diátaxis):

• Tutorials
  • Create a package and setting up a repository (currently, cookiecutter readme + parts of "setting up the template")
• How to guides
  • Making a pull request/ running tests scanpy dev guide
  • Modifying CI (CI template docs like "how to disable pre-commit checks")
• Reference
  • Documentation on what the various files in the template do (e.g. CI, pyproject.toml)
• Explanation
  • How to design an API
  • How to write useful tests
  • Considerations for dependency management

[ivirshup]

Aside: While I want to have a conversation about better developer documentation, I'm not sure the scverse tutorials site is the right place for them to be presented.

[emdann]

The proposed developer docs structure looks great to me. Potentially additions to the "Explanation" section could be

• How to write good docstrings and type hints
• Tips for writing tutorials
• Recommendations for including data in package

Happy to help on this if needed!