Discussion around tooling for better triaging/focusing on important things/dealing with the repo activity

[lesteve]

Below are some ideas of things that would make my life easier as a maintainer. Right now I think of it as some kind of web dashboard, but it could well be the case that parts of it are more automation/CI.

I guess at this point I am mostly looking for a discussion around tooling, not saying that we should build some tools right away at the Scientific Python ecosystem level, in particular because likely each project have their own slightly different way of doing things.

Things that would be useful (not provided by GitHub issue/PRs/searches as far as I know):

• diff +/-. Is this a smallish PR or a huge one?
• maintainers who were already involved and already gave a review
• who commented last and is it a maintainer or not. This would help to clarify whether it is this waiting on the contributor or on a reviewer
• quick GitHub profile summary to have an idea about the contributor. I want to know if the person has contributed to "serious" projects, in particular I don't care about the person's own repos. These days I go to the GitHub profile and then in the search and type involves:<user> to get a quick feeling, it's still a few clicks away and doesn't do any summary.
• which kind of CIs is passing failing , in a scikit-learn context, it's very different if the linting is failing or whether it's a more exotic one (say macOS without OpenMP for example)
• anything that would help us focus on "importance" rather than "recent activity". For example, make the difference between something that has been active consistently in the past month, compared to something that has no activity for 8 years and one person commented "can I work on this?" 2 days ago.
• probably more things that I forget

Different modes

• Who needs feed-back in my close collaborators: PRs from organization members
• Who needs feed-back amongst regular contributors to help them level-up: PRs from regular contributors (not necessarily part of the org)
• give feed-back to external contributors, in particular new contributors. I personally see this as some kind of community service, and right now I would personally aim to do it something like a couple of hours per week.
• quick PRs with a small diff for when I don't have much time or not much brain power

Interesting things I have bumped into

• SPARK has a nice dashboard to manage PRs. It has the kind of things I want diff +/-, who is involved, is it waiting on a reviewer or the PR author
• nice dashboard across many repos ipfs-shipyard. E.g. recent PRs by non-core contributors. You can look at active contributors (in the last week, 30 days, year ...):. You can see recent issues. Here is the repo.
• leanprover-community triage dashboard
• hugovk has a dashboard to help Python triagers
• triage-party Google webapp that you need to deploy yourself
• gh-dash seemed nice (CLI where you can register GitHub searches) but it would be a personal thing, a dashboard would be more easily usable by different maintainers. Also I never actually managed to make gh-dash stick in my workflow 😅.

[adrinjalali]

A dashboard for checking PR status stuff:

https://thomasjpfan.github.io/sklearn-pr-tracker/

[lesteve]

For @betatim, here is Thomas scikit-learn board that was used at Quansight: Quansight's scikit-learn Project Board (view)

[lesteve]

Ideas from the discussion:

• minimal dashboard PoC from GitHub search with additional info e.g. PR diff +/-
• set-up a manually updated GitHub project board with per-person ongoing stuff. Quansight's scikit-learn Project Board (view) could be used as inspiration?
• investigate Google triage party which seems interesting
• scikit-learn specific: get the linting bot to add/remove a "linting failed" label
• scikit-learn specific: saved reply (or bot comment based on label) that says "unless there is a maintainer who is willing to champion this issue/PR, this issue/PR is unlikely to move forward. If you are such a maintainer please comment below".

[bsipocz]

With the brief investigating @Cadair did triage party looks extremely interesting, especially when one has issues/PRs scattered around multiple repos. I would be interested to see if it could be deployed statically on GH pages?

[betatim]

An additional topic we discussed is the difficulty of work getting finished. Maintainers get pulled here, there and everywhere over the course of a day/week. As a result PRs that you might have commented on or reviewed get lost. As a result they stall. Getting things to the point where they are finished isn't a strength of us open-source maintainers :-/ A well known idea in managing projects is to limit the amount of in progress work (per person) to help with this problem/make it easier to finish things.

I created a script that you can use to see which currently open PRs you were active on last week. It is a lightweight way to remind yourself on Monday morning what you worked on last week that has the potential for more work (it isn't yet finished). I uploaded it as a gist https://gist.github.com/betatim/289339530b43c05becd11a7493317a24

There are ways to extend it, for example

• you could use it to create a summary for a whole group of users,
• you could create a webpage via a cron job
• you could mark each PR to indicate if anyone else has been active since you were last active
• highlight PRs by users that are important to you (other maintainers, recurring contributors, first timer contributors, etc)

Let me know what y'all think.

[agriyakhetarpal]

There is a similar script that we use for conda: https://gist.github.com/jaimergp/ff295fecb7e19f862ea3c8ce875a98eb, which is less extensible than the above gist. However, it creates a short-lived token via the gh CLI (using the gh auth token command) instead of having to obtain one from the GitHub settings, which could be helpful!

[betatim]

it creates a short-lived token via the gh CLI (using the gh auth token command) instead of having to obtain one from the GitHub settings, which could be helpful!

That is a cool idea!

[betatim]

The mention of gh made me realise that gh pr list --search "is:pr involves:betatim is:open updated:>2025-11-01" would do a similar thing as the script above. I couldn't find a way to write the date in a "dynamic way" (e.g. updated:@today-7d), but could be done via bash I guess.

[adrinjalali]

@betatim you can do

gh pr list --search "is:pr involves:betatim is:open updated:>$(date --date='30 days ago' +'%Y-%m-%d')"

[betatim]

Well, actually, on macOS you need to use date -v -30d +'%Y-%m-%d'

[bsipocz]

Here is another project board-based solution that the jupyter-book team started to use as an experiment:

https://github.com/jupyter-book/.github/blob/main/.github%2Fworkflows%2Fpr-triage-bot.yml along with PR triage (experimental)

[betatim]

The code for the bot lives in https://github.com/jupyter/pr-triage-board-bot