feat: use an opinionated project structure for the testing code#172
feat: use an opinionated project structure for the testing code#172dwilding wants to merge 21 commits intocanonical:mainfrom
Conversation
There was a problem hiding this comment.
I quite like the switch to project.toml and uv.
I ran through everything described in HACKING.md (test cases, make commands) and had no issues - everything was very straightforward, and I feel confident about contributing tests.
I'm not heavily involved in the development efforts at the moment, so I may not have all the context necessary to give this a thorough review. But from the perspective of a casual contributor, I think these changes make it easy and intuitive to run and modify tests.
|
I just merged the latest testing code from upstream and ran the formatter and linter. Linting picked up a duplicate test function name, which I'd missed when reviewing the relevant PR. I've pushed c994ecf to rename one of the test functions. It seems that my extra commits have dismissed the prior approvals, so I'm re-asking for approvals - sorry for the noise. Side question: How necessary is it for us to have that branch protection rule enabled? |
SecondSkoll
left a comment
There was a problem hiding this comment.
I agree with Andreia and JJ. These are some great improvements to the existing framework, and they align nicely with the professionalisation of our development work. As we discussed yesterday, this is going to make development and testing of our Vale rules a lot easier.
The purpose of this PR is to be more opinionated about how the testing code is maintained, while trying to keep the project accessible for contributors who only want to run the tests.
Main change
I've switched the testing code to a Python project with
project.toml(instead ofrequirements.txt) and introduced a makefile to standardize how the testing code is run and maintained.My opinionated view: Our makefile should prefer uv for running tools in the Python project. We shouldn't be in the business of managing virtual environments, but we shouldn't turn away people who want to bring their own virtual environment. In practice, this means:
If a contributor has uv installed, they can use
make runto run the tests, with uv managing the virtual environment. If they don't want to install uv, we document (inHACKING.md) how to create & activate a virtual environment before usingmake run.Maintainers of the testing code are more strongly encouraged to have uv installed if they want the smoothest development experience. The development commands
make formatandmake lintwill still work if the maintainer brings their own venv, but we don't document how to create a venv that contains the dev dependencies.Documentation improvements
A new
HACKING.mdguide (preview) that covers how to run the testing code and has guidance about maintaining the testing code. This guide is linked from "Testing the rules" inREADME.md(preview).Updated "Introduction to Vale rule development" to mention adding test cases (preview) and to clarify that the preexisting testing instructions are for manual testing (preview).
New requirements for the testing code
I propose we adopt ruff for formatting the testing code. I also propose that we require the testing code to be linted using ruff and pyright. The new makefile has
make formatandmake lintaccordingly.I'm also adding a workflow that runs
make linton PRs that touch thetestsdirectory.This PR includes some minor adjustments to the testing code to make sure that the code passes linting.
Out of scope
Linting of the YAML manifest of test cases and the style guide rules themselves (also YAML). I think this would be worth adding in follow-on PRs.