Skip to content

Conversation

@LunaStev
Copy link
Member

This PR introduces tools/run_tests.py, a robust Python-based automation script designed to streamline the integration testing process for the wavec compiler. This script replaces manual test execution, providing a standardized and efficient way to verify compiler stability and catch regressions.

Key Features

1. Automated Test Discovery

  • Scans the test/ directory for all files matching the test*.wave pattern.
  • Special Case Handling: Includes logic to manage directory-based tests (e.g., test28), ensuring complex test structures are executed correctly.

2. Execution & Safety

  • Execution Target: Runs tests against the release build of the compiler (target/release/wavec).
  • Timeout Protection: Implements a strict 5-second timeout per test case to prevent the testing suite from hanging due to infinite loops or deadlocks in the generated code.

3. Enhanced Terminal UI/UX

Uses colored output for immediate visual feedback:

  • BLUE: Indicates a test has started.
  • GREEN: Successful execution (PASS).
  • RED: Execution failed (FAIL) with the specific exit code displayed.
  • YELLOW: Test exceeded the time limit (TIMEOUT).

4. Post-Run Reporting

  • Provides a comprehensive summary at the end of the session.
  • Categorizes and lists every test by its result status (Passed, Failed, or Timed Out) for quick debugging.

Workflow Requirements

To ensure the script functions correctly, the compiler must be built in release mode beforehand:

cargo build --release
python3 tools/run_tests.py

Benefits

  • Stability: Easily verify that new features (like the recent expression parser refactor) don't break existing functionality.
  • Efficiency: Reduces the time required for pre-release verification from minutes to seconds.
  • Visibility: Clear, color-coded logs make it obvious where the compiler is failing.

Introduce `tools/run_tests.py` to automate integration testing of the
`wavec` compiler.

Key features:
- Automatically detects and runs all `test*.wave` files in the `test/` directory.
- Includes special handling for directory-based tests (e.g., `test28`).
- Implements a 5-second timeout for each test execution to prevent hanging.
- Provides colored terminal output for better readability:
  - BLUE for test execution start
  - GREEN for PASS
  - RED for FAIL (with exit code)
  - YELLOW for TIMEOUT
- Summarizes results at the end, listing all passed, failed, and timed-out tests.
- Requires `cargo build --release` to be run beforehand.

This script simplifies the testing workflow and provides quick feedback
on compiler stability and regressions.

Signed-off-by: LunaStev <luna@lunastev.org>
@LunaStev LunaStev self-assigned this Dec 24, 2025
@LunaStev LunaStev merged commit 7968cd5 into wavefnd:master Dec 24, 2025
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant