Convert unittests in .d files to use d-lang unittests

[Alex-Muirhead]

Motivation

Current unittests in D source files use tcl scripts for testing, and generate a separate executable for each unittest (enabled using specific version flags). This leaves 3 points of failure:

1. Not creating a recipe for the specific unittest
2. Not enabling the correct version flags for the specific unittest
3. Not adding the target to the tcl script for testing

By converting all unittests to use the specific unittest block (enabled via the --unittest flag), we can compile an entire module to a single executable that runs all tests. As long as the file is marked as a source file, any unittests it contains will be correctly run.

Additions

However, the default test-runner for D is extremely noisy (listing entire stack traces for assertion failures), without adding much information.

Example verbose output

For a single assert fail:

core.exception.AssertError@equilibrium_update.d(370): Assertion failure
----------------
??:? [0x574b2e669c9e]
??:? [0x574b2e669902]
??:? [0x574b2e68eb8e]
??:? [0x574b2e67173c]
??:? [0x574b2e668952]
equilibrium_update.d:370 [0x574b2e5f9a1d]
??:? [0x574b2e669bb7]
??:? [0x574b2e6748d6]
??:? [0x574b2e674e08]
??:? [0x574b2e67487b]
??:? [0x574b2e66e1ee]
??:? [0x574b2e669a63]
??:? [0x574b2e67142a]
??:? [0x574b2e671357]
??:? [0x574b2e6711ac]
/home/alex/.dlang/ldc2-v1.41.0/bin/../import/core/internal/entrypoint.d:42 [0x574b2e4337a1]
??:? [0x71a47902a1c9]
??:? __libc_start_main [0x71a47902a28a]
??:? [0x574b2e4336a4]
33/66 modules FAILED unittests

I've added a separate test_runner.d file to src/utils/ that aims to emulate the output of PyTest (already used in examples/).

Example test_runner output

============================= test session starts =============================
collected 9 test modules

kinetics.chemistry_update::unittest PASSED
kinetics.equilibrium_update::unittest FAILED
kinetics.rate_constant::unittest PASSED
kinetics.reaction::unittest PASSED
kinetics.reaction_mechanism::unittest PASSED
kinetics.two_temperature_argon_kinetics::unittest PASSED
kinetics.two_temperature_argon_with_ideal_gas::unittest SKIPPED
kinetics.two_temperature_gasgiant_kinetics::unittest PASSED
kinetics.vib_specific_co_kinetics::unittest PASSED

================================== FAILURES ===================================
____________________ kinetics.equilibrium_update::unittest ____________________
T: 2499.99
  equilibrium_update.d:L370 - Assertion failure

=================== 1 failed, 7 passed, 1 skipped in 0.07s ====================

Features

• Module filtering: By default, all files added to the compiler call (with --unittest) will have their unittest run. This is usually more information than required. The executable generated by test_runner.d has a --module <name> option, which filters tests down to those belonging to modules starting with <name> (i.e. --module gas.diffusion will only run tests within gas/diffusion/).
• stdout capturing: The previous unittesting would fail a test if it wrote to stdout (via writeln etc). Within the new framework, any output is captured, and only displayed in the error description of failed/errored tests. This helps with debugging tests without cluttering the output / causing false positives.
• Skipped tests: In some cases, a test is known to not succeed, but the failure is well known and not causing any immediate problems. The util.test_runner module contains a skip(string message = "Skipped test") function, and the relevant unittest won't count as failed or errored. This function won't cause any problems with existing test runners.

Notes

• As an entire module is compiled to a single executable, two separate compilations are needed if complex numbers are tested separately. Currently, the makefiles for these modules contain combinations of test, test-all, test-real, and test-complex targets. A unified naming convention would be best.

[Alex-Muirhead]

Removed all the final version(..._test) and added a basic makefile for ntypes unittests.