Benchmarking configuration files

[leonlan]

I think it would be nice to have benchmark configuration files. My SLURM scripts have become a bit messy because we have 6 different benchmark runs, each with different instances, rounding funcs, stopping criteria, etc.

Instead, it would be convenient to have these benchmark configurations in a TOML file, so that I could run benchmarks like this:

poetry run bench solve --bench_loc cvrp.toml

The TOML configuration files could follow the args interface like this:

instances = ["instances/CVRP/*.vrp"]
round_func = "round"
max_runtime = 2.4
per_client = true

It should be possible to specify other args combined with the config:

poetry run bench solve --bench_loc cvrp.toml --seed 1 --sol_dir out/cvrp/

An error should be raised when an argument is both passed and in the configuration file.

---

Example TOML file:

instances = [
    "instances/CVRP/X-n101-k25.vrp",
    "instances/CVRP/X-n1001-k43.vrp"
]
solutions = [
    "instances/CVRP/X-n101-k25.sol",
    "instances/CVRP/X-n1001-k43.sol"
]
seed = 42
num_procs = 4
stats_dir = "stats"
sol_dir = "solutions"
round_func = "dimacs"
config_loc = "configs/params.toml"
max_runtime = 60.0
max_iterations = 25000
no_improvement = 5000
per_client = false
display = false

• Introduce bench_config_loc
• Rename config_loc to solve_params_loc