Benchmarking

What is the bench-marking test doing?

In the bench-marking test of the component, we are testing how well the component can perform a certain amount of different tasks by recording the time it took to complete each task. Built into the algorithm, it will run the same test 3 (by default) times and average the time out for a more accurate representation for how well the component is at that task.

Why does the bench-marking test run the same test more than once?

It's a good question. The purpose is to find the average amount of time it takes to complete the task. You might think we can run more iterations of the task to increase the time overall and then that test of in itself should be more accurate. However increasing the iterations is a different question so to speak. It doesn't tell us information about specific intervals of iterations.

How accurate are the tests?

The tests are very accurate but not perfect. The reason they are not perfect is because at an given iteration, there may be more resources available for the component to consume or less resources. Taking the average improves the accuracy of your rating. You also want to have as little of programs running while taking the tests because of the statement before.