RandomSampler

Introduction

RandomSampler takes, as an input, a file as writter by the profiler, listing hot methods. For a specified number of iterations, RandomSampler will choose a hot method uniformly at random, and construct a patch comprising a fixed number of edits. Each patch is tested and the results saved to CSV.

Arguments

TODO

reps, baselineRepeats: there are two places that testing of patches can be repeated. Each test can be repeated immediately, and the whole test suite can be repeated, interleaved with testing of the unpatched code, for timing measurements. The loop looks like this:

// p: patched code
// u: unpatched code
for (BaselineRepeat = 1 to baselineRepeats) {
  for (t in tests) {
    for (repNumber = 1 to reps) {
      test(p, t) // run p on test t
    }
    if (baselineRepeats > 1) {
      for (RepNumber = 1 to reps) {
        test(u, t) // run u on test t
      }    
    }
  }
}

(note, RepNumber and BaselineRepeat correspond with the columns in the output CSV)

Output:

A CSV with one row per unit test and the following columns:

• "PatchIndex" - unique ID for each testing of each patch (use this to group unit tests)
• "PatchSize" - number of edits in this patch
• "Patch" - string representation of the patch; each edit is separated by a vertical bar |
• "MethodIndex" - index of the hot method this patch was applied to (i.e. line number in profiler file)
• "TestIndex" - index of the unit test that this row corresponds to (based on the order the tests are listed in the profiler file)
• "UnitTest" - the name of the unit test
• "RepNumber" - repeat for this test (see above)
• "PatchValid" - the patch could be applied to the source code using JavaParser
• "PatchCompiled" - patched code compiled
• "TestPassed" - unit test passed
• "TestExecutionTime(ns)" - beginning to end of test, measured by System.nanoTime()
• "TestCPUTime(ns)" - beginning to end of test, measured by threadMXBean.getCurrentThreadCpuTime()
• "TestTimedOut" - true if timed out, false otherwise
• "TestExceptionType" - from junit result
• "TestExceptionMessage" - from junit result
• "AssertionExpectedValue" - from junit result
• "AssertionActualValue" - from junit result
• "NoOp" - the net effect of this patch leaves the code unchanged (also true if edits were invalid)
• "EditsValid" - boolean string showing which edits were valid (1) or invalid (0)
• "Iteration" - iteration number for the outer loop of the sampler
• "BaselineRepeat" - repeat for this test (see above)
• "IsBaseline" - was this test run against the baseline (1) or the patched code (0)