Only update running averages during training

[kreid415]

Batch-Renormalization-PyTorch/batch_renormalization.py

Line 43 in 72bb6d9

self.running_avg_mean = self.running_avg_mean + self.momentum * (batch_ch_mean.data.to(device) - self.running_avg_mean)

As per the paper, https://arxiv.org/pdf/1702.03275.pdf, during inference in both functions, you should not update running averages. Otherwise you will diverge from the learned parameters over time.