R1, R2 computing too slow

[Peterande]

        R1Penalty = self.ZeroCenteredGradientPenalty(real_images, logits_real)
        R2Penalty = self.ZeroCenteredGradientPenalty(fake_images, logits_fake)

With this R1, R2 calculation, my training is many times slower

[nick-bvc]

For practical applications, you can try the lazy regularization trick in StyleGAN2 where R1 and R2 are evaluated once per 8 minibatches. It works on most datasets, and only worsens FID slightly.
However in theory the convergence results of R1 and R2 no longer hold with this trick, and it can fail on certain datasets. Use at your own risk.

[dingjibang]

+1same, It makes my network 6~7x slower, this frustrates me :(