Try doing just one intersection check, but rotate it 4 times. Does it work? Maybe some rotations aren't useful, so it'd be wasted work.