Update Weights Operation

[EscVM]

Hi @eric-mitchell,

Thank you for the repo and the great paper. I was looking at your code and I have a doubt on this line of code: why is there a plus? Shouldn't be a minus?
As you also wrote in the paper, the final edited weights should be:

$$\tilde{W} = W_l − α * \tilde{\nabla} W_l$$

Thanks for your help.

[eric-mitchell]

You're right about this- good catch! Turns out this is indeed a bug in the implementation. Have you been able to train a model with this corrected, and seen if there is a difference in performance?

[EscVM]

Actually, I've never run your code, but I adapted it for another use case. Anyway, the code works also with the plus. The optimization process makes the network flip all gradient signs. However, I expect slightly better results with the right sign from the beginning, simply because the task should be a little simpler from the point of view of hypernetworks.