About Performance.

[KelSolaar]

Hi @crowsonkb/Katherine,

I wanted to discuss performance here, what are the gain compared to a classic Numpy implementation, especially in regard to the following comment:

Theano can compile it to use a GPU but this was found to run slower.

Cheers,

Thomas

[crowsonkb]

I implemented it in Numpy first and the result was extremely slow, especially since I had to compute derivatives via finite differences. The performance gain from switching to Theano on the CPU was at least a hundredfold. I am not sure which part is general Theano optimization and which part is having an analytical gradient. I suspect it runs slower on the GPU because the batch size is relatively small (for a 30-step color gradient, only 90 parameters!) but haven't checked the relative performance at absurd numbers of steps yet.

[crowsonkb]

At 16384 steps the GPU starts to close the gap: it took 23 seconds for the default gradient to converge vs 20 on the CPU. I am inclined to think batch size is the answer here. GPUs are massively parallel and quite inefficient on small amounts of data.

[KelSolaar]

Thanks for the details, it is appreciated.