[Feature Request] GPU decoding in FFVShip

[superyu1337]

I believe it would be possible to decode on the GPU with NVDEC/AMF and avoid decode on the CPU.
Obviously not all inputs would support it, but it could be a huge speedup.

[Line-fr]

Hi, this is something that crossed my mind when I started FFVship specifically
but the reason why I backed off actually doing it is because when processing video metadata with ffmpeg API myself, I realized that almost every videos I had had bad metadata set and they were badly set differently from one to another making an ultimate plug and play solution almost impossible. In practice it is feasible to make it work, but I ended up switching entirely for ffms2 which does it way better than I would have done it myself. It is honestly super convenient.
However, it does not allow retrieving raw package to my knowledge which rules out GPU decoding with direct GPU memory usage in HIP/Cuda. Speaking of that, using these librairies would be quite tedious, they add potential compilation complexity (requires ROCm 6.3.3+ for example) and they are a bit different between HIP and Cuda making the usual preprocessor way of switching between HIP and Cuda a bit hard and it is already not trivial to make it work.
To conclude, I will also say that the bottleneck is not really at decoding nor at sending the frame to the GPU at the moment. It is more at the ZIMG part for FFVship and vapoursynth directly for Vship. As such, reimplementing the color conversion toolchain would prove more beneficial. Currently, you should be able to saturate your GPU already if your CPU/GPU combination is decent which shows FFVship is able to maximize the potential of Vship internal metric system. Still, there are scenarios in which ZIMG becomes extremely slow for some reasons and Ideally I would need to get rid of it and do it all on GPU, that is something I have in mind since quite a while but it needs quite a big commitment to actually do.
Also, if you feel like coding that kind of thing, don't hesitate to make a PR or even create a project using Vship C API.