Question: Project status, PR acceptance, and GGML backend/device support

[DenisMontes]

Hi,

I have a few questions regarding the current status of the project. Is it still actively maintained? Are you open to new pull requests at this time?

I noticed the project seems to be a bit behind the latest updates in GGML, and I believe it could benefit from upgrading to the latest GGML version.
In my local branch, I made some changes to allow selecting the backend and device at runtime for inference. Unfortunately, I couldn't get inference to work successfully yet. I'm not sure if this is due to recent changes in GGML or if I'm using an incompatible model for testing.

Could you provide any guidance on this?
Also, if I manage to get it working, would you be open to reviewing and accepting a PR for these changes?

Thanks in advance.

[PABannier]

Hello @DenisMontes ! Thanks for your interest in the project :)

Yes you can open a PR, happy to have a look and guide you in updating to the latest ggml version!

[DenisMontes]

Thank you very much for your response. I have a few questions I need to address in order to update the GGML version and add support for dynamic backend initialization, as well as the option to select devices at runtime when creating the context.

I tried downloading the Encodec model from this link: https://huggingface.co/facebook/encodec_24khz. Then, I converted it using the convert.py utility and ran the "main" example, but I got the output shown in the attached image. Could you please clarify which model is compatible with the current version of the project? This is preventing me from properly testing the version upgrade I implemented.

Do you think it would be beneficial to implement GPU splitting? With the new Vulkan backend and the reference implementation from llama.cpp, I believe this support could be feasible. However, I am unsure whether it makes sense given the model's architecture.