Suggestion: Add support for local Ollama models

[ayoubelouardi]

Description:
Right now, the project is designed for OpenAI endpoints. I propose adding support for Ollama, which runs LLMs locally via http://localhost:11434. This will allow users to run models offline without requiring API keys.

Proposed Changes:

• Detect when Base URL points to Ollama (http://localhost:11434).
• Update /fetch-models route to query Ollama’s /api/tags endpoint.
• Adjust /save-settings so it doesn’t require an API key when using Ollama.
• Ensure dropdown populates with installed Ollama models (e.g., tinyllama:latest, llama3.2:3b...).

Next Steps:
Patch app.py to support Ollama endpoints, update README with instructions, and test with installed models.

Note: I’m open to implementing this feature myself.

[Nurb4000]

I myself mostly use llama.cpp locally so it works, but a feature like that might open things up for a lot of people ( no pun intended there )

[ayoubelouardi]

Thanks for sharing your perspective! I know a lot of people use llama.cpp locally and it definitely works well. My thought with adding Ollama support is that it lowers the barrier for folks who aren’t as comfortable compiling or configuring things; Ollama makes it really easy to pull and run models with one command, and exposing that through the WebUI could make the project more approachable for a wider audience.

I’m happy to take on the implementation myself so it doesn’t add extra work for you. If you’re open to it, I can draft a PR that adds Ollama support alongside the existing setup, so users can choose whichever backend they prefer.

[Nurb4000]

I have heard, that Ollama has it on their agenda to add an OpenAI endpoint. That would go a long way to allow many projects to connect, without any work. Unsure their status. ( ran into a couple of interesting projects that were the exact opposite and they only supported Ollama, which basically cut me out :( )