Locals LLM

[brunob31880]

I will be a good think to allow using locals llms (ollama vllm)

[bet0x]

I will be a good think to allow using locals llms (ollama vllm)

It does support. Check the .env file...

[navr32]

Hi ! I just come to try but with local this give errors and in llm.py i found just azureopenai call from llm.py no just pure openai or llamacpp ollama call ?

My env file ! i have just try to set the openai direct api

=============================================================================

AI MODEL SETTINGS (Choose ONE option)

=============================================================================

OPTION 1: Azure OpenAI (Recommended for production)

#AZURE_OPENAI_API_KEY=your_azure_openai_key
#AZURE_OPENAI_ENDPOINT=https://your-resource.openai.azure.com
#AZURE_OPENAI_DEPLOYMENT=gpt-4o # Your deployment name
#AZURE_OPENAI_API_VERSION=2023-05-15
#AZURE_EMBEDDING_DEPLOYMENT=text-embedding-3-large
#AZURE_EMBEDDING_API_VERSION=2023-05-15

OPTION 2: OpenAI Direct API

OPENAI_API_KEY=localllama
OPENAI_API_BASE=https://0.0.0.0:11434/v1