model-runner.docker.internal

[doringeman]

Hello,

A Docker folk here part of the Docker Model Runner team. 👋
Happy to see you're integrating Docker Model Runner in your workflow! 🙌

The Docker Model Runner's OpenAI endpoint is available for the containers (within the VM) at model-runner.docker.internal.
Hence, you don't need Enable host-side TCP support if you want to access it from within a container.

How do you launch Open WebUI? If you're using a container, can you try with -e OPENAI_API_BASE_URL=http://model-runner.docker.internal/engines/v1?

Let me know if this helps.
We're happy to hear more from you on how we can improve your experience! 😁

Thanks,
Dorin

[kilgore5]

Hi @doringeman

Thanks for the response! I am indeed running OpenWebUi with Docker.

I've not yet looked deeply into how the connections are handled differently (setting OPENAI_API_BASE_URL vs adding a manual connection in OpenWebUI settings), but the approach you describe seems to work correctly.

I'm setting in my docker-compose.yml OPENAI_API_BASE_URLS so I can maintain the default URL also:

environment:
      - OPENAI_API_BASE_URLS=https://api.openai.com/v1;http://model-runner.docker.internal/engines/v1

Would you say this Is this the preferred method at this point to connect Docker Model Runner and OpenWebUI? If so I could open a PR in the OpenWebUI docs to document this approach and help others who may go down this road.

[doringeman]

Hey @kilgore5,

Yes, that's the preferred method!

If so I could open a PR in the OpenWebUI docs to document this approach and help others who may go down this road.

Sounds nice, thank you!

[kilgore5]

Thank you @doringeman for the quick reply... will let you know when I do.