Add LiteLLM provider support #1

metodn · 2026-01-14T17:49:22Z

Summary

Add new litellm provider that proxies through a local LiteLLM server
Provider reads LITELLM_MODEL env var to override model selection
Adds run.sh convenience script

Test plan

Tested with LiteLLM proxy running locally
Verified environment variable override works correctly

Add new litellm provider that proxies through a local LiteLLM server. This allows using models from various providers through a unified interface. The provider reads LITELLM_MODEL env var to override model selection.

Copilot

Pull request overview

This PR adds support for LiteLLM as a model provider, enabling Laser to proxy requests through a local LiteLLM server. The integration follows the existing provider pattern and includes environment variable configuration for flexible deployment.

Changes:

Added litellm provider configuration using OpenAILegacy with customizable base URL and API key
Implemented environment variable override for model selection via LITELLM_MODEL
Added convenience script run.sh for local testing with LiteLLM
Modified .gitignore to exclude all markdown files

Reviewed changes

Copilot reviewed 2 out of 3 changed files in this pull request and generated 5 comments.

File	Description
laser/main.py	Adds LiteLLM provider configuration and environment variable-based model override logic
run.sh	Provides convenience script for running Laser with LiteLLM configuration
.gitignore	Adds pattern to ignore all markdown files

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-01-14T17:52:11Z

run.sh

+
+exit
+
+o use:


Incomplete text. Line 8 appears to be missing the first character. Should be 'To use:' instead of 'o use:'.

Suggested change

o use:

To use:

Copilot · 2026-01-14T17:52:11Z

run.sh

+$: uv tool install git+https://github.com/ExpressGradient/laser
+$: laser
+
+or 


Missing capital letter. Should be 'Or' instead of 'or' for proper sentence structure.

Suggested change

or

Or

Copilot · 2026-01-14T17:52:12Z

run.sh

+export LITELLM_MODEL=glm-4.7
+export LITELLM_PROXY_API_KEY=sk-1234
+export LITELLM_PROXY_API_BASE=http://192.168.43.179:4001


The script contains hardcoded credentials (API key) and a specific IP address. This convenience script should not be committed with real credentials or environment-specific configuration. Consider creating a template file (e.g., run.sh.example) instead, or documenting that users should customize these values.

Suggested change

export LITELLM_MODEL=glm-4.7

export LITELLM_PROXY_API_KEY=sk-1234

export LITELLM_PROXY_API_BASE=http://192.168.43.179:4001

# Default model can be overridden via LITELLM_MODEL

export LITELLM_MODEL="${LITELLM_MODEL:-glm-4.7}"

# Require the API key to be provided via the environment; do not hardcode it here.

if [ -z "${LITELLM_PROXY_API_KEY:-}" ]; then

echo "Error: LITELLM_PROXY_API_KEY is not set. Please export it in your environment before running this script." >&2

exit 1

fi

export LITELLM_PROXY_API_KEY

# Default API base can be overridden via LITELLM_PROXY_API_BASE; use localhost by default.

export LITELLM_PROXY_API_BASE="${LITELLM_PROXY_API_BASE:-http://127.0.0.1:4001}"

Copilot · 2026-01-14T17:52:12Z

run.sh

+exit
+
+o use:
+
+$: uv tool install git+https://github.com/ExpressGradient/laser
+$: laser
+
+or 
+$: uvx --from git+https://github.com/ExpressGradient/laser laser


The exit statement on line 6 makes all subsequent documentation (lines 8-14) unreachable and non-functional. If this is intentional to prevent accidental execution, the documentation should be moved above the exit or the exit should be removed.

Suggested change

exit

o use:

$: uv tool install git+https://github.com/ExpressGradient/laser

$: laser

or

$: uvx --from git+https://github.com/ExpressGradient/laser laser

# To use:

#

# $: uv tool install git+https://github.com/ExpressGradient/laser

# $: laser

#

# or

# $: uvx --from git+https://github.com/ExpressGradient/laser laser

exit

Copilot · 2026-01-14T17:52:12Z

laser/main.py

    provider_cls, provider_kwargs = PROVIDER_CONFIGS[provider]
    if provider_cls is Anthropic:
        provider_kwargs = {**provider_kwargs, "default_max_tokens": max_tokens}
+    if provider == "litellm":


The LITELLM_MODEL environment variable override behavior should be documented. Add a comment explaining that this allows runtime model override for litellm provider, as this is unique behavior not present in other providers.

Suggested change

if provider == "litellm":

if provider == "litellm":

# Allow runtime model override specifically for the litellm provider via

# the LITELLM_MODEL environment variable. This override behavior is unique

# to litellm and is not supported by other providers in this CLI.

Add LiteLLM provider support

bc51e1a

Add new litellm provider that proxies through a local LiteLLM server. This allows using models from various providers through a unified interface. The provider reads LITELLM_MODEL env var to override model selection.

Copilot AI review requested due to automatic review settings January 14, 2026 17:49

Copilot started reviewing on behalf of metodn January 14, 2026 17:49 View session

Copilot AI reviewed Jan 14, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Add LiteLLM provider support #1

Add LiteLLM provider support #1

metodn commented Jan 14, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Jan 14, 2026

Uh oh!

Copilot AI Jan 14, 2026

Uh oh!

Copilot AI Jan 14, 2026

Uh oh!

Copilot AI Jan 14, 2026

Uh oh!

Copilot AI Jan 14, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

-export LITELLM_MODEL=glm-4.7
-export LITELLM_PROXY_API_KEY=sk-1234
-export LITELLM_PROXY_API_BASE=http://192.168.43.179:4001
+# Default model can be overridden via LITELLM_MODEL
+export LITELLM_MODEL="${LITELLM_MODEL:-glm-4.7}"
+# Require the API key to be provided via the environment; do not hardcode it here.
+if [ -z "${LITELLM_PROXY_API_KEY:-}" ]; then
+    echo "Error: LITELLM_PROXY_API_KEY is not set. Please export it in your environment before running this script." >&2
+    exit 1
+fi
+export LITELLM_PROXY_API_KEY
+# Default API base can be overridden via LITELLM_PROXY_API_BASE; use localhost by default.
+export LITELLM_PROXY_API_BASE="${LITELLM_PROXY_API_BASE:-http://127.0.0.1:4001}"

-exit
-o use:
-$: uv tool install git+https://github.com/ExpressGradient/laser
-$: laser
-or
-$: uvx --from git+https://github.com/ExpressGradient/laser laser
+# To use:
+#
+# $: uv tool install git+https://github.com/ExpressGradient/laser
+# $: laser
+#
+# or
+# $: uvx --from git+https://github.com/ExpressGradient/laser laser
+exit

-    if provider == "litellm":
+    if provider == "litellm":
+        # Allow runtime model override specifically for the litellm provider via
+        # the LITELLM_MODEL environment variable. This override behavior is unique
+        # to litellm and is not supported by other providers in this CLI.

Add LiteLLM provider support #1

Are you sure you want to change the base?

Add LiteLLM provider support #1

Conversation

metodn commented Jan 14, 2026

Summary

Test plan

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Jan 14, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 14, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 14, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 14, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 14, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant