Skip to content

Conversation

@ThanhNguyxn
Copy link

Summary

Users on Z.AI coding plan subscription need to use a different API endpoint (https://api.z.ai/api/coding/paas/v4) instead of the general API (https://api.z.ai/api/paas/v4). Currently, when users select the "zai" provider, it uses the general API URL, causing errors for coding plan users.

Root Cause

Z.AI (Zhipu AI) has two different API endpoints:

  • General API: https://api.z.ai/api/paas/v4 - for standard subscriptions
  • Coding Plan API: https://api.z.ai/api/coding/paas/v4 - for coding plan subscriptions

LiteLLM's zai/ prefix uses the general API by default, but coding plan users need the dedicated endpoint.

Solution

This PR adds a new zai_coding/ provider prefix that:

  1. Detects zai_coding/ model prefix in the model validator
  2. Rewrites it to zai/ for LiteLLM compatibility
  3. Sets the correct base_url to the coding API endpoint

Usage

Users on Z.AI coding plan can now use models like zai_coding/glm-4.5 without needing to manually configure the base URL.

Related PRs

Fixes

Resolves OpenHands/OpenHands#12268

Users on Z.AI coding plan subscription need to use a different API
endpoint (https://api.z.ai/api/coding/paas/v4) instead of the general
API (https://api.z.ai/api/paas/v4).

This adds a new 'zai_coding/' provider prefix that:
- Detects 'zai_coding/' model prefix
- Rewrites it to 'zai/' for LiteLLM compatibility
- Sets the correct base_url to the coding API endpoint

Fixes: OpenHands/OpenHands#12268
Copy link
Collaborator

@all-hands-bot all-hands-bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The implementation looks good overall and correctly handles the model prefix rewrite. However, there are two issues: an important inconsistency with base_url handling and missing test coverage.

if model_val.startswith("zai_coding/"):
model_name = model_val.removeprefix("zai_coding/")
d["model"] = f"zai/{model_name}"
d["base_url"] = "https://api.z.ai/api/coding/paas/v4"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟠 Important: The base_url is unconditionally set here, which differs from the openhands/ provider pattern (line 366) and will override any custom base_url that users provide.

The existing openhands/ provider uses d.get("base_url") or ... to respect user-provided custom URLs, and there are explicit tests (test_base_url_for_openhands_provider_with_custom_url) demonstrating this expected behavior.

Suggested change
d["base_url"] = "https://api.z.ai/api/coding/paas/v4"
d["base_url"] = d.get("base_url") or "https://api.z.ai/api/coding/paas/v4"

This change ensures:

  • If no base_url is provided → sets the coding API endpoint
  • If base_url is explicitly None → sets the coding API endpoint
  • If a custom base_url is provided → respects the user's choice

This maintains consistency with other provider rewrites and allows for testing/debugging with custom endpoints.

# - Coding Plan API: https://api.z.ai/api/coding/paas/v4 (for coding plan subscriptions)
# LiteLLM's zai/ prefix uses the general API by default, but coding plan users
# need the dedicated endpoint. See: https://github.com/All-Hands-AI/OpenHands/issues/12268
if model_val.startswith("zai_coding/"):
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Suggestion: Consider adding tests for this new provider rewrite to ensure it works correctly and consistently. The openhands/ provider has comprehensive test coverage in tests/sdk/llm/test_llm.py:

  • test_base_url_for_openhands_provider() - automatic base_url setting
  • test_base_url_for_openhands_provider_with_explicit_none() - explicit None handling
  • test_base_url_for_openhands_provider_with_custom_url() - respecting custom URLs

Similar tests for zai_coding/ would help verify:

  1. Model prefix rewriting (zai_coding/glm-4.5zai/glm-4.5)
  2. Automatic base_url setting to the coding endpoint
  3. Custom base_url handling (especially after fixing the issue above)

Copy link
Collaborator

@enyst enyst left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you! Using coding plan is very helpful.

Could you please tell, does it work for you, or doesn't it require auth?

@ThanhNguyxn
Copy link
Author

@enyst Thank you for the review!

To answer your question: Yes, it does require authentication. The Z.AI Coding Plan API requires an API key just like the standard Z.AI API. Users need to:

  1. Get their API key from the Z.AI platform (same key works for both endpoints)
  2. Set the LLM_API_KEY environment variable or configure it through the OpenHands settings

The only difference between the two endpoints is:

  • Standard API (https://api.z.ai/api/paas/v4): For users with regular Z.AI subscriptions
  • Coding Plan API (https://api.z.ai/api/coding/paas/v4): For users who specifically subscribed to the coding plan

Both endpoints use the same authentication mechanism (Bearer token with API key). This PR just ensures that users with the coding plan subscription get routed to the correct endpoint automatically when they use the zai_coding/ prefix.


Regarding the bot's suggestions:

  1. ✅ I will update the base_url assignment to use d.get("base_url") or ... pattern for consistency with the openhands/ provider
  2. ✅ I will add test coverage for the zai_coding/ prefix handling

I'll push these fixes shortly.

@ThanhNguyxn
Copy link
Author

Pushed the fixes!

Changes made in commit 0464cc0f:

  1. Updated base_url assignment to use d.get("base_url") or ... pattern for consistency
  2. Added 4 test cases for zai_coding/ provider:
    • test_base_url_for_zai_coding_provider - Tests default base_url
    • test_base_url_for_zai_coding_provider_with_explicit_none - Tests explicit None handling
    • test_base_url_for_zai_coding_provider_with_custom_url - Tests custom URL respect
    • test_zai_coding_provider_model_rewrite - Tests model prefix rewriting

All tests pass locally. Ready for re-review! 🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Unable to run z.ai (zai) models if you're on a z.ai coding plan, works fine if you used Advanced and enter the right API url manually

3 participants