Remove modelRouter and add model_store #224

philipph-askui · 2026-01-21T07:20:44Z

This PR:

removes the modelRouter
adds a new model_store
removes chat-related code, and
refactors the doc
(especially the last 2 bloat the PR a bit, sorry for that)

Summary

Removed the ModelRouter abstraction and string-based model selection
Introduced direct model injection and a discoverable model_store
Clarified model vs. MessagesApi responsibilities
Significantly reduced complexity and type errors
refactor the docs

Key Changes

Model Store (model_store): Central discovery + creation of models
Direct Injection: Models passed explicitly to VisionAgent
Per-call Overrides: act_model, get_model, locate_model
API and Codebase Cleanup:
- Removed router, facade
- - Settings Architecture: Single source of truth at agent level
- Clear parameter naming (*_settings, *_model)
- Renamed Gemini models for clarity
- Removes chat-related code
- Removes support for unused models, e.g. UI-TARS

Breaking Change

all code that does not use the default models will likely break
String-based model selection is removed.
Users must migrate to direct model injection via model_store.

…del store BREAKING CHANGE: Removed ModelRouter and ModelRegistry classes. Users must now use direct model injection.

…modelrouter

…raises an Error when executing from cache)

…move() methods

…nd AskUiInferenceLocateApi

mlikasam-askui · 2026-01-30T10:17:48Z

docs/01_Setup.md

+**Problem**: Error connecting to Agent OS
+
+**Solutions**:
+1. Check if Agent OS is running (look for the system tray icon)


The agent OS doesn’t have a tray icon.

mlikasam-askui · 2026-01-30T10:18:21Z

docs/01_Setup.md

+
+**Solutions**:
+1. Check if Agent OS is running (look for the system tray icon)
+2. Restart Agent OS from your applications menu


The agent OS is not listed in the application menu.

mlikasam-askui · 2026-01-30T10:18:48Z

docs/03_Using-Models-and-BYOM.md

+    custom_settings = ActSettings(
+        messages=MessageSettings(
+            max_tokens=8192,
+            temperature=0.5,
+            betas=["computer-use-2025-01-24"],
+        )
+    )
+


Which system prompt is used in this example?
Could you please remove the betas?

mlikasam-askui · 2026-01-28T09:04:30Z

docs/01_Setup.md

+
+## Python Package Installation
+
+AskUI Vision Agent requires Python 3.10 or higher.


requires-python = ">=3.10,<3.14"

mlikasam-askui · 2026-01-28T09:06:39Z

docs/01_Setup.md

+```bash
+pip install askui[anthropic]  # Anthropic Claude support
+pip install askui[openrouter]  # OpenRouter support
+pip install askui[documents]  # PDF, Excel, Word support
+```


SDK dosent support these targets

mlikasam-askui · 2026-01-30T08:39:34Z

src/askui/models/anthropic/settings.py

+        super().__init__(self.message)
+
+
+class AnthropicModelSettings(BaseSettings):


Currently unused.

mlikasam-askui · 2026-01-30T08:42:29Z

src/askui/models/askui/locate_api.py

+        self,
+        locator: str | Locator,
+        image: ImageSource,
+        locate_settings: LocateSettings,  # noqa: ARG002


Are the Locate settings only needed for LLM-based locators?

idea was to have a general settings object for all locate commands here

mlikasam-askui · 2026-01-30T09:11:19Z

src/askui/models/shared/settings.py

+    max_tokens: int = 4096
+    temperature: float = Field(default=0.5, ge=0.0, le=1.0)
+    system_prompt: GetSystemPrompt | None = None
+    timeout: float | None = None
+
+
+class LocateSettings(BaseModel):
+    """Settings for LocateModel operations (UI element location)."""
+
+    model_config = ConfigDict(arbitrary_types_allowed=True)
+
+    query_type: str | None = None
+    confidence_threshold: float = Field(default=0.8, ge=0.0, le=1.0)
+    max_detections: int = 10
+    timeout: float | None = None
+    system_prompt: LocateSystemPrompt | None = None


Currently, only the system prompt is being used.

mlikasam-askui · 2026-01-30T09:12:32Z

src/askui/models/shared/settings.py

+    confidence_threshold: float = Field(default=0.8, ge=0.0, le=1.0)
+    max_detections: int = 10
+    timeout: float | None = None
+    system_prompt: LocateSystemPrompt | None = None


The Locate system prompt should not be configurable because the expected return is currently hard-coded. Changing the system prompt would cause the Locate code to fail.

mlikasam-askui · 2026-01-30T09:58:18Z

src/askui/models/shared/settings.py

+    timeout: float | None = None
+
+
+class LocateSettings(BaseModel):


What do you think about removing the Locate and Get settings?

I think we should generally discuss how "configurable" get and locate should be. This also includes if we want to support BYOM for these commands or if that should only be possible for act

philipph-askui added 3 commits January 21, 2026 06:57

chore: add plan and claude rules

b99056d

refactor: simplify model selection with direct model injection and mo…

7a23ca9

…del store BREAKING CHANGE: Removed ModelRouter and ModelRegistry classes. Users must now use direct model injection.

resolve merge conflict

421d043

philipph-askui changed the title ~~Chore/modelrouter~~ Remove Modelrouter Jan 21, 2026

philipph-askui and others added 12 commits January 21, 2026 08:23

Merge branch 'main' into chore/modelrouter

188ddd2

feat: remove chat related dependencies and components

5c3ade4

Merge remote-tracking branch 'origin/feat/deprecate-chat' into chore/…

dfcda56

…modelrouter

chore: remove dependency on anthropic types

00d359c

fix(agentOS): set default value of name for Display to None (else it …

339717f

…raises an Error when executing from cache)

chore: fix typecheck errors

1d5929b

fix linting errors

021c90b

fix linting errors

2598a46

feat: add model_store

f1460a9

fix linting errors

d8ce5b2

Merge branch 'main' into chore/modelrouter

d4ea0d8

fix linting errors

a0a75b1

philipph-askui changed the title ~~Remove Modelrouter~~ Remove modelRouter and add mode_store Jan 22, 2026

philipph-askui added 11 commits January 22, 2026 15:16

remove UITARS

93ea9ee

remove ModelComposition

b2c79be

update get model interface

f4c3cb2

remove UITars, modelcomposition, and model definition

6222282

Add locate_settings and locate_model parameters to click() and mouse_…

fca6234

…move() methods

Remove AskUiBaseLocateModel class and create LocateApi (Base class) a…

e3dc24a

…nd AskUiInferenceLocateApi

chore: remove GenericActModel

4ac0df4

move HF models and OpenRouter models to model_store

25a3ad8

chore: fix docs

19c8968

fix: make models serializable for telemetry

6217b11

chore: update docs

8f3a9ea

philipph-askui marked this pull request as ready for review January 26, 2026 06:53

philipph-askui requested a review from mlikasam-askui January 26, 2026 06:54

philipph-askui requested a review from programminx-askui January 26, 2026 06:54

philipph-askui changed the title ~~Remove modelRouter and add mode_store~~ Remove modelRouter and add model_store Jan 26, 2026

mlikasam-askui requested changes Jan 30, 2026

View reviewed changes

address review remakrs

0732ac8

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Remove modelRouter and add model_store #224

Remove modelRouter and add model_store #224

Uh oh!

philipph-askui commented Jan 21, 2026 •

edited

Loading

Uh oh!

mlikasam-askui Jan 30, 2026

Uh oh!

mlikasam-askui Jan 30, 2026

Uh oh!

mlikasam-askui Jan 30, 2026

Uh oh!

mlikasam-askui Jan 28, 2026

Uh oh!

mlikasam-askui Jan 28, 2026

Uh oh!

mlikasam-askui Jan 30, 2026

Uh oh!

mlikasam-askui Jan 30, 2026

Uh oh!

philipph-askui Jan 30, 2026

Uh oh!

mlikasam-askui Jan 30, 2026

Uh oh!

mlikasam-askui Jan 30, 2026

Uh oh!

mlikasam-askui Jan 30, 2026

Uh oh!

philipph-askui Jan 30, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants


		## Python Package Installation

		AskUI Vision Agent requires Python 3.10 or higher.

		super().__init__(self.message)


		class AnthropicModelSettings(BaseSettings):

		timeout: float \| None = None


		class LocateSettings(BaseModel):

Remove modelRouter and add model_store #224

Are you sure you want to change the base?

Remove modelRouter and add model_store #224

Uh oh!

Conversation

philipph-askui commented Jan 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

philipph-askui commented Jan 21, 2026 •

edited

Loading