A performance-optimized, framework-agnostic system for AI agents to understand and interact with web interfaces.
UUICS bridges the gap between AI models and web UIs by providing structured context about page elements and enabling AI-driven interactions through a simple action execution system.
UUICS scans your web page, extracts all interactive elements (buttons, inputs, dropdowns, etc.), and provides this information in AI-friendly formats. When an AI decides to take action, UUICS executes it safely on the page.
┌────────────────────────────────────────────────────────────────────────────────────┐
│ HOW UUICS WORKS │
├────────────────────────────────────────────────────────────────────────────────────┤
│ │
│ ┌─────────────────────┐ ┌─────────────────────┐ ┌──────────────┐ │
│ │ YOUR WEB PAGE │ │ UUICS ENGINE │ │ AI MODEL │ │
│ │ │ │ │ │ │ │
│ │ • Buttons │ scan │ ┌───────────────┐ │ send │ Claude, │ │
│ │ • Inputs │ ───────▶│ │ DOM Scanner │ │ ───────▶│ GPT-4, │ │
│ │ • Forms │ │ │ │ │ │ or any LLM │ │
│ │ • Dropdowns │ │ │ • Elements │ │ │ │ │
│ │ • Radio buttons │ │ │ • States │ │ │ Analyzes │ │
│ │ • Checkboxes │ │ │ • Attributes │ │ │ context & │ │
│ │ • Links │ │ │ • Labels │ │ │ decides │ │
│ │ • Custom elements │ │ └───────────────┘ │ │ action │ │
│ │ │ │ │ │ │ │ │
│ │ Framework Support: │ │ ▼ │ └──────┬───────┘ │
│ │ ✓ Radix UI │ │ ┌───────────────┐ │ │ │
│ │ ✓ Shadcn/ui │ │ │ Serializer │ │ │ │
│ │ ✓ MUI/Chakra │ │ │ │ │ │ │
│ │ ✓ Headless UI │ │ │ JSON/Natural/ │ │ │ │
│ │ ✓ Vanilla HTML │ │ │ OpenAPI/MCP │ │ │ │
│ └─────────────────────┘ │ └───────────────┘ │ │ │
│ ▲ │ │ │ │
│ │ │ ┌───────────────┐ │ │ │
│ │ │ │Action Executor│ │ │ │
│ │ execute │ │ │◀─┼────────────────┘ │
│ └──────────────────────┼──│ click, type, │ │ command │
│ │ │ select, check │ │ │
│ Element Detection: │ │ hover, submit │ │ │
│ ───────────────── │ └───────────────┘ │ │
│ • data-uuics-* attributes │ │ │
│ • aria-label/aria-checked └─────────────────────┘ │
│ • data-state (Radix UI) │
│ • role="button/radio/..." │
│ • Standard HTML attributes │
│ │
└────────────────────────────────────────────────────────────────────────────────────┘
- 🔍 DOM Scanning: Automatically detects all interactive elements
- 📝 Context Serialization: JSON, Natural Language, or OpenAPI formats
- ⚡ Action Execution: Click, type, select, check, hover, and more
- 🔄 State Tracking: Track JavaScript variables alongside DOM state
- 🛡️ Sensitive Data Protection: Automatically exclude passwords and tokens
- 🔌 MCP Support: Native Model Context Protocol integration for Claude
- Vanilla JavaScript: Direct engine usage
- React: Provider pattern with hooks (see examples)
- Vue, Angular, etc.: Engine works with any framework
- Claude: Full support with MCP tool calling + natural language context
- GPT-4/OpenAI: Function calling with OpenAPI format
- Any LLM: JSON format works universally
npm install @angelerator/uuics-coreimport { UUICSEngine } from '@angelerator/uuics-core';
// Create engine
const uuics = new UUICSEngine({
scan: { interval: 0 }, // Manual scanning
track: { mutations: true, clicks: true }
});
// Initialize
await uuics.initialize();
// Scan the page
const context = await uuics.scan();
// Get AI-friendly context
const naturalLanguage = uuics.serialize('natural');
console.log(naturalLanguage);
// Output:
// # Page Context
// ## Interactive Elements
// ### Buttons (3)
// - **Submit** → `#submit-btn`
// - **Cancel** → `#cancel-btn`
// ...
// Execute an action from AI response
await uuics.execute({
action: 'setValue',
target: '#email',
parameters: { value: 'user@example.com' }
});import { UUICSEngine } from '@angelerator/uuics-core';
const uuics = new UUICSEngine();
await uuics.initialize();
// Get page context
await uuics.scan();
const context = uuics.serialize('natural');
// Send to Claude (using your preferred method)
const response = await claude.messages.create({
model: 'claude-sonnet-4-20250514',
system: `You are a web automation assistant. Here's the page context:\n${context}`,
messages: [{ role: 'user', content: 'Fill in the email field with test@example.com' }]
});
// Parse and execute Claude's action
const action = parseActionFromResponse(response);
await uuics.execute(action);UUICS includes native MCP support for Claude's tool calling capabilities:
import {
UUICSEngine,
MCPToolsGenerator,
MCPToolHandler
} from '@angelerator/uuics-core';
// Initialize UUICS
const engine = new UUICSEngine();
await engine.initialize();
await engine.scan();
// Generate MCP tools from current page context
const generator = new MCPToolsGenerator();
const tools = generator.generateTools(engine.getContext());
// Send tools to Claude
const response = await anthropic.messages.create({
model: 'claude-sonnet-4-20250514',
max_tokens: 1024,
tools: tools.map(t => ({
name: t.name,
description: t.description,
input_schema: t.input_schema,
})),
messages: [{ role: 'user', content: 'Click the submit button' }]
});
// Handle tool calls from Claude
const handler = new MCPToolHandler(engine);
for (const block of response.content) {
if (block.type === 'tool_use') {
const result = await handler.handleToolCall({
name: block.name,
input: block.input,
id: block.id,
});
console.log('Tool result:', result);
}
}The main class for interacting with UUICS.
new UUICSEngine(config?: UUICSConfig)| Method | Description |
|---|---|
initialize() |
Initialize the engine |
scan(root?, config?) |
Scan DOM and update context |
getContext() |
Get current page context |
serialize(format?) |
Serialize context (json/natural/openapi) |
subscribe(callback) |
Subscribe to context updates |
| Method | Description |
|---|---|
execute(command) |
Execute a single action |
executeBatch(commands) |
Execute multiple actions sequentially |
| Method | Description |
|---|---|
trackState(name, obj) |
Track object with auto-updates |
registerState(name, getter) |
Register computed state |
untrackState(name) |
Stop tracking an object |
type ActionType =
| 'click' // Click an element
| 'setValue' // Set input/textarea value
| 'select' // Select dropdown option(s)
| 'check' // Check a checkbox
| 'uncheck' // Uncheck a checkbox
| 'submit' // Submit a form
| 'focus' // Focus an element
| 'scroll' // Scroll to element
| 'hover' // Hover over element
| 'custom'; // Execute custom scriptUUICS provides native MCP integration for Claude and other MCP-compatible AI models.
| Class | Description |
|---|---|
MCPToolsGenerator |
Generates MCP tool definitions from page context |
MCPToolHandler |
Handles MCP tool calls and executes UI actions |
| Tool | Category | Description |
|---|---|---|
ui_scan |
context | Scan page to discover interactive elements |
ui_get_context |
context | Get current context without re-scanning |
ui_get_element |
context | Get details about a specific element |
ui_get_state |
context | Query tracked JavaScript application state |
ui_click |
interaction | Click on an element |
ui_type |
interaction | Type text into an input field |
ui_select |
interaction | Select an option from dropdown |
ui_check |
interaction | Check a checkbox |
ui_uncheck |
interaction | Uncheck a checkbox |
ui_submit |
interaction | Submit a form |
ui_scroll |
interaction | Scroll to bring element into view |
ui_focus |
interaction | Set focus on an element |
ui_hover |
interaction | Hover over element (dropdowns/tooltips) |
ui_wait_for |
utility | Wait for element or condition |
ui_screenshot |
debug | Capture element/page for visual debugging |
ui_execute_batch |
interaction | Execute multiple actions in sequence |
import { MCPToolsGenerator, MCPToolHandler } from '@angelerator/uuics-core';
const generator = new MCPToolsGenerator({
includeCoreTools: true, // Include all 16 core tools
generateDynamicTools: true, // Generate element-specific tools
maxDynamicTools: 50, // Limit dynamic tools
toolPrefix: 'ui_', // Tool name prefix
elementTypes: ['button', 'input', 'select', 'checkbox', 'link'],
customTools: [], // Add your own tools
});
// Generate tools for current page state
const tools = generator.generateTools(engine.getContext());
// Create handler to execute tool calls
const handler = new MCPToolHandler(engine);MCP automatically generates element-specific tools based on the current page:
# For a page with:
# - Submit button (#submit-btn)
# - Email input (#email)
# - Country dropdown (#country)
Generated dynamic tools:
- ui_click_submit → Clicks the Submit button
- ui_set_email → Sets the Email input value
- ui_select_country → Selects a Country option
| Condition | Description |
|---|---|
visible |
Element is visible in viewport |
hidden |
Element is hidden or removed |
exists |
Element exists in DOM |
not_exists |
Element does not exist |
enabled |
Form element is enabled |
disabled |
Form element is disabled |
// Wait for loading spinner to disappear
await handler.handleToolCall({
name: 'ui_wait_for',
input: {
selector: '.loading-spinner',
condition: 'hidden',
timeout: 10000,
}
});// Capture element bounds
const result = await handler.handleToolCall({
name: 'ui_screenshot',
input: {
selector: '#main-form',
format: 'png',
scale: 2,
}
});
// Returns: { dataUrl: 'data:image/png;base64,...', width, height }// Query tracked application state
const result = await handler.handleToolCall({
name: 'ui_get_state',
input: {
key: 'user', // Optional: specific key
include_metadata: true, // Include timestamp
}
});
// Returns: { key: 'user', value: {...}, timestamp: 1234567890 }const config: UUICSConfig = {
scan: {
interval: 0, // Auto-scan interval (0 = manual)
depth: 10, // Max DOM depth
includeHidden: false, // Include hidden elements
rootSelectors: ['#app'], // Scan specific areas only
excludeSelectors: ['.ads'] // Skip certain elements
},
track: {
mutations: true, // Track DOM mutations
clicks: true, // Track click events
changes: true, // Track input changes
debounceDelay: 100 // Debounce delay (ms)
},
state: {
enabled: true, // Enable state tracking
exclude: ['*password*'] // Exclude sensitive fields
},
performance: {
enableCache: true, // Cache scanned elements
maxElements: 1000 // Max elements to scan
}
};# Page Context
Page: My Application
URL: https://example.com
## Interactive Elements
### Inputs (3)
- **Email** → `#email`
- **Password** → `#password`
- **Name** → `#name`
### Buttons (2)
- **Submit** → `#submit-btn`
- **Cancel** → `#cancel-btn`
### Selects (1)
- **Country** [OPTIONS: "USA" (value: us), "UK" (value: uk)] → `#country`
## Application State
- user: { name: "John", loggedIn: true }
{
"url": "https://example.com",
"title": "My Application",
"elements": [
{
"type": "input",
"selector": "#email",
"label": "Email",
"value": ""
}
],
"actions": [
{ "type": "setValue", "target": "#email" },
{ "type": "click", "target": "#submit-btn" }
],
"state": {
"user": { "name": "John" }
}
}{
"tools": [
{
"type": "function",
"function": {
"name": "ui_click",
"parameters": {
"properties": {
"target": { "enum": ["#submit-btn", "#cancel-btn"] }
}
}
}
}
]
}uuics/
├── packages/
│ └── core/ # @angelerator/uuics-core
│ ├── src/
│ │ ├── scanner/ # DOM scanning
│ │ ├── tracker/ # Mutation & state tracking
│ │ ├── aggregator/ # Context aggregation
│ │ ├── serializer/ # Output formatting
│ │ ├── executor/ # Action execution
│ │ ├── mcp/ # MCP (Model Context Protocol) support
│ │ │ ├── types.ts
│ │ │ ├── MCPToolsGenerator.ts
│ │ │ ├── MCPToolHandler.ts
│ │ │ └── index.ts
│ │ └── UUICSEngine.ts
│ └── package.json
│
└── examples/
├── react-app/ # React + Claude integration
│ ├── src/
│ │ ├── UUICSProvider.tsx # React context
│ │ ├── ClaudeAdapter.ts # Claude integration
│ │ └── ...
│ └── claude-cli-proxy.cjs # Proxy for Claude Code
│
└── vanilla/ # Vanilla JavaScript example
└── index.html
# Clone the repository
git clone https://github.com/Angelerator/UUICS.git
cd UUICS
# Install dependencies
pnpm install
# Build packages
pnpm build# Terminal 1: Start the React app
cd examples/react-app
pnpm dev
# Opens at http://localhost:5173
# Terminal 2: Start Claude proxy (uses Claude Code subscription)
cd examples/react-app
node claude-cli-proxy.cjs
# Proxy runs at http://localhost:3100Then open the app and click the 🤖 button to chat with Claude!
cd examples/vanilla
pnpm dev
# Opens at http://localhost:5173// Track objects with automatic change detection
const user = uuics.trackState('user', {
name: 'John',
preferences: { theme: 'dark' }
});
user.name = 'Jane'; // Automatically tracked!
// Register computed values
uuics.registerState('metrics', () => ({
pageViews: analytics.getPageViews(),
sessionDuration: Date.now() - startTime
}));// Scan only specific areas
await uuics.scan(null, {
rootSelectors: ['#main-form', '#sidebar'],
excludeSelectors: ['.advertisement', 'footer']
});const uuics = new UUICSEngine({
state: {
enabled: true,
exclude: ['*password*', '*token*', '*secret*', '*key*']
}
});
// Sensitive fields automatically excluded from context
const auth = uuics.trackState('auth', {
username: 'john', // ✅ Included
password: 'secret123', // ❌ Excluded as '[EXCLUDED]'
apiKey: 'sk-abc123' // ❌ Excluded as '[EXCLUDED]'
});// Execute multiple actions in sequence
const results = await uuics.executeBatch([
{ action: 'click', target: '#menu-button' },
{ action: 'click', target: '#settings' },
{ action: 'setValue', target: '#theme', parameters: { value: 'dark' } },
{ action: 'click', target: '#save' }
]);
results.forEach((r, i) => {
console.log(`Step ${i + 1}: ${r.success ? '✓' : '✗'} ${r.message}`);
});- AI-Powered Testing: Automated testing with natural language
- Intelligent Automation: Smart bots that understand page context
- Accessibility Tools: AI assistants for users with disabilities
- Form Auto-Fill: Context-aware form completion
- Browser Extensions: AI-powered browser tools
- RPA: Intelligent workflow automation
If you were using the separate packages (@angelerator/uuics-react, @angelerator/uuics-models-claude, @angelerator/uuics-models-openai), these have been deprecated. The React integration and model adapters are now provided as examples that you can copy into your project.
See the examples/react-app/ directory for:
UUICSProvider.tsx- React context and hooksClaudeAdapter.ts- Claude integration
Contributions are welcome!
- Fork the repository
- Create a feature branch:
git checkout -b feature/amazing-feature - Commit changes:
git commit -m 'feat: add amazing feature' - Push to branch:
git push origin feature/amazing-feature - Open a Pull Request
MIT - see LICENSE for details.
- NPM: @angelerator/uuics-core
- GitHub: Angelerator/UUICS
- Issues: GitHub Issues