Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
82 changes: 81 additions & 1 deletion packages/llmpane-py/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,14 @@

A lightweight, embeddable chat UI component for Python/FastAPI backends serving LLM features.

## Features

- **Pydantic AI integration** — `ChatSession` wraps any Pydantic AI agent with conversation persistence and streaming
- **Tool calling support** — Automatic tool execution with status streaming to the frontend
- **Conversation persistence** — Pluggable stores (in-memory, Redis, SQL) for multi-turn conversations
- **Multimodal support** — Send images alongside text for vision-capable models
- **Type-safe streaming** — Pydantic models for requests, responses, and SSE chunks

## Installation

```bash
Expand All @@ -10,8 +18,11 @@ uv add llmpane
# With FastAPI support
uv add llmpane[fastapi]

# With Pydantic AI agent support
uv add llmpane[fastapi,agent]

# Or with pip
pip install llmpane[fastapi]
pip install llmpane[fastapi,agent]
```

## Quick Start
Expand Down Expand Up @@ -131,6 +142,75 @@ async def chat(request: ChatRequest):
)
```

## Pydantic AI Agents

The `agent` extra provides deep integration with [Pydantic AI](https://ai.pydantic.dev/):

```bash
uv add llmpane[fastapi,agent]
```

### ChatSession

`ChatSession` combines a Pydantic AI agent with conversation persistence:

```python
from pydantic_ai import Agent
from llmpane import ChatRequest, create_sse_response
from llmpane.agent import ChatSession

agent = Agent(
"anthropic:claude-sonnet-4-20250514",
system_prompt="You are a helpful assistant.",
)

# Tools are defined on the agent as usual
@agent.tool_plain
def get_weather(city: str) -> str:
"""Get the current weather for a city."""
return f"Sunny, 72°F in {city}"

session = ChatSession(agent=agent)

@app.post("/chat")
async def chat(request: ChatRequest):
# Handles persistence, tool execution, and streaming automatically
return create_sse_response(
session.run(request.conversation_id, request.message)
)
```

### Conversation Stores

Conversations are persisted automatically. The default is in-memory, but you can use Redis or SQL:

```python
from llmpane.store import create_store

# In-memory (default)
store = create_store("memory", max_conversations=1000)

# Redis
store = create_store("redis", url="redis://localhost:6379")

# SQLite/PostgreSQL
store = create_store("sql", url="sqlite+aiosqlite:///chats.db")

session = ChatSession(agent=agent, store=store)
```

### Tool Call Streaming

When the agent calls tools, llmpane streams `ToolUseMetadata` to the frontend so you can show pending/completed states:

```python
from llmpane.agent import ToolUseMetadata, ToolStatus

# Streamed automatically during agent execution:
# {"metadata": {"tools": [{"name": "get_weather", "status": "pending"}]}}
# {"metadata": {"tools": [{"name": "get_weather", "status": "completed", "result": "Sunny..."}]}}
```

## Custom Metadata

Use generics to add type-safe custom metadata:
Expand Down