diff --git a/packages/llmpane-py/README.md b/packages/llmpane-py/README.md index 221528f..11ca6cc 100644 --- a/packages/llmpane-py/README.md +++ b/packages/llmpane-py/README.md @@ -2,6 +2,14 @@ A lightweight, embeddable chat UI component for Python/FastAPI backends serving LLM features. +## Features + +- **Pydantic AI integration** — `ChatSession` wraps any Pydantic AI agent with conversation persistence and streaming +- **Tool calling support** — Automatic tool execution with status streaming to the frontend +- **Conversation persistence** — Pluggable stores (in-memory, Redis, SQL) for multi-turn conversations +- **Multimodal support** — Send images alongside text for vision-capable models +- **Type-safe streaming** — Pydantic models for requests, responses, and SSE chunks + ## Installation ```bash @@ -10,8 +18,11 @@ uv add llmpane # With FastAPI support uv add llmpane[fastapi] +# With Pydantic AI agent support +uv add llmpane[fastapi,agent] + # Or with pip -pip install llmpane[fastapi] +pip install llmpane[fastapi,agent] ``` ## Quick Start @@ -131,6 +142,75 @@ async def chat(request: ChatRequest): ) ``` +## Pydantic AI Agents + +The `agent` extra provides deep integration with [Pydantic AI](https://ai.pydantic.dev/): + +```bash +uv add llmpane[fastapi,agent] +``` + +### ChatSession + +`ChatSession` combines a Pydantic AI agent with conversation persistence: + +```python +from pydantic_ai import Agent +from llmpane import ChatRequest, create_sse_response +from llmpane.agent import ChatSession + +agent = Agent( + "anthropic:claude-sonnet-4-20250514", + system_prompt="You are a helpful assistant.", +) + +# Tools are defined on the agent as usual +@agent.tool_plain +def get_weather(city: str) -> str: + """Get the current weather for a city.""" + return f"Sunny, 72°F in {city}" + +session = ChatSession(agent=agent) + +@app.post("/chat") +async def chat(request: ChatRequest): + # Handles persistence, tool execution, and streaming automatically + return create_sse_response( + session.run(request.conversation_id, request.message) + ) +``` + +### Conversation Stores + +Conversations are persisted automatically. The default is in-memory, but you can use Redis or SQL: + +```python +from llmpane.store import create_store + +# In-memory (default) +store = create_store("memory", max_conversations=1000) + +# Redis +store = create_store("redis", url="redis://localhost:6379") + +# SQLite/PostgreSQL +store = create_store("sql", url="sqlite+aiosqlite:///chats.db") + +session = ChatSession(agent=agent, store=store) +``` + +### Tool Call Streaming + +When the agent calls tools, llmpane streams `ToolUseMetadata` to the frontend so you can show pending/completed states: + +```python +from llmpane.agent import ToolUseMetadata, ToolStatus + +# Streamed automatically during agent execution: +# {"metadata": {"tools": [{"name": "get_weather", "status": "pending"}]}} +# {"metadata": {"tools": [{"name": "get_weather", "status": "completed", "result": "Sunny..."}]}} +``` + ## Custom Metadata Use generics to add type-safe custom metadata: