refactor/mcp-server-design

Containerfile

@@ -35,16 +35,10 @@ ENV TORERO_MCP_TRANSPORT_TYPE=sse
 ENV TORERO_MCP_TRANSPORT_HOST=0.0.0.0
 ENV TORERO_MCP_TRANSPORT_PORT=8080
 ENV TORERO_MCP_TRANSPORT_PATH=/sse
-ENV TORERO_API_BASE_URL=http://localhost:8000
-ENV TORERO_API_TIMEOUT=30
+ENV TORERO_CLI_TIMEOUT=30
 ENV TORERO_LOG_LEVEL=INFO
-ENV TORERO_MCP_PID_FILE=/tmp/torero-mcp.pid
 ENV TORERO_MCP_LOG_FILE=/home/admin/.torero-mcp.log
 
-# api server is disabled by default
-ENV ENABLE_API=false
-ENV API_PORT=8000
-
 # ui server is disabled by default
 ENV ENABLE_UI=false
 ENV UI_PORT=8001
@@ -60,12 +54,11 @@ COPY configure.sh /configure.sh
 COPY entrypoint.sh /entrypoint.sh
 
 # copy torero projects to image
-COPY opt/torero-api /opt/torero-api
 COPY opt/torero-ui /opt/torero-ui
 COPY opt/torero-mcp /opt/torero-mcp
 
 # install Python dependencies at build time
-RUN pip install --no-cache-dir -e /opt/torero-api /opt/torero-ui /opt/torero-mcp
+RUN pip install --no-cache-dir -e /opt/torero-ui /opt/torero-mcp
 
 # set up Django static files at build time
 ENV DJANGO_SETTINGS_MODULE=torero_ui.settings
@@ -85,9 +78,6 @@ RUN chmod +x /configure.sh && /configure.sh && \
 # expose ssh port (only used if SSH is enabled)
 EXPOSE 22
 
-# expose API port (only used if API is enabled)
-EXPOSE 8000
-
 # expose UI port (only used if UI is enabled)
 EXPOSE 8001
 

README.md

@@ -12,8 +12,8 @@ Container image for [torero](https://torero.dev), built using vendor-neutral Con
 - Includes _torero_ installed and ready to go
 - Optional [OpenTofu](https://opentofu.org/) installation at runtime
 - Optional SSH administration for testing convenience + labs
-- Integrated torero-api service for REST API access
-- Integrated torero-mcp service for Model Context Protocol server
+- Integrated torero-mcp service for Model Context Protocol server with direct CLI integration
+- Optional torero-ui service for web-based dashboard
 
 ## Inspiration
 Managing and automating a hybrid, _multi-vendor_ infrastrcuture that encompasses _on-premises systems, private and public clouds, edge computing, and colocation environments_ poses significant challenges. How can you experiment to _learn_ without breaking things? How can you test new and innovative products like _torero_ on the test bench without friction to help in your evaluation? How do you test the behavior of changes in lower level environments before making changes to production? I use [containerlab](https://containerlab.dev/) for all of the above! This project makes it easy to insert _torero_ in your _containerlab_ topology file, connect to the container, and run your experiments -- the sky is the limit!
@@ -28,7 +28,7 @@ docker run -d -p 2222:22 ghcr.io/torerodev/torero-container:latest
 
 ![docker cli](./img/docker-cli.gif)
 
-### docker compose _(with custom OpenTofu version)_
+### docker compose _(with MCP server enabled)_
 ```yaml
 ---
 services:
@@ -37,14 +37,13 @@ services:
     container_name: torero
     ports:
       - "22:22"                  # use when ENABLE_SSH_ADMIN=true
-      - "8000:8000"              # use when ENABLE_API=true
       - "8080:8080"              # use when ENABLE_MCP=true
+      - "8001:8001"              # use when ENABLE_UI=true
     volumes:
       - ./data:/home/admin/data
     environment:
-      - ENABLE_API=true          # enable and run torero api
-      - API_PORT=8000            # api port
       - ENABLE_MCP=true          # enable torero-mcp server
+      - ENABLE_UI=true           # enable torero web dashboard
       - ENABLE_SSH_ADMIN=true    # enable ssh admin at runtime
       - OPENTOFU_VERSION=1.9.0   # override OpenTofu version at runtime (optional)
       - PYTHON_VERSION=3.13.0
@@ -72,28 +71,24 @@ The following environment variables can be set at runtime:
 
 | Variable | Default | Description |
 |----------|---------|-------------|
-| `ENABLE_API`       | `false`  | Enable torero API          |
-| `API_PORT`         | `8000`   | Set API port               |
-| `ENABLE_UI`        | `false`  | Enable torero UI           |
-| `UI_PORT`          | `8001`   | Set UI port                |
-| `ENABLE_MCP`       | `false`  | Enable torero MCP server   |
-| `ENABLE_SSH_ADMIN` | `false`  | Enable SSH admin user      |
-| `OPENTOFU_VERSION` | `1.10.5` | Override OpenTofu version  |
-| `PYTHON_VERSION`   | `3.13.0` | Set Python version         |
+| `ENABLE_UI`        | `false`  | Enable torero web dashboard    |
+| `UI_PORT`          | `8001`   | Set UI port                    |
+| `ENABLE_MCP`       | `false`  | Enable torero MCP server       |
+| `ENABLE_SSH_ADMIN` | `false`  | Enable SSH admin user          |
+| `OPENTOFU_VERSION` | `1.10.5` | Override OpenTofu version      |
+| `PYTHON_VERSION`   | `3.13.0` | Set Python version             |
 
 #### MCP Server Environment Variables
 When `ENABLE_MCP=true`, the following additional environment variables are available:
 
 | Variable | Default | Description |
 |----------|---------|-------------|
-| `TORERO_MCP_TRANSPORT_TYPE` | `sse`                         | MCP transport type              |
+| `TORERO_MCP_TRANSPORT_TYPE` | `sse`                         | MCP transport type (stdio, sse, streamable_http) |
 | `TORERO_MCP_TRANSPORT_HOST` | `0.0.0.0`                     | MCP server host                 |
 | `TORERO_MCP_TRANSPORT_PORT` | `8080`                        | MCP server port                 |
 | `TORERO_MCP_TRANSPORT_PATH` | `/sse`                        | SSE endpoint path               |
-| `TORERO_API_BASE_URL`       | `http://localhost:8000`       | torero API base URL             |
-| `TORERO_API_TIMEOUT`        | `30`                          | API request timeout in seconds  |
+| `TORERO_CLI_TIMEOUT`        | `30`                          | CLI command timeout in seconds  |
 | `TORERO_LOG_LEVEL`          | `INFO`                        | Logging level                   |
-| `TORERO_MCP_PID_FILE`       | `/tmp/torero-mcp.pid`         | PID file location               |
 | `TORERO_MCP_LOG_FILE`       | `/home/admin/.torero-mcp.log` | Log file path                   |
 
 ### OpenTofu Version Management
@@ -113,6 +108,28 @@ services:
 
 The container will automatically download and install the requested version at startup if it differs from the pre-installed version. If no `OPENTOFU_VERSION` is specified, it uses the pre-installed version (1.10.5).
 
+## MCP Server Integration
+
+The torero-mcp service provides Model Context Protocol integration with direct CLI access for optimal performance. This allows AI assistants and other MCP clients to interact with torero services without the overhead of an intermediate REST API.
+
+### Key Features:
+- **Direct CLI Integration**: MCP tools execute torero commands directly for optimal performance
+- **Comprehensive Tool Coverage**: Full access to torero services, repositories, decorators, secrets, and execution capabilities
+- **Multiple Transport Options**: Support for stdio, Server-Sent Events (SSE), and streamable HTTP
+- **Automatic Tool Discovery**: Dynamic loading of MCP tools with proper parameter mapping
+
+### Connecting MCP Clients:
+```bash
+# For stdio transport (direct process communication)
+torero-mcp run --transport stdio
+
+# For SSE transport (web-based clients)
+torero-mcp run --transport sse --host 0.0.0.0 --port 8080
+
+# For streamable HTTP transport
+torero-mcp run --transport streamable_http --host 0.0.0.0 --port 8080
+```
+
 ## Local Development
 
 ### Using docker-compose for local testing
@@ -150,6 +167,11 @@ docker compose -f docker-compose.dev.yml down -v
 ## Container Architecture
 This project uses vendor-neutral Containerfile specifications for maximum compatibility across container runtimes. The image is built and distributed through GitHub Container Registry (GHCR) for reliable access and version management.
 
+The architecture has been simplified to use direct CLI integration for optimal performance:
+- **torero CLI**: Core torero functionality accessed directly
+- **torero-mcp**: MCP server providing direct CLI integration for AI assistants
+- **torero-ui**: Optional web dashboard for execution monitoring (uses CLI wrapper data)
+
 ### Multi-Architecture Support
 The container images support both AMD64 and ARM64 architectures. The appropriate architecture will be selected automatically based on your host system when pulling the image.
 

docker-compose.dev.yml

@@ -8,14 +8,11 @@ services:
     container_name: torero
     ports:
       - "22:22"
-      - "8000:8000"
       - "8001:8001"
       - "8080:8080"
     volumes:
       - ./data:/home/admin/data
     environment:
-      - ENABLE_API=true
-      - API_PORT=8000
       - ENABLE_UI=true
       - UI_PORT=8001
       - UI_REFRESH_INTERVAL=15

docker-compose.yml

@@ -5,14 +5,11 @@ services:
     container_name: torero
     ports:
       - "22:22"
-      - "8000:8000"
       - "8001:8001"
       - "8080:8080"
     volumes:
       - ./data:/home/admin/data
     environment:
-      - ENABLE_API=true
-      - API_PORT=8000
       - ENABLE_UI=true
       - UI_PORT=8001
       - UI_REFRESH_INTERVAL=15
@@ -25,4 +22,4 @@ services:
       timeout: 10s
       retries: 3
       start_period: 5s
-...
+...

docs/README.md

@@ -3,21 +3,33 @@ This directory contains documentation for the torero Container project.
 
 ## Documentation Structure
 - **[tests.md](./tests.md)** - Step-by-step guide for running local tests
-- **[openapi.json](./openapi.json)** - OpenAPI specification for the torero API (JSON format)
-- **[openapi.yaml](./openapi.yaml)** - OpenAPI specification for the torero API (YAML format)
+- **[global-context.md](../config/global-context.md)** - Comprehensive architecture documentation
 
 ## Components
 
-### torero API
-RESTful API service that provides programmatic access to torero functionality. Located in `opt/torero-api/`.
-
 ### torero MCP
 Model Context Protocol server for integration with AI assistants. Located in `opt/torero-mcp/`.
+- Direct CLI integration for optimal performance
+- FastMCP-based implementation
+- Support for SSE, stdio, and streamable HTTP transports
 
 ### torero UI
 Web-based user interface for managing torero operations. Located in `opt/torero-ui/`.
+- Django-based dashboard with auto-refresh
+- Automatic service synchronization
+- Direct CLI integration via ToreroCliClient
+
+## Architecture Highlights
+
+The torero-container has been optimized for performance and simplicity:
+- **Direct CLI Integration**: Both MCP and UI services interact directly with the torero CLI
+- **Automatic Synchronization**: UI automatically detects service changes from any source
+- **Universal CLI Wrapper**: Captures all executions for comprehensive tracking
+- **Supervisor Management**: All services managed through a single supervisor configuration
 
 ## Quick Links
 - [Running Tests](./tests.md)
-- [API Documentation](./openapi.yaml)
-- [Main README](../README.md)
+- [Architecture Documentation](../config/global-context.md)
+- [Main README](../README.md)
+- [MCP Service Documentation](../opt/torero-mcp/README.md)
+- [UI Service Documentation](../opt/torero-ui/README.md)