From 97f0583c662e954349241e2490cf49b79beef5ce Mon Sep 17 00:00:00 2001 From: Charith Nuwan Bimsara <59943919+nuwangeek@users.noreply.github.com> Date: Tue, 2 Dec 2025 12:59:42 +0530 Subject: [PATCH] Complete langfuse setup (#182) * partialy completes prompt refiner * integrate prompt refiner with llm_config_module * fixed ruff lint issues * complete prompt refiner, chunk retriver and reranker * remove unnesessary comments * updated .gitignore * Remove data_sets from tracking * update .gitignore file * complete vault setup and response generator * remove ignore comment * removed old modules * fixed merge conflicts * Vault Authentication token handling (#154) (#70) * partialy completes prompt refiner * integrate prompt refiner with llm_config_module * fixed ruff lint issues * complete prompt refiner, chunk retriver and reranker * remove unnesessary comments * updated .gitignore * Remove data_sets from tracking * update .gitignore file * complete vault setup and response generator * remove ignore comment * removed old modules * fixed merge conflicts * added initial setup for the vector indexer * initial llm orchestration service update with context generation * added new endpoints * vector indexer with contextual retrieval * fixed requested changes * fixed issue * initial diff identifier setup * uncommment docker compose file * added test endpoint for orchestrate service * fixed ruff linting issue * Rag 103 budget related schema changes (#41) * Refactor llm_connections table: update budget tracking fields and reorder columns * Add budget threshold fields and logic to LLM connection management * Enhance budget management: update budget status logic, adjust thresholds, and improve form handling for LLM connections * resolve pr comments & refactoring * rename commonUtils --------- * Rag 93 update connection status (#47) * Refactor llm_connections table: update budget tracking fields and reorder columns * Add budget threshold fields and logic to LLM connection management * Enhance budget management: update budget status logic, adjust thresholds, and improve form handling for LLM connections * resolve pr comments & refactoring * rename commonUtils * Implement LLM connection status update functionality with API integration and UI enhancements --------- * Rag 99 production llm connections logic (#46) * Refactor llm_connections table: update budget tracking fields and reorder columns * Add budget threshold fields and logic to LLM connection management * Enhance budget management: update budget status logic, adjust thresholds, and improve form handling for LLM connections * resolve pr comments & refactoring * rename commonUtils * Add production connection retrieval and update related components * Implement LLM connection environment update and enhance connection management logic --------- * Rag 119 endpoint to update used budget (#42) * Refactor llm_connections table: update budget tracking fields and reorder columns * Add budget threshold fields and logic to LLM connection management * Enhance budget management: update budget status logic, adjust thresholds, and improve form handling for LLM connections * resolve pr comments & refactoring * Add functionality to update used budget for LLM connections with validation and response handling * Implement budget threshold checks and connection deactivation logic in update process * resolve pr comments --------- * Rag 113 warning and termination banners (#43) * Refactor llm_connections table: update budget tracking fields and reorder columns * Add budget threshold fields and logic to LLM connection management * Enhance budget management: update budget status logic, adjust thresholds, and improve form handling for LLM connections * resolve pr comments & refactoring * Add budget status check and update BudgetBanner component * rename commonUtils * resove pr comments --------- * rag-105-reset-used-budget-cron-job (#44) * Refactor llm_connections table: update budget tracking fields and reorder columns * Add budget threshold fields and logic to LLM connection management * Enhance budget management: update budget status logic, adjust thresholds, and improve form handling for LLM connections * resolve pr comments & refactoring * Add cron job to reset used budget * rename commonUtils * resolve pr comments * Remove trailing slash from vault/agent-out in .gitignore --------- * Rag 101 budget check functionality (#45) * Refactor llm_connections table: update budget tracking fields and reorder columns * Add budget threshold fields and logic to LLM connection management * Enhance budget management: update budget status logic, adjust thresholds, and improve form handling for LLM connections * resolve pr comments & refactoring * rename commonUtils * budget check functionality --------- * gui running on 3003 issue fixed * gui running on 3003 issue fixed (#50) * added get-configuration.sqpl and updated llmconnections.ts * Add SQL query to retrieve configuration values * Hashicorp key saving (#51) * gui running on 3003 issue fixed * Add SQL query to retrieve configuration values --------- * Remove REACT_APP_NOTIFICATION_NODE_URL variable Removed REACT_APP_NOTIFICATION_NODE_URL environment variable. * added initil diff identifier functionality * test phase1 * Refactor inference and connection handling in YAML and TypeScript files * fixes (#52) * gui running on 3003 issue fixed * Add SQL query to retrieve configuration values * Refactor inference and connection handling in YAML and TypeScript files --------- * Add entry point script for Vector Indexer with command line interface * fix (#53) * gui running on 3003 issue fixed * Add SQL query to retrieve configuration values * Refactor inference and connection handling in YAML and TypeScript files * Add entry point script for Vector Indexer with command line interface --------- * diff fixes * uncomment llm orchestration service in docker compose file * complete vector indexer * Add YAML configurations and scripts for managing vault secrets * Add vault secret management functions and endpoints for LLM connections * Add Test Production LLM page with messaging functionality and styles * fixed issue * fixed merge conflicts * fixed issue * fixed issue * updated with requested chnages * fixed test ui endpoint request responses schema issue * fixed dvc path issue * added dspy optimization * filters fixed * refactor: restructure llm_connections table for improved configuration and tracking * feat: enhance LLM connection handling with AWS and Azure embedding credentials * fixed issues * refactor: remove redundant Azure and AWS credential assignments in vault secret functions * fixed issue * intial vault setup script * complete vault authentication handling * review requested change fix * fixed issues according to the pr review * fixed issues in docker compose file relevent to pr review --------- Co-authored-by: Charith Nuwan Bimsara <59943919+nuwangeek@users.noreply.github.com> Co-authored-by: erangi-ar * fixed number chunk issue * fixed ruff format issue * complete inference result update and budget updating * fixed issues * fixed ruff format issue * fixed endpoint issue * fixed format issues * fixed issue * fixed issue * complete langfuse setup * fixed review comments --------- Co-authored-by: erangi-ar <111747955+erangi-ar@users.noreply.github.com> Co-authored-by: erangi-ar --- README.md | 16 ++ env.example | 24 ++- src/llm_orchestration_service.py | 2 +- src/utils/production_store.py | 4 +- store-langfuse-secrets.sh | 163 +++++++++++++++++++++ vault/README.md | 242 ------------------------------- 6 files changed, 194 insertions(+), 257 deletions(-) create mode 100644 store-langfuse-secrets.sh delete mode 100644 vault/README.md diff --git a/README.md b/README.md index 08ef0b6..fd6ab79 100644 --- a/README.md +++ b/README.md @@ -24,3 +24,19 @@ The **BYK-RAG Module** is part of the Burokratt ecosystem, designed to provide * - External **Langfuse dashboard** for API usage, inference trends, cost analysis, and performance logs. - Agencies can configure cost alerts and view alerts via LLM Alerts UI. - Logs integrated with **Grafana Loki**. + +### Storing Langfuse Secrets + +1. **Generate API keys from Langfuse UI** (Settings → Project → API Keys) + +2. **Copy the script to vault container:** +```bash +docker cp store-langfuse-secrets.sh vault:/tmp/store-langfuse-secrets.sh +``` + +3. **Execute the script with your API keys:** +```bash +docker exec -e LANGFUSE_INIT_PROJECT_PUBLIC_KEY= \ + -e LANGFUSE_INIT_PROJECT_SECRET_KEY= \ + vault sh -c "chmod +x /tmp/store-langfuse-secrets.sh && /tmp/store-langfuse-secrets.sh" +``` diff --git a/env.example b/env.example index f77f0f8..65f4f1f 100644 --- a/env.example +++ b/env.example @@ -6,8 +6,8 @@ S3_ENDPOINT_NAME=minio:9000 S3_DATA_BUCKET_PATH=resources S3_DATA_BUCKET_NAME=rag-search FS_DATA_DIRECTORY_PATH=/app -S3_SECRET_ACCESS_KEY=changeme -S3_ACCESS_KEY_ID=changeme +S3_SECRET_ACCESS_KEY=minioadmin +S3_ACCESS_KEY_ID=minioadmin S3_HEALTH_ENDPOINT=http://minio:9000/minio/health/live MINIO_BROWSER_REDIRECT_URL=http://localhost:9091 GF_SECURITY_ADMIN_USER=admin @@ -16,8 +16,8 @@ GF_USERS_ALLOW_SIGN_UP=false PORT=3000 POSTGRES_USER=postgres POSTGRES_PASSWORD=dbadmin -POSTGRES_DB=rag-search -NEXTAUTH_URL=http://localhost:3000 +POSTGRES_DB=rag-search-langfuse +NEXTAUTH_URL=http://localhost:3005 DATABASE_URL=postgresql://postgres:dbadmin@rag_search_db:5432/rag-search SALT=changeme ENCRYPTION_KEY=changeme @@ -27,7 +27,7 @@ LANGFUSE_ENABLE_EXPERIMENTAL_FEATURES=true CLICKHOUSE_MIGRATION_URL=clickhouse://clickhouse:9000 CLICKHOUSE_URL=http://clickhouse:8123 CLICKHOUSE_USER=clickhouse -CLICKHOUSE_PASSWORD=changeme +CLICKHOUSE_PASSWORD=clickhouse CLICKHOUSE_CLUSTER_ENABLED=false LANGFUSE_USE_AZURE_BLOB=false LANGFUSE_S3_EVENT_UPLOAD_BUCKET=rag-search @@ -41,7 +41,7 @@ LANGFUSE_S3_MEDIA_UPLOAD_BUCKET=rag-search LANGFUSE_S3_MEDIA_UPLOAD_REGION=auto LANGFUSE_S3_MEDIA_UPLOAD_ACCESS_KEY_ID=changeme LANGFUSE_S3_MEDIA_UPLOAD_SECRET_ACCESS_KEY=changeme -LANGFUSE_S3_MEDIA_UPLOAD_ENDPOINT=http://localhost:9090 +LANGFUSE_S3_MEDIA_UPLOAD_ENDPOINT=http://minio:9000 LANGFUSE_S3_MEDIA_UPLOAD_FORCE_PATH_STYLE=true LANGFUSE_S3_MEDIA_UPLOAD_PREFIX=langfuse/media/ LANGFUSE_S3_BATCH_EXPORT_ENABLED=false @@ -49,7 +49,7 @@ LANGFUSE_S3_BATCH_EXPORT_BUCKET=rag-search LANGFUSE_S3_BATCH_EXPORT_PREFIX=langfuse/exports/ LANGFUSE_S3_BATCH_EXPORT_REGION=auto LANGFUSE_S3_BATCH_EXPORT_ENDPOINT=http://minio:9000 -LANGFUSE_S3_BATCH_EXPORT_EXTERNAL_ENDPOINT=http://localhost:9090 +LANGFUSE_S3_BATCH_EXPORT_EXTERNAL_ENDPOINT=http://minio:9000 LANGFUSE_S3_BATCH_EXPORT_ACCESS_KEY_ID=changeme LANGFUSE_S3_BATCH_EXPORT_SECRET_ACCESS_KEY=changeme LANGFUSE_S3_BATCH_EXPORT_FORCE_PATH_STYLE=true @@ -64,9 +64,7 @@ REDIS_TLS_CERT=/certs/redis.crt REDIS_TLS_KEY=/certs/redis.key EMAIL_FROM_ADDRESS= SMTP_CONNECTION_URL= -AZURE_OPENAI_ENDPOINT=your_azure_openai_endpoint_here -AZURE_OPENAI_API_KEY=your_azure_openai_api_key_here -AZURE_OPENAI_DEPLOYMENT_NAME=gpt-4o-mini -AWS_REGION=us-east-1 -AWS_ACCESS_KEY_ID=your_aws_access_key_here -AWS_SECRET_ACCESS_KEY=your_aws_secret_key_here \ No newline at end of file +VAULT_ADDR=http://localhost:8200 +S3_FERRY_URL=http://rag-s3-ferry:3000/v1/files/copy +DATASETS_PATH=/app/datasets +METADATA_FILENAME=processed-metadata.json \ No newline at end of file diff --git a/src/llm_orchestration_service.py b/src/llm_orchestration_service.py index 59417d5..a6cc98e 100644 --- a/src/llm_orchestration_service.py +++ b/src/llm_orchestration_service.py @@ -1884,7 +1884,7 @@ def _refine_user_prompt( ) output_json = validated_output.model_dump() logger.info( - f"Prompt refinement output: {json.dumps(output_json, indent=2)}" + f"Prompt refinement output: {json_module.dumps(output_json, indent=2)}" ) logger.info("Prompt refinement completed successfully") diff --git a/src/utils/production_store.py b/src/utils/production_store.py index 4d15f21..f0f30fe 100644 --- a/src/utils/production_store.py +++ b/src/utils/production_store.py @@ -12,7 +12,9 @@ import requests import aiohttp from src.utils.connection_id_fetcher import get_connection_id_fetcher -from ..llm_orchestrator_config.llm_ochestrator_constants import RAG_SEARCH_RUUTER_PUBLIC +from src.llm_orchestrator_config.llm_ochestrator_constants import ( + RAG_SEARCH_RUUTER_PUBLIC, +) class ProductionInferenceStore: diff --git a/store-langfuse-secrets.sh b/store-langfuse-secrets.sh new file mode 100644 index 0000000..234457e --- /dev/null +++ b/store-langfuse-secrets.sh @@ -0,0 +1,163 @@ +#!/bin/sh +set -e + +# ============================================================================ +# Langfuse Secrets Storage Script for Vault +# ============================================================================ +# This script stores Langfuse configuration secrets in HashiCorp Vault. +# Run this script AFTER vault-init.sh has completed successfully. +# +# Prerequisites: +# 1. Vault must be initialized and unsealed +# 2. Environment variables must be set (LANGFUSE_INIT_PROJECT_PUBLIC_KEY, etc.) +# 3. Root token must be available in /vault/file/unseal-keys.json +# +# Usage: +# ./store-langfuse-secrets.sh +# +# Or with custom values: +# LANGFUSE_INIT_PROJECT_PUBLIC_KEY=pk-xxx \ +# LANGFUSE_INIT_PROJECT_SECRET_KEY=sk-xxx \ +# LANGFUSE_HOST=http://langfuse-web:3000 \ +# ./store-langfuse-secrets.sh +# ============================================================================ + +VAULT_ADDR="${VAULT_ADDR:-http://vault:8200}" +UNSEAL_KEYS_FILE="/vault/file/unseal-keys.json" + +echo "========================================" +echo "Langfuse Secrets Storage Script" +echo "========================================" + +# Check if Vault is available +echo "Checking Vault availability..." +if ! wget -q -O- "$VAULT_ADDR/v1/sys/health" >/dev/null 2>&1; then + echo "Error: Vault is not available at $VAULT_ADDR" + echo " Please ensure Vault is running and accessible." + exit 1 +fi +echo "Vault is available" + +# Check if Vault is sealed +SEALED=$(wget -q -O- "$VAULT_ADDR/v1/sys/seal-status" | grep -o '"sealed":[^,}]*' | cut -d':' -f2) +if [ "$SEALED" = "true" ]; then + echo "Error: Vault is sealed" + echo " Please unseal Vault first using vault-init.sh or manual unseal process." + exit 1 +fi +echo "Vault is unsealed" + +# Get root token +echo "Loading Vault root token..." +if [ ! -f "$UNSEAL_KEYS_FILE" ]; then + echo "Error: Unseal keys file not found at $UNSEAL_KEYS_FILE" + echo " Please run vault-init.sh first to initialize Vault." + exit 1 +fi + +ROOT_TOKEN=$(grep -o '"root_token":"[^"]*"' "$UNSEAL_KEYS_FILE" | cut -d':' -f2 | tr -d '"') +if [ -z "$ROOT_TOKEN" ]; then + echo "Error: Could not extract root token from $UNSEAL_KEYS_FILE" + exit 1 +fi +echo "Root token loaded" + +# Check required environment variables +echo "Checking Langfuse environment variables..." +if [ -z "$LANGFUSE_INIT_PROJECT_PUBLIC_KEY" ]; then + echo "Error: LANGFUSE_INIT_PROJECT_PUBLIC_KEY is not set" + echo " Please set this environment variable before running the script." + echo "" + echo " Example:" + echo " export LANGFUSE_INIT_PROJECT_PUBLIC_KEY='pk-lf-...'" + exit 1 +fi + +if [ -z "$LANGFUSE_INIT_PROJECT_SECRET_KEY" ]; then + echo "Error: LANGFUSE_INIT_PROJECT_SECRET_KEY is not set" + echo " Please set this environment variable before running the script." + echo "" + echo " Example:" + echo " export LANGFUSE_INIT_PROJECT_SECRET_KEY='sk-lf-...'" + exit 1 +fi + +# Use default host if not specified +LANGFUSE_HOST="${LANGFUSE_HOST:-http://langfuse-web:3000}" + +echo "Langfuse environment variables found" +echo " Public Key: ${LANGFUSE_INIT_PROJECT_PUBLIC_KEY:0:10}..." +echo " Secret Key: ${LANGFUSE_INIT_PROJECT_SECRET_KEY:0:10}..." +echo " Host: $LANGFUSE_HOST" + +# Update Vault policy to include Langfuse secrets access +echo "" +echo "Updating llm-orchestration policy to include Langfuse secrets..." +POLICY='path "secret/metadata/llm/*" { capabilities = ["list", "delete"] } +path "secret/data/llm/*" { capabilities = ["create", "read", "update", "delete"] } +path "secret/metadata/embeddings/*" { capabilities = ["list", "delete"] } +path "secret/data/embeddings/*" { capabilities = ["create", "read", "update", "delete"] } +path "secret/metadata/langfuse/*" { capabilities = ["list", "delete"] } +path "secret/data/langfuse/*" { capabilities = ["create", "read", "update", "delete"] } +path "auth/token/lookup-self" { capabilities = ["read"] }' + +# Create JSON without jq (using printf for proper escaping) +POLICY_ESCAPED=$(printf '%s' "$POLICY" | sed 's/\\/\\\\/g' | sed 's/"/\\"/g' | sed ':a;N;$!ba;s/\n/\\n/g') +POLICY_JSON='{"policy":"'"$POLICY_ESCAPED"'"}' + +if wget -q -O- --post-data="$POLICY_JSON" \ + --header="X-Vault-Token: $ROOT_TOKEN" \ + --header='Content-Type: application/json' \ + "$VAULT_ADDR/v1/sys/policies/acl/llm-orchestration" >/dev/null 2>&1; then + echo "Policy updated successfully" +else + echo "Warning: Policy update failed (may already be updated)" +fi + +# Store Langfuse secrets in Vault +echo "" +echo "Storing Langfuse secrets in Vault..." + +# Create JSON payload +LANGFUSE_SECRET='{"data":{"public_key":"'"$LANGFUSE_INIT_PROJECT_PUBLIC_KEY"'","secret_key":"'"$LANGFUSE_INIT_PROJECT_SECRET_KEY"'","host":"'"$LANGFUSE_HOST"'"}}' + +# Store in Vault +if wget -q -O- --post-data="$LANGFUSE_SECRET" \ + --header="X-Vault-Token: $ROOT_TOKEN" \ + --header='Content-Type: application/json' \ + "$VAULT_ADDR/v1/secret/data/langfuse/config" >/dev/null 2>&1; then + echo "Langfuse secrets stored successfully" +else + echo "Error: Failed to store Langfuse secrets" + exit 1 +fi + +# Verify secrets were stored +echo "" +echo "Verifying stored secrets..." +VERIFICATION=$(wget -q -O- \ + --header="X-Vault-Token: $ROOT_TOKEN" \ + "$VAULT_ADDR/v1/secret/data/langfuse/config" 2>/dev/null) + +if echo "$VERIFICATION" | grep -q '"public_key"'; then + echo "Secrets verified successfully" + echo "" + echo "========================================" + echo "SUCCESS" + echo "========================================" + echo "Langfuse secrets have been stored in Vault at:" + echo " Path: secret/data/langfuse/config" + echo "" + echo "The LLM Orchestration Service will now be able to:" + echo " - Initialize Langfuse client automatically" + echo " - Track LLM usage and costs" + echo " - Monitor orchestration pipelines" + echo "" + echo "Next steps:" + echo " 1. Restart llm-orchestration-service container (if running)" + echo " 2. Check logs for 'Langfuse client initialized successfully'" + echo "========================================" +else + echo "Warning: Secrets stored but verification failed" + echo " The secrets may still be accessible, but verification could not confirm." +fi diff --git a/vault/README.md b/vault/README.md deleted file mode 100644 index f6890b4..0000000 --- a/vault/README.md +++ /dev/null @@ -1,242 +0,0 @@ -# HashiCorp Vault Setup for LLM Orchestration Service - -This document explains how to set up and configure HashiCorp Vault for the LLM Orchestration Service, including Vault Agent for automatic token management. - -## 🏗️ Architecture Overview - -``` -┌─────────────────┐ ┌──────────────────┐ ┌─────────────────────┐ -│ Vault Server │ │ Vault Agent │ │ LLM Orchestration │ -│ │ │ │ │ Service │ -│ - Storage │◄───┤ - AppRole Auth │◄───┤ │ -│ - Auth Methods │ │ - Token Sink │ │ - Reads from │ -│ - KV v2 Engine │ │ - Auto Renewal │ │ /run/vault/token │ -│ - Policies │ │ │ │ │ -└─────────────────┘ └──────────────────┘ └─────────────────────┘ -``` - -## 📋 Prerequisites - -1. **Docker and Docker Compose** installed -2. **PowerShell** (Windows) or **Bash** (Linux/Mac) -3. **Network connectivity** between containers - -## 🚀 Quick Start - -### Step 1: Start Vault Server - -```bash -# Start only the Vault server first -docker-compose up -d vault -``` - -### Step 2: Initialize and Configure Vault - -**For Windows (PowerShell):** -```powershell -.\setup-vault.ps1 -``` - -**For Linux/Mac (Bash):** -```bash -chmod +x setup-vault.sh -./setup-vault.sh -``` - -### Step 3: Start Vault Agent and LLM Service - -```bash -# Start Vault Agent -docker-compose up -d vault-agent-llm - -# Start LLM Orchestration Service -docker-compose up -d llm-orchestration-service -``` - -## 📁 Directory Structure - -After setup, your vault directory will look like this: - -``` -vault/ -├── config/ -│ └── vault.hcl # Vault server configuration -├── agents/ -│ └── llm/ -│ ├── agent.hcl # Vault Agent configuration -│ ├── role_id # AppRole role ID (auto-generated) -│ └── secret_id # AppRole secret ID (auto-generated) -├── logs/ # Vault server logs -└── .vault-token # Root token (keep secure!) -``` - -## 🔐 Secret Schema - -Secrets are stored in Vault using this path structure: - -``` -secret/llm-config/{provider}/{environment}/{model} -``` - -### Azure OpenAI Secret Example - -**Path:** `secret/llm-config/azure-openai/production/gpt-4` - -```json -{ - "connection_id": "azure-prod-gpt4", - "model": "gpt-4", - "environment": "production", - "endpoint": "https://your-azure-openai.openai.azure.com/", - "api_key": "your-azure-api-key", - "deployment_name": "gpt-4", - "api_version": "2024-05-01-preview", - "tags": ["production", "gpt-4"] -} -``` - -### AWS Bedrock Secret Example - -**Path:** `secret/llm-config/aws-bedrock/production/claude-3` - -```json -{ - "connection_id": "aws-prod-claude3", - "model": "anthropic.claude-3-sonnet-20240229-v1:0", - "environment": "production", - "region": "us-east-1", - "access_key_id": "your-aws-access-key", - "secret_access_key": "your-aws-secret-key", - "tags": ["production", "claude-3"] -} -``` - -## 🔧 Manual Configuration - -If you prefer to configure Vault manually, follow these steps: - -### 1. Initialize Vault - -```bash -# Initialize Vault (only needed once) -docker exec vault vault operator init -key-shares=1 -key-threshold=1 - -# Unseal Vault with the unseal key -docker exec vault vault operator unseal - -# Login with root token -docker exec -e VAULT_TOKEN= vault vault auth -``` - -### 2. Enable Auth Methods and Secrets Engine - -```bash -# Set root token -export VAULT_TOKEN= - -# Enable AppRole authentication -docker exec -e VAULT_TOKEN=$VAULT_TOKEN vault vault auth enable approle - -# Enable KV v2 secrets engine -docker exec -e VAULT_TOKEN=$VAULT_TOKEN vault vault secrets enable -version=2 -path=secret kv -``` - -### 3. Create Policy and AppRole - -```bash -# Create policy for LLM service -docker exec -e VAULT_TOKEN=$VAULT_TOKEN vault vault policy write llm-policy - << 'EOF' -path "secret/data/llm-config/*" { - capabilities = ["read"] -} -path "secret/metadata/llm-config/*" { - capabilities = ["list", "read"] -} -EOF - -# Create AppRole -docker exec -e VAULT_TOKEN=$VAULT_TOKEN vault vault write auth/approle/role/llm-service \ - token_policies="llm-policy" \ - token_ttl=1h \ - token_max_ttl=4h -``` - -### 4. Get AppRole Credentials - -```bash -# Get role ID -docker exec -e VAULT_TOKEN=$VAULT_TOKEN vault vault read -field=role_id auth/approle/role/llm-service/role-id > ./vault/agents/llm/role_id - -# Generate secret ID -docker exec -e VAULT_TOKEN=$VAULT_TOKEN vault vault write -field=secret_id auth/approle/role/llm-service/secret-id > ./vault/agents/llm/secret_id -``` - -## 🔍 Troubleshooting - -### Common Issues - -1. **"Vault Agent token file not found"** - - Ensure Vault Agent is running: `docker-compose logs vault-agent-llm` - - Check if token is being written: `docker exec vault-agent-llm ls -la /agent/out/` - -2. **"Connection refused to vault:8200"** - - Verify Vault server is running: `docker-compose ps vault` - - Check Vault server logs: `docker-compose logs vault` - -3. **"Permission denied" errors** - - Verify AppRole credentials are correct - - Check policy permissions in Vault UI - -### Verification Commands - -```bash -# Check Vault server status -docker exec vault vault status - -# Check if secrets exist -docker exec -e VAULT_TOKEN= vault vault kv list secret/llm-config/ - -# Test AppRole authentication -docker exec vault vault write auth/approle/login \ - role_id=@/agent/in/role_id \ - secret_id=@/agent/in/secret_id -``` - -### Logs - -```bash -# Vault server logs -docker-compose logs vault - -# Vault Agent logs -docker-compose logs vault-agent-llm - -# LLM service logs -docker-compose logs llm-orchestration-service -``` - -## 🔒 Security Considerations - -1. **Root Token**: Store securely and rotate regularly -2. **AppRole Credentials**: Auto-generated and rotated by Vault Agent -3. **Network**: Vault is only accessible within Docker network (no external ports) -4. **TLS**: In production, enable TLS for all Vault communications -5. **Policies**: Follow principle of least privilege - -## 🎯 Production Deployment - -For production environments: - -1. **Enable TLS** in vault.hcl and agent.hcl -2. **Use external storage** (Consul, database) instead of Raft for HA -3. **Configure proper** backup and disaster recovery -4. **Set up monitoring** and alerting -5. **Implement proper** secret rotation policies -6. **Use Vault namespaces** for multi-tenancy - -## 📚 Additional Resources - -- [HashiCorp Vault Documentation](https://www.vaultproject.io/docs) -- [Vault Agent Documentation](https://www.vaultproject.io/docs/agent) -- [AppRole Auth Method](https://www.vaultproject.io/docs/auth/approle) -- [KV v2 Secrets Engine](https://www.vaultproject.io/docs/secrets/kv/kv-v2) \ No newline at end of file