Skip to content

Commit c29bd2f

Browse files
nuwangeekerangi-arerangi-ar
authored
Bug fixes in Deployment environments (buerokratt#164)
* partialy completes prompt refiner * integrate prompt refiner with llm_config_module * fixed ruff lint issues * complete prompt refiner, chunk retriver and reranker * remove unnesessary comments * updated .gitignore * Remove data_sets from tracking * update .gitignore file * complete vault setup and response generator * remove ignore comment * removed old modules * fixed merge conflicts * Vault Authentication token handling (buerokratt#154) (#70) * partialy completes prompt refiner * integrate prompt refiner with llm_config_module * fixed ruff lint issues * complete prompt refiner, chunk retriver and reranker * remove unnesessary comments * updated .gitignore * Remove data_sets from tracking * update .gitignore file * complete vault setup and response generator * remove ignore comment * removed old modules * fixed merge conflicts * added initial setup for the vector indexer * initial llm orchestration service update with context generation * added new endpoints * vector indexer with contextual retrieval * fixed requested changes * fixed issue * initial diff identifier setup * uncommment docker compose file * added test endpoint for orchestrate service * fixed ruff linting issue * Rag 103 budget related schema changes (#41) * Refactor llm_connections table: update budget tracking fields and reorder columns * Add budget threshold fields and logic to LLM connection management * Enhance budget management: update budget status logic, adjust thresholds, and improve form handling for LLM connections * resolve pr comments & refactoring * rename commonUtils --------- * Rag 93 update connection status (#47) * Refactor llm_connections table: update budget tracking fields and reorder columns * Add budget threshold fields and logic to LLM connection management * Enhance budget management: update budget status logic, adjust thresholds, and improve form handling for LLM connections * resolve pr comments & refactoring * rename commonUtils * Implement LLM connection status update functionality with API integration and UI enhancements --------- * Rag 99 production llm connections logic (#46) * Refactor llm_connections table: update budget tracking fields and reorder columns * Add budget threshold fields and logic to LLM connection management * Enhance budget management: update budget status logic, adjust thresholds, and improve form handling for LLM connections * resolve pr comments & refactoring * rename commonUtils * Add production connection retrieval and update related components * Implement LLM connection environment update and enhance connection management logic --------- * Rag 119 endpoint to update used budget (#42) * Refactor llm_connections table: update budget tracking fields and reorder columns * Add budget threshold fields and logic to LLM connection management * Enhance budget management: update budget status logic, adjust thresholds, and improve form handling for LLM connections * resolve pr comments & refactoring * Add functionality to update used budget for LLM connections with validation and response handling * Implement budget threshold checks and connection deactivation logic in update process * resolve pr comments --------- * Rag 113 warning and termination banners (#43) * Refactor llm_connections table: update budget tracking fields and reorder columns * Add budget threshold fields and logic to LLM connection management * Enhance budget management: update budget status logic, adjust thresholds, and improve form handling for LLM connections * resolve pr comments & refactoring * Add budget status check and update BudgetBanner component * rename commonUtils * resove pr comments --------- * rag-105-reset-used-budget-cron-job (#44) * Refactor llm_connections table: update budget tracking fields and reorder columns * Add budget threshold fields and logic to LLM connection management * Enhance budget management: update budget status logic, adjust thresholds, and improve form handling for LLM connections * resolve pr comments & refactoring * Add cron job to reset used budget * rename commonUtils * resolve pr comments * Remove trailing slash from vault/agent-out in .gitignore --------- * Rag 101 budget check functionality (#45) * Refactor llm_connections table: update budget tracking fields and reorder columns * Add budget threshold fields and logic to LLM connection management * Enhance budget management: update budget status logic, adjust thresholds, and improve form handling for LLM connections * resolve pr comments & refactoring * rename commonUtils * budget check functionality --------- * gui running on 3003 issue fixed * gui running on 3003 issue fixed (#50) * added get-configuration.sqpl and updated llmconnections.ts * Add SQL query to retrieve configuration values * Hashicorp key saving (#51) * gui running on 3003 issue fixed * Add SQL query to retrieve configuration values --------- * Remove REACT_APP_NOTIFICATION_NODE_URL variable Removed REACT_APP_NOTIFICATION_NODE_URL environment variable. * added initil diff identifier functionality * test phase1 * Refactor inference and connection handling in YAML and TypeScript files * fixes (#52) * gui running on 3003 issue fixed * Add SQL query to retrieve configuration values * Refactor inference and connection handling in YAML and TypeScript files --------- * Add entry point script for Vector Indexer with command line interface * fix (#53) * gui running on 3003 issue fixed * Add SQL query to retrieve configuration values * Refactor inference and connection handling in YAML and TypeScript files * Add entry point script for Vector Indexer with command line interface --------- * diff fixes * uncomment llm orchestration service in docker compose file * complete vector indexer * Add YAML configurations and scripts for managing vault secrets * Add vault secret management functions and endpoints for LLM connections * Add Test Production LLM page with messaging functionality and styles * fixed issue * fixed merge conflicts * fixed issue * fixed issue * updated with requested chnages * fixed test ui endpoint request responses schema issue * fixed dvc path issue * added dspy optimization * filters fixed * refactor: restructure llm_connections table for improved configuration and tracking * feat: enhance LLM connection handling with AWS and Azure embedding credentials * fixed issues * refactor: remove redundant Azure and AWS credential assignments in vault secret functions * fixed issue * intial vault setup script * complete vault authentication handling * review requested change fix * fixed issues according to the pr review * fixed issues in docker compose file relevent to pr review --------- Co-authored-by: Charith Nuwan Bimsara <59943919+nuwangeek@users.noreply.github.com> Co-authored-by: erangi-ar <erangika.ariyasena@rootcode.io> * initial streaming updates * fixed requested chnges * fixed issues * complete stream handling in python end * remove unnesasary files * fix test environment issue * fixed constant issue --------- Co-authored-by: erangi-ar <111747955+erangi-ar@users.noreply.github.com> Co-authored-by: erangi-ar <erangika.ariyasena@rootcode.io>
1 parent 67f7c05 commit c29bd2f

File tree

5 files changed

+17
-15
lines changed

5 files changed

+17
-15
lines changed

‎DSL/CronManager/script/store_secrets_in_vault.sh‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -68,7 +68,7 @@ build_vault_path() {
6868
model=$(get_model_name)
6969
fi
7070

71-
if [ "$deploymentEnvironment" = "test" ]; then
71+
if [ "$deploymentEnvironment" = "testing" ]; then
7272
echo "secret/$secret_type/connections/$platform/$deploymentEnvironment/$connectionId"
7373
else
7474
echo "secret/$secret_type/connections/$platform/$deploymentEnvironment/$model"

‎DSL/Ruuter.private/rag-search/POST/inference/test.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -62,7 +62,7 @@ call_orchestrate_endpoint:
6262
body:
6363
connectionId: ${connectionId}
6464
message: ${message}
65-
environment: "test"
65+
environment: "testing"
6666
headers:
6767
Content-Type: "application/json"
6868
result: orchestrate_result

‎src/llm_orchestration_service.py‎

Lines changed: 10 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -27,6 +27,7 @@
2727
INPUT_GUARDRAIL_VIOLATION_MESSAGE,
2828
OUTPUT_GUARDRAIL_VIOLATION_MESSAGE,
2929
GUARDRAILS_BLOCKED_PHRASES,
30+
TEST_DEPLOYMENT_ENVIRONMENT,
3031
)
3132
from src.utils.cost_utils import calculate_total_costs, get_lm_usage_since
3233
from src.guardrails import NeMoRailsAdapter, GuardrailCheckResult
@@ -770,7 +771,7 @@ def handle_input_guardrails(
770771

771772
if not input_check_result.allowed:
772773
logger.warning(f"Input blocked by guardrails: {input_check_result.reason}")
773-
if request.environment == "test":
774+
if request.environment == TEST_DEPLOYMENT_ENVIRONMENT:
774775
logger.info(
775776
"Test environment detected – returning input guardrail violation message."
776777
)
@@ -941,7 +942,7 @@ def _initialize_guardrails(
941942
Initialize NeMo Guardrails adapter.
942943
943944
Args:
944-
environment: Environment context (production/test/development)
945+
environment: Environment context (production/testing/development)
945946
connection_id: Optional connection identifier
946947
947948
Returns:
@@ -1257,7 +1258,7 @@ def _initialize_llm_manager(
12571258
Initialize LLM Manager with proper configuration.
12581259
12591260
Args:
1260-
environment: Environment context (production/test/development)
1261+
environment: Environment context (production/testing/development)
12611262
connection_id: Optional connection identifier
12621263
12631264
Returns:
@@ -1480,7 +1481,7 @@ def _generate_rag_response(
14801481
logger.warning(
14811482
"Response generator unavailable – returning technical issue message."
14821483
)
1483-
if request.environment == "test":
1484+
if request.environment == TEST_DEPLOYMENT_ENVIRONMENT:
14841485
logger.info(
14851486
"Test environment detected – returning technical issue message."
14861487
)
@@ -1547,7 +1548,7 @@ def _generate_rag_response(
15471548
)
15481549
if question_out_of_scope:
15491550
logger.info("Question determined out-of-scope – sending fixed message.")
1550-
if request.environment == "test":
1551+
if request.environment == TEST_DEPLOYMENT_ENVIRONMENT:
15511552
logger.info(
15521553
"Test environment detected – returning out-of-scope message."
15531554
)
@@ -1568,7 +1569,7 @@ def _generate_rag_response(
15681569

15691570
# In-scope: return the answer as-is (NO citations)
15701571
logger.info("Returning in-scope answer without citations.")
1571-
if request.environment == "test":
1572+
if request.environment == TEST_DEPLOYMENT_ENVIRONMENT:
15721573
logger.info("Test environment detected – returning generated answer.")
15731574
return TestOrchestrationResponse(
15741575
llmServiceActive=True,
@@ -1598,7 +1599,7 @@ def _generate_rag_response(
15981599
}
15991600
)
16001601
# Standardized technical issue; no second LLM call, no citations
1601-
if request.environment == "test":
1602+
if request.environment == TEST_DEPLOYMENT_ENVIRONMENT:
16021603
logger.info(
16031604
"Test environment detected – returning technical issue message."
16041605
)
@@ -1635,7 +1636,7 @@ def create_embeddings_for_indexer(
16351636
16361637
Args:
16371638
texts: List of texts to embed
1638-
environment: Environment (production, development, test)
1639+
environment: Environment (production, development, testing)
16391640
connection_id: Optional connection ID for dev/test environments
16401641
batch_size: Batch size for processing
16411642
@@ -1691,7 +1692,7 @@ def get_available_embedding_models_for_indexer(
16911692
"""Get available embedding models for vector indexer.
16921693
16931694
Args:
1694-
environment: Environment (production, development, test)
1695+
environment: Environment (production, development, testing)
16951696
16961697
Returns:
16971698
Dictionary with available models and default model info

‎src/llm_orchestrator_config/llm_cochestrator_constants.py‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -24,3 +24,4 @@
2424

2525
# Streaming configuration
2626
STREAMING_ALLOWED_ENVS = {"production"}
27+
TEST_DEPLOYMENT_ENVIRONMENT = "testing"

‎src/models/request_models.py‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -33,7 +33,7 @@ class OrchestrationRequest(BaseModel):
3333
..., description="Previous conversation history"
3434
)
3535
url: str = Field(..., description="Source URL context")
36-
environment: Literal["production", "test", "development"] = Field(
36+
environment: Literal["production", "testing", "development"] = Field(
3737
..., description="Environment context"
3838
)
3939
connection_id: Optional[str] = Field(
@@ -66,7 +66,7 @@ class EmbeddingRequest(BaseModel):
6666
"""
6767

6868
texts: List[str] = Field(..., description="List of texts to embed", max_length=1000)
69-
environment: Literal["production", "development", "test"] = Field(
69+
environment: Literal["production", "development", "testing"] = Field(
7070
..., description="Environment for model resolution"
7171
)
7272
batch_size: Optional[int] = Field(
@@ -97,7 +97,7 @@ class ContextGenerationRequest(BaseModel):
9797
..., description="Document content for caching", max_length=100000
9898
)
9999
chunk_prompt: str = Field(..., description="Chunk-specific prompt", max_length=5000)
100-
environment: Literal["production", "development", "test"] = Field(
100+
environment: Literal["production", "development", "testing"] = Field(
101101
..., description="Environment for model resolution"
102102
)
103103
use_cache: bool = Field(default=True, description="Enable prompt caching")
@@ -138,7 +138,7 @@ class TestOrchestrationRequest(BaseModel):
138138
"""Model for simplified test orchestration request."""
139139

140140
message: str = Field(..., description="User's message/query")
141-
environment: Literal["production", "test", "development"] = Field(
141+
environment: Literal["production", "testing", "development"] = Field(
142142
..., description="Environment context"
143143
)
144144
connectionId: Optional[int] = Field(

0 commit comments

Comments
 (0)