iOps — Automated Insight Discovery for Operational Data

Problem

Teams collect large operational datasets (sales, product usage, operations) but struggle to consistently identify which patterns are meaningful enough to act on.
Most dashboards surface metrics, not decisions—leaving analysts to manually search for signal in noise.

What this project explores

This project investigates how automated statistical analysis combined with constrained LLM interpretation can surface actionable, defensible insights from tabular business data while minimizing false positives.

The goal is not to replace analysts, but to accelerate decision discovery.

---

Decisions this system supports

• Which metrics show statistically significant change vs normal variation
• Where trends justify deeper investigation or experimentation
• Which segments contribute disproportionately to observed outcomes
• When observed patterns are likely noise and should be ignored

If the system cannot support a decision, it does not surface the insight.

---

Analytical approach

Methods used

• Descriptive statistics and distribution analysis
• Trend detection on time-indexed data
• Correlation discovery with configurable thresholds
• Heuristic-based insight ranking to reduce noise
• Rule-constrained LLM explanations (LLM does not generate raw insights)

What the system does

• Surfaces candidate insights ranked by statistical relevance
• Generates plain-language explanations grounded in computed results
• Flags uncertainty and low-confidence findings
• Preserves analyst oversight at every step

What the system does NOT do

• Perform causal inference
• Replace controlled experiments (A/B testing)
• Make autonomous business decisions
• Guarantee insight correctness on sparse or biased data

---

Example: End-to-end insight generation

Dataset
Monthly sales and operational data across regions and product categories.

Question
“Why did total revenue decline in Q3 despite stable traffic?”

Automated findings

• Revenue decline concentrated in two regions (–18% and –22%)
• No corresponding traffic drop → demand unchanged
• Significant increase in order cancellations (p < 0.05)
• Strong correlation between cancellations and delayed fulfillment times

Interpretation Revenue decline is unlikely due to demand loss.
Operational delays appear to be the dominant contributing factor.

Recommended action Prioritize fulfillment process review before adjusting pricing or marketing spend.

Limitation Correlation does not imply causation. Controlled experiments are required to validate operational impact.

---

System overview

• Data ingestion: CSV, Excel, Parquet, Google Sheets
• Analysis engine: Python (statistical computation and heuristics)
• Explanation layer: LLM constrained to computed outputs
• Interface: Interactive visualizations for inspection and validation

---

Architecture (compressed)

• Backend: Python 3.11, FastAPI, SQLAlchemy
• Frontend: Next.js, TypeScript
• Database: PostgreSQL
• LLM: Groq API (explanations only, not inference)

Infrastructure, authentication, and billing exist but are not the focus of this project.

---

Why this matters for Data Science

This project emphasizes:

• Framing ambiguous problems
• Choosing appropriate analytical methods
• Quantifying uncertainty
• Translating results into decisions

Not model complexity. Not feature count.

---

Project Structure

iOps/
├── backend/                 # FastAPI backend
│   ├── routers/            # API route handlers
│   ├── utils/              # Business logic utilities
│   ├── middleware/         # Custom middleware
│   ├── migrations/         # Database migrations
│   └── tests/              # Test suite
├── frontend/               # Next.js frontend
│   └── src/
│       ├── app/            # App router pages
│       ├── components/     # React components
│       └── lib/            # Utilities
├── docs/                   # Documentation
└── frontend-examples/      # OAuth integration examples

🔧 Quick Start

Prerequisites

• Python 3.11+
• Node.js 18+
• Git

Installation

1. Clone the repository

git clone https://github.com/Emart29/iOpsAI.git
cd iOpsAI

2. Backend Setup

cd backend
python -m venv venv
venv\Scripts\activate        # Windows
# source venv/bin/activate   # Linux/Mac
pip install -r requirements.txt
cp .env.example .env         # Configure your environment variables
python migrate.py upgrade
python run.py

3. Frontend Setup (new terminal)

cd frontend
npm install
cp .env.example .env.local   # Configure your environment variables
npm run dev

4. Access the application

• Frontend: http://localhost:3000
• Backend API: http://localhost:8000
• API Docs: http://localhost:8000/docs

📖 For detailed setup instructions, see SETUP_GUIDE.md

Author

Emmanuel Nwanguma
Data Scientist focused on experimentation, forecasting, and decision-making under uncertainty.