• 💼 Data Engineer at Celebal Technologies | Building enterprise data platforms
• 🚀 Specialized in Databricks, Azure Data Factory, Delta Lake — Handling billion-scale workloads
• 🔭 Currently building AI-powered code migration solutions with MCP servers and GenAI
• 🏆 Databricks Certified Data Engineer Professional | GenAI Engineer Certified
• 👯 Seeking to collaborate on innovative data engineering and AI automation projects
• 📫 Connect: manuddn15@gmail.com | Portfolio

I architect and optimize enterprise-grade data platforms that process massive data volumes reliably and efficiently. My focus is on building scalable solutions that solve complex data engineering challenges at scale.

Core Expertise:

• 🏗️ Designing metadata-driven data pipelines and governance frameworks
• ⚡ Optimizing large-scale data operations and reducing pipeline latency
• 🤖 Building AI-powered automation tools for data engineering workflows
• 🔐 Implementing enterprise data governance, security, and observability

Led end-to-end design and optimization of enterprise data platforms handling massive data volumes from multiple sources.

Key Deliverables:

• ⚡ Optimized pipeline runtime from 2 hours → 4 minutes (97% reduction) through advanced ADF parallel execution patterns
• 🚀 Engineered data ingestion framework reducing Sentinel extraction from 3 weeks → 2 days with zero data loss
• 🔐 Implemented comprehensive data governance with Microsoft Purview, RLS/CLS, and data lineage tracking
• 🐛 Identified critical data duplication bug affecting data quality — escalated to Microsoft product team
• 💰 Designed cost-optimized cluster configurations reducing infrastructure spend while maintaining SLA

Technical Stack: Databricks, Azure Data Factory, PySpark, Delta Lake, Microsoft Sentinel, Microsoft Purview, Apache Spark

Business Impact: Reduced operational costs, improved data quality to 100% accuracy, enabled enterprise-scale analytics

Architected comprehensive governance solutions enabling secure, traceable, and observable data flows across enterprise.

Key Deliverables:

• 📊 Custom data lineage implementation with Atlas APIs and Microsoft Purview integration
• 👁️ Built observability dashboards with table-level monitoring for Lakehouse environments
• 🔄 Designed Delta Sharing framework for secure external data exchange without direct access
• 🔧 Engineered metadata-driven ingestion pipelines enabling dynamic, scalable configurations

Technical Stack: Microsoft Purview, Delta Lake, Databricks, Atlas APIs, Delta Live Tables

Building intelligent automation platform to accelerate cloud migration by converting legacy ETL code to modern cloud-native pipelines.

Current Work:

• 🤖 Architecting MCP server-based framework for AI-driven code transformation
• 🔄 Automating migration from QLIK to Databricks PySpark reducing manual effort by 80%+
• 📈 Enabling faster cloud adoption and modernization for enterprise clients

Technical Stack: MCP Servers, Generative AI, PySpark, Databricks, Python

• 🥇 Databricks Certified Data Engineer Professional — Advanced certification demonstrating enterprise expertise
• 🤖 Databricks Certified Generative AI Engineer Associate — Expertise in AI/ML automation
• ⭐ 5-Star Performance Rating — Consistent excellence across all project deliverables
• 🔍 Identified Microsoft Product Bug — Discovered critical data issue, confirmed by Microsoft engineering team
• 👥 Mentor — Guiding next generation of data engineers within organization