Production-grade REST API ingestion framework for Databricks. Transform any endpoint into Delta Lake tables with metadata-driven state machines, auto-incrementals, flexible pagination, idempotent retries, and immutable audit logging. Works with any API: SaaS platforms, internal services, data vendors, microservices, webhooks, or custom endpoints.
Enterprise-grade three-tier distributed pipeline for ingesting massive Azure Log Analytics data into Databricks. Adaptive windowing, intelligent checkpoint merging, zero-duplicate guarantee. 5.6x faster, 90% cost savings, 100% reliable. Complete documentation included.
A framework that eliminates the dependency on Apache Spark by leveraging delta-rs for the creation and management of Delta Lake tables. This framework follows Medallion architecture.
ingestion framework for project sonar [archived] 👣 🕵🏼 🚛
Add a description, image, and links to the ingestion-framework topic page so that developers can more easily learn about it.
To associate your repository with the ingestion-framework topic, visit your repo's landing page and select "manage topics."