Skip to content
HHemlineAnalytics
Request a call
All workCase study · Healthcare services · Fabric platform

One Fabric pipeline for every source.

Metadata-Driven Lakehouse: a medallion lakehouse where adding a source is a row of configuration, not a new pipeline, and drifting data fails loudly before it reaches a report.

15+ hrs
a week of manual reporting retired, worth $35K+ a year
6+ sources
loaded by one pipeline, from APIs to AWS S3
50+ tables
onboarded with a config row each
01The problem

Reporting ran on exports. Each source had its own extract, its own schedule and its own way of breaking, and someone spent hours every week pulling files together by hand before any report could refresh.

02What we built
  1. 1One metadata-driven pipeline. Every source and table is a row of configuration, so a new table onboards without new pipeline code.
  2. 2Bronze, silver and gold layers. Raw data lands as it arrives, gets cleaned and conformed, then shaped into business-ready tables.
  3. 3Schema contracts. Silver checks each source against its expected shape and fails the run when something drifts, instead of passing bad data along.
  4. 4Credentials kept safe. Connection secrets live in Azure Key Vault, never in notebooks or pipeline settings.
03The result

Millions of rows land on schedule from every source, a new table onboards with a config row, and more than fifteen hours a week of manual reporting is gone.

04 · Inside the platform

Rebuilt on fictional data, from source to report.

The same workspace, pipeline and checks, with every name and number made up. Each screen plays through on its own; pick one or pause it any time.

Lumenwell Data PlatformSample data

Lumenwell Data Platform

main · in sync with Git · 6:41 AM New item New folder Source control
Filter by keyword Filter
Data sourcesIngestionLakehousesSemantic modelsReportsCRMBulk APIContact centerAmazon S3Case managementSFTPPartner filesSharePointMember appREST APIWeb eventsREST APIIdentityREST APISurveysREST APIpl_masterData pipelinedf_survey_questionsDataflow Gen2qa_consoleQA workspacelh_bronzeLakehouselh_silverLakehouselh_goldLakehouselh_archiveLakehouseSales DashboardSemantic modelContact Center ReportSemantic modelCases ReportSemantic modelUtilization ReportSemantic model14 more modelsSemantic modelSales DashboardReportContact Center ReportReportCases ReportReportUtilization ReportReport14 more reportsReport
  1. Data sourcesCRMContact centerCase managementPartner filesMember appWeb eventsIdentitySurveys
  2. Ingestionpl_masterdf_survey_questionsqa_console
  3. Lakehouseslh_bronzelh_silverlh_goldlh_archive
  4. Semantic modelsSales DashboardContact Center ReportCases ReportUtilization Report14 more models
  5. ReportsSales DashboardContact Center ReportCases ReportUtilization Report14 more reports

Workspace. One Git-backed workspace: dataflows, lakehouses, notebooks by source and reports by team, with lineage from source to report.

Next case studySales Pipeline Intelligence Daily snapshots of every open deal, and a pipeline waterfall that checks its own math.
Still pulling exports together by hand every week?Talk to us