The Story
Follow data on its journey through Microsoft Fabric โ from raw arrival to polished insight.
The Arrival
Raw data arrives at the grand harbour of OneLake, the one lake to rule them all. Every piece of data in Microsoft Fabric automatically docks here โ no exceptions. OneLake is built atop Azure Data Lake Storage Gen2, providing a single unified namespace for the entire organisation.
The Factory Floor
Inside the Data Factory, pipelines hum with 170+ connectors pulling data from every corner of the enterprise. The Copy Activity moves raw materials from source to destination, while orchestration pipelines coordinate the entire operation.
The Cleaning Crew
Dataflows Gen2 workers scrub, transform, and polish the data using Power Query Online. They watch the folding indicators carefully โ 5 states: Folding, Not Folding, Might Fold, Opaque, and Unknown. When steps fold, transformations push back to the source.
The Laboratory
Data scientists in the Notebook Lab run PySpark experiments. They broadcast small DataFrames to every node, use withColumn to engineer features, and maintain their Delta tables with regular VACUUM and OPTIMIZE cycles.
The Twin Vaults
Data reaches two great vaults. The Lakehouse welcomes all comers โ structured, semi-structured, unstructured โ with a flexible schema-on-read philosophy. The Warehouse, more selective, demands structured data and rewards it with full T-SQL power and multi-table transactions.
The Shortcut
A magical portal โ the OneLake Shortcut โ lets data appear in multiple places without being copied. Zero-copy access across workspaces, across lakehouses, even across clouds.
The Architect's Workshop
The Semantic Model architect shapes raw tables into analytical gold. They choose storage modes wisely: Import for speed, DirectQuery for freshness, Direct Lake for the best of both worlds. Calculation groups reduce redundancy with SELECTEDMEASURE().
The Security Sentinels
Three sentinels guard the data. RLS filters rows with DAX expressions. CLS hides specific columns. OLS โ the most powerful โ makes entire tables and columns invisible, as if they never existed.
The Gallery
Finally, data is displayed in the Gallery of Reports. The Performance Analyzer watches for slow visuals. Viewers browse with confidence, knowing Promoted items are team-approved, Certified items meet org quality standards, and Master data items are the organisation's single source of truth.
The Cycle Continues
Deployment pipelines move everything from Dev to Test to Prod. Git integration via PBIP format enables PR reviews. The XMLA endpoint opens the door to enterprise tools like Tabular Editor and DAX Studio.