The Story
Follow the narrative to build a mental model of how everything connects.
The Arrival
Raw data arrives at the grand harbour of OneLake, the one lake to rule them all. Every piece of data in Microsoft Fabric automatically docks here -no exceptions. OneLake is built atop Azure Data Lake Storage Gen2, providing a single unified namespace for the entire organisation.
The Factory Floor
Inside the Data Factory, pipelines hum with 170+ connectors pulling data from every corner of the enterprise. The Copy Activity moves raw materials from source to destination, while orchestration pipelines coordinate the entire operation.
The Cleaning Crew
Dataflows Gen2 workers scrub, transform, and polish the data using Power Query Online. They watch the folding indicators carefully -5 states: Folding, Not Folding, Might Fold, Opaque, and Unknown. When steps fold, transformations push back to the source.
The Laboratory
Data scientists in the Notebook Lab run PySpark experiments. They broadcast small DataFrames to every node, use withColumn to engineer features, and maintain their Delta tables with regular VACUUM and OPTIMIZE cycles.
The Twin Vaults
Data reaches two great vaults. The Lakehouse welcomes all comers -structured, semi-structured, unstructured -with a flexible schema-on-read philosophy. The Warehouse, more selective, demands structured data and rewards it with full T-SQL power and multi-table transactions.
The Shortcut
A magical portal -the OneLake Shortcut -lets data appear in multiple places without being copied. Zero-copy access across workspaces, across lakehouses, even across clouds.
The Architect's Workshop
The Semantic Model architect arranges tables into a perfect star schema -fact tables at the centre recording transactions and events, dimension tables radiating outward with descriptive attributes. Relationships flow one-to-many from dimension to fact, with single-direction cross-filtering keeping queries clean and predictable.
The Power Source
Storage modes determine how the model draws its power. Import caches everything in memory for blistering speed. DirectQuery reaches through live portals to the source. Direct Lake -the newest and most elegant -reads Delta tables directly from OneLake without importing or querying. Composite models mix all three.
The Formula Chamber
In the Formula Chamber, DAX expressions bring the model to life. CALCULATE -the most powerful function -transforms filter context, while context transition converts row context into filter context inside iterators. Time intelligence functions like TOTALYTD and SAMEPERIODLASTYEAR unlock temporal analysis, but only when the date table is properly marked.
The Security Sentinels
Three sentinels guard the data. RLS filters rows with DAX expressions. CLS hides specific columns. OLS -the most powerful -makes entire tables and columns invisible, as if they never existed.
The Gallery
Finally, data is displayed in the Gallery of Reports. The Performance Analyzer watches for slow visuals. Viewers browse with confidence, knowing Promoted items are team-approved, Certified items meet org quality standards, and Master data items are the organisation's single source of truth.
The Cycle Continues
Deployment pipelines move everything from Dev to Test to Prod. Git integration via PBIP format enables PR reviews. The XMLA endpoint opens the door to enterprise tools like Tabular Editor and DAX Studio.
