An interactive study guide built on 7 memory techniques to help you pass the Microsoft Fabric Data Engineer Associate exam.
Lakehouses, Spark notebooks (PySpark/SQL), data pipelines, mirroring, Delta Lake, medallion architecture (bronze/silver/gold), and monitoring & optimisation.
Data engineers, ETL developers, platform engineers, and anyone building data pipelines and lakehouses in Microsoft Fabric.
You're a SQL developer or data analyst wanting to move into data engineering, or an Azure data engineer transitioning to Fabric.
Tap any component to see what you need to know for the exam.
Follow the narrative to build a mental model of how everything connects.
Everything begins with OneLake — the unified data lake that underpins all of Microsoft Fabric. Built on ADLS Gen2, every piece of data automatically lands here. Shortcuts extend its reach across clouds without copying a single byte.
Data flows in through multiple channels. Pipelines orchestrate bulk movements with Copy Activity and its 170+ connectors. Dataflows Gen2 provide no-code Power Query transformations. For heavy engineering, Spark notebooks run PySpark at scale.
Some data doesn't need pipelines at all. Mirroring creates near-real-time replicas of external databases — Azure SQL, Cosmos DB, Snowflake — as read-only Delta tables in OneLake. Change Data Capture ensures only differences flow, not full copies.
Raw data touches down in the Bronze layer — the first stop in the medallion architecture. Data arrives as-is from source systems. Engineers add metadata columns (ingestion time, source, batch ID) but resist the urge to transform. Schema-on-read rules here.
In the Silver layer, data gets scrubbed. Nulls are handled, types are cast, duplicates are removed. Column names are standardised, dates normalised. Related tables join together. This is where data quality rules enforce consistency.
Gold is where data becomes a product. Aggregations are pre-computed, star schemas are formed, and partitioning strategies align with downstream query patterns. Z-Order optimisation ensures filter columns are read-efficient.
Under the hood, Spark does the heavy lifting. V-Order optimisation on writes ensures downstream reads are fast. mssparkutils handles file operations and credential management. High Concurrency mode shares sessions across notebooks for efficiency.
Data reaches consumers through two paths: the Lakehouse (Spark + SQL analytics endpoint) and the Warehouse (full T-SQL). Direct Lake semantic models read Delta tables without copying, while the SQL analytics endpoint provides familiar T-SQL access.
Engineers watch from the monitoring tower. The Monitoring Hub tracks pipeline runs, Spark jobs, and refresh operations. Capacity metrics reveal compute usage. Apache Spark Advisor suggests performance improvements.
Before data leaves, governance ensures it's trustworthy. Fabric domains organise workspaces by business area. Sensitivity labels flow downstream automatically. OneLake RBAC controls who accesses what, while Purview catalogues everything.
Memorable acronyms and phrases to anchor key exam concepts in your memory.
Side-by-side comparisons to sharpen your understanding of similar concepts.
| Aspect | Lakehouse | Warehouse |
|---|---|---|
| Schema | Schema-on-read | Schema-on-write |
| Language | Spark (PySpark/SQL) | T-SQL |
| Data types | Structured + semi + unstructured | Structured only |
| Transactions | Single table (Delta) | Multi-table |
| SQL access | SQL analytics endpoint (read-only) | Full DML (read-write) |
| Best for | Data engineering, ML, data science | BI analytics, T-SQL workloads |
| Files section | Yes (unstructured storage) | No |
| Aspect | Pipeline | Dataflow | Notebook |
|---|---|---|---|
| Purpose | Orchestration | No-code transform | Code-based transform | |
| Language | Activities (visual) | Power Query (M) | PySpark/SQL/Scala | |
| Best for | Workflow coordination | Simple ETL | Complex engineering | |
| Compute | Pipeline engine | Dataflow engine | Spark cluster | |
| Reusability | Template pipelines | Reusable dataflows | Parameterised notebooks | |
| Monitoring | Pipeline runs | Refresh history | Spark UI + jobs |
| Aspect | Shortcuts | Copy | Mirror |
|---|---|---|---|
| Data movement | Zero-copy (pointer) | Full copy | CDC replication | |
| Latency | Real-time | Batch (scheduled) | Near real-time | |
| Storage cost | None (source only) | Duplicated | Delta tables in OneLake | |
| Transformation | None | In pipeline | None (read-only) | |
| Sources | ADLS, S3, GCS, internal | 170+ connectors | Azure SQL, Cosmos, Snowflake | |
| Best for | Cross-workspace access | ETL workflows | Always-fresh replicas |
| Aspect | Bronze | Silver | Gold |
|---|---|---|---|
| Data state | Raw, as-is | Cleansed, conformed | Business-ready | |
| Schema | Schema-on-read | Validated schema | Star schema / aggregated | |
| Transformations | Metadata only | Clean, deduplicate, join | Aggregate, model, partition | |
| Quality | Source fidelity | Data quality rules | Business rules applied | |
| Users | Data engineers | Data engineers/analysts | Analysts, semantic models | |
| Format | Delta (or raw files) | Delta tables | Delta tables (optimised) |
| Aspect | Batch | Streaming Ingestion |
|---|---|---|
| Timing | Scheduled intervals | Continuous / real-time |
| Tools | Pipelines, Notebooks | Eventstream, Mirroring |
| Latency | Minutes to hours | Seconds |
| Volume | Large bulk loads | Event-by-event |
| Complexity | Simpler, predictable | Stateful, complex |
| Use case | Nightly ETL, reports | IoT, fraud detection, alerts |
| Aspect | OPTIMIZE | VACUUM | Z-Order |
|---|---|---|---|
| OPTIMIZE | Compact small files into larger ones | After many small writes/appends | |
| VACUUM | Remove old files no longer referenced | Regularly to reclaim storage | |
| Z-ORDER BY | Co-locate related data for faster reads | On frequently filtered columns | |
| V-Order | Write-time optimisation (Fabric default) | Automatic in Fabric notebooks | |
| DESCRIBE HISTORY | View table version history | Auditing, debugging, time-travel | |
| RESTORE | Roll back to a previous version | Recovering from bad writes |
| Aspect | Delta | Parquet | CSV |
|---|---|---|---|
| ACID transactions | Yes | No | No | |
| Schema evolution | Yes | Limited | No | |
| Time travel | Yes (versioned) | No | No | |
| Compression | Excellent | Excellent | Poor | |
| Read performance | Best (with V-Order) | Good (columnar) | Slow | |
| Default in Fabric | Yes (tables) | Files area | Import only |
| Aspect | Workspace Roles | OneLake RBAC |
|---|---|---|
| Scope | Entire workspace | Item/folder level |
| Granularity | Coarse (4 roles) | Fine-grained |
| Roles | Admin, Member, Contributor, Viewer | Custom per item |
| Controls | Create, edit, delete items | Read/write data in OneLake |
| Use case | Team access management | Data-level security |
| Combined with | OneLake RBAC for data access | Workspace roles for item access |
| Aspect | Full Load | Incremental Load |
|---|---|---|
| Data scope | All data every time | Only new/changed records |
| Performance | Slower, resource-heavy | Faster, efficient |
| Complexity | Simple implementation | Needs watermark/CDC tracking |
| Storage | Replace or overwrite | Merge/upsert (MERGE INTO) |
| Use case | Small tables, initial load | Large tables, ongoing sync |
| Fabric pattern | Overwrite mode | Watermark + merge pattern |
Organised reference grouped by exam domain — everything you need on one page.
Walk through themed rooms — each object anchors a concept in spatial memory.
Ingestion — Where data first arrives
Architecture — Where data is layered
Compute — Where transformation happens
Monitoring — Where engineers watch
Security — Where trust is enforced
Decision trees and trigger-answer pairs — see the pattern, know the answer.
Lucid Labs delivers Microsoft certification training led by Microsoft Certified Trainers (MCTs) and grounded in real-world project experience. We adapt every session to your team's environment, data stack, and business objectives — because the best exam prep comes from engineers who build these solutions every day.
Custom training for teams & individuals — remote or on-site across Australia
A free, hands-on Microsoft Fabric workshop on Real-Time Intelligence: ingest, analyse and act on streaming data. Labs run in an environment provided on the day, so there is nothing to set up beforehand.
Microsoft Solutions Partner architect specialising in Fabric, Azure Data & AI, and GitHub Enterprise. 18+ years delivering data platforms for Australian businesses — building the systems these exams test every day.