Master the MLOps

An interactive study guide built on 7 memory techniques to help you pass the Machine Learning Operations (MLOps) Engineer Associate exam.

Design ML Solutions 25-30%Implement ML Training 25-30%Deploy & Operationalise 25-30%Responsible AI 15-20%

What it covers

Azure Machine Learning workspace, compute targets, datasets, ML pipelines, AutoML, hyperparameter tuning, model registry, managed endpoints (online/batch), Azure AI Foundry, model monitoring, A/B testing, responsible AI dashboard, fairness/explainability.

Ideal for

ML engineers, data scientists, and platform engineers building production ML pipelines on Azure.

Aspire to this if

You’re a data scientist who trains models in notebooks and wants to learn how to productionise them, or a DevOps engineer expanding into ML infrastructure.

Section 1 / Spatial Memory

The Map

Tap any component to see what you need to know for the exam.

🏢 AML Workspace

Central Hub · Resource Group

📊 Data Assets

Datasets · Datastores · MLTable

⚙️ Compute Targets

Instances · Clusters · Serverless

🤖 AutoML

Automated Feature Eng · Model Selection

🔧 ML Pipelines

Components · Jobs · Orchestration

🎯 Training Jobs

Experiments · Sweeps · Distributed

📦 Model Registry

Versioning · Lineage · Staging

🚀 Managed Endpoints

Online · Batch · A/B Testing

🔍 Model Monitoring

Drift · Performance · Alerts

🛡️ Responsible AI

Fairness · Explainability · Safety

🏭 Azure AI Foundry

Multi-Model · Prompt Flow · Evaluation

Section 2 / Narrative Memory

The Story

Follow the narrative to build a mental model of how everything connects.

🏢

The Workspace

Every ML journey starts with an Azure Machine Learning workspace — the central hub that ties together compute, data, models, and endpoints. It’s backed by a storage account, key vault, container registry, and Application Insights. Managed virtual networks keep everything secure.

Exam IntelWorkspace = top-level resource. Linked services: Storage, Key Vault, ACR, App Insights. RBAC roles: Owner, Contributor, AzureML Data Scientist. AI Foundry projects extend workspaces.

📊

The Data Layer

Data enters through datastores — abstractions over Blob, ADLS Gen2, and SQL. Data assets version your references so experiments are reproducible. MLTable defines tabular schemas with built-in transformations. Drift detection watches for distribution shifts between training and production.

Exam IntelDatastores = connection abstractions. Data asset types: uri_file, uri_folder, mltable. MLTable = tabular schema + transforms. Data drift detection compares training vs serving distributions.

⚙️

The Compute Engine

Training needs power. Compute instances serve as dev workstations. Compute clusters auto-scale from 0 to N nodes for distributed jobs. Serverless compute eliminates cluster management entirely. GPU SKUs tackle deep learning while CPUs handle classical ML.

Exam IntelCompute instance = single-node dev. Compute cluster = multi-node, auto-scale (min 0). Serverless = on-demand, no management. GPU: NC/ND series. Attached: Spark, Databricks, AKS.

🤖

The AutoML Shortcut

AutoML accelerates experimentation. It tries multiple algorithms, featurisation strategies, and hyperparameters automatically. Guard rails catch class imbalance and high cardinality. The best models are ensembled, and explainability comes built in.

Exam IntelAutoML tasks: classification, regression, forecasting, NLP, vision. Auto featurisation: impute, encode, scale, datetime decompose. Guard rails: class balance, cardinality, split validation. Ensembles combine top models.

🔧

The Pipeline Factory

Production ML isn’t a single script — it’s a pipeline. Components encapsulate reusable steps: data prep, feature engineering, training, evaluation, and registration. Pipelines orchestrate these steps with dependency tracking, scheduled triggers, and parallel batch processing.

Exam IntelPipelines = multi-step orchestration. Components = reusable steps. ParallelRunStep for batch. Schedule: cron, recurrence, data triggers. Authored with CLI v2 (YAML) or SDK v2 (Python).

🎯

The Training Loop

Command jobs run scripts on remote compute with reproducible environments. Sweep jobs search hyperparameter spaces using Bayesian, grid, or random strategies. Early termination kills underperforming runs. Distributed training scales across nodes with PyTorch DDP or DeepSpeed. MLflow captures everything automatically.

Exam IntelCommand job = single training run. Sweep job = hyperparameter tuning. Early termination: bandit, median stopping, truncation. Distributed: PyTorch DDP, TF MirroredStrategy, DeepSpeed. MLflow autolog for metrics/params/artefacts.

📦

The Model Vault

Trained models are registered in the model registry with version history and full lineage back to the training data and experiment. Models progress through stages — Staging, Production, Archived. Registry-level models can be shared across workspaces for enterprise-wide collaboration.

Exam IntelModel registry = versioned catalogue. Lineage: data → experiment → model. Stages: None → Staging → Production → Archived. Registry models = cross-workspace sharing. Formats: MLflow, custom, Hugging Face.

🚀

The Deployment Runway

Models go live through managed endpoints. Online endpoints serve real-time predictions with auto-scaling. Batch endpoints process bulk data asynchronously. Traffic splitting enables A/B testing, while blue-green deployments allow safe rollouts by shifting traffic after validation.

Exam IntelManaged online = real-time, auto-scale. Batch = async, ParallelRunStep. Traffic split = A/B testing (% routing). Blue-green: deploy, validate, shift. Kubernetes endpoints for custom infra.

🔍

The Watchtower

Deployed models need monitoring. Data drift detection compares production inputs against training baselines. Prediction drift tracks output distribution changes. Feature attribution drift spots shifts in what drives predictions. Application Insights tracks latency, errors, and throughput.

Exam IntelData drift = input distribution shift. Prediction drift = output distribution shift. Feature attribution drift = feature importance shift. App Insights = latency, errors, throughput. Alert thresholds trigger notifications.

🛡️

The Ethics Gate

Before any model reaches users, it passes through responsible AI review. The RAI dashboard surfaces error hotspots, fairness gaps across demographics, counterfactual examples, and causal analysis. Model cards document intended use, limitations, and evaluation results.

Exam IntelRAI dashboard: error analysis, fairness, counterfactuals, causal analysis. Fairlearn: demographic parity, equalised odds. InterpretML: global/local explanations. Content safety for GenAI. Model cards for documentation.

Section 3 / Acronym Memory

Mnemonic Wall

Memorable acronyms and phrases to anchor key exam concepts in your memory.

🏢

SKCA

The four linked services of an AML workspace. Every workspace depends on these.

📊

UFM

Three data asset types. URI file/folder for unstructured. MLTable for tabular with schema.

⚙️

ICSA

Four compute target types. Instance = dev. Cluster = training. Serverless = on-demand. Attached = external.

🤖

CRFNV

Five AutoML task types. Covers both tabular and unstructured data.

🎯

BGR

Hyperparameter search strategies for sweep jobs. Bayesian is most efficient for expensive runs.

✂️

BMT

Early termination policies for sweep jobs. Bandit = slack-based. Median = below-median cut. Truncation = bottom %.

🚀

OBK

Three endpoint types. Online = real-time. Batch = async bulk. Kubernetes = custom infra (AKS).

🔄

ABGD

Deployment strategies. Traffic split for A/B. Blue-green for safe swap. Gradual rollout for controlled migration.

🔍

DPF

Three monitoring signals. Data = input shift. Prediction = output shift. Feature = importance shift.

🛡️

EFCC

Four components of the Responsible AI dashboard. Diagnose, measure, explain, and understand.

🏭

SPAC

Model lifecycle stages in the registry. Promote through stages for governance.

🧠

DDD

Distributed training frameworks. DDP = data parallel. DeepSpeed = memory-efficient. TF = MirroredStrategy.

📝

YAML+SDK

Two primary authoring methods for AML resources. YAML for declarative, SDK for programmatic.

🔐

MVNET

Workspace network isolation. Managed VNet with private endpoints for secure AML deployments.

🎲

MLflow

MLflow is the de facto experiment tracking standard in AML. Autolog captures everything.

Section 4 / Contrast Memory

Versus Arena

Side-by-side comparisons to sharpen your understanding of similar concepts.

Online EndpointvsBatch Endpoint

Online Endpoint vs Batch Endpoint

Aspect	Online Endpoint	Batch Endpoint
Latency	Real-time (milliseconds)	Asynchronous (minutes–hours)
Scaling	Auto-scale on request load	Parallel nodes for throughput
Input	Single request / small payload	Large dataset / file URI
Use case	API serving, chatbots, fraud detection	Bulk scoring, nightly predictions
Traffic split	Yes (A/B testing)	No
Compute	Always-on or scale-to-zero	Provisioned per job

Compute InstancevsClustervsServerless

Compute Instance vs Cluster vs Serverless

Aspect	Compute Instance	Cluster
Nodes	Single VM \| Multi-node	On-demand (auto)
Purpose	Dev/experimentation \| Training	Any job, no management
Scaling	None (fixed) \| Auto-scale 0–N	Automatic per job
Cost	Pay while running \| Min nodes cost	Pay per job only
Setup	Quick \| Configure SKU + limits	Zero config
Best for	Notebooks/IDE \| Large training	Simplicity, burst workloads

AutoMLvsCustom Training

AutoML vs Custom Training

Aspect	AutoML	Custom Training
Effort	Low — automated pipeline	High — manual code
Flexibility	Constrained to supported tasks	Unlimited — any framework
Featurisation	Automatic (impute, encode, scale)	Manual feature engineering
Algorithms	Pre-selected set per task	Any custom algorithm
Explainability	Built-in for best model	Requires InterpretML / SHAP
Best for	Baselines, tabular data, speed	Custom architectures, DL, research

MLflowvsAML Native Tracking

MLflow vs AML Native Tracking

Aspect	MLflow	AML Native Tracking
Portability	Open standard, multi-cloud	Azure-native only
Autologging	Yes — scikit-learn, PyTorch, etc.	Manual log_metric calls
Model format	MLflow model flavours	Custom model packaging
Registry	MLflow Model Registry	AML Model Registry
Recommendation	Default in AML — preferred	Legacy / specific use cases
Serving	Direct deployment from MLflow	Score.py + environment

Managed OnlinevsKubernetes Endpoint

Managed Online vs Kubernetes Endpoint

Aspect	Managed Online	Kubernetes Endpoint
Infrastructure	Fully managed by Azure	Self-managed AKS cluster
Setup	Simple — specify SKU	Complex — configure AKS + AML extension
Scaling	Built-in auto-scale rules	Kubernetes HPA/VPA
Networking	Managed VNet / public	Full Kubernetes networking
Cost	Pay per compute time	Always-on cluster cost
Best for	Most production workloads	Custom infra, multi-cloud, compliance

Data DriftvsPrediction Drift

Data Drift vs Prediction Drift

Aspect	Data Drift	Prediction Drift
What shifts	Input feature distributions	Model output distributions
Detects	Upstream data changes	Model behaviour changes
Baseline	Training dataset	Historical predictions
Metrics	PSI, Wasserstein, Jensen-Shannon	Distribution distance on outputs
Action	Investigate data pipeline issues	Retrain or roll back model
Related	Feature attribution drift	Performance degradation

Sweep Job: BayesianvsGridvsRandom

Bayesian vs Grid vs Random

Aspect	Sweep Job: Bayesian	Grid
Strategy	Model-guided search \| Exhaustive	Random sampling
Efficiency	Most efficient \| Least efficient	Moderate
Search space	Continuous preferred \| Discrete	Any distribution
Parallelism	Limited (sequential learning) \| Full	Full
Best for	Expensive runs \| Small spaces	Quick exploration, large spaces
Early termination	Bandit policy recommended	Any policy or none

RAI Dashboard Components

Error Analysis vs Fairness vs Counterfactuals vs Causal

Aspect	RAI Dashboard Components
Error analysis	Find cohorts with high error rates	Tree map + heat map views
Fairness	Measure disparity across groups	Demographic parity, equalised odds
Counterfactuals	What-if: minimal feature changes	Flip prediction outcome
Causal	Causal inference on treatment effects	What interventions help?
Explanations	Global + local feature importance	InterpretML / SHAP values
Goal	Diagnose, measure, explain	Build trust and compliance

CLI v2 (YAML)vsSDK v2 (Python)

CLI v2 vs SDK v2

Aspect	CLI v2 (YAML)	SDK v2 (Python)
Format	Declarative YAML	Imperative Python code
Best for	CI/CD pipelines, GitOps	Notebooks, dynamic logic
Portability	Easy to version in Git	Embedded in application code
Flexibility	Template-driven	Full programming language
Learning curve	Lower for DevOps engineers	Lower for data scientists
Both support	Jobs, pipelines, endpoints, models	Jobs, pipelines, endpoints, models

Section 5 / Grouping Memory

Cheat Sheet

Organised reference grouped by exam domain — everything you need on one page.

Design ML Solutions

25-30%

Workspace & Infrastructure

AML workspace: hub for compute, data, models, endpoints
Linked services: Storage Account, Key Vault, ACR, App Insights
Managed VNet with private endpoints for network isolation
RBAC: Owner, Contributor, AzureML Data Scientist, custom roles
AI Foundry projects extend workspaces for multi-model apps

Compute Targets

Compute instance: single-node dev VM (notebooks, IDE)
Compute cluster: auto-scale 0–N for training (GPU: NC/ND)
Serverless compute: on-demand, zero management
Attached compute: Spark pools, Databricks, AKS
Low-priority VMs: up to 80% discount, preemptible

Data Architecture

Datastores: abstract connections to Blob, ADLS, SQL
Data asset types: uri_file, uri_folder, mltable
MLTable: tabular schema + transforms for reproducibility
Data drift detection: training vs serving distribution comparison
Feature store: centralised feature engineering and sharing

Authoring & DevOps

CLI v2 (YAML): declarative, CI/CD-friendly, GitOps
SDK v2 (Python): programmatic, notebook-friendly
Azure DevOps / GitHub Actions for ML CI/CD pipelines
Git integration for code, YAML definitions, and notebooks
Environment definitions: curated, custom Docker, conda

Implement ML Training

25-30%

Training Jobs

Command job: single script on remote compute with environment
Sweep job: hyperparameter tuning (Bayesian, grid, random)
Pipeline job: multi-step orchestration with dependencies
AutoML job: automated model selection + featurisation
MLflow autolog: captures metrics, params, artefacts automatically

Hyperparameter Tuning

Search strategies: Bayesian (most efficient), grid, random
Parameter spaces: choice, uniform, loguniform, quniform
Early termination: bandit (slack), median stopping, truncation
Primary metric: accuracy, AUC, RMSE (defines best run)
Max total trials + max concurrent trials control cost

Distributed Training

PyTorch DDP: data-parallel across GPU nodes
DeepSpeed: memory-efficient large model training
TensorFlow MirroredStrategy: synchronous data parallel
Horovod: framework-agnostic ring-allreduce
Configure node count + process count per node

ML Pipelines & Components

Components: reusable, versioned steps (YAML or Python)
Pipeline DAG: data prep → train → evaluate → register
ParallelRunStep: batch inference across nodes
Schedule: cron, recurrence, or data-trigger based
Pipeline parameters for dynamic configuration

Deploy & Operationalise

25-30%

Model Registry

Versioned model catalogue with metadata and lineage
Stages: None → Staging → Production → Archived
Registry models shared across workspaces
Formats: MLflow, custom, Hugging Face, ONNX
Model lineage: data asset → experiment → registered model

Managed Endpoints

Online endpoint: real-time, auto-scale, managed infra
Batch endpoint: async bulk scoring with ParallelRunStep
Kubernetes endpoint: AKS for custom infrastructure
Traffic split: route % to deployments for A/B testing
Blue-green: deploy new, validate, shift 100% traffic

Model Monitoring

Data drift: compare serving features to training baseline
Prediction drift: track output distribution changes
Feature attribution drift: detect feature importance shifts
App Insights: latency, error rate, throughput metrics
Configurable alert thresholds with email/webhook notifications

CI/CD for ML

GitHub Actions / Azure Pipelines for automated ML workflows
Model validation gates: accuracy threshold, fairness checks
Infrastructure-as-code: Bicep/Terraform for workspace + endpoints
Automated retraining triggered by data drift alerts
Deployment approval gates for production promotion

Responsible AI

15-20%

RAI Dashboard

Error analysis: cohort-level error distribution (tree/heat map)
Fairness assessment: disparity metrics across sensitive groups
Counterfactual explanations: minimal changes to flip prediction
Causal analysis: treatment effect estimation for interventions
Generate RAI insights as a pipeline component

Fairness & Explainability

Fairlearn: demographic parity, equalised odds metrics
InterpretML: global and local feature importance
SHAP values for individual prediction explanations
Mitigations: post-processing, resampling, constraint optimisation
Model cards: document intended use, limitations, eval results

Content Safety & Governance

Content safety filters for generative AI (Azure AI Foundry)
Prompt shields for jailbreak and injection detection
Groundedness detection to reduce hallucination
Human-in-the-loop review processes for high-stakes decisions
Audit logging and compliance documentation

Section 6 / Method of Loci

The Memory Palace

Walk through themed rooms — each object anchors a concept in spatial memory.

The Workspace Lobby

Infrastructure — Where the ML environment lives

🏢

AML Workspace

Central hub. Linked to Storage, Key Vault, ACR, App Insights

⚙️

Compute Instance

Single-node dev VM. Notebooks, IDE, experimentation

🔥

Compute Cluster

Auto-scale 0–N nodes. GPU (NC/ND) for training. Low-priority for savings

☁️

Serverless Compute

On-demand, no management. Pay per job only

🔐

Managed VNet

Private endpoints for workspace isolation. Secure by default

📝

CLI v2 + SDK v2

YAML (declarative, GitOps) + Python (programmatic)

The Data Vault

Data — Where features are stored and versioned

📊

Datastores

Abstract Blob, ADLS Gen2, SQL connections. Credential-free

📁

Data Assets

Versioned references: uri_file, uri_folder, mltable. Lineage tracked

📃

MLTable

Tabular schema + transforms. Reproducible feature loading

📉

Data Drift Monitor

Compare training vs serving distributions. Alert on shift

🏭

Feature Store

Centralised feature engineering. Shared across teams and models

🔗

Data Lineage

Track data → experiment → model → endpoint

The Training Arena

Training — Where models learn from data

🤖

AutoML

Classification, regression, forecasting, NLP, vision. Auto featurisation

🎯

Sweep Jobs

Bayesian, grid, random search. Bandit/median/truncation termination

🔧

Pipeline Components

Reusable steps: prep → train → evaluate → register

⚡

Distributed Training

PyTorch DDP, DeepSpeed, TF MirroredStrategy, Horovod

📊

MLflow Tracking

Autolog metrics, params, artefacts. Open standard. Default in AML

📅

Environments

Curated, custom Docker, conda YAML. Reproducible training

The Model Registry

Registration — Where models are catalogued

📦

Model Versions

Versioned catalogue. Metadata, tags, description per version

🔄

Lifecycle Stages

None → Staging → Production → Archived

🌐

Registry Models

Cross-workspace sharing for enterprise-wide collaboration

📎

Model Formats

MLflow, custom, Hugging Face, ONNX, TensorFlow SavedModel

🔗

Full Lineage

Data asset → training job → registered model → endpoint

📝

Model Cards

Document intended use, limitations, evaluation metrics, bias checks

The Deployment Runway

Serving — Where models meet production traffic

⚡

Online Endpoints

Real-time inference. Auto-scale. Managed load balancing

📦

Batch Endpoints

Async bulk scoring. ParallelRunStep. Large datasets

🔀

Traffic Splitting

Route % to deployments. A/B testing model variants

🟢

Blue-Green Deploy

Deploy new version, validate, then shift all traffic

⚓

Kubernetes Endpoints

AKS-based. Custom infra, networking, compliance

🔍

Model Monitoring

Data drift, prediction drift, feature attribution drift. Alerts

Section 7 / Pattern Recognition

Pattern Spotter

Decision trees and trigger-answer pairs — see the pattern, know the answer.

Which Compute Target?

Which Compute Target?
  ├── Interactive development / notebooks → Compute Instance
  ├── Large-scale training / distributed → Compute Cluster (GPU)
  ├── No cluster management, burst workload → Serverless Compute
  ├── External Spark / Databricks → Attached Compute
  └── Cost-sensitive, can tolerate preemption → Low-Priority VMs

Which Training Approach?

Which Training Approach?
  ├── Quick baseline, tabular data → AutoML
  ├── Custom model, single run → Command Job
  ├── Tune hyperparameters → Sweep Job (Bayesian/grid/random)
  ├── Multi-step workflow with dependencies → Pipeline Job
  └── Large model, multi-GPU needed → Distributed Training (DDP/DeepSpeed)

Which Endpoint Type?

Which Endpoint Type?
  ├── Real-time API predictions → Managed Online Endpoint
  ├── Bulk scoring large datasets → Batch Endpoint
  ├── Custom infra / compliance / AKS → Kubernetes Endpoint
  ├── A/B test model variants → Online Endpoint + Traffic Split
  └── Safe rollout, zero downtime → Blue-Green Deployment

Which Monitoring Signal?

Which Monitoring Signal?
  ├── Input features changed vs training → Data Drift
  ├── Model outputs shifting over time → Prediction Drift
  ├── Feature importance rankings changed → Feature Attribution Drift
  ├── Latency / error rate / throughput → Application Insights
  └── Fairness metrics degrade for subgroups → RAI Dashboard → Fairness

Decision Cards

"managed online endpoint" or "real-time inference"→Auto-scaling REST API for predictions

"batch endpoint" or "bulk scoring"→Async processing with ParallelRunStep

"traffic split" or "A/B testing"→Route % of requests to deployment variants

"blue-green deployment"→Deploy new, validate, shift traffic, delete old

"sweep job" or "hyperparameter tuning"→Bayesian / grid / random with early termination

"AutoML" or "automated machine learning"→Auto featurise + model selection + ensemble

"MLflow" or "experiment tracking"→Autolog metrics, params, artefacts. AML default

"data drift" or "distribution shift"→Compare serving vs training feature distributions

"model registry" or "model versioning"→Versioned catalogue: Staging → Production → Archived

"responsible AI dashboard"→Error analysis + fairness + counterfactuals + causal

"Fairlearn" or "demographic parity"→Fairness metrics and disparity mitigation

"InterpretML" or "SHAP values"→Global + local feature importance explanations

"compute cluster" or "auto-scale training"→Multi-node, 0–N auto-scale, GPU: NC/ND series

"mltable" or "tabular data asset"→Schema + transforms for reproducible feature loading

"managed virtual network" or "private endpoints"→Workspace network isolation for secure deployments

"distributed training" or "PyTorch DDP"→Multi-node: DDP, DeepSpeed, TF Mirrored, Horovod

"pipeline component" or "reusable ML step"→Versioned step: prep → train → eval → register

"content safety" or "prompt shields"→GenAI filters: jailbreak, injection, groundedness

Ready to certify?

Train with practitioners, not presenters

Lucid Labs delivers Microsoft certification training led by Microsoft Certified Trainers (MCTs) and grounded in real-world project experience. We adapt every session to your team's environment, data stack, and business objectives — because the best exam prep comes from engineers who build these solutions every day.

🎯

Tailored Content

Training built around your actual ML workloads, your Azure environment, and your deployment patterns — not generic slides.

🛠️

Hands-On Labs

Build real ML pipelines, deploy managed endpoints, and configure model monitoring with expert guidance.

📈

Exam + Capability

Pass the exam and build lasting MLOps skills your team can apply from day one.

Talk to us about MLOps Engineer Associate (Azure ML) training

Custom training for teams & individuals — remote or on-site across Australia

Keith Oak

Chief Technology Officer — Lucid Labs

Microsoft Solutions Partner architect specialising in Fabric, Azure Data & AI, and GitHub Enterprise. 18+ years delivering data platforms for Australian businesses — building the systems these exams test every day.

LinkedIn ↗lucidlabs.com.au ↗Published 29-03-2026