A machine learning (ML) pipeline is the series of stages that turns a business problem and raw data into a working model that keeps performing well. The exam expects you to name each stage, say what it produces, and recognise which stage a scenario is describing.
From Business Goal to ML Problem
Every pipeline starts before any data is touched. First, the team agrees on the business goal: the problem to solve and how success will be measured, such as "cut late deliveries by 10%". Then comes problem framing: turning that goal into something a model can predict. For late deliveries, the model might predict "will this order arrive late: yes or no?" The thing being predicted is called the label or target. If the goal can't be measured, or the problem can't be framed as a prediction, ML may not be the right tool.
Data Collection and Exploration
Next, the team gathers data from wherever it lives: databases, logs, files, or third-party sources. Exploratory data analysis (EDA) comes next. It means looking at the data with charts and summary statistics to understand it before building anything. EDA reveals missing values, odd outliers, and whether the data actually contains signals related to the label. Skipping it often means discovering problems only after a model has been trained on bad data.
Pre-processing and Feature Engineering
Raw data is rarely ready for training. Pre-processing cleans it: filling or removing missing values, fixing formats, removing duplicates, and splitting the data into training, validation and test sets. Feature engineering creates the inputs the model learns from. A feature is one measurable input, such as "distance to customer" or "day of week". Engineering features might mean combining columns, converting dates into weekdays, or turning text categories into numbers. Good features often matter more than the choice of algorithm.
Training, Tuning and Evaluation
During training, an algorithm learns patterns from the training data and produces a model. Hyperparameter tuning adjusts the settings that control how training happens, such as how long it runs, to find the best-performing version. Evaluation then tests the model on data it hasn't seen, using metrics such as accuracy or recall, to decide whether it's good enough for the business goal.
Deployment and Monitoring
Deployment puts the model where applications can send it new data and get predictions back. That stage is called inference. Monitoring watches the deployed model over time. Real-world data drifts: customer behaviour changes, new products appear, and accuracy quietly drops. When monitoring spots a drop, the team goes back to earlier stages, often retraining with fresh data. That's why the pipeline is drawn as a loop, not a straight line.
On the Exam
| Stage | What it produces |
|---|---|
| Data collection | Raw data gathered in one place |
| EDA | Understanding of the data and its problems |
| Pre-processing | Clean data split into training, validation and test sets |
| Feature engineering | Model-ready input features |
| Training and tuning | A trained model with chosen hyperparameters |
| Evaluation | Metrics showing whether the model meets the goal |
| Deployment | An endpoint or job that serves predictions |
| Monitoring | Alerts about drift that trigger retraining |
A common trap is mixing up pre-processing (cleaning what's there) with feature engineering (creating new inputs). Another is forgetting that monitoring feeds back into retraining.