Skip to main content

Module 10: ML and DL Projects Pipeline

The pipeline includes a high-level machine-learning route, six detailed ML stages, and a 17-step deep-learning workflow. The diagrams below present these steps as responsive, readable cards.

Full-color diagrams showing the ML and DL project pipelines
ML and DL project pipeline

Module 10: ML Project Pipeline

  1. 1Problem Definition
  2. 2Dataset
  3. 3Cleaning
  4. 4EDA
  5. 5Feature Engineering
  6. 6Training
  7. 7Evaluation
  8. 8Deployment
  9. 9Monitoring

The 6 Stages of an ML Project

  1. 1

    Data Loading

    Collect data from a CSV, database or API.

  2. 2

    Preprocessing

    Clean and visualize the data (EDA).

  3. 3

    Feature Engineering

    Create features and split the data into train and test sets.

  4. 4

    Model Training

    Choose a model and train it.

  5. 5

    Evaluation

    Make predictions, evaluate and tune, use cross-validation and build a pipeline.

  6. 6

    Deployment

    Run the model for users through an app or API.

Module 10: DL Project Pipeline

Data Preparation

  1. 1Data Loading (Pandas)
  2. 2Data Cleaning
  3. 3EDA
  4. 4Feature Engineering
  5. 5Train / Val / Test Split
  6. 6Feature Scaling

Tensors and DataLoader

  1. 7Convert Data to PyTorch Tensors
  2. 8Create Dataset & DataLoader

Building the Model

  1. 9Build Neural Network (nn.Module)
  2. 10Define Loss Function
  3. 11Define Optimizer

Training

  1. 12Training Loop
  2. 13Validation Loop

Evaluation and Improvement

  1. 14Model Evaluation
  2. 15Hyperparameter Tuning

Delivery

  1. 16Save Model
  2. 17Deployment

ML project stages​

The overview runs from Problem Definition through Dataset, Cleaning, EDA, Feature Engineering, Training, Evaluation, Deployment and Monitoring.

  1. Data Loading: collect data from a CSV, database or API.
  2. Preprocessing: clean and visualize the data (EDA).
  3. Feature Engineering: create features and split the data into train and test sets.
  4. Model Training: select and train a model.
  5. Evaluation: make predictions, evaluate and tune, use cross-validation, and build a pipeline.
  6. Deployment: run the model for users through an app or API.

DL project stages​

The 17-step sequence starts with data preparation: load data with Pandas, clean it, perform EDA and feature engineering, split it into train/validation/test sets, and scale features. Convert data to PyTorch tensors, then create a Dataset and DataLoader. Build a neural network with nn.Module, define its loss function and optimizer, and run training and validation loops. Finish with model evaluation, hyperparameter tuning, saving the model and deployment.

Module Test​

Continue to Module 10 Test for eight review questions.