AI: Making an ML Model

DAIM Team

Module 4: Part 1 Learning Objectives

• Appreciate what is meant by “machine learning” and “AI”
• Understand the high-level steps needed to train a statistical model
• Rationale for “verifying your model findings” and how to use a test set
• Understand dataset bias and clinical factors

What is machine learning?

• Machine learning describes a collection of algorithms that allow computers to “learn” from data.
• This stands in contrast to desccribing exactly what an algorithm should detect with code

Big Data, Machine Learning, and Artificial Intelligence

• Big data support the development of ML/AI
• The six Vs of big data: volume, variety, velocity, veracity, value, and variability
• I would like to add vulnerability

Machine Learning vs. Artificial Intelligence

• There are no set, global definitions for either term!
• Machine learning (ML) often refers to a specific subset of statistical algorithms.
  • Artificial neural networks
  • Decision trees
  • Support vector machines
  • And more - it is a loose category

Machine Learning vs. Artificial Intelligence

• There are different types of ML:
  • Supervised learning
  • Unsupervised learning
  • Reinforcement Learning
• Deep Learning refers to neural networks with multiple layers
  • We will be building one in the workshop.

Supervised / Unsupervised Learning

• Supervised learning uses labelled data.
  • For instance, a neurologist labels data with different types of EEG patterns.
  • The model learns to predict which pattern is present.

See this Kaggle Competition.

Supervised / Unsupervised Learning

• Unsupervised learning uses unlabelled data
  • For instance, an ML system is given a series of normal chest radiographs and learns how best to compress and uncompress them.
  • This is known as an autoencoder.

Variational Autoencoder - By EugenioTL - Own work, CC BY-SA 4.0, link here

Appreciate

Right??

How to train an ML model

What steps are needed to train a machine learning model?

• Collecting or identifying a dataset
• Partitioning a dataset into groups
• Preprocessing your dataset
• Designing the architecture of the model
• Training the model
• Evaluating the model
• Deploying the model

Machine Learning Process

The MLOps Lifecycle

• MLOps = Machine Learning + Operations
• Key stages include:
  • ETL pipeline: Extract, Transform, and Load data
  • Train the model: Run computations to optimize model performance
  • Deploy the model: Serve it to users in real-time, with continuous deployment

How do you split up a dataset?

• A dataset is typically split into three parts
  1. Training data (training the model)
  2. Testing data (testing the model after training)
  3. Validation data (testing the model during training)
• The reasons for this will be discussed over the course of these two seminars.

Bias in Machine Learning

Bias in clinical applications

• Bias can come from many places during creation of a clinical dataset.
  • Missing data for certain groups
  • Lack of robust data collection protocols for researchers
  • Biased sampling
• This can lead to algorithms that underperform for certain groups.

Source: [1] (see sources slide.)

Case Study 1 - Dermatology

• Convolutional neural networks can be trained to classify skin lesions.
• Dataset contains 5-10% images of lesions on black skin.
• Diagnostic accuracy halves when trialled with images of black skin.

Source: [1] (see sources slide.)

Case Study 2 - Heart Disease Predictors

• Predictive models for heart disease are often trained on datasets that have more data from male patients.
• How might this affect the clinical application of the tool?

Source: [1] (see sources slide.)

How to preprocess data for an ML model?

Data preprocessing

• Data preprocessing refers to optimising the data in your dataset for the model that you have chosen to train.
• Different models have different formats which the data must be in
• Data is often passed to the model as a NumPy array.
  • These are often referred to as tensors

What kind of NumPy Array?

• This will depend on the type of data being used.
• For images, it is likely to be an image with certain:
  • Dimensions (e.g. 256 x 256)
  • Number of channels (e.g. 1 or 3)
• We will explore this further in the workshop.

What is data augmentation?

• Often, datasets are small and machine learning models need a large amount of data to be robust to natural variations.
• This includes changes in brightness, contrast, rotation, etc.
• This can be simulated in a dataset by varying these parameters for each image.

Example of data augmentation

Examples of different data augementation techniques. Source: [2]

Going further than this…

• In imaging, further data augmentation techniques can be applied to simulate anatomical variation
• For instance, elastic deformation
• Subtle image distortion to simulate data acquired from a different patient.

Data augmentation

• There are many more techniques that can be applied to extend a small dataset
• There are more techniques for other types of data (e.g. time series data).

Break!

The workshop task

• In the workshop, we will be building a machine learning system to predict pneumonia on chest XR.
• This will demonstrate the process of training a model on a dataset.
• The technique is a cornerstone of building neural networks that interpret images
  • We will be using tried-and-tested techniques to demonstrate first principles.

What is learning?

Machine Learning and Philosophy of Learning

• Learning? Why do we learn?
• Difference between learning and memorizing
• What does it mean mathematically?
• What does training mean for you and for a machine learning model?

Learning vs. Memorizing

• Using an example of adding and the symbol +:
  • You learn to add and use the symbol +
  • This allows you to generalize rather than memorizing specific sums, like 1002 + 2003 = 3005
  • You train by practicing simple exercises like 1 + 1 = 2
  • Your teacher corrects if you get 1 + 1 = 3

Learning vs. Memorizing

• You are then examined on new material to ensure you can generalize and apply your learnings
• Similarly: You use a training dataset to train the model so it can learn and apply in a general context.

What does learning mean for a computer?

\[ Y = f(x_1, x_2, x_3, ...) \]

• Find the set of variables (x1, etc.) and that minimises the outcome measure (Y) of a function (f(x))
  • Variables - the weights, biases, and other parameters of your network
  • f(x) - the loss function, which tells the model how incorrect it is by calculating the loss (Y)

What does learning mean for a computer?

• How do we know how to change the variables?
  • Optimizer = acts like a teacher, with the goal to minimize the loss function

How does the optimiser make the network “learn”?

• A common analogy is one of a ball rolling down a hill.
• The height of the hill (z) represents the loss (error), and the two other directions (x1, x2) are model parameters

How does the optimiser make the network “learn”?

• The “ball” (the model) rolls down the hill to find the value of the two values that minimises the loss.
• This is called gradient descent.

How does the optimiser make the network “learn”?

• Explanation
• Diagram

• The animation demonstrates the process for 2 parameters in 2D.
• Machine learning models have 100,000s of parameters.
• It’s difficult to visualise a ball rolling around in a multi-million dimensional space!

Gradient descent. By Gpeyre - Own work, CC BY-SA 4.0, link here

Break!

How do we create an ML model?

Creating an ML model

• Many ML algorithms are designed and trained with frameworks within Python
• Frameworks are Python packages that allow for simplified use of high-level network building blocks.
  • This saves the user from dealing with low-level details.
  • This is called abstraction.
• Popular frameworks include TensorFlow and PyTorch.

What are these “building blocks”?

• Various layers are used to build a neural network.
• A layer is a set of neurons with inputs/outputs and parameters which can be learned.
• For simple networks, layers are connected to each other sequentially.
• The presence of multiple layers is where the term deep learning originates.

What types of layer are there?

• Dense layer
  • This is the most common layer.
  • Consists of neurons that are connected to every neuron in the preceding and subsequent layer (hence Dense)
• Therefore they take up lots of memory and computing power!
  • Think about how the number of connections grows between neurons as the number increases.

Dense layer

A diagram of a dense layer with 5 neurons, connected to another dense layer with 3 neurons.

Convolutional layers

• Convolutional layers involve convolving a kernel with the input image
  • Look back to module 1 for a refresher.
• However, the parameters of the kernel are learned
• Multiple different kernels are used in the layer to evaluate different aspects
• As convolution uses the same kernel for the whole image, they have fewer parameters than Dense layers.

Pooling layers

• Pooling layers reduce the size of the image through the network
• This is often done by a factor of 2 e.g. (256, 256, 3) to (128, 128, 3)
• This allows for the model to “shed” less important, fine detail information

Why do we need to reduce the image size throughout the network?

• When images are connected to Dense layers, they are flattened.
  • e.g. (256, 256) to (65536)
• Even small images will result in huge networks, as all neurons between Dense layers are connected
  • (256, 256) to a Dense layer with 256 neurons: 16.8 million parameters!

What are the inputs to the network?

• The input to neural networks which process images will be an image.
  • The image will have a fixed resolution and number of channels
  • e.g. (128, 128, 3)

What are the outputs?

• The output will vary depending on the architecture of the network:
  • Probabilities that certain features are present (e.g. pneumonia)
  • A processed image, showing the locations of objects/features
    • This is called semantic segmentation
  • It can even be the same image!
    • This is an autoencoder which is used for compression.

How to train an ML model?

Batches of data

• Data is fed to the network in batches.
  • A batch is a collection of datapoints e.g. 16 images.
• The overall direction to change the network parameters is established after asking the network to predict each batch item.
• This gradually minimises the loss and therefore the error of the network.

Epochs of training

• After a certain number of batches are fed to the network, an epoch of training has been completed.
• The network is then usually evaluated on the validation dataset.
• Repeat… until a set number of epochs has been completed.

The process of training

Number of epochs plotted against loss in an ideal situation.

What is a hyperparameter?

• Hyperparameters determine how your network learns and the network structure.
• Any parameter that cannot be modified by the network optimiser during training can be seen as a hyperparameter.
• Tuning hyperparameters is important for extracting the best performance from your network.

What is a hyperparameter?

• Examples include:
  • The number of neurons in each layer.
  • The number of kernels in a convolutional layer.
  • The learning rate of the network.

Conclusion

• We have covered a lot of information in this lecture.
• We will recap nearly all of it during the workshop.

Further resources

• Further resources
  • 3Blue1Brown’s series on the basics of neural networks. This covers more mathematical detail about how these networks work.
  • Google’s Machine Learning Crash Course. This provides more insight into different types of model and different datasets.

Thank you!

Any questions?

Sources

[1] - Norori N, Hu Q, Aellen FM, Faraci FD, Tzovara A. Addressing bias in big data and AI for health care: A call for open science. Patterns (N Y). 2021;2(10):100347. Published 2021 Oct 8. doi:10.1016/j.patter.2021.100347

[2] - Kermany, Daniel; Zhang, Kang; Goldbaum, Michael (2018), “Labeled Optical Coherence Tomography (OCT) and Chest X-Ray Images for Classification”, Mendeley Data, V2, doi: 10.17632/rscbjbr9sj.2. Data adapted under CC BY 4.0 Licence.