GPU An Essential Requirement For Deep Learning

GPUs or graphics processing units is a processor that is good at handling computations.

Ok!! but the same things are done by CPUs or (Central Processing Unit), about which we have been learning from our school days as the “brain of the computer”.

So an important question what differentiates GPUs from CPUs?

To answer this question lets tweak our earlier definition a bit -
GPUs are good at handling “specialized computations” and CPUs are good at handling “general computations”.

Now another question what is an example of special and general computation?
When you train a deep learning model, two main operations are performed:

1. Forward Pass
2. Backward Pass

In the forward pass, the input is passed through the neural network and after processing the input, an output is generated. Whereas in the backward pass, we update the weights of the neural networks on the basis of error we get in forward pass.

Both of these operations are essentially matrix multiplications. Simple matrix multiplication can be represented by the image below

Here, we can see that each element in one row of the first array is multiplied with one column of the second array. So in a neural network, we can consider the first array as input to the neural network, and the second array can be considered as weights of the network.

This seems to be a simple task. Now just to give you a sense of what kind of scale deep learning — VGG16 (a convolutional neural network of 16 hidden layers which is frequently used in deep learning applications) has ~140 million parameters; aka weights and biases. Now think of all the matrix multiplications you would have to do to pass just one input to this network! It would take years to train this kind of system if we take traditional approaches.

How to train your neural net faster?
We saw that the computationally intensive part of the neural network is made up of multiple matrix multiplications. So how can we make it faster?

We can simply do this by doing all the operations at the same time instead of doing it one after the other. This is in a nutshell why we use GPU (graphics processing units) instead of a CPU (central processing unit) for training a neural network.

Training of the neural network was a specialized computation and here GPU is faster than CPU but it’s not always so. You yourself can check this by taking an example of simple scalar multiplication and you will see that CPU is faster than GPU. Scalar multiplication is an example of general computation.

Now you might be wondering why GPU is faster in neural networks then?

The answer is:-

1. Larger Memory Bandwidth
2. They use Parallelization
3. Fast(or more )memory access

In CPUs, a smaller amount of data is fetched at a faster rate whereas in GPUs a larger amount of data are fetched at a slower rate. Memory Bandwidth is the amount of data transferred in a single trip to and from memory. Therefore we can say GPUs has higher memory bandwidth than CPUs.

Coupling GPU with Parallelization we increase the speed of GPUs.

GPUs have faster memory since GPUs L1 and L2 cache are smaller than CPUs L1 and L2 cache and smaller cache means they are accessed much faster.Also, the streamlined processors consist of registers that are super fast.

References:-

Why are GPUs more powerful than CPUs?

Why are GPUs necessary for training Deep Learning models?

CUDA — GPU Memory Architecture

If you gained any piece of information from this blog don’t forget to 👏!!