Deep Learning Using Python (DLUP01)

Session 1 – 02:00:00 – Introduction to Artificial Neural Networks
This session establishes the conceptual and historical foundations of neural networks. We begin with biological neurons and how they inspired artificial neurons, tracing the history from early perceptrons through AI winters to the modern deep learning revolution. The perceptron model is introduced: weighted sums of inputs, bias terms, and activation functions that introduce non-linearity. We explore why single-layer perceptrons are limited and how multilayer networks overcome these limitations through hidden layers. Various activation functions are covered (sigmoid, tanh, ReLU, GELU) with their mathematical properties and practical trade-offs. Network architecture is explained: input layers receiving features, hidden layers extracting representations, and output layers making predictions. The universal approximation theorem is discussed, showing that neural networks can approximate any continuous function given sufficient capacity. The session concludes with the modern deep learning revolution enabled by GPUs, large datasets, and algorithmic improvements.

Break – 01:00:00

Session 2 – 02:00:00 – Training Neural Networks
This session explains how neural networks learn from data through backpropagation and gradient descent. We begin with loss functions that quantify prediction errors: mean squared error for regression, cross-entropy for classification. Backpropagation is explained as the method for computing gradients of the loss with respect to every parameter in the network, using the chain rule to propagate errors backward through layers. Gradient descent and its variants are covered: stochastic gradient descent (SGD), momentum methods that accelerate convergence, and adaptive methods like Adam that adjust learning rates per parameter. The mechanics of batch training are explained: mini-batches, epochs, and iterations. Overfitting in neural networks and regularization techniques are covered in detail: dropout (randomly disabling neurons during training), weight decay (L2 regularization penalizing large weights), and early stopping (halting training when validation performance plateaus). Train/validation/test splitting strategies for deep learning are discussed. The session introduces PyTorch’s fundamental concepts: tensors (multidimensional arrays), autograd (automatic differentiation for computing gradients), and computational graphs that track operations for backpropagation.

Break – 01:00:00

Session 3 – 01:00:00 – Multilayer Perceptrons with PyTorch
This session provides hands-on implementation of multilayer perceptrons using PyTorch. We begin with PyTorch fundamentals: creating and manipulating tensors, understanding devices (CPU vs GPU), and data types for numerical computation. Building networks with nn.Module is covered in detail: defining the network class, specifying layers in init, implementing the forward pass that computes predictions. Network architecture decisions are discussed: number of hidden layers, layer widths, activation functions between layers. The complete training loop is implemented step-by-step: moving data to device, forward pass through the network, computing loss, backward pass to compute gradients, and optimizer step to update parameters. We apply these concepts to MNIST digit classification, building a complete MLP (784 input units → hidden layers → 10 output units) and training it to recognize handwritten digits. Monitoring training progress through loss curves and validation accuracy is demonstrated. The skorch library is introduced as a scikit-learn wrapper for PyTorch, providing familiar fit/predict interfaces. Practical tips are covered: debugging network architecture errors, common mistakes (forgetting to zero gradients, incorrect tensor shapes), and using GPU acceleration to speed up training. By the end of this session, participants have trained a working neural network and understand the full pipeline from data to trained model.