Deep Learning Syllabus
(1) Introduction To Deep Learning:
- Overview of deep learning and its applications.
- Historical development and milestones in deep learning.
- Basics of neural networks and their components.
(2) Artificial Neural Networks:
- Perceptrons and activation functions
- Feedforward neural networks
- Training algorithms: gradient descent, backpropagation
- Regularization techniques: dropout, weight decay
- Optimization algorithms: stochastic gradient descent, Adam
- Initialization strategies
(3) Convolutional Neural Networks (CNNs):
- Introduction to CNNs and their architecture.
- Convolutional layers, pooling layers, and fully connected layers
- CNN training and optimization
- CNN applications in computer vision tasks (e.g., image classification, object detection)
(4) Recurrent Neural Networks (RNNs):
- Introduction to RNNs and their architecture.
- Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU).
- Training and optimization of RNNs.
- RNN applications in natural language processing, speech recognition, and sequence generation
(5) Encoder-Decoder Architecture:
- Introduction to Encoder and Decoder.
- Exploring different variations of the encoder architecture, such as uni-directional and bi-directional encoders.
- Understanding how the encoder processes the input sequence and generates a fixed-length representation (context vector).
- Understanding the working of the decoder, which generates the output sequence based on the context vector.
- Different decoding strategies, including teacher forcing, scheduled sampling, and beam search decoding.
(6) Deep Learning for Natural Language Processing (NLP):
- Word embeddings: Word2Vec, GloVe
- Recurrent Neural Networks for NLP tasks (e.g., language modelling, sentiment analysis).
- Attention mechanisms and Transformer architecture.
- Pretrained language models (e.g., BERT, GPT)
(7) Generative Adversarial Networks (GANs):
- Introduction to GANs and their architecture.
- Training GANs: adversarial learning, loss functions.
- Applications of GANs in image generation, style transfer, and data augmentation
(8) Transformers:
- Attention Mechanism.
- Transformer Architecture.
- Positional Encoding.
- Transformer Encoder.
- Transformer Decoder.
- Training Transformers.
- Transfer Learning and Pre-trained Transformers.
- Advanced Transformer Models and Variants.
- Applications of Transformers.
(8) Transfer Learning and Fine-tuning:
- Transfer learning concepts and techniques.
- Pretrained models and their adaptation to new tasks.
- Fine-tuning strategies
(9) Reinforcement Learning (RL):
- Introduction to RL and its connection to deep learning.
- Understanding the RL framework, including agents, environments, and rewards.
- Deep Q-Networks (DQNs) and policy gradient methods in RL.
(10) Advanced Topics:
- Reinforcement Learning and Deep Q-Networks (DQN).
- Autoencoders and Variational Autoencoders (VAEs).
- Deep Reinforcement Learning.
- Deep Learning for time series analysis
(11) Practical Implementation and Tools:
- Deep learning frameworks (e.g., TensorFlow, PyTorch).
- Model development and training.
- Best practices in deep learning projects.
- Model deployment and serving.