Skip to main content
Search by keyword

Deep Learning Advanced Learning Path - MAI4CAREU Master in AI

Deep Learning Advanced Learning Path - MAI4CAREU Master in AI

This Deep Learning (DL) learning path offers a structured curriculum that delves into Deep Learning, a subfield that has been associated with the revolution of AI in the last decade, with advancements in vision, natural language processing, attention and artificial general intelligence. The impact of DL extends in societal, technological, industrial, and financial sectors – and is poised to shape our future even more profoundly.

It is based on the elective advanced course offered at the Master in Artificial Intelligence of the University of Cyprus, which was developed with co-funding from the MAI4CAREU European project. The course offers a guided exploration of DL’s fundamental principles and diverse applications, and consists of twelve units organized in three parts as shown below, which are further arranged into Introductory (eight units), and Advanced (four units) according to their level of difficulty. The recommended order for studying all materials is the one shown below (from 1 to 12).

Part I: Introduction

  1. Introduction to Deep Learning
  2. Deep Learning Fundamentals
  3. Mathematics of Deep Learning

Part II: Deep Neural Networks

  1. Principles of Deep Neural Networks
  2. Convolutional Neural Networks
  3. Transfer Learning and Residual Networks
  4. Optimizing Deep Neural Networks
  5. Recurrent Deep Neural Networks

Part III: Advanced and Emerging Topics in Deep Learning

  1. Attention and Transformers
  2. Generative Adversarial Networks
  3. Deep Reinforcement Learning
  4. Emerging Topics in Deep Learning 
Introductory learning materials

MAI4CAREU - Deep Learning - Introduction to Deep Learning

Deep learning has evolved rapidly over the past decade, driven by advances in computing power, the availability of vast amounts of data, and breakthroughs in neural network architectures. Initially inspired by the structure and function of the human brain, deep learning algorithms, particularly neural networks with multiple layers, have demonstrated unprecedented performance in various tasks such as image recognition, natural language processing, and speech recognition.  In the first introductory lecture, students are introduced to the foundational concepts and principles of this powerful subset of machine learning. The lecture begins by defining deep learning and its significance in modern artificial intelligence, highlighting its ability to automatically learn representations from data. Basic components of neural networks, including neurons, layers, and activation functions, are explained, and key training techniques like backpropagation and gradient descent are also introduced, emphasizing their role in optimizing neural network parameters. The unit also introduces basic structures of deep neural network models and architectures. 

MAI4CAREU - Deep Learning - Fundamentals

Prior to understanding the basics of deep learning, students will be introduced to foundational concepts in mathematics, including calculus, linear algebra, and probability theory, as these form the basis of many deep learning algorithms. They will also reiterate data representation, data preparation, dimensions, and tensors. The unit also covers machine learning principles, such as supervised and unsupervised learning, as well as knowledge of optimization algorithms like gradient descent. Additionally, an overview of neural networks and their components, like activation functions and layers, are included to set the stage for deeper exploration in the course. 

MAI4CAREU - Deep Learning - Mathematics of Deep Learning

This Unit introduces the students to the mathematics of deep learning, with fundamental concepts in calculus, linear algebra, and probability theory covered. This includes derivatives and gradients for optimization, matrix operations for handling data transformations, and probability distributions for understanding uncertainty in predictions. Additionally, students will learn about key mathematical operations within neural networks, such as activation functions and their derivatives. Overall, the Unit aims to equip students with the mathematical tools necessary to understand and implement deep learning algorithms effectively. 

Advanced learning materials

MAI4CAREU - Deep Learning - Attention and Transformers

The first of the advanced units, traces the evolution of recurrent neural networks (RNNs) from basic architectures to the emergence of transformers, underlining the necessity for more efficient and scalable models. As sequences grew longer, these models struggled to capture long-range dependencies effectively. The introduction of attention mechanisms mitigated this challenge by allowing models to focus on relevant information, paving the way for transformers. Transformers revolutionized sequence modelling by solely relying on attention mechanisms, enabling parallelization and efficient handling of long-range dependencies. This narrative underscores the iterative progression towards attention-based architectures driven by the need for scalable models capable of handling diverse sequential data efficiently across domains like machine translation, sentiment analysis, and speech recognition. 

MAI4CAREU - Deep Learning - Generative Adversarial Networks

Following up the unit on transformers, the next advanced topic introduces generative modelling, where students will learn the principles of generative adversarial networks (GANs) alongside other influential approaches like deep generative adversarial networks (DGANs), deep belief networks (DBNs), and encoder-decoder architectures. The unit elucidates the foundational principles of GANs, emphasizing the adversarial training process between a generator and discriminator to produce realistic data samples. Additionally, students learn about the advancements introduced by DGANs, which leverage deep architectures to enhance the quality and diversity of generated data samples. DBNs are introduced as probabilistic generative models capable of learning intricate data distributions, while encoder-decoder architectures enable tasks such as image-to-image translation and text-to-image synthesis. Throughout the lecture, the profound impact of these generative models on diverse domains, from computer vision to natural language processing, is showcased, highlighting their transformative role in modern machine learning research and applications. 

MAI4CAREU - Deep Learning – Deep Reinforcement Learning

The second-to-the-last unit focuses on deep reinforcement learning (DRL), where students are introduced to the principles and theory behind training agents to make sequential decisions in dynamic environments. The lecture covers the foundational concepts of reinforcement learning (RL), highlighting the interaction between an agent, its environment, and rewards. Students learn how deep learning techniques are integrated with RL algorithms to handle high-dimensional input spaces and complex decision-making tasks. Applications of DRL in domains such as robotics, game playing, and autonomous systems are explored, showcasing its potential for solving real-world problems. However, the lecture also addresses the limitations and challenges of DRL, including sample inefficiency, stability issues, and the need for exploration strategies. Despite these challenges, the lecture underscores the transformative impact of DRL on various fields, from healthcare to entertainment, and its potential to revolutionize the way machines learn and interact with the world. Through this comprehensive exploratory unit, students will gain a deep understanding of the principles, applications, challenges, and impact of DRL in modern artificial intelligence research and practice. 

Learning path Details

Digital skill level
Digital technology / specialisation