Deep Learning

Module aims

This module addresses the fundamental concepts and advanced methodologies of deep learning and relates them to real-world problems in a variety of domains. The aim is to provide an overview of different approaches, both classical and emerging. The module will equip you with the necessary knowledge and skills to work in the field of deep learning and to contribute to ongoing research in the area.

Learning outcomes

Upon successful completion of this module you will be able to:
- express and relate the underlying theoretical concepts of modern deep learning methods
- compare, characterise and quantitively evaluate various deep learning approaches
- evaluate the limitations of deep learning
- apply deep learning techniques to real-world problems in computer vision, speech, text analysis, and graph processing

Module syllabus

• Supervised vs unsupervised learning, generalisation, overfitting
• Perceptrons, including deep vs shallow models
• Stochastic gradient descent and backpropagation
• Convolutional neural networks (CNN) and underlying mathematical principles
• CNN architectures and applications in image analysis
• Recurrent neural networks (RNN), long-short term memory (LSTM), gated recurrent units (GRU)
• Applications on RNNs in speech analysis and machine translation
• Mathematical principles of generative networks; variational autoencoders (VAE); generative adversarial networks (GAN)
• Applications of generative networks in image generation
• Graph neural networks (GNN): spectral and spatial domain methods, message passing
• Applications of GNNs in computational social sciences, high-energy physics, and medicine

Pre-requisites

Teaching methods

The material is taught through traditional lectures, backed up by unassessed, formative tutorial exercises and online quizzes, all designed to reinforce understanding of the comprehensive course notes that accompany the module. The tutorial exercises will be accompanied by specimen answers, and the tutorials will be supported by Graduate Teaching Assistants (GTAs). The tutorial questions will include past exam questions, in preparation for the final exam.
Direct communication channels are provided through MS Teams and a discussion forum. Lecturers and GTAs discuss lecture content and answer questions on these platforms.

Assessments

The coursework consists of multiple tasks contributing a total of 50% of the marks for the module. There will be a final written exam, contributing the remaining 50% of the module marks, which will test both theoretical and practical aspects of the subject.

There will be written feedback for the assessed coursework and class-wide feedback explaining common pitfalls and suggestions for improvement. For unassessed tutorials with the lecturers and GTAs, written solutions are provided after an in-class tutorial session. Unassessed online quizzes give direct feedback online on an individual level.   

Module leaders

Dr Yingzhen Li
Professor Bernhard Kainz