Intro and Overview Machine Learning Lecture#

Goals of this Lecture#

Goals of this lecture are:

  • Understand the basic concepts, procedures and categories of Machine Learning

  • For any task or problem: Know, which algorithm-type is suitable for this task

  • Understand conventional ML-algorithms

  • Understand Deep Learning

  • Understand the currently most important types of Deep Neural Networks

  • Implementations are provided in order to support understanding

Contents#

Intro#

  1. Basic Concepts of Data Mining and Machine Learning

    • Definition

    • Categories

    • Validation

  2. Basics of Probability Theory (external Link)

Conventional ML Algorithms#

  1. K-Nearest Neighbours

  2. Bayes Classification

  3. Linear Regression

  4. Linear Classification

  5. Support Vector Machines (SVM)

  6. Gaussian Process

Neural Networks and Deep Neural Networks#

  1. Conventional Neural Networks

    • Natural Neuron

    • General Notions for Artificial Neural Networks

    • Single Layer Perceptron (SLP)

      • Architectures for Regression and Classification

      • Gradient Descent- and Stochastic Gradient Descent Learning

    • Gradient Descent- and Stochastic Gradient Descent Learning

    • Multilayer Perceptron (MLP) Architectures for Regression and Classification

    • Backpropagation-Algorithm for Learning

  2. Recurrent Neural Networks (RNN)

    • Simple Recurrent Neural Networks (RNNs)

    • Long short-term Memory Networks (LSTMs)

    • Gated Recurrent Units (GRUs)

    • Application Categories of Recurrent Networks

  3. Deep Neural Networks: Convolutional Neural Networks (CNN)

    • Overall Architecture of CNNs

    • General concept of convolution filtering

    • Layer-types of CNNs:

      • Convolution,

      • Pooling,

      • Fully-Connected

  4. MLP and CNN for Object Classification

    • Example Data: Cifar-10 Image Dataset

    • Image Representation in numpy

    • Define, train and evaluate MLP in Keras

    • Define, train and evaluate CNN in Keras

  5. Apply pretrained CNNs for object classification - original task

    • Access image from local file system

    • Download and apply pretrained CNNs for object recognition in arbitrary images

  6. Use of pretrained CNNs for object classification - new task: Classify x-ray images of lungs into healthy and covid-19

    • Download pretrained feature-extractor (CNN without the classifier part)

    • Define new classifier architecture and concatenate it with pretrained classifier

    • Fine-tune network with task-specific data

    • Apply the fine-tuned network for object-recognition

Autoencoder#

  1. Autoencoder

GAN#

  1. Generative Adversarial Networks

Reinforcement Learning#

  1. Reinforcement Learning

Modelling of Text#

  1. Modelling of Words and Texts / Word Embeddings

    • Concept of Word-Embeddings

    • Skip-Gram and CBOW

    • Working with pretrained word-embeddings

  2. Text Classification with CNNs and LSTMs

    • Example Data: IMDB-Movie Reviews for Sentiment Classification

    • Text preprocessing and representation with Keras

    • Load and apply pretrained word-embedding

    • News classification with CNN

    • News classification with LSTM

Graph Neural Networks#

  1. Graph Neural Networks

    • Concepts of GNNs

    • Implementation of GNN with Keras

    • Document Classification with GNNs

Attention, Transformer, BERT#

  1. Attention, Transformer, BERT