Intro and Overview Machine Learning Lecture

  • Author: Prof. Dr. Johannes Maucher

  • Email: maucher@hdm-stuttgart.de

  • Last Update: October, 4th 2021

Goals of this Lecture

Goals of this lecture are:

  • Understand the basic concepts, procedures and categories of Machine Learning

  • For any task or problem: Know, which algorithm-type is suitable for this task

  • Understand conventional ML-algorithms

  • Understand Deep Learning

  • Understand the currently most important types of Deep Neural Networks

  • Implementations are provided in order to support understanding

Contents

Neural Networks and Deep Neural Networks

  1. Conventional Neural Networks

    • Natural Neuron

    • General Notions for Artificial Neural Networks

    • Single Layer Perceptron (SLP)

      • Architectures for Regression and Classification

      • Gradient Descent- and Stochastic Gradient Descent Learning

    • Gradient Descent- and Stochastic Gradient Descent Learning

    • Multilayer Perceptron (MLP) Architectures for Regression and Classification

    • Backpropagation-Algorithm for Learning

  2. Recurrent Neural Networks (RNN)

    • Simple Recurrent Neural Networks (RNNs)

    • Long short-term Memory Networks (LSTMs)

    • Gated Recurrent Units (GRUs)

    • Application Categories of Recurrent Networks

  3. Deep Neural Networks: Convolutional Neural Networks (CNN)

    • Overall Architecture of CNNs

    • General concept of convolution filtering

    • Layer-types of CNNs:

      • Convolution,

      • Pooling,

      • Fully-Connected

  4. MLP and CNN for Object Classification

    • Example Data: Cifar-10 Image Dataset

    • Image Representation in numpy

    • Define, train and evaluate MLP in Keras

    • Define, train and evaluate CNN in Keras

  5. Apply pretrained CNNs for object classification - original task

    • Access image from local file system

    • Download and apply pretrained CNNs for object recognition in arbitrary images

  6. Use of pretrained CNNs for object classification - new task: Classify x-ray images of lungs into healthy and covid-19

    • Download pretrained feature-extractor (CNN without the classifier part)

    • Define new classifier architecture and concatenate it with pretrained classifier

    • Fine-tune network with task-specific data

    • Apply the fine-tuned network for object-recognition

Autoencoder

  1. Autoencoder

Reinforcement Learning

  1. Reinforcement Learning

Modelling of Text

  1. Modelling of Words and Texts / Word Embeddings

    • Concept of Word-Embeddings

    • Skip-Gram and CBOW

    • Working with pretrained word-embeddings

  2. Text Classification with CNNs and LSTMs

    • Example Data: IMDB-Movie Reviews for Sentiment Classification

    • Text preprocessing and representation with Keras

    • Load and apply pretrained word-embedding

    • News classification with CNN

    • News classification with LSTM

Attention, Transformer, BERT

  1. Attention, Transformer, BERT