Basic Concepts of Probability Theory for AI and ML
Basic Concepts of Probability Theory for AI and ML#
Author: Prof. Dr. Johannes Maucher
Institution: Stuttgart Media University
Last Update: 04.08.2022
In this document the basic notions, rules and theorems of Probability Theory are introduced. The goal is to provide the skills from this subject, which are fundamental for Artificial Intelligence and Machine Learning. This means that this introduction does not claim to present the field in its entirety.
The topics of this document are summarized in the picture below.
In Probability Univariate only a single random variable, either discrete or continuous, is considered. These concepts are then extended to the multivariate case in Probability Multivariate. Here, joint- and conditional probabilitis as well as the corresponding fundamental laws such as chain-rule, marginalisation and Bayes Theorem are introduced.
In Estimate Probability it is shown how Probability Distributions can be estimated from data. In Example Estimate Probability it is shown how the Python package Pandas can be applied to estimate joint- and conditional probabilities from discrete datasets.
Section Bayes Classifier builds the bridge to Machine Learning. Concrete, is shown how the Bayes Rule is applied for Bayesian Inference and how a corresponding classification algorithm can be learned from data.
Bayesian Networks constitute an important framework to model uncertainty in Artifical Intelligence. They allow to represent uncertain knowledge and provide means for causal- and diagnostic- probabilistic inference. Section Bayesian Networks demonstrates how Bayesian Networks can be constructed and applied with the Python package pyAgrum. In contrast to Machine Learning algorithms, Bayesian Networks provide the important capability to integrate knowledge from data with expert-knwoledge. In Learn Bayesian Networks it is shown how the parameters of a Bayesian Network can be learned from data.