Unit 5 of 15

Classification, Probabilities, and Decisions

  • 9 lessons
  • Videos 27 to 35

Classification predicts a category, often through a probability. This unit fits probabilistic classifiers, turns their scores into decisions and checks whether the probabilities can be trusted.

Logistic regression turns a linear score into a probability through the log-odds, maximum likelihood gives the cross-entropy loss and softmax handles several classes. A threshold turns a score into a decision, a confusion matrix counts the outcomes and expected cost compares thresholds. Precision, recall, ROC and precision-recall curves answer different questions when positives are rare, and class weights and resampling are kept apart from the threshold. Calibration checks whether predicted probabilities match observed frequencies. Naive Bayes and Gaussian discriminant analysis close the unit.

Every lesson will be one video. The videos are coming soon, and each lesson page already has the definitions and an example for every key term.

Logistic Regression and Likelihood

  1. Logistic Regression

    How logistic regression turns a linear score into a probability with the sigmoid function, how odds, log-odds and odds ratios read its coefficients, and where a threshold puts the decision boundary.

    Video 27

  2. Maximum Likelihood and Cross-Entropy

    How the Bernoulli likelihood of observed labels leads through the log likelihood to binary cross-entropy, how maximum likelihood estimation fits a model, and how a prior distribution gives MAP estimation and penalties.

    Video 28

  3. Multiclass Classification and Softmax

    How softmax turns class scores into a class probability vector, how multiclass cross-entropy and the log-sum-exp trick train it stably, and how multiclass, multilabel and one-vs-rest setups differ.

    Video 29

Decisions, Metrics and Calibration

  1. Confusion Matrices and Decision Thresholds

    How a decision threshold turns scores into actions, how a confusion matrix counts right and wrong decisions, what sensitivity and specificity measure, and how expected cost sets the threshold.

    Video 30

  2. Precision, Recall, and Classification Metrics

    What accuracy, precision, recall and the F1 score measure at one threshold, and how the ROC curve, ROC AUC, the precision-recall curve and average precision compare score rankings across thresholds.

    Video 31

  3. Imbalanced Classes and Cost-Sensitive Learning

    Why accuracy misleads when one class is rare, how prevalence, balanced accuracy and macro and micro averages describe performance, and how class weights, SMOTE and cost-sensitive thresholds differ.

    Video 32

  4. Probability Calibration

    How reliability diagrams, the Brier score and log loss check predicted probabilities against event rates, how calibration differs from discrimination, and how Platt scaling and isotonic regression use a calibration set.

    Video 33

Generative Classifiers

  1. Naive Bayes Classification

    How naive Bayes combines class priors with per-class feature probabilities, what its conditional independence assumption says, why Laplace smoothing prevents zero probabilities, and why scores use log probabilities.

    Video 34

  2. Gaussian Discriminant Analysis

    How Gaussian discriminant analysis models each class with a multivariate Gaussian density, why the covariance must be positive definite, and how shared or separate covariances give linear or quadratic boundaries.

    Video 35