Machine Learning

Video 2

feature, target, label, unit of observation, prediction time, prediction horizon, data schema and target leakage

Builds on What Is Machine Learning?.

Video 3

NumPy array, pandas DataFrame, array shape, indexing and slicing, array axis, elementwise multiplication, broadcasting and notebook

Builds on Features, Targets, and Prediction Time. Also uses basic Python variables and functions.

Video 4

training set, validation set, test set, model parameter, hyperparameter, constant baseline, generalization and deployment population

Builds on What Is Machine Learning?, Features, Targets, and Prediction Time and Python Arrays and Tables for Machine Learning.

Video 5

data leakage, information boundary, random row split, group split, temporal split, duplicate leakage and preprocessing leakage

Builds on Features, Targets, and Prediction Time, Python Arrays and Tables for Machine Learning and Training, Validation, and Test Sets.

Video 6

exploratory data analysis, data quality, empirical distribution, mean, median, quantile, histogram and outlier

Builds on Features, Targets, and Prediction Time, Python Arrays and Tables for Machine Learning, Training, Validation, and Test Sets and Data Leakage and Honest Splitting.

Video 7

missing value, missing completely at random, missing at random, missing not at random, imputation, missingness indicator, complete-case analysis and missing target

Builds on Exploratory Data Analysis and Data Quality and Data Leakage and Honest Splitting.

Video 8

feature scaling, standardization, standard deviation, min-max scaling, train-only fit, one-hot encoding, ordinal category and unknown category

Builds on Python Arrays and Tables for Machine Learning, Data Leakage and Honest Splitting and Missing Data and Reliable Imputation.

Video 9

k-nearest neighbors, Euclidean distance, majority vote, k-nearest neighbor regression, number of neighbors and tie-breaking rule

Builds on Python Arrays and Tables for Machine Learning, Training, Validation, and Test Sets, Data Leakage and Honest Splitting and Feature Scaling and Categorical Encoding. Also uses points on a coordinate plane and the Pythagorean theorem.

Video 10

overfitting, underfitting, model complexity, training error, validation error and memorizing predictor

Builds on Training, Validation, and Test Sets, Data Leakage and Honest Splitting and K-Nearest Neighbors.

Video 11

feature vector, dot product, matrix, design matrix, matrix-vector product, intercept column and linear prediction

Builds on Python Arrays and Tables for Machine Learning and K-Nearest Neighbors.

Video 12

Euclidean norm, orthogonal projection, span, column space, linear independence, rank, singular matrix and conditioning

Builds on Vectors, Matrices, and Linear Predictions.

Video 13

derivative, differentiability, partial derivative, gradient, chain rule, directional derivative and finite-difference approximation

Builds on Vectors, Matrices, and Linear Predictions.

Video 14

objective function, gradient descent, learning rate, stationary point, local minimum, global minimum, convex function and Hessian matrix

Builds on Derivatives, Partial Derivatives, and the Chain Rule and Vectors, Matrices, and Linear Predictions.

Video 15

event, joint probability, conditional probability, Bayes' rule, base rate, posterior probability, independence and conditional independence

Builds on Features, Targets, and Prediction Time and Feature Scaling and Categorical Encoding.

Video 16

random variable, summation notation, probability mass function, cumulative distribution function, integral as area, probability density function, binomial distribution and normal distribution

Builds on Conditional Probability and Bayes' Rule. Also uses exponential functions.

Video 17

expected value, linearity of expectation, variance, covariance, correlation, covariance matrix and conditional expectation

Builds on Random Variables and Probability Distributions and Vectors, Matrices, and Linear Predictions.

Video 18

estimator, sampling distribution, independent and identically distributed, law of large numbers, standard error, confidence interval and selection bias

Builds on Random Variables and Probability Distributions and Expectation, Variance, and Covariance.

Video 19

loss function, squared error, absolute error, empirical risk, population risk, conditional mean, training objective and evaluation metric

Builds on Training, Validation, and Test Sets, Vectors, Matrices, and Linear Predictions, Random Variables and Probability Distributions and Expectation, Variance, and Covariance.

Video 20

simple linear regression, least squares, slope, intercept, fitted value, residual and extrapolation

Builds on Vectors, Matrices, and Linear Predictions, Loss Functions and Empirical Risk, Gradient Descent and Optimization and Expectation, Variance, and Covariance.

Video 21

multiple linear regression, normal equations, least-squares projection, full column rank, partial effect and QR decomposition

Builds on Simple Linear Regression and Vector Geometry, Rank, and Projection.

Video 22

stochastic gradient descent, stochastic gradient, mini-batch, epoch, full-batch gradient descent, learning-rate schedule and convergence

Builds on Data Leakage and Honest Splitting, Gradient Descent and Optimization, Feature Scaling and Categorical Encoding and Sampling, Estimation, and Statistical Uncertainty.

Video 23

feature engineering, polynomial feature, interaction term, basis function, linear in parameters, indicator variable and spline

Builds on Multiple Linear Regression and Least Squares and Feature Scaling and Categorical Encoding.

Video 24

regularization, ridge regression, lasso, elastic net, regularization strength, coefficient shrinkage and regularization path

Builds on Loss Functions and Empirical Risk, Multiple Linear Regression and Least Squares, Gradient Descent and Optimization, Stochastic Gradient Descent and Mini-Batches, Polynomial Features and Interactions and Overfitting, Underfitting, and Model Complexity.

Video 25

mean absolute error, mean squared error, root mean squared error, R-squared, mean baseline, median baseline and mean absolute percentage error

Builds on Loss Functions and Empirical Risk, Simple Linear Regression and Expectation, Variance, and Covariance.

Video 26

robust regression, Huber loss, pinball loss, quantile regression, conditional quantile and asymmetric cost

Builds on Loss Functions and Empirical Risk, Gradient Descent and Optimization, Random Variables and Probability Distributions, Expectation, Variance, and Covariance and Regression Metrics and Baseline Comparisons.

Video 27

binary classification, logistic regression, sigmoid function, odds, log-odds, odds ratio and decision boundary

Builds on Multiple Linear Regression and Least Squares, Conditional Probability and Bayes' Rule, Random Variables and Probability Distributions and Gradient Descent and Optimization. Also uses exponential functions and natural logarithms.

Video 28

likelihood, maximum likelihood estimation, Bernoulli likelihood, log likelihood, negative log likelihood, binary cross-entropy, maximum a posteriori estimation and prior distribution

Builds on Loss Functions and Empirical Risk, Gradient Descent and Optimization, Conditional Probability and Bayes' Rule, Random Variables and Probability Distributions, Logistic Regression and Ridge, Lasso, and Elastic Net. Also uses rules of logarithms.

Video 29

multiclass classification, multilabel classification, softmax function, class probability vector, multiclass cross-entropy, log-sum-exp trick and one-vs-rest

Builds on Vectors, Matrices, and Linear Predictions, Logistic Regression and Maximum Likelihood and Cross-Entropy.

Video 30

positive class, decision threshold, confusion matrix, false positive, false negative, sensitivity, specificity and expected cost

Builds on Training, Validation, and Test Sets, Conditional Probability and Bayes' Rule, Logistic Regression and Expectation, Variance, and Covariance.

Video 31

accuracy, precision, recall, F1 score, ROC curve, ROC AUC, precision-recall curve and average precision

Builds on Confusion Matrices and Decision Thresholds.

Video 32

class imbalance, prevalence, balanced accuracy, macro average, micro average, class weight, SMOTE and cost-sensitive learning

Builds on Confusion Matrices and Decision Thresholds, Precision, Recall, and Classification Metrics and Data Leakage and Honest Splitting.

Video 33

probability calibration, reliability diagram, Brier score, log loss, discrimination, Platt scaling, isotonic regression and calibration set

Builds on Data Leakage and Honest Splitting, Maximum Likelihood and Cross-Entropy, Confusion Matrices and Decision Thresholds, Precision, Recall, and Classification Metrics and Imbalanced Classes and Cost-Sensitive Learning.

Video 34

naive Bayes classifier, generative classifier, class prior, naive Bayes assumption, Laplace smoothing and log-probability score

Builds on Conditional Probability and Bayes' Rule, Maximum Likelihood and Cross-Entropy, Confusion Matrices and Decision Thresholds and Random Variables and Probability Distributions.

Video 35

Gaussian discriminant analysis, class-conditional density, multivariate Gaussian distribution, positive-definite covariance matrix, linear discriminant analysis, quadratic discriminant analysis and shared covariance matrix

Builds on Vectors, Matrices, and Linear Predictions, Conditional Probability and Bayes' Rule, Naive Bayes Classification, Random Variables and Probability Distributions, Expectation, Variance, and Covariance and Vector Geometry, Rank, and Projection.

Video 36

bias-variance decomposition, squared bias, prediction variance, irreducible error, bias-variance tradeoff and training randomness

Builds on Overfitting, Underfitting, and Model Complexity, Expectation, Variance, and Covariance, Sampling, Estimation, and Statistical Uncertainty and Ridge, Lasso, and Elastic Net.

Video 37

k-fold cross-validation, stratified folds, grouped folds, temporal folds, preprocessing pipeline and fold leakage

Builds on Data Leakage and Honest Splitting, Ridge, Lasso, and Elastic Net, Precision, Recall, and Classification Metrics, Probability Calibration, Feature Scaling and Categorical Encoding and Bias, Variance, and Irreducible Error.

Video 38

hyperparameter tuning, grid search, random search, search budget, nested cross-validation, inner and outer folds and optimistic selection bias

Builds on Ridge, Lasso, and Elastic Net and Cross-Validation and Preprocessing Pipelines.

Video 39

learning curve, training set size, error analysis, error slice and subgroup evaluation

Builds on Precision, Recall, and Classification Metrics, Overfitting, Underfitting, and Model Complexity, Cross-Validation and Preprocessing Pipelines, Hyperparameter Search and Nested Evaluation and Regression Metrics and Baseline Comparisons.

Video 40

bootstrap resampling, bootstrap distribution, percentile interval, metric uncertainty, paired comparison and independent sampling unit

Builds on Precision, Recall, and Classification Metrics, Cross-Validation and Preprocessing Pipelines, Sampling, Estimation, and Statistical Uncertainty and Regression Metrics and Baseline Comparisons.

Video 41

prediction interval, aleatoric uncertainty, epistemic uncertainty, conformity score, split conformal prediction, exchangeability and marginal coverage

Builds on Bootstrap Uncertainty and Model Comparisons, Sampling, Estimation, and Statistical Uncertainty and Robust Regression and Quantile Prediction.

Video 42

finite hypothesis class, Hoeffding's inequality, union bound, uniform convergence, generalization bound, capacity, inductive bias and no free lunch theorem

Builds on Loss Functions and Empirical Risk, Overfitting, Underfitting, and Model Complexity, Random Variables and Probability Distributions, Sampling, Estimation, and Statistical Uncertainty, Bias, Variance, and Irreducible Error and Hyperparameter Search and Nested Evaluation.

Video 43

collinearity, feature redundancy, coefficient interpretation, feature selection and post-selection inference

Builds on Multiple Linear Regression and Least Squares, Polynomial Features and Interactions, Ridge, Lasso, and Elastic Net, Cross-Validation and Preprocessing Pipelines and Hyperparameter Search and Nested Evaluation.

Video 44

supervised learning workflow, data contract, baseline comparison, development data, refitting after selection, final evaluation and reproducibility

Builds on Probability Calibration, Hyperparameter Search and Nested Evaluation, Learning Curves and Error Analysis, Bootstrap Uncertainty and Model Comparisons, Feature Selection and Collinearity, Regression Metrics and Baseline Comparisons and Prediction Intervals and Split Conformal Prediction.

Video 45

linear classifier, separating hyperplane, perceptron, perceptron update rule, linear separability and perceptron convergence theorem

Builds on Vectors, Matrices, and Linear Predictions, Logistic Regression, Overfitting, Underfitting, and Model Complexity and Vector Geometry, Rank, and Projection.

Video 46

support vector machine, functional margin, geometric margin, hinge loss, soft margin and support vector

Builds on Loss Functions and Empirical Risk, Ridge, Lasso, and Elastic Net and Perceptrons and Separating Hyperplanes.

Video 47

feature space, kernel function, kernel trick, Gram matrix, positive semidefinite kernel, polynomial kernel, radial basis function kernel and kernel SVM

Builds on Polynomial Features and Interactions, Hyperparameter Search and Nested Evaluation, Support Vector Machines and Soft Margins and Vector Geometry, Rank, and Projection.

Video 48

classification tree, Gini impurity, entropy, information gain, weighted child impurity, greedy splitting and leaf probability

Builds on Conditional Probability and Bayes' Rule, Confusion Matrices and Decision Thresholds, Overfitting, Underfitting, and Model Complexity and Random Variables and Probability Distributions.

Video 49

regression tree, squared-error split, leaf mean, tree depth, minimum leaf size and cost-complexity pruning

Builds on Loss Functions and Empirical Risk, Simple Linear Regression, Hyperparameter Search and Nested Evaluation and Classification Trees.

Video 50

bagging, bootstrap sample, random forest, random feature subsets, out-of-bag error, ensemble variance and correlation between trees

Builds on Bias, Variance, and Irreducible Error, Bootstrap Uncertainty and Model Comparisons, Classification Trees and Regression Trees and Pruning.

Video 51

gradient boosting, stagewise additive model, weak learner, negative gradient, residual fitting, shrinkage, number of stages and AdaBoost

Builds on Gradient Descent and Optimization, Hyperparameter Search and Nested Evaluation, Regression Trees and Pruning and Bagging and Random Forests.

Video 52

model comparison, stacking, meta-model, out-of-fold prediction and ensemble selection

Builds on Hyperparameter Search and Nested Evaluation, Learning Curves and Error Analysis, Bagging and Random Forests and Boosting and Gradient-Boosted Trees.

Video 53

distance measure, cosine similarity, feature weighting, curse of dimensionality and distance concentration

Builds on K-Nearest Neighbors, Vectors, Matrices, and Linear Predictions, Feature Scaling and Categorical Encoding, Expectation, Variance, and Covariance and Vector Geometry, Rank, and Projection.

Video 54

k-means clustering, centroid, within-cluster sum of squares, assignment step, update step, k-means initialization and number of clusters

Builds on Loss Functions and Empirical Risk, Hyperparameter Search and Nested Evaluation and Distance and the Curse of Dimensionality.

Video 55

agglomerative clustering, linkage, single linkage, complete linkage, Ward's criterion, dendrogram and cluster stability

Builds on Distance and the Curse of Dimensionality and K-Means Clustering.

Video 56

DBSCAN, epsilon neighborhood, core point, border point, noise point, density reachability and density connectivity

Builds on Distance and the Curse of Dimensionality, K-Means Clustering and Hierarchical Clustering.

Video 57

Gaussian mixture model, latent variable, soft assignment, responsibility, expectation-maximization algorithm, E step, M step and covariance collapse

Builds on Conditional Probability and Bayes' Rule, Maximum Likelihood and Cross-Entropy, Gaussian Discriminant Analysis, K-Means Clustering, Random Variables and Probability Distributions and Expectation, Variance, and Covariance.

Video 58

centering, sample covariance matrix, projected variance, eigenvector, eigenvalue, principal direction and principal component analysis

Builds on Vectors, Matrices, and Linear Predictions, Distance and the Curse of Dimensionality, Expectation, Variance, and Covariance and Vector Geometry, Rank, and Projection.

Video 59

singular value decomposition, singular value, principal component scores, explained variance, low-rank reconstruction and reconstruction error

Builds on Multiple Linear Regression and Least Squares and Covariance, Eigenvectors, and PCA.

Video 60

dimensionality reduction, PCA pipeline, information loss, t-SNE, perplexity, UMAP and local neighborhood structure

Builds on Cross-Validation and Preprocessing Pipelines, Hyperparameter Search and Nested Evaluation, Feature Selection and Collinearity and PCA with SVD and Reconstruction.

Video 61

anomaly detection, novelty detection, reference population, anomaly score, isolation forest, training contamination and alert threshold

Builds on Confusion Matrices and Decision Thresholds, Precision, Recall, and Classification Metrics, Distance and the Curse of Dimensionality, Density-Based Clustering with DBSCAN and Dimensionality Reduction and Visualization.

Video 62

neural network, hidden layer, activation function, ReLU, forward propagation, output layer, tensor shape and universal approximation theorem

Builds on Vectors, Matrices, and Linear Predictions, Logistic Regression, Multiclass Classification and Softmax and Perceptrons and Separating Hyperplanes.

Video 63

computational graph, backpropagation, multivariable chain rule, reverse-mode automatic differentiation and gradient check

Builds on Gradient Descent and Optimization, Maximum Likelihood and Cross-Entropy, Neural Networks and Forward Propagation and Derivatives, Partial Derivatives, and the Chain Rule.

Video 64

training loop, weight initialization, vanishing gradient, exploding gradient, momentum, Adam optimizer and batch normalization

Builds on Stochastic Gradient Descent and Mini-Batches, Cross-Validation and Preprocessing Pipelines, Neural Networks and Forward Propagation and Backpropagation and Automatic Differentiation.

Video 65

weight decay, decoupled weight decay, dropout, early stopping, validation loss, training and evaluation modes and checkpoint selection

Builds on Ridge, Lasso, and Elastic Net, Overfitting, Underfitting, and Model Complexity, Cross-Validation and Preprocessing Pipelines and Training Neural Networks.

Video 66

neural network classifier, logistic regression baseline, network architecture, data roles and random seed

Builds on Precision, Recall, and Classification Metrics, Probability Calibration, Learning Curves and Error Analysis and Neural Regularization and Early Stopping.

Video 67

convolution, convolutional neural network, filter, feature map, stride and padding, receptive field and pooling

Builds on Python Arrays and Tables for Machine Learning, Neural Networks and Forward Propagation, Backpropagation and Automatic Differentiation and Training Neural Networks.

Video 68

embedding, representation learning, pretrained model, transfer learning, frozen feature extractor, prediction head and fine-tuning

Builds on Vectors, Matrices, and Linear Predictions, Neural Networks and Forward Propagation, Neural Regularization and Early Stopping and Convolutional Neural Networks.

Video 69

autoencoder, encoder, decoder, bottleneck, latent representation, reconstruction loss, denoising autoencoder and nonlinear dimensionality reduction

Builds on Neural Networks and Forward Propagation, Backpropagation and Automatic Differentiation, Neural Regularization and Early Stopping and PCA with SVD and Reconstruction.

Video 70

semi-supervised learning, pseudo-labeling, consistency regularization, self-supervised learning, contrastive learning, positive and negative pairs, representation collapse and active learning

Builds on Embeddings and Transfer Learning, Autoencoders and Nonlinear Representations and Cross-Validation and Preprocessing Pipelines.

Video 71

image classification, data augmentation, label-preserving augmentation, subject-level split and frozen-feature model

Builds on Data Leakage and Honest Splitting, Precision, Recall, and Classification Metrics, Learning Curves and Error Analysis, Convolutional Neural Networks, Embeddings and Transfer Learning and Semi-Supervised and Self-Supervised Learning.

Video 72

tokenization, vocabulary, bag of words, document-term matrix, term frequency, document frequency, TF-IDF and sparse matrix

Builds on Python Arrays and Tables for Machine Learning, Vectors, Matrices, and Linear Predictions, Conditional Probability and Bayes' Rule, Naive Bayes Classification and Distance and the Curse of Dimensionality.

Video 73

text classification, multinomial naive Bayes, regularized logistic regression, sparse pipeline, text leakage and collection artifact

Builds on Data Leakage and Honest Splitting, Precision, Recall, and Classification Metrics, Probability Calibration, Cross-Validation and Preprocessing Pipelines and Text Features and TF-IDF.

Video 74

recurrent neural network, hidden state, backpropagation through time, long short-term memory, gated recurrent unit, sequence padding, sequence mask and teacher forcing

Builds on Neural Networks and Forward Propagation, Backpropagation and Automatic Differentiation, Training Neural Networks and Text Features and TF-IDF.

Video 75

query vector, key vector, value vector, scaled dot-product attention, attention weights, attention mask, causal mask and multi-head attention

Builds on Vectors, Matrices, and Linear Predictions, Multiclass Classification and Softmax, Neural Networks and Forward Propagation, Embeddings and Transfer Learning, Text Features and TF-IDF and Sequence Models and Recurrent Neural Networks.

Video 76

transformer block, token embedding, positional encoding, layer normalization, residual connection, masked language modeling, next-token prediction and decoder-only model

Builds on Attention, Queries, Keys, and Values, Backpropagation and Automatic Differentiation, Training Neural Networks, Embeddings and Transfer Learning and Semi-Supervised and Self-Supervised Learning.

Video 77

generative model, autoregressive factorization, latent-variable model, variational autoencoder, evidence lower bound, generative adversarial network, mode collapse and diffusion model

Builds on Random Variables and Probability Distributions, Maximum Likelihood and Cross-Entropy, Gaussian Mixtures and Expectation-Maximization, Autoencoders and Nonlinear Representations and Transformers and Self-Supervised Language Models.

Video 78

recommender system, content-based recommendation, collaborative filtering, matrix factorization, explicit and implicit feedback, cold start, precision at k and normalized discounted cumulative gain

Builds on Vectors, Matrices, and Linear Predictions, Ridge, Lasso, and Elastic Net, Hyperparameter Search and Nested Evaluation, Distance and the Curse of Dimensionality and PCA with SVD and Reconstruction.

Video 79

time series, forecast origin, forecast horizon, lag feature, seasonality, naive forecast and seasonal-naive forecast

Builds on Features, Targets, and Prediction Time, Data Leakage and Honest Splitting, Polynomial Features and Interactions, Cross-Validation and Preprocessing Pipelines and Regression Metrics and Baseline Comparisons.

Video 80

rolling-origin evaluation, time-series backtesting, sliding window, expanding window, forecast error, drift and update policy

Builds on Learning Curves and Error Analysis, Bootstrap Uncertainty and Model Comparisons, Boosting and Gradient-Boosted Trees, Time-Series Features and Forecasting and Prediction Intervals and Split Conformal Prediction.

Video 81

multi-armed bandit, action value, incremental mean update, exploration and exploitation, epsilon-greedy policy, upper confidence bound and regret

Builds on Random Variables and Probability Distributions, Expectation, Variance, and Covariance and Sampling, Estimation, and Statistical Uncertainty.

Video 82

Markov decision process, Markov property, transition probability, policy, discounted return, value function, Bellman expectation equation and Bellman optimality equation

Builds on Random Variables and Probability Distributions, Expectation, Variance, and Covariance and Multi-Armed Bandits and Exploration.

Video 83

temporal-difference learning, TD error, bootstrapping, Q-learning, on-policy learning, off-policy learning and terminal state

Builds on Markov Decision Processes and Bellman Equations, Multi-Armed Bandits and Exploration and Gradient Descent and Optimization.

Video 84

algorithmic fairness, subgroup metric, group base rate, selection rate, demographic parity, equality of opportunity, equalized odds and decision tradeoff

Builds on Features, Targets, and Prediction Time, Conditional Probability and Bayes' Rule, Confusion Matrices and Decision Thresholds, Precision, Recall, and Classification Metrics and Bootstrap Uncertainty and Model Comparisons.

Video 85

interpretability, global importance, local explanation, permutation importance, partial dependence, individual conditional expectation, SHAP values and background distribution

Builds on Feature Selection and Collinearity, Bagging and Random Forests, Boosting and Gradient-Boosted Trees and Dimensionality Reduction and Visualization.

Video 86

distribution shift, covariate shift, label shift, concept shift, stress test, out-of-distribution input, feedback loop and abstention

Builds on Data Leakage and Honest Splitting, Probability Calibration, Learning Curves and Error Analysis, Bootstrap Uncertainty and Model Comparisons, Fairness and Subgroup Evaluation and Model Interpretation and Feature Importance.

Video 87

personal data, dataset license, memorization, membership inference, differential privacy, privacy budget, data poisoning and adversarial input

Builds on Features, Targets, and Prediction Time, Data Leakage and Honest Splitting and Distribution Shift and Robustness.

Video 88

inference pipeline, training-serving consistency, schema validation, model versioning, drift monitoring, delayed label, rollback and model card

Builds on A Complete Supervised Learning Workflow, Stacking and Ensemble Comparison, Rolling Forecast Evaluation, Model Interpretation and Feature Importance, Distribution Shift and Robustness and Privacy, Data Rights, and Model Security.

Video 89

data audit, protected test set, regularized linear model, tree ensemble, residual analysis, signed mean error and out-of-range inputs

Builds on A Complete Supervised Learning Workflow, Stacking and Ensemble Comparison, Model Interpretation and Feature Importance, Distribution Shift and Robustness and Deployment, Monitoring, and Model Documentation.

Video 90

rare-event classification, event definition, calibration sample, cost-based threshold, review capacity, frozen procedure and subgroup audit

Builds on Probability Calibration, A Complete Supervised Learning Workflow, Stacking and Ensemble Comparison, Fairness and Subgroup Evaluation, Model Interpretation and Feature Importance, Distribution Shift and Robustness and Deployment, Monitoring, and Model Documentation.

Video 91

study protocol, post-outcome information, independent unit of analysis, failed alternative, reproducibility versus validity, stated limitation and critical review

Builds on Deployment, Monitoring, and Model Documentation, Regression Project from Data to Evaluation and Rare-Event Classification Project.