Unit 8 of 15

Clustering and Dimensionality Reduction

  • 9 lessons
  • Videos 53 to 61

Without labels, learning looks for structure in the inputs themselves. This unit groups observations into clusters, finds low-dimensional representations and flags unusual cases.

Distances depend on feature scaling and can become less informative in many dimensions. K-means, hierarchical clustering, DBSCAN and Gaussian mixtures define clusters by different criteria, and none of them shows that the groups are true classes. Principal component analysis finds directions of maximum variance from the covariance matrix or the singular value decomposition and measures what a low-rank reconstruction keeps. PCA, t-SNE and UMAP serve prediction, compression and visualization in different ways. Anomaly and novelty detection score new observations against a reference population.

Every lesson will be one video. The videos are coming soon, and each lesson page already has the definitions and an example for every key term.

Distance and Clustering

  1. Distance and the Curse of Dimensionality

    How Euclidean distance and cosine similarity compare feature vectors, how units and feature weighting decide what counts as close, and how the curse of dimensionality and distance concentration affect similarity.

    Video 53

  2. K-Means Clustering

    How k-means clustering alternates an assignment step and an update step to lower the within-cluster sum of squares around each centroid, and why the result depends on initialization and on the number of clusters.

    Video 54

  3. Hierarchical Clustering

    How agglomerative clustering builds nested groups under single linkage, complete linkage or Ward's criterion, how to read and cut a dendrogram, and how cluster stability checks whether the groups of a cut persist.

    Video 55

  4. Density-Based Clustering with DBSCAN

    How DBSCAN builds clusters from epsilon neighborhoods and a minimum count, how it labels core, border and noise points, and how density reachability and density connectivity define its clusters.

    Video 56

  5. Gaussian Mixtures and Expectation-Maximization

    How a Gaussian mixture model describes data with hidden component labels, how responsibilities give soft assignments, how the E step and M step of expectation-maximization fit it, and why covariance collapse needs safeguards.

    Video 57

Principal Components and Dimensionality Reduction

  1. Covariance, Eigenvectors, and PCA

    How centering and the sample covariance matrix lead to principal component analysis, why a principal direction is an eigenvector with the largest eigenvalue, and how projected variance, units, sign and ties enter.

    Video 58

  2. PCA with SVD and Reconstruction

    How the singular value decomposition of a centered data matrix gives principal component loadings, scores and explained variance, and why keeping k components gives the best low-rank reconstruction with a known error.

    Video 59

  3. Dimensionality Reduction and Visualization

    How dimensionality reduction serves compression, prediction or visualization, how to fit a PCA pipeline inside training folds, why high explained variance can hide information loss, and what t-SNE, perplexity and UMAP preserve.

    Video 60

Detecting Unusual Observations

  1. Anomaly and Novelty Detection

    How anomaly and novelty detection score observations against a reference population, how a distance score and an isolation forest work, why training contamination matters, and how to set and evaluate an alert threshold.

    Video 61