Anomaly and Novelty Detection

Definition Anomaly detection is the task of finding observations that depart markedly from a stated reference population. A detector has three parts, an anomaly score fitted on data from the reference population, an alert threshold that turns scores into flags, and an action that a flag triggers, such as a manual review or a recheck of the sensor. Anomalous here means unusual relative to the reference. Whether a flagged observation is a fault, a harmful event or a rare but legitimate condition is a separate question, answered by review or by later outcomes. A detector is useful only to the extent that its flags agree with those answers on data it was neither fitted nor tuned on.

Definition The reference population is the population of observations that the detector should treat as normal. It is defined by the context in which the scores will be used, such as the machine type, the operating mode, the sensor and its calibration, and the time period. The reference sample is a set of observations from that population, available at fitting time, from which the parameters of the score are estimated. Every statement that an observation is unusual is relative to the reference sample, and it carries over to the reference population only as far as the sample represents it.

Definition An anomaly score is a function $s(\mathbf{x})$ fitted on a reference sample, with a stated direction, here larger meaning more unusual. Its operational definition names the method, the features and their scaling, the reference sample and the direction. It ranks observations by their departure from the reference, and it is not a probability of failure or of harm unless it has been calibrated against labeled outcomes. A simple score for a single feature is the standardized distance from the reference mean,

where $m$ is the mean and $d$ the standard deviation of the reference sample, with divisor $n$ and $d > 0$, both computed once and then frozen. This score assumes that normal behavior forms one central range. A reference with several separate modes needs a different score, such as a mixture density, and correlated features call for a distance that accounts for the correlation, such as the Mahalanobis distance.

Definition Novelty detection fits the score on a reference sample that is intended to contain only normal observations, then scores new observations made later for departure from it. Outlier detection, by contrast, fits on data that may already contain anomalies and tries to find them within the same data. Training contamination is the presence, in the data used to fit the score, of observations from outside the reference population, such as faults or recording errors, and the contamination rate is their fraction. Contamination pulls the fitted center toward the anomalies and inflates the fitted spread, so anomalies resembling the contaminating ones receive lower scores, an effect called masking, and the detector learns that some failure behavior is normal. Authors use these terms differently, so a description of a detector should state whether the fitting data are meant to be clean, how contamination is handled, and whether scores are used on the fitting data or on future observations.

Definition An isolation forest scores observations by how quickly random splits separate them from a reference sample. Each of its $t$ trees is grown on a subsample of $\psi \ge 2$ reference observations drawn without replacement, and $\psi = 256$ is a common default when enough data exist. At each node a feature is chosen at random, in common implementations among the features that still vary in the node, and a split value is drawn uniformly between the smallest and largest values of that feature in the node. Observations below the split go left and the rest go right. A node becomes a leaf when it holds one observation, when all its observations are identical, or when its depth reaches the height limit $\lceil \log_2 \psi \rceil$. To score $\mathbf{x}$ in one tree, follow its splits to a leaf at depth $e$ that holds $m$ reference observations, and set its path length to $h(\mathbf{x}) = e + c(m)$, where $c(1) = 0$ and

for $m \ge 2$, with the harmonic numbers $H_r = 1 + \tfrac12 + \dots + \tfrac1r$. The correction $c(m)$ is the average path length of an unsuccessful search in a binary search tree with $m$ keys, and it stands in for the splits the leaf did not make. With $\bar{h}(\mathbf{x})$ the average path length over the trees, the score is

a number between 0 and 1 that is larger for shorter paths. Implementations often approximate $H_r$ by $\ln r + 0.5772$, and some report the negative of this score or subtract an offset, so check which direction means more unusual before setting a threshold. A score of 0.6 is not a 60 percent probability of anything.

Definition An alert threshold $\tau$ turns a score in which larger means more unusual into a decision, flag $\mathbf{x}$ when $s(\mathbf{x}) \ge \tau$. It is chosen on development data from the review capacity and the costs of missed events and of false alarms, and it is frozen before the final evaluation. On a protected evaluation set with reliable labels, the flags give the counts $\mathrm{TP}$, $\mathrm{FP}$, $\mathrm{FN}$ and $\mathrm{TN}$, and

The false-alarm rate is $\mathrm{FP}/(\mathrm{FP} + \mathrm{TN})$. Precision needs at least one flag, recall at least one harmful case and the false-alarm rate at least one harmless case. The review workload is the number of flags per operating period. A budget rule that flags the $r$ highest scores in each period is an alternative to a fixed threshold. A threshold derived from an assumed contamination rate builds in that assumption and does not measure how often harmful events occur.