Transformations and Curved Relationships

Definition A polynomial regression of degree $k$ models the conditional mean $m(x) = E(Y \mid x)$ of a response $Y$ given a numerical predictor $x$ as a polynomial in $x$,

that is, $\beta_0 + \beta_1 x + \cdots + \beta_k x^k$. It estimates the coefficients by least squares with the columns $1, x, x^2, \dots, x^k$ as predictors. The estimates are unique when the data contain at least $k + 1$ distinct values of $x$. When $k \ge 2$ and $\beta_k \ne 0$ the mean function is curved, so the change in the mean for a one-unit step in $x$ depends on where the step starts. For a quadratic

Definition A regression mean function is linear in the coefficients when it has the form

where $f_0, \dots, f_k$ are known functions of the predictors that contain no unknown parameters. Each $f_j(x)$ can be computed from the data and used as a column of the design matrix, so least squares and the usual linear-model inference apply, under the same assumptions as for any other linear regression. Polynomials, the function $\beta_0 + \beta_1 \log x$ for positive $x$, and models with interaction terms are all linear in the coefficients. The mean function $\beta_0 e^{\beta_1 x}$ is not, because $\beta_1$ sits inside the exponential.

Definition Here $\log$ is the natural logarithm, the inverse of $\exp(u) = e^u$ with $e \approx 2.71828$. It is defined only for positive arguments, and $\log(ab) = \log a + \log b$ for $a, b > 0$. A log transformation replaces a positive variable by its logarithm. For a response $Y > 0$ measured in units, write $\log(Y/y_0)$ with a fixed positive reference amount $y_0$ in the same units, such as one minute, so that the logarithm is taken of a pure number. A response transformation models $\log(Y/y_0)$ in place of $Y$, which changes the scale on which the mean, the errors and their assumptions are stated. A predictor transformation uses $\log(x/x_0)$ as a predictor, which changes the shape of the mean function in $x$ and leaves the response scale alone.

Definition A positive random variable $W$ is lognormal with parameters $\mu$ and $\sigma^2$ when $\log W \sim N(\mu, \sigma^2)$. The parameters are the mean and variance of $\log W$, not of $W$. A regression with lognormal error states that

where, given $x$, the error is $\varepsilon \sim N(0, \sigma^2)$ with the same $\sigma^2$ for every $x$, and errors of different observations are independent. On the original scale this reads

so the error multiplies the typical value instead of adding to it, and given $x$ the ratio $Y/y_0$ is lognormal with parameters $\beta_0 + \beta_1 x$ and $\sigma^2$.

Definition A conditional median of $Y$ given $x$ is a number $M$ with $P(Y \le M \mid x) \ge 1/2$ and $P(Y \ge M \mid x) \ge 1/2$. For a continuous conditional distribution whose density is positive on an interval and zero outside it, such as a lognormal one, it is the unique value with $P(Y \le M \mid x) = 1/2$. A strictly increasing function carries the median of a variable to the median of the new variable, and in the lognormal error model the log error has median $0$. Hence

The same holds for any continuous log error with unique median $0$, normal or not.

Definition Back-transformation converts a value on the log scale to the original scale by exponentiating it and multiplying by $y_0$. Which original-scale quantity the result describes depends on the error model. Suppose the conditional mean of $\log(Y/y_0)$ is $\mu(x)$, so that $\log(Y/y_0) = \mu(x) + \varepsilon$ with an error of mean $0$. Unless the error equals $0$ with probability $1$, Jensen’s inequality gives $E(e^{\varepsilon} \mid x) > 1$, so $y_0 e^{\mu(x)}$ is smaller than $E(Y \mid x)$. Under the lognormal error model with constant variance $\sigma^2$,

and therefore

that is, $E(Y \mid x) = y_0 \exp(\beta_0 + \beta_1 x + \sigma^2/2)$. If the log-error variance changes with $x$, the factor changes with $x$ too, and for a log error that is not normal the factor that turns $y_0 e^{\mu(x)}$ into the mean is $E(e^{\varepsilon} \mid x)$ under that distribution, while $y_0 e^{\mu(x)}$ is the median only if the error has median $0$. When the coefficients and $\sigma^2$ are estimated, the values obtained by back-transformation are estimates and carry that uncertainty.