torchcell.losses.distributional.gaussian_nll

gaussian_nll(mu: Tensor, sigma: Tensor, target: Tensor) → Tensor

Gaussian negative log-likelihood, WITHOUT the constant term.

\[\mathrm{NLL}(\mu,\sigma;y) = \tfrac{1}{2}\Big(\log\sigma^2 + \frac{(y-\mu)^2}{\sigma^2}\Big) .\]

The dropped additive constant is \(\tfrac12\log(2\pi)\); it is omitted because it shifts every value by the same amount and so changes neither gradients nor the ranking of models. Values are therefore NOT comparable to a full log-likelihood in absolute terms, and unlike CRPS they can be negative.

KNOWN PATHOLOGY – read before using. The \(1/\sigma^2\) factor multiplies the gradient on \(\mu\):

\[\frac{\partial\,\mathrm{NLL}}{\partial \mu} = -\frac{y-\mu}{\sigma^2} .\]

A model can therefore lower its loss on points it fits badly by INFLATING \(\sigma\) there, and doing so simultaneously SHRINKS those same points’ gradient on \(\mu\) – so the hard points stop teaching the mean, the fit stalls, and \(\sigma\) keeps growing to cover the residual. This positive feedback is “sigma-collapse” (Seitzer et al. 2022, On the Pitfalls of Heteroscedastic Uncertainty Estimation with Probabilistic Neural Networks; the beta-NLL fix reweights by \(\sigma^{2\beta}\)). CRPS has no such factor, which is why it is the default.

It is implemented here UNPATCHED and on purpose: it is the negative control of the distributional sweep. Rather than assuming the pathology occurs, we MEASURE it with pit_values() / coverage() – sigma-collapse shows up as an n-shaped (hump-at-0.5) PIT histogram and over-coverage.

Parameters:

• mu – Predicted mean, any shape.

• sigma – Predicted standard deviation (STRICTLY positive), broadcastable to mu.

• target – Observations, broadcastable to mu.

Returns:

Elementwise NLL with the broadcast shape of the inputs (callers reduce).