Statistical Model
The automated foehn classification foehnix
is based on a
two-component mixture model. The basic idea is that two unobservable
components (or clusters) exist. One component for situations without
foehn, one component for situations with foehn. foehnix
uses an unsupervised statistical method to identify the two components
based on a set of observed values (e.g., wind speed, gust speed,
potential temperature differences) to model the probability whether or
not a specific observation is related to a foehn event.
The statistical model consists of two parts: one part to identify the
two components, and a second part modelling the probability whether or
not a specific observation belongs to component 1 or component 2. The
latter is known as the concomitant model.
The density of a two-component mixed distribution
in its general form is specified as follows for a specific observation
:
β¦ where
is the covariate for the first part of the statistical model to identify
components 1 and 2, and
the covariates for the concomitant model. The density of the mixed
distribution
is the sum (or superposition) of the densities of the two components
(;
i.e., Gaussian distribution) times the probability
from the concomitant model which describes whether or not a specific
observation belongs to component 2.
are the distribution parameters of the components,
the parameters of the concomitant model.
The concomitant model can be any model which fulfills
,
e.g., an constant value or intercept only model (mixture model
without concomitants), or any kind of probability model.
foehnix
uses a logistic regression model of the following
form:
The final foehn probability of the two-component mixture
model, also known as the a-posteriori probability, is given by:
β¦ where
in our case represents the probability of foehn. All one has to know are
the parameters
and
which can be estimated using an appropriate M-estimator such as maximum
likelihood.
Parameter Estimation
The maximum likelihood of a mixture model can usually not be
maximized directly. One possibility to estimate the coefficients of is
an iterative expectation maximization (EM) algorithm. The EM
algorithm otimizes the following log-likelihood:
-
.
with
as specified above (a-posteriori probability).
represents the number of observations.
The EM algorithm is specified as follows:
Initialization: initialize values for
and
.
Estimation: compute the posterior class
probability
Maximize: estimate
and
which maximize the likelihood using the posterior class probability
from the estimation step as weights:
The EM steps are repeated until the likelihood improvement falls
below a certain threshold or the maximum number of iterations is
reached.
Gaussian Mixture Model Without Concomitants
The simplest case is a Gaussian two-component mixture model without
concomitants. In this case the density of the two components is the
density
of the Gaussian distribution with its parameters
and
where
and
are the location and scale parameter of the Gaussian
distribution, or mean and standard deviation.
Initialization step
First, initial values for the parameters
()
and the posterior weights
()
have to be specified.
does not have to be initialized as no concomitant model is used in this
case! To be able to do so we have to attribute each observation
to one of the two components. This initial membership will be denoted as
and takes 1 if observation
belongs to component 2 and 0 else. This initial attribution
defines that observations with high values of
belong to component 2, observations with low values of
to component 1.
Note: Depending on the model specification this can
lead to models where the probability for no foehn will be
returned by foehnix
rather than posteriori probability of
foehn. However, the switch
argument of the
foehnix(...)
function allows you to control this behavior
(see foehnix
manual
page).
foehnix
uses the following initialization for the
two-component Gaussian mixture model without concomitants:
- Initialize class membership:
- Initial parameters for
using weighted empirical moments for
,
and the standard deviation of
as initial guess for
and
:
- Initialize
- Given
and
:
calculate a-posteriory probability:
Once the required elements have been initialized start the EM
algorithm for
:
- Update
- Obtain new
using
:
- Update posterior probabilities
:
- Calculate likelihood:
.
If
proceed with step 5.
- For
:
if
the likelihood could not have been improved in iteration
(converged or stuck): stop EM algorithm and return parameters of
iteration
.
If
:
maximum number of iterations reached, stop EM algorithm, return
parameters of iteration
.
Else proceed with step 5 until one of the stopping
criteria is reached.
Gaussian Mixture Model With Concomitants
The optimizer for a two-component Gaussian mixture model with
additional concomitants is very similar except that we also have to
update the concomitant model (logistic regression model). For mixed
models with concomitants the probabilities
are a function of the concomitant covariates
and the regression coefficients
.
The following algorithm is used:
- Initialize class membership
as for the Gaussian mixture model without concomitants.
- Initialize coefficients
as for the Gaussian mixture model without concomitants.
- Given
and
:
estimate logistic regression model to obtain the parameters
,
calculate
(see logistic regression
vignette).
- Calculate a-posteriori probability
as for the Gaussian mixture model without concomitants.
The EM algorithm for
:
- Update
by updating the concomitant model (logistic regression model) using
as response for the concomitant model (see logistic regression vignette).
- Obtain new
as for the Gaussian mixture model without concomitants.
- Update posterior probabilities
as for the Gaussian mixture model without concomitants.
- Calculate likelihood as for the Gaussian mixture model without
concomitants.
-
As for the Gaussian mixture model without concomitants:
proceed with step 5 until one of the stopping criteria
is reached.
Logistic Mixture Model
The logistic two-component mixture models can be estimated as the
Gaussian ones except that component density is the density of the
logistic distribution, and that the weighted empirical moments for
and
,
the scale of the logistic distribution, is now:
Censored and Truncated Models
In case of a censored or truncated mixed model the distributional
parameters
of the components of the mixture model cannot be calculated using
weighted empirical moments. In these cases a numreical likelihood-based
solver is used to estimate
,
,
,
and
.