Example 4.3: Normal Data, ctd.
Following on from Example 3.3, we have . Recall that the log-likelihood is
From Theorem 6, is asymptotically
since
It follows that the standard error for is and that the standard error for is .
Since the covariance terms in the variance
matrix are zero, the parameters of the model are orthogonal —
thus, the standard errors of each of and
are unaffected by the uncertainty in the value of the other
parameter, i.e.
Inference on one parameter is the same whether the other parameter is known or not.
To think about this more explicitly, suppose that is known. Then the (one-dimensional) likelihood for is
and so
Thus
giving
This is what we expected due to the parameters in the bivariate model being orthogonal (off-diagonal terms in the variance matrix being zero).
However, looking again at the Fisher information matrix (below)
also note that this variance is the reciprocal of the top-left entry in the Fisher information matrix for the bivariate model. Similarly, the if were known, then the asymptotic variance of would be the reciprocal of the bottom-right entry of the Fisher information matrix.
This gives us a quicker way of computing the variance (or standard error) if we assume other parameters are known.
Example 4.4: Gamma distribution.
Following Example 3.3 we have
.
From Theorem 3,
is asymptotically
where
is the determinant of .
It follows that the standard error of is
and the standard error of is
Because the model is non-orthogonal, the standard error for each parameter is different from that which would have been obtained had the other parameter been known.
For example, if were known, then the standard error for would have been
Thus, in this example, we see that in non-orthogonal models,
inferences for each parameter are affected by the uncertainty in the
value of other parameters.
Finally as
the correlation between and is positive which is consistent with the likelihood contours seen in Figure 7.
Example 4.5: Linear Regression with unknown variance.
Now consider the simple linear regression model with unknown. Let
with (known) explanatory variables , i.e. .
Similar to Example 3.9, we have
By looking at the pairwise contour plots of parameters, the following observations were made:
The likelihood has near-elliptical contours, with principal axes not parallel to the coordinate axes.
The likelihoods for and are also near-elliptical, but with axes parallel to the coordinate axes.
The score function is made up of the components:
The second partial derivatives are:
Thus the Fisher information matrix is
Taking the inverse, we have the variance matrix as
which simplifies to
where .
The asymptotic variance of the third estimator, , is .
This variance is the same as in the previous example. This should not be a surprise, since it not correlated with the other two estimators, and thus inference using this estimator is not influenced by that of the others.
Exercise 6: What are the asymptotic variances of and ?
Exercise 7: What is the asymptotic correlation between and , ?
Exercise 8: If the other parameters were known, what would the standard
error of be?
Exercise 9: If the other parameters were known, what would the standard error of be?