The term non-stationary covers a host of stochastic processes. The simplest forms arise when either the mean of the series is varying, or if the process is represented by a random walk.
The variations observed in many series are explained as the cumulative sum
of independent variables from IID() distributions,
the two parameters
being the mean and variance of the distribution. Reverting now to calling the
observed series , if the mean is zero the series is called a simple
random walk and the independent variables are white noise :
4.2.1
A non-zero mean represents a mean increase per unit time and the
model is called a random walk with drift. To see if this model is
appropriate for a time series the first differences of the series:
4.2.2
are calculated, and should appear to be white noise , or if .
The typical appearance of a simple random walk is difficult to appreciate intuitively. The series is best predicted by the most recent level and is equally likely to move up or down, yet the eye persists in seeing patterns of changing trends and cycles. It may be considered as a borderline case of the AR() model as . Figure 24 shows a real series of daily interest rates, and an artificially generated random walk. The similarities in behaviour are quite striking. Random walk models are widely used in financial time series analysis, typically after taking logarithms of the original series. An added feature, though, is that the variance of the white noise disturbance (or returns) , which is called the volatility of the series, may change slowly over time. It then becomes important to model the process .
Forecasting is a topic yet to be covered, but is one of the main application
areas of time series modelling. One of the earliest and simplest forecasting
schemes, apart from fitting trend lines, was the EWMA predictor. This does not assume any particular structure of the observations
and can be routinely used for any data. However, this was particularly
designed to predict future values of a series which had a fluctuating level.
The compromise between averaging many values to get a good estimate of the
level, and using only a few values so as to estimate the recent level rather
than an out-of-date level, was achieved by a choice of discounting factor
applied to past values. The average used at time , to predict
was therefore of the theoretical form:
4.3.1
where the factor is present so that the weights sum to one, giving
a true average. The practical point is that this average could easily be
calculated by a recursive or updating formula:
4.3.2
As each new ‘day’ came with its most recent record of , the old EWMA
was updated by this to give the new EWMA.
These last two equations are conveniently written using operator notation:
4.3.3
Consideration of when this might be ‘the best thing’ to do leads to the
assumption that the prediction errors are white noise,
completely unpredictable from (independent of) past information. On substituting
for this becomes:
4.3.4
Thus the first difference of follows a MA() model.
This is known as the Integrated Moving Average or IMA() model for the series . The series of weekly business transactions is well represented by this model - recall that the first differences (after a log transformation) followed a MA() model (see Figures 6–8).
Alternatively, in an IMA() series given by where is white noise, is predicted using EWMA, then the errors in prediction is the white noise . To see it, note that
Collecting the coefficients of , the right hand side is equal to .
Figure 25 shows a plot of that series, with the EWMA prediction of each term in the series added to the figure. The smoothing parameter was estimated to be , a typical value. For the purposes of illustration, another plot is shown using . The EWMA is then much smoother - but a less accurate predictor.