4 Linear models for non-stationary and seasonal time series.

The term non-stationary covers a host of stochastic processes. The simplest forms arise when either the mean of the series is varying, or if the process is represented by a random walk.

The variations observed in many series are explained as the cumulative sum of independent variables from IID($\mu ,{\sigma }^2$) distributions, the two parameters being the mean and variance of the distribution. Reverting now to calling the observed series $x_t$, if the mean $\mu$ is zero the series is called a simple random walk and the independent variables are white noise $e_t$:
4.2.1

$$x_t=x_{t-1}+e_t.$$

A non-zero mean $\mu$ represents a mean increase per unit time and the model is called a random walk with drift. To see if this model is appropriate for a time series $x_t$ the first differences of the series:
4.2.2

$$x_t-x_{t-1}=\left(1-B\right)x_t=\nabla x_t$$

are calculated, and should appear to be white noise $e_t$, or $\mu +e_t$ if $\mu \ne 0$.

The typical appearance of a simple random walk is difficult to appreciate intuitively. The series is best predicted by the most recent level and is equally likely to move up or down, yet the eye persists in seeing patterns of changing trends and cycles. It may be considered as a borderline case of the AR($1$) model as $\varphi \to 1$. Figure 24 shows a real series of daily interest rates, and an artificially generated random walk. The similarities in behaviour are quite striking. Random walk models are widely used in financial time series analysis, typically after taking logarithms of the original series. An added feature, though, is that the variance of the white noise disturbance (or returns) $e_t$, which is called the volatility of the series, may change slowly over time. It then becomes important to model the process $e_t$.

Forecasting is a topic yet to be covered, but is one of the main application areas of time series modelling. One of the earliest and simplest forecasting schemes, apart from fitting trend lines, was the EWMA predictor. This does not assume any particular structure of the observations and can be routinely used for any data. However, this was particularly designed to predict future values of a series which had a fluctuating level. The compromise between averaging many values to get a good estimate of the level, and using only a few values so as to estimate the recent level rather than an out-of-date level, was achieved by a choice of discounting factor $\theta$ applied to past values. The average used at time $t$, ${\overline{x}}_t$ to predict $x_{t+1}$ was therefore of the theoretical form:
4.3.1

$${\overline{x}}_t=\left(1-\theta \right)\left(x_t+\theta x_{t-1}+{\theta }^2{x}_{t-2}+\mathrm{\cdots }\right),$$

where the factor $\left(1-\theta \right)$ is present so that the weights sum to one, giving a true average. The practical point is that this average could easily be calculated by a recursive or updating formula:
4.3.2

$${\overline{x}}_t=\left(1-\theta \right)x_t+\theta {\overline{x}}_{t-1}.$$

As each new ‘day’ came with its most recent record of $x_t$, the old EWMA ${\overline{x}}_{t-1}$ was updated by this to give the new EWMA. These last two equations are conveniently written using operator notation:
4.3.3

$${\overline{x}}_t=\left(1-\theta \right){\left(1-\theta B\right)}^{-1}{x}_t\text{ and }\left(1-\theta B\right){\overline{x}}_t=\left(1-\theta \right)x_t.$$

Consideration of when this might be ‘the best thing’ to do leads to the assumption that the prediction errors $e_t=x_t-{\overline{x}}_{t-1}$ are white noise, completely unpredictable from (independent of) past information. On substituting for ${\overline{x}}_{t-1}$ this becomes:
4.3.4

	$x_t$	$=$	${\overline{x}}_{t-1}+e_t$
	$\Rightarrow \left(1-\theta B\right)x_t$	$=$	$\left(1-\theta \right)x_{t-1}+\left(1-\theta B\right)e_t$
	$\Rightarrow \left(1-B\right)x_t$	$=$	$\left(1-\theta B\right)e_t.$

Thus the first difference $\nabla x_t$ of $x_t$ follows a MA($1$) model.

This is known as the Integrated Moving Average or IMA($1,1$) model for the series $x_t$. The series of weekly business transactions is well represented by this model - recall that the first differences (after a log transformation) followed a MA($1$) model (see Figures 6–8).

Alternatively, in an IMA($1,1$) series $\left\{x_t\right\}$ given by $\left(1-B\right)x_t=\left(1-\theta B\right)e_t$ where $\left\{e_t\right\}$ is white noise, is predicted using EWMA, then the errors in prediction $x_t-{\overline{x}}_{t-1}$ is the white noise $e_t$. To see it, note that

$$e_t=\frac{\left(1-B\right)}{\left(1-\theta B\right)}{x}_t=\left(1-B\right)\left(1+\theta B+\theta B^2+\mathrm{\dots }\right)x_t.$$

Collecting the coefficients of $\left\{B^j;j\ge 0\right\}$, the right hand side is equal to $x_t-{\overline{x}}_{t-1}$.

Figure 25 shows a plot of that series, with the EWMA prediction of each term in the series added to the figure. The smoothing parameter was estimated to be $\theta =0.66$, a typical value. For the purposes of illustration, another plot is shown using $\theta =0.9$. The EWMA is then much smoother - but a less accurate predictor.