The Particle Filter … this doesn’t look like a phrase you’d expect to see on a statistics and operational research blog, maybe in a fluid dynamics textbook. Despite this, the particle filter is an incredibly current method of inference in statistics. The name is just a result of an old mathematical convention that has stuck around.
This blog post could be considered a follow up to my recent post about the Kalman filter, the particle filter is designed around tackling a very similar statistical problem albeit in a more general fashion. What is this statistical problem? We have a process that is evolving through time that we cannot directly observe and an accompanying set of measurements that are related to this underlying process. We want to try and figure out what the values of the underlying process are based on the values of the process that we can actually observe. This is because it is usually the underlying process that actually dictates the behaviour of the observations.
An example of this is in the stock market. We have a situation where we can observe the returns of shares over time but what we really want to know is how we expect these returns to vary over time. Do we expect wild swings in value or do we expect it to be fairly stable? This is referred to as the volatility of the returns. This is a perfect application case for the particle filter. There are several different models that have been created that give mathematical relationships between the volatility observed at one point in time and the volatility observed at the next point in time and how the value of the return depends on the value of the volatility. Using these equations, the observation data and the particle filter it is possible to obtain estimates of the values of the underlying volatility at different points in time.
The particle filter begins by drawing a large number (e.g. \( N = 100 \)) of samples (known as particles) from an initial probability distribution of values for the volatility. This initial distribution is typically a gaussian distribution around zero because we don’t know much before seeing any observations of the returns. The particle filter then uses the mathematical model of the volatility to produce \( N \) estimates of the volatility at the next time point. These estimates are compared with the values of the observations of the returns and the particles are given a weighting based on how well they correspond to that observation in terms of the mathematical model. The particle filter then uses bootstrap sampling to resample a new set of \( N \) particles from the current particles where the particles with higher weights have a bigger chance of being resampled (possibly multiple times). Some particles with low weights will not be resampled at all. The point of this resampling step is to try and make it so that the particles that give the best approximation of the volatility continue to the next step of the particle filter. The next step begins with the estimation a.k.a. propagation step where the particles are moved along to the next point in time and thus the process is repeated over time.
The gif above shows the steps of the particle filter for \( N = 20 \) particles in a stochastic volatility model for a few time steps. In the weighting step, the particles are sized based on their weight. The function shown in red is the likelihood of the observed return being caused by each value of the volatility. Where this red line is further from the dotted line, particles are assigned a higher weight. In the resampling step, the particles have been plotted in a way that you can see the particles that are resampled multiple times. They are still all estimates of the volatility at the same time. The red cross on the reweighting step shows what the true volatility is at that given time.
Hopefully, this blog has given you a better idea of the general steps a particle filter takes to try and estimate an unseen process or at the very least it’s given me a chance to show off my fancy gif. I’m going to end with a few closing remarks. This method can get very computationally intensive when the number of particles used increases but increasing the number of particles increases the precision of the estimate made by the filter so there is a trade-off to be made here. The particle filter can be applied to many statistical problems where there is an unobserved underlying process that is related to some observed data (generally known as Hidden Markov Models). One of my favourites is in neuroscience where the motion of a subject’s limbs is treated as unobserved and is then inferred based on observed spikes in brain activity.
Relevant Links