Computer scientists from Lancaster University have developed new AI technology that takes autonomous cars a step closer to our roads.

Funded by global car manufacturer Ford, the three-year research project provides a step-change in AI car technology by enabling autonomous cars to recognise new and unexpected situations.

Around the world, many different automotive brands, computing companies, and research teams, are developing autonomous car technologies and many of these are using a machine learning technique called ‘Deep Learning’. Deep Learning works by recognising patterns after the computer system has been shown a large number of different training examples.

However, a fundamental drawback with Deep Learning algorithms is that they are unable to recognise scenarios that differ significantly from training examples and, unlike humans, they are incapable of exploring, improving and improvising.

On the road, Deep Learning’s lack of situational awareness has led to life-threatening situations.

The new algorithms developed by Lancaster scientists overcome these weaknesses by using example images, called prototypes. They process the prototypes in a similar way to how humans reason - by comparing examples and looking for similarities. This results in a fundamental improvement to the AI system, making it truly intelligent and capable of recognising unknown and potentially threatening situations, informing safety procedures. The AI will also learn from these experiences, so that it recognises similar scenarios in the future.

If a new encounter is significantly different to prior knowledge then the system treats it as unknown and unexpected, learns from the experience and updates the system autonomously. This ensures that autonomous vehicles are continually improving – unlike mainstream Deep Learning systems, which would require extensive retraining to cope with a new environment.

The AI technology developed by the Lancaster researchers focuses on improving perception and classification by driverless systems. It informs, and would be integrated with, the system’s separate decision-making technology.

Professor Plamen Angelov, Chair of Intelligent Systems at Lancaster University, said: “In recent years Deep Learning has become the gold standard in computer vision and machine learning. However, it has several key weaknesses that make it especially problematic for driverless vehicles where wrong decisions can lead to loss of life.

“Our project proposed a system aware of its own limitations, so the computer is not only able to recognise previously known scenarios, but also to recognise the unexpected and inform appropriate safety actions.”

The new system can learn much faster and, importantly, is interpretable by design. This means developers can easily understand how and why any decision is made including the mistakes.

This differs significantly from existing Deep Learning systems, which are often called ‘black box’ systems because it is very difficult for people to understand and interpret how and why certain decisions have been made.

Eduardo Soares, who’s PhD was funded by Ford, said: “In some situations Deep Learning systems fail to correctly recognise objects or scenes, but what is worse is that it is not clear why this happened and so it becomes very difficult to fix.”

The three-year research project, called ‘Transparent Deep Learning Classifier of Driving Scenarios able to Identify and Learn from Unseen Situations’, has been conducted by Professor Plamen Angelov and his Ph.D. student, Eduardo Soares, at Lancaster University, and by Dr Dimitar Filev, Dr Marcos P. G. Castro, Dr Subramanya Nageshrao, and Dr Bruno Costa from Ford Research and Innovate.

As a result of the project, a series research articles have been published in top tier journals and conferences:

https://ieeexplore.ieee.org/document/9107404/authors#authors

https://www.sciencedirect.com/science/article/pii/S0893608020302513

https://link.springer.com/article/10.1007/s00521-021-06137-w

https://ieeexplore.ieee.org/document/8851842

https://ieeexplore.ieee.org/document/8999088

https://ieeexplore.ieee.org/document/9282812

The project has also been presented to the Ford global team at the Ford Controls & Automated System Technical Forum, where it has been well praised.