A high-resolution mass spectrometer device. Image: WUR
Artificial Intelligence may yield impressive results, but its inner workings often remain a mystery. Is our only option to blindly trust these algorithms? Fortunately, it isn’t. Innovations in the field of Explainable AI can offer us more insight into AI, or even allow AI to explain itself.
Artificial Intelligence (AI) has access to immense computing power, making it highly suitable for complex analyses. Vast data sets or extremely precise measurements can be processed faster and with greater accuracy through the application of AI technology. Medical diagnoses or food safety checks are two good examples of this. At the same time, the new technology is also giving rise to doom scenarios among laymen, and some scientists are likewise sceptical about the innovations in this field. ‘An important reason for these misgivings is the black box: the calculations and analytical steps a neural network carries out often remain unknown. Scientist especially want to properly understand their digital tools. What if an AI is prejudiced towards certain conclusions, but we don’t initially notice this?’ We interviewed Bas van der Velden, team lead Data Science at Wageningen Food Safety Research (WFSR). He and his colleagues are investigating ways to create more insight into the inner workings of AI.
Explainable AI
‘What is colloquially known as AI are usually self-learning algorithms, or so-called “deep learning” models. They are trained using relevant data and can program themselves,’ Van der Velden explains. ‘AI systems create many so-called ‘non-linear’ correlations. These are complex mathematical equations that can be used to plot the movements of a pendulum or the multiplication of cells. Deep learning models can feature millions of these equations.’ The high complexity of deep learning models makes it virtually impossible to manually check the conclusions of an AI system. ‘Even if someone were to check all those equations, this wouldn’t necessarily lead to deeper insight into the inner workings of the network,’ Van der Velden claims. This complex network of correlations also allows AI systems to perform highly in-depth analyses. Be we don’t quite know how.
"What if an AI is biased in drawing conclusions?"
Image: Shutterstock
‘Under the heading ‘Explainable AI’, data scientists are proposing various solutions to improve the transparency of such neural networks.’ To experiment with one of these solutions, Van der Velden and his colleagues set up a project, also known as a Small Innovative Project. ‘These are projects in which we try to develop a proof of concept for research of a potentially larger scope.’ The objective: explainable AI.
Checking for growth hormones
With the AI they aimed to build for this project, the researchers wanted to contribute to the process of checking cattle for growth hormones. These substances, which make cattle unnaturally bulky, are prohibited. Currently, there are various chemical-analytical methods to test animals for growth hormones. Could the AI contribute to these checks? Van der Velden and his colleagues trained an algorithm to test cow urine for traces of banned substances. In order to ensure accurate results, the scientists fed the system with data from a mass spectrometer. This is a device that can accurately measure the molecular mass, allowing it to determine the structure of chemical connections in a urine sample, for example.
Artificial intelligence is able to perform myriad exceptional tasks, especially when it comes to analysing large quantities of data. At the same time, many AI models a black box; they output answers without substantiation. But it doesn’t have to be this way. Researchers Bas van der Velden and Zuzanne Fendor are working on Explainable AI: artificial intelligence that explains itself. These models offer insight into the way they function. This not only makes them more reliable, but also more informative.
The neural network learned to analyse mass spectrometer data and separate the samples containing traces of growth hormones from the ones that did not. The results were impressive: the AI had an accuracy rate of 90%, which is about as good as the current statistical methods. ‘We expect that AI will work even better at a larger scale,’ Van der Velden says. ‘The algorithm may allow us to find unknown growth promotors as well, and could also be used for other applications, such as food safety checks.
No AI for straightforward issues
These are lovely results, but thus far remain unsubstantiated. What properties did the AI draw from to formulate its conclusions? ‘To find out, we used a common framework from game theory, named SHAP. This framework establishes connections between the data entering the model and the results it produces,’ Van der Velden explains. The framework pointed out a specific chemical structure as the key property of detecting growth hormones in urine samples. ‘Domain experts confirmed that this analysis is correct. SHAP managed to show what the AI based its analysis on.’
Example of an LC-MS spectrum, which can be used as input for the AI analysis. Source: Wikipedia
‘Explanations like these not only allow us to understand how a neural network works, but could also help pinpoint the origins of potential mistakes made by the AI. These insights can be used to improve algorithms, so they don’t continue to make the same mistakes.’ This is important, since AI will be used for large-scale, complicated tasks. ‘Straightforward issues likely don’t require Explainable AI, but if a neural network is going to make medical diagnoses or estimate the risk of relapses, transparency becomes increasingly important.’
AI that explains itself
While Van der Velden is proud of the results of the project, he does not consider game theory solutions to be the future of explainable AI. ‘There are roughly two main varieties of Explainable AI: subsequent explanation, which is what we used in this project, and an explanation functionality built into the AI itself. AI that teaches itself to explain, so to speak.’ Methods that provide subsequent explanation certainly have their merits, but they come with some drawbacks, too. ‘Of course, it is great that these systems can be applied to any AI, but the results aren’t of the same high quality.’
'Data scientists are proposing various solutions to improve the transparency of such neural networks'
A high-resolution mass spectrometer is used to measure high-resolution mass spectra, as applied in the specific use case. Photo: WUR
‘The integrated explanation is more elegant,’ Van der Velden explains, using a word with a very specific meaning to mathematicians. ‘With elegance, I mostly mean that the explanation is a lot more specific, tailored to the purpose of the AI. Since the underlying functionalities are developed along with the AI, they can be built to suit the user’s needs. I believe this is a major benefit. After all, biologists need different information than food processing companies.’ Self-explaining AI already offers more insight into its own inner workings even when the algorithm is still being trained. ‘What you get is a feedback loop, allowing you to further optimise the neural network.’
Social responsibility
‘Many scientists who work with artificial intelligence believe that this technology will significantly change our future. By this, they don’t mean smart kitchen appliances, but far-reaching, significant innovations in the way we process data,’ Van der Velden explains. ‘This is the time to determine the direction of that transformation. As an AI researcher, I consider it my social responsibility to take the utmost care when it comes to the risks of artificial intelligence.’ Van der Velden doesn’t suggest we’ll encounter science-fiction doom scenarios. ‘No, what I’m talking about is the responsible development of these powerful algorithms. Opening the black box of AI will allow us to more consciously deal with artificial intelligence.’
Share this article