Investigating layer activation patterns in neural networks for classification error detection

Neural networks are intricate systems that learn to classify and recognize complex patterns. They yield incredible results in many fields like computer vision, speech recognition, text categorization and many others. It is therefore tempting to use them in difficult and important applications such as road tracking for self-driving cars, medical diagnosis, stock market predictions, etc. But our understanding of the intrinsic mechanisms that govern their behavior is still not fully understood. Neural networks are still mostly black-boxes, and blindly trusting their decision on important matters could have disastrous consequences. Therefore, a lot of research is made to find ways to make neural networks more reliable. In this thesis, we look at the particular problem of detecting when the decision of a network is likely to be incorrect. Most of today's post training methods focus on the last layer of the network. We propose two different approaches that instead look at the entirety of the network. In the first one we search for individual neurons that could indicate a correct or incorrect decision. The second approach looks at patterns in the activation of neurons. It measures the similarity between samples regarding these activation patterns and tries to detect erroneous samples by looking at how similar they are to supposedly close training samples. We then measure the performances of our methods and compare them between each other and to the state-of-the-art.