Decision Fusion in Identity Verificationusing Facial Images

Automatic verification of personal identity using facial images is the central topic of the thesis. This problem can be stated as follows. Given two face images, it must be determined automatically whether they are images of the same person or of different persons. Due to many factors such as variability of facial appearance, sensitivity to noise, template aging, etc., the problem is difficult. We can overcome some of these difficulties by combining different information sources for the classification/recognition task. In this thesis we propose strategies on how to combine the different information sources, i.e. fusion strategies, in order to improve the verification accuracy. We have designed and thoroughly optimised a number of face verification algorithms. Their individual properties such as how their accuracy depends on algorithm parameters, image size, or sensitivity to mis-registrations have been studied. We have also studied how to combine the outputs of the different algorithms in order to reduce the verification error rates. Another decision fusion aspect considered in this thesis is the fusion of confidences obtained sequentially on several video frames of the same person's face. Finally multimodal fusion has been studied. In this case, the speech and face of the same subject are recorded and processed by different algorithms which output separate opinions. These two opinions are then conciliated at the fusion stage. It is shown that in all cases, information fusion allows a considerable performance improvement if the fusion stage is carefully designed.