A maximum entropy approach to multiple classifiers combination

Fouss, François [UCL] Saerens, Marco [UCL]

In this paper,we present amaximumentropy (maxent) approach to the fusion of experts opinions, or classifiers outputs, problem. Themaxent approach is quite versatile and allows us to express in a clear, rigorous,way the a priori knowledge that is available on the problem. For instance, our knowledge about the reliability of the experts and the correlations between these experts can be easily integrated: Each piece of knowledge is expressed in the form of a linear constraint. An iterative scaling algorithm is used in order to compute the maxent solution of the problem. The maximum entropy method seeks the joint probability density of a set of random variables that has maximum entropy while satisfying the constraints. It is therefore the “most honest” characterization of our knowledge given the available facts (constraints). In the case of conflicting constraints, we propose to minimise the “lack of constraints satisfaction” or to relax some constraints and recompute the maximum entropy solution. The maxent fusion rule is illustrated by some simulations.