Contrast properties of entropic criteria for blind source separation : a unifying framework based on information-theoretic inequalities/

Vrins, Frédéric [UCL]

In the recent years, Independent Component Analysis (ICA) has become a fundamental tool in adaptive signal and data processing, especially in the field of Blind Source Separation (BSS). Even though there exist some methods for which an algebraic solution to the ICA problem may be found, other iterative methods are very popular. Among them is the class of information-theoretic approaches, laying on entropies. The associated objective functions are maximized based on optimization schemes, and on gradient-ascent techniques in particular. Two major issues in this field are the following: 1) Does the global maximum point of these entropic objectives correspond to a satisfactory solution of BSS ? and 2) as gradient techniques are used, optimization algorithms look in fact for local maximum points, so what about the meaning of these local optima from the BSS problem point of view? Even though there are some partial answers to these questions in the literature, most of them are based on simulation and conjectures; formal developments are often lacking. This thesis aims at filling this lack and providing intuitive justifications, too. We focus the analysis on Rényi's entropy-based contrast functions. Our results show that, generally speaking, Rényi's entropy is not a suitable contrast function for BSS, even though we recover the well-known results saying that Shannon's entropy-based objectives are contrast functions. We also show that the range-based contrast functions can be built under some conditions on the sources. The BSS problem is stated in the first chapter, and viewed under the information (theory) angle. The two next chapters address specifically the above questions. Finally, the last chapter deals with range-based ICA, the only ``entropy-based contrast' which, based on the enclosed results, is also a discriminant contrast function, in the sense that it is theoretically free of spurious local optima. Geometrical interpretations and surprising examples are given. The interest of this approach is confirmed by testing the algorithm on the MLSP 2006 data analysis competition benchmark; the proposed method outperforms the previously obtained results on large-scale and noisy mixture samples obtained through ill-conditioned mixing matrices.