Real-time segmentation of video objects for mixed-reality interactive applications

The present paper introduces a very specific and pragmatic approach to segmentation. It is driven by a particular application context: in the framework of mixed-reality, Tranfiction ("transportation into fictional spaces") is designed to mix synthetic and natural images in real time while allowing users to interact in these input/output screens. Segmentation is therefore used to provide both the immersion and interaction capabilities. The former aspect is achieved by composing the image of the user within the projected virtual scenes, while the later is achieved thanks to basic body/gesture analysis on the segmented silhouettes. According to indoor or outdoor usages, two real-time techniques are developed. Results are analyzed with respect to the overall application, not only in terms of absolute quality but also in terms of perception by the users.