Towards unified gesture sets : structure, transformation, acquisition, and persistence

Gesture-based user interface of interactive applications have today become ubiquitous as soon as gesture interaction has become effective and efficient on a wide variety of devices and platforms, ranging from smart watches to wall screens. For this purpose, gesture recognition has been a major topic of research and development since more than a decade, mainly through developing gesture recognizers to be tested on gesture sets. The common shortcoming of all these works is that they are largely inconsistent with each other. Their gesture sets are largely varying in terms of format (e.g., XML, TXT, BIN, CSV, JSON, or any proprietary format), of structure (e.g., flat, array, hierarchy), of level of details (e.g., more or less features are extracted from the sensor), of sampling, units (e.g., Euclidean coordinates vs quaternions), and definitions. Moreover, these gesture sets are scattered through the literature, rarely accessible on the web, if not damaged. Consequently, the gesture recognizers working on these gesture sets are no longer compatible and interoperable between themselves, and reusing them is almost impossible such as for comparing recognizers against a same gesture set.\\ To leverage these shortcomings, this thesis pursues four goals: (1) to introduce, define, and motivate a unified format for storing gesture sets (3 versions) according to a consistent structure; (2) to transform a large amount of existing gesture sets into this unified format according to a model-based approach implemented through various modules; (3) develop an online web application that acquires gestures directly into the unified format for both 2D (through a surface computing pointer) and 3D (through a Myo Armband) so as to create compatible gesture sets; and (4) make the transformed gesture sets along with the application permanently available on two web sites ensuring dataset persistence (i.e., Kaggle and OSF.io) to make them widely available to the community of data science. Hence, structure, transformation, acquisition, and persistence are the four main goals of this work.