Modelization of limb coordination for human action analysis

Kosta, Gaitanis [UCL] Pedro, C. Macq, Benoît [UCL]

This paper analyzes the movements of the human body limbs (hands, feet and head) and center of gravity in order to detect simple actions such as walking, jumping and displacing an object. Correlation between these points is guaranteed by considering them as cooperative agents forming a cooperative team : the whole body. The movements are analyzed at individual level and at team level using a hierarchical structure. We make use of a novel framework for online probabilistic plan recognition in cooperative multiagent systems (the multiagent abstract hidden Markov memory model, M-AHMEM) to modelize the human body and detect the actions performed. Knowledge of the high-level team actions (such as "walking") improves the pertinence of our predictions on the low-level individual actions (hand is moving back and forth) and allows us to compensate for missing or erroneous data produced by the feature extraction system. Experiments on real video sequences show the feasibility of the approach.