Gesture recognition in cooking video based on image features and motion features using Bayesian network classifier

Published in Emerging Trends in Image Processing, Computer Vision and Pattern Recognition, 2015

Abstract: In this chapter, we propose an advanced method, which combines image features and motion features, for gesture recognition in cooking video. First of all, the image features including global and local features of Red-Green-Blue color images are extracted, then represented using bag of features method. Concurrently, motion features are also extracted from these videos and represented through some dense trajectories descriptors such as histogram of oriented gradient, histogram of optical flow, or motion boundary histogram. In addition, we use relative positions between objects and also their positions are detected in each frame to decrease misclassification. Next, we combine both image features and motion features to describe the cooking gestures. At the last step, Bayesian network models are applied to predict which action class a certain frame belongs to, base on the action class of previous frames and the cooking gesture in current frame. Our method has been approved through Actions for Cooking Eggs dataset that it can recognize human cooking actions as we expected. We evaluate our method as a general method for solving many different action recognition problems. In near future, therefore, we are going to apply it to solve other action recognition problems.

Recommended citation: Nguyen Hung, Pham Bao, Jin Kim, "Gesture recognition in cooking video based on image features and motion features using Bayesian network classifier." Emerging Trends in Image Processing, Computer Vision and Pattern Recognition, 2015. http://www.sciencedirect.com/science/article/pii/B9780128020456000247