Real-time finger detection and tracking systems have been growing rapidly in the past decade. Among those methods, Appearance-based and Model-based methods have produced excellent results. However, the occlusion issue is one of the main challenges in this field. In this study, we address this issue by considering the repeating-finite gestures of a guitar-strumming or a hand puppet and, represent using a Finite State Machine model. Also we proposed a novel finger pose tracking system using FSM Model combining with the appearance-based method.. The proposed system consists of two parts: FSM-FT builder creates the FSM hand, and the FSM-FT runner controls the FSM-FT system. Empirically, we conducted an experimental study involving one sample repeating hand gesture and our approach achieved a significance recognition rate of 82% in the testing phase.
