Abstract—In this paper, we propose a vision based human computer interaction (HCI) system, and user can interact with the computer by using the fingertip. And a novel algorithm is proposed, which robustly detects fingertip and archives touch-operated controls with a high precision of few millimeters. The fingertip detection algorithm is developed based on the prediction of the contents which is projected by projector, and a binocular vision system based on two cameras is applied for detecting the depth of fingertips and touch operation. The experiment shows good results and good performance. We have two important contributions: first, we develop a simple and robust hand detection method which can work well in a wide variety of situations including dark and luminous ambient illumination. Second, the two cameras comprise a stereo vision system. We determine whether a physical touch takes places by triangulation. 

Index Terms—hands detection, fingertip detection, stereo vision, touch detection, touch-operated controls

Cite: Qun Wang, Jun Cheng, and Jianxin Pang, "A Novel Projector-Camera Interaction System with the Fingertip," Journal of Image and Graphics, Vol. 1, No. 2, pp. 80-84, June 2013. doi: 10.12720/joig.1.2.80-84