For complex background, a hybrid hand detection algorithm based on the hand motion and the skin color is introduced. Gaussian Mixture Model (GMM) is used for the Foreground Detection. Skin Detection is performed on Foreground objects. For the purpose of Skin Detection a Look-up Table (LUT) is generated by collecting large number of skin color pixels in YCbCr color space. Skin segmentation is performed by thresholding each frame to obtain a binary image of the skin color area. Morphological operations are used to enhance the segmentation results. Experimental results show that the proposed algorithm can accurately detect different hand shapes under complex background, which is the most necessary step in the HCI.