In this letter, we propose a method of monitoring driver's movement using a small-sized millimeter-wave radar sensor. First, we install a frequency-modulated continuous wave radar sensor on the steering wheel and collect reflected signals for various driver motions. Because the reflection patterns are different for each motion, we can distinguish different movements of the driver by analyzing the received signal. In our method, we use the spectrogram of the received signal obtained through multiple measurements and apply a window function to extract the radar signal in the form of an image. Then, we use the convolutional neural network architecture to classify the radar signal images. The classification results show that our proposed method can classify four different movements of the driver with an accuracy higher than 80%.