Convolution and Recurrent Neural Network Fusion for Human Action Recognition in Controlled Environment

Human Action Recognition from video can play a big role in intelligent video surveillance. Majority of the approaches in the literature depends on pre environmental knowledge to extract complex handcrafted features from inputs. Convolutional neural networks (CNN) are one of the promising approaches which automate the process of feature extraction. Whereas, Recurrent Neural Network (RNN) is one of the promising technique in sequence modeling. This paper proposes a novel approach for Human action recognition by fusing the CNN and RNN architecture. The input video is converted to a sequence of images and the conjugative image difference sequence is passed to a sequence CNN and the outputs of the CNN passed to the RNN. To evaluate the performance of the proposed approach we have collected real-time walking videos of 100 persons. 10 Sec video of each person is considered as one sample. 70% of the samples are used for training purpose and remaining 30% samples are used for testing purpose. The accuracy achieved by the proposed approach is 78.32%.