Herein, we address transitional actions class as a class between actions. Transitional actions should be useful for producing short-term action predictions while an action is transitive. However, transitional action recognition is difficult because actions and transitional actions partially overlap each other. To deal with this issue, we propose a subtle motion descriptor (SMD) that identifies the sensitive differences between actions and transitional actions. The two primary contributions in this paper are as follows: (i) defining transitional actions for short-term action predictions that permit earlier predictions than early action recognition, (see Figure below) and (ii) utilizing convolutional neural network (CNN) based SMD to present a clear distinction between actions and transitional actions.
Transitional actions should be used to predict human actions while an action is transitive because symptoms of upcoming actions appear in transitional actions. Discriminative temporal CNN feature with Subtle Motion Descriptor (SMD) is used for understanding a transitional action (see Figure below): Multi-channel input from RGB and differential image is divided into two streams. At each frame, a CNN-based feature (V^t) is extracted with the first fully connected layer of 16-layer VGGNet (N=4,096). The consecutive subtractions (Delta(V^t)) are pooled into four vectors, namely x^Delta (V^+), x^Delta(V^0^+), x^Delta(V^-), x^Delta(V^0^-). Here, the x^Delta(V^0^+) and x^Delta(V^0^-) are the proposed SMD. The feature concatenation of RGB and differential image streams is the final classification vector.
References
- Hirokatsu Kataoka, Yudai Miyashita, Masaki Hayashi, Kenji Iwata, Yutaka Satoh, "Recognition of Transitional Action for Short-Term Action Prediction using Discriminative Temporal CNN Feature", British Machine Vision Conference (BMVC), Sep. 2016. (Acceptance rate: 39.4%) [PDF][Abstract][Poster][Slide][Project][Video]
