In this paper, we propose the simple technique of using "acceleration images'' to represent a change of a flow image. The acceleration images must be significant because the representation is different from position (RGB) and speed (flow) images. We apply two-stream CNN [Simonyan+, NIPS2014] as the baseline; then, we employ an acceleration stream, in addition to the spatial and the temporal streams. The acceleration images are generated by differential calculations from a sequence of flow images. Although the sparse representation tends to be noisy data (see Figure below), automatic feature learning with CNN can significantly pick up a necessary feature in the acceleration images.