Home > Published Issues > 2018 > Volume 6, No. 1, June 2018 >

3D Human Action Recognition with Siamese-LSTM Based Deep Metric Learning

Seyma Yucer and Yusuf Sinan Akgul
VisLab, Department of Computer Engineering, Gebze Technical University, Kocaeli, Turkey

Abstract—This paper proposes a new 3D Human Action Recognition system as a two-phase system: (1) Deep Metric Learning Module which learns a similarity metric between two 3D joint sequences using Siamese-LSTM networks; (2) A Multi-class Classification Module that uses the output of the first module to produce the final recognition output. This model has several advantages: the first module is trained with a larger set of data because it uses many combinations of sequence pairs. Our deep metric learning module can also be trained independently of the datasets, which makes our system modular and generalizable. We tested the proposed system on standard and newly introduced datasets that showed us that initial results are promising. We will continue developing this system by adding more sophisticated LSTM blocks and by cross-training between different datasets.

Index Terms—3D human action, action recognition, similarity learning, siamese networks, LSTM, deep metric learning

Cite: Seyma Yucer and Yusuf Sinan Akgul, "3D Human Action Recognition with Siamese-LSTM Based Deep Metric Learning," Journal of Image and Graphics, Vol. 6, No. 1, pp. 21-26, June 2018. doi: 10.18178/joig.6.1.21-26