First-Person View Hand Parameter Estimation Based on Fully Convolutional Neural Network

En Te Chou, Yun Chih Guo, Ya Hui Tang, Pei Yung Hsiao, Li Chen Fu*

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review


In this paper, we propose a real-time framework that can not only estimate location of hands within a RGB image but also their corresponding 3D joint coordinates and their hand side determination of left or right handed simultaneously. Most of the recent methods on hand pose analysis from monocular images only focus on the 3D coordinates of hand joints, which cannot give a full story to users or applications. Moreover, to meet the demands of applications such as virtual reality or augmented reality, a first-person viewpoint hand pose dataset is needed to train our proposed CNN. Thus, we collect a synthetic RGB dataset captured in an egocentric view with the help of Unity, a 3D engine. The synthetic dataset is composed of hands with various posture, skin color and size. We provide 21 joint annotations including 3D coordinates, 2D locations, and corresponding hand side which is left hand or right hand for each hand within an image.

Original languageEnglish
Title of host publicationPattern Recognition - 5th Asian Conference, ACPR 2019, Revised Selected Papers
EditorsShivakumara Palaiahnakote, Gabriella Sanniti di Baja, Liang Wang, Wei Qi Yan
Number of pages14
ISBN (Print)9783030412982
StatePublished - 2020
Externally publishedYes
Event5th Asian Conference on Pattern Recognition, ACPR 2019 - Auckland, New Zealand
Duration: 26 11 201929 11 2019

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume12047 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349


Conference5th Asian Conference on Pattern Recognition, ACPR 2019
Country/TerritoryNew Zealand

Bibliographical note

Publisher Copyright:
© 2020, Springer Nature Switzerland AG.


  • Convolutional neural network
  • Hand pose estimation
  • Synthetic data


Dive into the research topics of 'First-Person View Hand Parameter Estimation Based on Fully Convolutional Neural Network'. Together they form a unique fingerprint.

Cite this