A mobile robot generating video summaries of seniors' indoor activities

Chih Yuan Yang; Srenavis Varadaraj; Heeseung Yun; Jane Yung Jen Hsu

doi:10.1145/3338286.3344401

A mobile robot generating video summaries of seniors' indoor activities

Chih Yuan Yang
, Srenavis Varadaraj
, Heeseung Yun
, Jane Yung Jen Hsu

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

We develop a system which generates summaries from seniors' indoor-activity videos captured by a social robot to help remote family members know their seniors' daily activities at home. Unlike the traditional video summarization datasets, indoor videos captured from a moving robot poses additional challenges, namely, (i) the video sequences are very long (ii) a significant number of videoframes contain no-subject or with subjects at ill-posed locations and scales (iii) most of the well-posed frames contain highly redundant information. To address this problem, we propose to exploit pose estimation for detecting people in frames. This guides the robot to follow the user and capture effective videos. We use person identification to distinguish a target senior from other people. We also make use of action recognition to analyze seniors' major activities at different moments, and develop a video summarization method to select diverse and representative keyframes as summaries.

Original language	English
Title of host publication	Proceedings of the 21st International Conference on Human-Computer Interaction with Mobile Devices and Services, MobileHCI 2019
Publisher	Association for Computing Machinery, Inc
ISBN (Electronic)	9781450368254
DOIs	https://doi.org/10.1145/3338286.3344401
State	Published - 01 10 2019
Externally published	Yes
Event	21st International Conference on Human-Computer Interaction with Mobile Devices and Services, MobileHCI 2019 - Taipei, Taiwan Duration: 01 10 2019 → 04 10 2019

Publication series

Name	Proceedings of the 21st International Conference on Human-Computer Interaction with Mobile Devices and Services, MobileHCI 2019

Conference

Conference	21st International Conference on Human-Computer Interaction with Mobile Devices and Services, MobileHCI 2019
Country/Territory	Taiwan
City	Taipei
Period	01/10/19 → 04/10/19

Bibliographical note

Publisher Copyright:
© 2019 Copyright is held by the owner/author(s).

Keywords

Indoor activity
Mobile robot
Senior
Video summary

Access to Document

10.1145/3338286.3344401

Cite this

Yang, C. Y., Varadaraj, S., Yun, H., & Hsu, J. Y. J. (2019). A mobile robot generating video summaries of seniors' indoor activities. In Proceedings of the 21st International Conference on Human-Computer Interaction with Mobile Devices and Services, MobileHCI 2019 Article a54 (Proceedings of the 21st International Conference on Human-Computer Interaction with Mobile Devices and Services, MobileHCI 2019). Association for Computing Machinery, Inc. https://doi.org/10.1145/3338286.3344401

Yang, Chih Yuan ; Varadaraj, Srenavis ; Yun, Heeseung et al. / A mobile robot generating video summaries of seniors' indoor activities. Proceedings of the 21st International Conference on Human-Computer Interaction with Mobile Devices and Services, MobileHCI 2019. Association for Computing Machinery, Inc, 2019. (Proceedings of the 21st International Conference on Human-Computer Interaction with Mobile Devices and Services, MobileHCI 2019).

@inproceedings{afaad4630430431588a2865ac4a187c4,

title = "A mobile robot generating video summaries of seniors' indoor activities",

abstract = "We develop a system which generates summaries from seniors' indoor-activity videos captured by a social robot to help remote family members know their seniors' daily activities at home. Unlike the traditional video summarization datasets, indoor videos captured from a moving robot poses additional challenges, namely, (i) the video sequences are very long (ii) a significant number of videoframes contain no-subject or with subjects at ill-posed locations and scales (iii) most of the well-posed frames contain highly redundant information. To address this problem, we propose to exploit pose estimation for detecting people in frames. This guides the robot to follow the user and capture effective videos. We use person identification to distinguish a target senior from other people. We also make use of action recognition to analyze seniors' major activities at different moments, and develop a video summarization method to select diverse and representative keyframes as summaries.",

keywords = "Indoor activity, Mobile robot, Senior, Video summary",

author = "Yang, \{Chih Yuan\} and Srenavis Varadaraj and Heeseung Yun and Hsu, \{Jane Yung Jen\}",

note = "Publisher Copyright: {\textcopyright} 2019 Copyright is held by the owner/author(s).; 21st International Conference on Human-Computer Interaction with Mobile Devices and Services, MobileHCI 2019 ; Conference date: 01-10-2019 Through 04-10-2019",

year = "2019",

month = oct,

day = "1",

doi = "10.1145/3338286.3344401",

language = "英语",

series = "Proceedings of the 21st International Conference on Human-Computer Interaction with Mobile Devices and Services, MobileHCI 2019",

publisher = "Association for Computing Machinery, Inc",

booktitle = "Proceedings of the 21st International Conference on Human-Computer Interaction with Mobile Devices and Services, MobileHCI 2019",

}

Yang, CY, Varadaraj, S, Yun, H & Hsu, JYJ 2019, A mobile robot generating video summaries of seniors' indoor activities. in Proceedings of the 21st International Conference on Human-Computer Interaction with Mobile Devices and Services, MobileHCI 2019., a54, Proceedings of the 21st International Conference on Human-Computer Interaction with Mobile Devices and Services, MobileHCI 2019, Association for Computing Machinery, Inc, 21st International Conference on Human-Computer Interaction with Mobile Devices and Services, MobileHCI 2019, Taipei, Taiwan, 01/10/19. https://doi.org/10.1145/3338286.3344401

A mobile robot generating video summaries of seniors' indoor activities. / Yang, Chih Yuan; Varadaraj, Srenavis; Yun, Heeseung et al.
Proceedings of the 21st International Conference on Human-Computer Interaction with Mobile Devices and Services, MobileHCI 2019. Association for Computing Machinery, Inc, 2019. a54 (Proceedings of the 21st International Conference on Human-Computer Interaction with Mobile Devices and Services, MobileHCI 2019).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - A mobile robot generating video summaries of seniors' indoor activities

AU - Yang, Chih Yuan

AU - Varadaraj, Srenavis

AU - Yun, Heeseung

AU - Hsu, Jane Yung Jen

PY - 2019/10/1

Y1 - 2019/10/1

N2 - We develop a system which generates summaries from seniors' indoor-activity videos captured by a social robot to help remote family members know their seniors' daily activities at home. Unlike the traditional video summarization datasets, indoor videos captured from a moving robot poses additional challenges, namely, (i) the video sequences are very long (ii) a significant number of videoframes contain no-subject or with subjects at ill-posed locations and scales (iii) most of the well-posed frames contain highly redundant information. To address this problem, we propose to exploit pose estimation for detecting people in frames. This guides the robot to follow the user and capture effective videos. We use person identification to distinguish a target senior from other people. We also make use of action recognition to analyze seniors' major activities at different moments, and develop a video summarization method to select diverse and representative keyframes as summaries.

AB - We develop a system which generates summaries from seniors' indoor-activity videos captured by a social robot to help remote family members know their seniors' daily activities at home. Unlike the traditional video summarization datasets, indoor videos captured from a moving robot poses additional challenges, namely, (i) the video sequences are very long (ii) a significant number of videoframes contain no-subject or with subjects at ill-posed locations and scales (iii) most of the well-posed frames contain highly redundant information. To address this problem, we propose to exploit pose estimation for detecting people in frames. This guides the robot to follow the user and capture effective videos. We use person identification to distinguish a target senior from other people. We also make use of action recognition to analyze seniors' major activities at different moments, and develop a video summarization method to select diverse and representative keyframes as summaries.

KW - Indoor activity

KW - Mobile robot

KW - Senior

KW - Video summary

UR - https://www.scopus.com/pages/publications/85073513230

U2 - 10.1145/3338286.3344401

DO - 10.1145/3338286.3344401

M3 - 会议稿件

AN - SCOPUS:85073513230

T3 - Proceedings of the 21st International Conference on Human-Computer Interaction with Mobile Devices and Services, MobileHCI 2019

BT - Proceedings of the 21st International Conference on Human-Computer Interaction with Mobile Devices and Services, MobileHCI 2019

PB - Association for Computing Machinery, Inc

T2 - 21st International Conference on Human-Computer Interaction with Mobile Devices and Services, MobileHCI 2019

Y2 - 1 October 2019 through 4 October 2019

ER -

Yang CY, Varadaraj S, Yun H, Hsu JYJ. A mobile robot generating video summaries of seniors' indoor activities. In Proceedings of the 21st International Conference on Human-Computer Interaction with Mobile Devices and Services, MobileHCI 2019. Association for Computing Machinery, Inc. 2019. a54. (Proceedings of the 21st International Conference on Human-Computer Interaction with Mobile Devices and Services, MobileHCI 2019). doi: 10.1145/3338286.3344401

A mobile robot generating video summaries of seniors' indoor activities

Abstract

Publication series

Conference

Bibliographical note

Keywords

Access to Document

Other files and links

Fingerprint

Cite this