Abstract
Endowing visual agents with predictive capability is a key step towards video intelligence at scale. Early action recognition aims to predict the action labels before fully observing the complete video frames. Unlike action recognition, the model is asked to forecast the future or the effects by only observing the initial few frames. The strong reasoning ability over the temporal dimension is the key to success. To this end, in this paper, we propose a novel recurrent network with decomposed space-time attention and higher-order design to capture the temporal dependency associated with the specific actions. Our method achieves state-of-the-art performance on Something-Something and EPIC-Kitchens datasets under the early action recognition setting, showing evidence of predictive capability that we attribute to our higher-order recurrent design with space-time attention.
| Original language | English |
|---|---|
| Title of host publication | 2022 IEEE International Conference on Image Processing, ICIP 2022 - Proceedings |
| Publisher | IEEE Computer Society |
| Pages | 1631-1635 |
| Number of pages | 5 |
| ISBN (Electronic) | 9781665496209 |
| DOIs | |
| State | Published - 2022 |
| Externally published | Yes |
| Event | 29th IEEE International Conference on Image Processing, ICIP 2022 - Bordeaux, France Duration: 16 10 2022 → 19 10 2022 |
Publication series
| Name | Proceedings - International Conference on Image Processing, ICIP |
|---|---|
| ISSN (Print) | 1522-4880 |
Conference
| Conference | 29th IEEE International Conference on Image Processing, ICIP 2022 |
|---|---|
| Country/Territory | France |
| City | Bordeaux |
| Period | 16/10/22 → 19/10/22 |
Bibliographical note
Publisher Copyright:© 2022 IEEE.
Keywords
- Video prediction
- early action recognition
- higher-order recurrent networks
- space-time attention