Optimization of GPU Memory Usage for Training Deep Neural Networks

Che Lun Hung; Chine fu Hsin; Hsiao Hsi Wang; Chuan Yi Tang

doi:10.1007/978-3-030-30143-9_23

Optimization of GPU Memory Usage for Training Deep Neural Networks

Che Lun Hung^*, Chine fu Hsin, Hsiao Hsi Wang, Chuan Yi Tang

^*Corresponding author for this work

Department of Computer Science and Information Engineering

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

3 Scopus citations

Abstract

Recently, Deep Neural Networks have been successfully utilized in many domains; especially in computer vision. Many famous convolutional neural networks, such as VGG, ResNet, Inception, and so forth, are used for image classification, object detection, and so forth. The architecture of these state-of-the-art neural networks has become deeper and complicated than ever. In this paper, we propose a method to solve the problem of large memory requirement in the process of training a model. The experimental result shows that the proposed algorithm is able to reduce the GPU memory significantly.

Original language	English
Title of host publication	Pervasive Systems, Algorithms and Networks - 16th International Symposium, I-SPAN 2019, Proceedings
Editors	Christian Esposito, Jiman Hong, Kim-Kwang Raymond Choo
Publisher	Springer
Pages	289-293
Number of pages	5
ISBN (Print)	9783030301422
DOIs	https://doi.org/10.1007/978-3-030-30143-9_23
State	Published - 2019
Event	16th International Symposium on Pervasive Systems, Algorithms and Networks, I-SPAN 2019 - Naples, Italy Duration: 16 09 2019 → 20 09 2019

Publication series

Name	Communications in Computer and Information Science
Volume	1080 CCIS
ISSN (Print)	1865-0929
ISSN (Electronic)	1865-0937

Conference

Conference	16th International Symposium on Pervasive Systems, Algorithms and Networks, I-SPAN 2019
Country/Territory	Italy
City	Naples
Period	16/09/19 → 20/09/19

Bibliographical note

Publisher Copyright:
© 2019, Springer Nature Switzerland AG.

Keywords

Convolutional Neural Networks
Deep Neural Network
GPU

Access to Document

10.1007/978-3-030-30143-9_23

Cite this

Hung, C. L., Hsin, C. F., Wang, H. H., & Tang, C. Y. (2019). Optimization of GPU Memory Usage for Training Deep Neural Networks. In C. Esposito, J. Hong, & K.-K. R. Choo (Eds.), Pervasive Systems, Algorithms and Networks - 16th International Symposium, I-SPAN 2019, Proceedings (pp. 289-293). (Communications in Computer and Information Science; Vol. 1080 CCIS). Springer. https://doi.org/10.1007/978-3-030-30143-9_23

Hung, Che Lun ; Hsin, Chine fu ; Wang, Hsiao Hsi et al. / Optimization of GPU Memory Usage for Training Deep Neural Networks. Pervasive Systems, Algorithms and Networks - 16th International Symposium, I-SPAN 2019, Proceedings. editor / Christian Esposito ; Jiman Hong ; Kim-Kwang Raymond Choo. Springer, 2019. pp. 289-293 (Communications in Computer and Information Science).

@inproceedings{dc9a1d83b9014c44b12bef0f8cdf0dc4,

title = "Optimization of GPU Memory Usage for Training Deep Neural Networks",

abstract = "Recently, Deep Neural Networks have been successfully utilized in many domains; especially in computer vision. Many famous convolutional neural networks, such as VGG, ResNet, Inception, and so forth, are used for image classification, object detection, and so forth. The architecture of these state-of-the-art neural networks has become deeper and complicated than ever. In this paper, we propose a method to solve the problem of large memory requirement in the process of training a model. The experimental result shows that the proposed algorithm is able to reduce the GPU memory significantly.",

keywords = "Convolutional Neural Networks, Deep Neural Network, GPU",

author = "Hung, {Che Lun} and Hsin, {Chine fu} and Wang, {Hsiao Hsi} and Tang, {Chuan Yi}",

note = "Publisher Copyright: {\textcopyright} 2019, Springer Nature Switzerland AG.; 16th International Symposium on Pervasive Systems, Algorithms and Networks, I-SPAN 2019 ; Conference date: 16-09-2019 Through 20-09-2019",

year = "2019",

doi = "10.1007/978-3-030-30143-9_23",

language = "英语",

isbn = "9783030301422",

series = "Communications in Computer and Information Science",

publisher = "Springer",

pages = "289--293",

editor = "Christian Esposito and Jiman Hong and Choo, {Kim-Kwang Raymond}",

booktitle = "Pervasive Systems, Algorithms and Networks - 16th International Symposium, I-SPAN 2019, Proceedings",

}

Hung, CL, Hsin, CF, Wang, HH & Tang, CY 2019, Optimization of GPU Memory Usage for Training Deep Neural Networks. in C Esposito, J Hong & K-KR Choo (eds), Pervasive Systems, Algorithms and Networks - 16th International Symposium, I-SPAN 2019, Proceedings. Communications in Computer and Information Science, vol. 1080 CCIS, Springer, pp. 289-293, 16th International Symposium on Pervasive Systems, Algorithms and Networks, I-SPAN 2019, Naples, Italy, 16/09/19. https://doi.org/10.1007/978-3-030-30143-9_23

Optimization of GPU Memory Usage for Training Deep Neural Networks. / Hung, Che Lun; Hsin, Chine fu; Wang, Hsiao Hsi et al.
Pervasive Systems, Algorithms and Networks - 16th International Symposium, I-SPAN 2019, Proceedings. ed. / Christian Esposito; Jiman Hong; Kim-Kwang Raymond Choo. Springer, 2019. p. 289-293 (Communications in Computer and Information Science; Vol. 1080 CCIS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Optimization of GPU Memory Usage for Training Deep Neural Networks

AU - Hung, Che Lun

AU - Hsin, Chine fu

AU - Wang, Hsiao Hsi

AU - Tang, Chuan Yi

PY - 2019

Y1 - 2019

N2 - Recently, Deep Neural Networks have been successfully utilized in many domains; especially in computer vision. Many famous convolutional neural networks, such as VGG, ResNet, Inception, and so forth, are used for image classification, object detection, and so forth. The architecture of these state-of-the-art neural networks has become deeper and complicated than ever. In this paper, we propose a method to solve the problem of large memory requirement in the process of training a model. The experimental result shows that the proposed algorithm is able to reduce the GPU memory significantly.

AB - Recently, Deep Neural Networks have been successfully utilized in many domains; especially in computer vision. Many famous convolutional neural networks, such as VGG, ResNet, Inception, and so forth, are used for image classification, object detection, and so forth. The architecture of these state-of-the-art neural networks has become deeper and complicated than ever. In this paper, we propose a method to solve the problem of large memory requirement in the process of training a model. The experimental result shows that the proposed algorithm is able to reduce the GPU memory significantly.

KW - Convolutional Neural Networks

KW - Deep Neural Network

KW - GPU

UR - http://www.scopus.com/inward/record.url?scp=85076846430&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-30143-9_23

DO - 10.1007/978-3-030-30143-9_23

M3 - 会议稿件

AN - SCOPUS:85076846430

SN - 9783030301422

T3 - Communications in Computer and Information Science

SP - 289

EP - 293

BT - Pervasive Systems, Algorithms and Networks - 16th International Symposium, I-SPAN 2019, Proceedings

A2 - Esposito, Christian

A2 - Hong, Jiman

A2 - Choo, Kim-Kwang Raymond

PB - Springer

T2 - 16th International Symposium on Pervasive Systems, Algorithms and Networks, I-SPAN 2019

Y2 - 16 September 2019 through 20 September 2019

ER -

Optimization of GPU Memory Usage for Training Deep Neural Networks

Abstract

Publication series

Conference

Bibliographical note

Keywords

Access to Document

Other files and links

Fingerprint

Cite this