Performance of convolution neural network based on multiple GPUs with different data communication models

Che Lun Hung; Yi Yang Lin; Chuan Yi Tang; Chilung Wang; Ming Chiang Chen

doi:10.1109/SNPD.2018.8441056

Performance of convolution neural network based on multiple GPUs with different data communication models

Che Lun Hung
, Yi Yang Lin
, Chuan Yi Tang
, Chilung Wang
, Ming Chiang Chen

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

1 Scopus citations

Abstract

Recently, deep learning technologies have been utilized in many scientific domains successfully. Convolution neural networks are common used in image understanding problems. However, to train a convolution neural network model with huge amount of images is time-consuming task. Most of deep learning frameworks, such as Caffe, TensorFlow, Torch, Keras, MxNet, and so forth, support GPU to train model fast; especially executing these models on multiple GPUs. In this work, we present the comparison of computation performance of AlexNet among different GPU servers and hyperparameters. The results shows that GPU servers with high bandwidth rate, NVLINK, can achieve better performance than others.

Original language	English
Title of host publication	Proceedings - 2018 IEEE/ACIS 19th International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing, SNPD 2018
Editors	Ha Jin Hwang, Lizhi Cai, Gun Huck Yeom, Tokuro Matsuo, Haeng Kon Kim, Hyun Yeo, Chung Sun Hong, Naoki Fukuta, Takayuki Ito, Huaikou Miao
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	87-92
Number of pages	6
ISBN (Print)	9781538658895
DOIs	https://doi.org/10.1109/SNPD.2018.8441056
State	Published - 20 08 2018
Externally published	Yes
Event	19th IEEE/ACIS International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing, SNPD 2018 - Busan, Korea, Republic of Duration: 27 06 2018 → 29 06 2018

Publication series

Name	Proceedings - 2018 IEEE/ACIS 19th International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing, SNPD 2018

Conference

Conference	19th IEEE/ACIS International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing, SNPD 2018
Country/Territory	Korea, Republic of
City	Busan
Period	27/06/18 → 29/06/18

Bibliographical note

Publisher Copyright:
© 2018 IEEE.

Keywords

Convolution Neural Networks
Deep Learning
GPU
Multiple GPUs

Access to Document

10.1109/SNPD.2018.8441056

Cite this

Hung, C. L., Lin, Y. Y., Tang, C. Y., Wang, C., & Chen, M. C. (2018). Performance of convolution neural network based on multiple GPUs with different data communication models. In H. J. Hwang, L. Cai, G. H. Yeom, T. Matsuo, H. K. Kim, H. Yeo, C. S. Hong, N. Fukuta, T. Ito, & H. Miao (Eds.), Proceedings - 2018 IEEE/ACIS 19th International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing, SNPD 2018 (pp. 87-92). Article 8441056 (Proceedings - 2018 IEEE/ACIS 19th International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing, SNPD 2018). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/SNPD.2018.8441056

Hung, Che Lun ; Lin, Yi Yang ; Tang, Chuan Yi et al. / Performance of convolution neural network based on multiple GPUs with different data communication models. Proceedings - 2018 IEEE/ACIS 19th International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing, SNPD 2018. editor / Ha Jin Hwang ; Lizhi Cai ; Gun Huck Yeom ; Tokuro Matsuo ; Haeng Kon Kim ; Hyun Yeo ; Chung Sun Hong ; Naoki Fukuta ; Takayuki Ito ; Huaikou Miao. Institute of Electrical and Electronics Engineers Inc., 2018. pp. 87-92 (Proceedings - 2018 IEEE/ACIS 19th International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing, SNPD 2018).

@inproceedings{da23e4476f99425dbffe2d409c4700a0,

title = "Performance of convolution neural network based on multiple GPUs with different data communication models",

abstract = "Recently, deep learning technologies have been utilized in many scientific domains successfully. Convolution neural networks are common used in image understanding problems. However, to train a convolution neural network model with huge amount of images is time-consuming task. Most of deep learning frameworks, such as Caffe, TensorFlow, Torch, Keras, MxNet, and so forth, support GPU to train model fast; especially executing these models on multiple GPUs. In this work, we present the comparison of computation performance of AlexNet among different GPU servers and hyperparameters. The results shows that GPU servers with high bandwidth rate, NVLINK, can achieve better performance than others.",

keywords = "Convolution Neural Networks, Deep Learning, GPU, Multiple GPUs",

author = "Hung, \{Che Lun\} and Lin, \{Yi Yang\} and Tang, \{Chuan Yi\} and Chilung Wang and Chen, \{Ming Chiang\}",

note = "Publisher Copyright: {\textcopyright} 2018 IEEE.; 19th IEEE/ACIS International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing, SNPD 2018 ; Conference date: 27-06-2018 Through 29-06-2018",

year = "2018",

month = aug,

day = "20",

doi = "10.1109/SNPD.2018.8441056",

language = "英语",

isbn = "9781538658895",

series = "Proceedings - 2018 IEEE/ACIS 19th International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing, SNPD 2018",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "87--92",

editor = "Hwang, \{Ha Jin\} and Lizhi Cai and Yeom, \{Gun Huck\} and Tokuro Matsuo and Kim, \{Haeng Kon\} and Hyun Yeo and Hong, \{Chung Sun\} and Naoki Fukuta and Takayuki Ito and Huaikou Miao",

booktitle = "Proceedings - 2018 IEEE/ACIS 19th International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing, SNPD 2018",

address = "美国",

}

Hung, CL, Lin, YY, Tang, CY, Wang, C & Chen, MC 2018, Performance of convolution neural network based on multiple GPUs with different data communication models. in HJ Hwang, L Cai, GH Yeom, T Matsuo, HK Kim, H Yeo, CS Hong, N Fukuta, T Ito & H Miao (eds), Proceedings - 2018 IEEE/ACIS 19th International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing, SNPD 2018., 8441056, Proceedings - 2018 IEEE/ACIS 19th International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing, SNPD 2018, Institute of Electrical and Electronics Engineers Inc., pp. 87-92, 19th IEEE/ACIS International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing, SNPD 2018, Busan, Korea, Republic of, 27/06/18. https://doi.org/10.1109/SNPD.2018.8441056

Performance of convolution neural network based on multiple GPUs with different data communication models. / Hung, Che Lun; Lin, Yi Yang; Tang, Chuan Yi et al.
Proceedings - 2018 IEEE/ACIS 19th International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing, SNPD 2018. ed. / Ha Jin Hwang; Lizhi Cai; Gun Huck Yeom; Tokuro Matsuo; Haeng Kon Kim; Hyun Yeo; Chung Sun Hong; Naoki Fukuta; Takayuki Ito; Huaikou Miao. Institute of Electrical and Electronics Engineers Inc., 2018. p. 87-92 8441056 (Proceedings - 2018 IEEE/ACIS 19th International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing, SNPD 2018).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Performance of convolution neural network based on multiple GPUs with different data communication models

AU - Hung, Che Lun

AU - Lin, Yi Yang

AU - Tang, Chuan Yi

AU - Wang, Chilung

AU - Chen, Ming Chiang

PY - 2018/8/20

Y1 - 2018/8/20

N2 - Recently, deep learning technologies have been utilized in many scientific domains successfully. Convolution neural networks are common used in image understanding problems. However, to train a convolution neural network model with huge amount of images is time-consuming task. Most of deep learning frameworks, such as Caffe, TensorFlow, Torch, Keras, MxNet, and so forth, support GPU to train model fast; especially executing these models on multiple GPUs. In this work, we present the comparison of computation performance of AlexNet among different GPU servers and hyperparameters. The results shows that GPU servers with high bandwidth rate, NVLINK, can achieve better performance than others.

AB - Recently, deep learning technologies have been utilized in many scientific domains successfully. Convolution neural networks are common used in image understanding problems. However, to train a convolution neural network model with huge amount of images is time-consuming task. Most of deep learning frameworks, such as Caffe, TensorFlow, Torch, Keras, MxNet, and so forth, support GPU to train model fast; especially executing these models on multiple GPUs. In this work, we present the comparison of computation performance of AlexNet among different GPU servers and hyperparameters. The results shows that GPU servers with high bandwidth rate, NVLINK, can achieve better performance than others.

KW - Convolution Neural Networks

KW - Deep Learning

KW - GPU

KW - Multiple GPUs

UR - https://www.scopus.com/pages/publications/85053506153

U2 - 10.1109/SNPD.2018.8441056

DO - 10.1109/SNPD.2018.8441056

M3 - 会议稿件

AN - SCOPUS:85053506153

SN - 9781538658895

T3 - Proceedings - 2018 IEEE/ACIS 19th International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing, SNPD 2018

SP - 87

EP - 92

BT - Proceedings - 2018 IEEE/ACIS 19th International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing, SNPD 2018

A2 - Hwang, Ha Jin

A2 - Cai, Lizhi

A2 - Yeom, Gun Huck

A2 - Matsuo, Tokuro

A2 - Kim, Haeng Kon

A2 - Yeo, Hyun

A2 - Hong, Chung Sun

A2 - Fukuta, Naoki

A2 - Ito, Takayuki

A2 - Miao, Huaikou

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 19th IEEE/ACIS International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing, SNPD 2018

Y2 - 27 June 2018 through 29 June 2018

ER -

Hung CL, Lin YY, Tang CY, Wang C, Chen MC. Performance of convolution neural network based on multiple GPUs with different data communication models. In Hwang HJ, Cai L, Yeom GH, Matsuo T, Kim HK, Yeo H, Hong CS, Fukuta N, Ito T, Miao H, editors, Proceedings - 2018 IEEE/ACIS 19th International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing, SNPD 2018. Institute of Electrical and Electronics Engineers Inc. 2018. p. 87-92. 8441056. (Proceedings - 2018 IEEE/ACIS 19th International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing, SNPD 2018). doi: 10.1109/SNPD.2018.8441056

Performance of convolution neural network based on multiple GPUs with different data communication models

Abstract

Publication series

Conference

Bibliographical note

Keywords

Access to Document

Other files and links

Fingerprint

Cite this