Improved Unsupervised Chinese Word Segmentation Using Pre-trained Knowledge and Pseudo-labeling Transfer

Hsiu Wen Li; Ying Jia Lin; Yi Ting Li; Chun Yi Lin; Hung Yu Kao

doi:10.18653/v1/2023.emnlp-main.564

Improved Unsupervised Chinese Word Segmentation Using Pre-trained Knowledge and Pseudo-labeling Transfer

Hsiu Wen Li, Ying Jia Lin, Yi Ting Li, Chun Yi Lin, Hung Yu Kao

National Cheng Kung University

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

1 Scopus citations

Abstract

Unsupervised Chinese word segmentation (UCWS) has made progress by incorporating linguistic knowledge from pre-trained language models using parameter-free probing techniques. However, such approaches suffer from increased training time due to the need for multiple inferences using a pre-trained language model to perform word segmentation. This work introduces a novel way to enhance UCWS performance while maintaining training efficiency. Our proposed method integrates the segmentation signal from the unsupervised segmental language model to the pre-trained BERT classifier under a pseudo-labeling framework. Experimental results demonstrate that our approach achieves state-of-the-art performance on the seven out of eight UCWS tasks while considerably reducing the training time compared to previous approaches.

Original language	English
Title of host publication	EMNLP 2023 - 2023 Conference on Empirical Methods in Natural Language Processing, Proceedings
Editors	Houda Bouamor, Juan Pino, Kalika Bali
Publisher	Association for Computational Linguistics (ACL)
Pages	9109-9118
Number of pages	10
ISBN (Electronic)	9798891760608
DOIs	https://doi.org/10.18653/v1/2023.emnlp-main.564
State	Published - 2023
Externally published	Yes
Event	2023 Conference on Empirical Methods in Natural Language Processing, EMNLP 2023 - Hybrid, Singapore, Singapore Duration: 06 12 2023 → 10 12 2023

Publication series

Name	EMNLP 2023 - 2023 Conference on Empirical Methods in Natural Language Processing, Proceedings

Conference

Conference	2023 Conference on Empirical Methods in Natural Language Processing, EMNLP 2023
Country/Territory	Singapore
City	Hybrid, Singapore
Period	06/12/23 → 10/12/23

Bibliographical note

Publisher Copyright:
© 2023 Association for Computational Linguistics.

Access to Document

10.18653/v1/2023.emnlp-main.564

Cite this

Li, H. W., Lin, Y. J., Li, Y. T., Lin, C. Y., & Kao, H. Y. (2023). Improved Unsupervised Chinese Word Segmentation Using Pre-trained Knowledge and Pseudo-labeling Transfer. In H. Bouamor, J. Pino, & K. Bali (Eds.), EMNLP 2023 - 2023 Conference on Empirical Methods in Natural Language Processing, Proceedings (pp. 9109-9118). (EMNLP 2023 - 2023 Conference on Empirical Methods in Natural Language Processing, Proceedings). Association for Computational Linguistics (ACL). https://doi.org/10.18653/v1/2023.emnlp-main.564

Li, Hsiu Wen ; Lin, Ying Jia ; Li, Yi Ting et al. / Improved Unsupervised Chinese Word Segmentation Using Pre-trained Knowledge and Pseudo-labeling Transfer. EMNLP 2023 - 2023 Conference on Empirical Methods in Natural Language Processing, Proceedings. editor / Houda Bouamor ; Juan Pino ; Kalika Bali. Association for Computational Linguistics (ACL), 2023. pp. 9109-9118 (EMNLP 2023 - 2023 Conference on Empirical Methods in Natural Language Processing, Proceedings).

@inproceedings{1c7ba069532646798d4880568b6f3d6e,

title = "Improved Unsupervised Chinese Word Segmentation Using Pre-trained Knowledge and Pseudo-labeling Transfer",

abstract = "Unsupervised Chinese word segmentation (UCWS) has made progress by incorporating linguistic knowledge from pre-trained language models using parameter-free probing techniques. However, such approaches suffer from increased training time due to the need for multiple inferences using a pre-trained language model to perform word segmentation. This work introduces a novel way to enhance UCWS performance while maintaining training efficiency. Our proposed method integrates the segmentation signal from the unsupervised segmental language model to the pre-trained BERT classifier under a pseudo-labeling framework. Experimental results demonstrate that our approach achieves state-of-the-art performance on the seven out of eight UCWS tasks while considerably reducing the training time compared to previous approaches.",

author = "Li, \{Hsiu Wen\} and Lin, \{Ying Jia\} and Li, \{Yi Ting\} and Lin, \{Chun Yi\} and Kao, \{Hung Yu\}",

note = "Publisher Copyright: {\textcopyright} 2023 Association for Computational Linguistics.; 2023 Conference on Empirical Methods in Natural Language Processing, EMNLP 2023 ; Conference date: 06-12-2023 Through 10-12-2023",

year = "2023",

doi = "10.18653/v1/2023.emnlp-main.564",

language = "英语",

series = "EMNLP 2023 - 2023 Conference on Empirical Methods in Natural Language Processing, Proceedings",

publisher = "Association for Computational Linguistics (ACL)",

pages = "9109--9118",

editor = "Houda Bouamor and Juan Pino and Kalika Bali",

booktitle = "EMNLP 2023 - 2023 Conference on Empirical Methods in Natural Language Processing, Proceedings",

address = "澳大利亚",

}

Li, HW, Lin, YJ, Li, YT, Lin, CY & Kao, HY 2023, Improved Unsupervised Chinese Word Segmentation Using Pre-trained Knowledge and Pseudo-labeling Transfer. in H Bouamor, J Pino & K Bali (eds), EMNLP 2023 - 2023 Conference on Empirical Methods in Natural Language Processing, Proceedings. EMNLP 2023 - 2023 Conference on Empirical Methods in Natural Language Processing, Proceedings, Association for Computational Linguistics (ACL), pp. 9109-9118, 2023 Conference on Empirical Methods in Natural Language Processing, EMNLP 2023, Hybrid, Singapore, Singapore, 06/12/23. https://doi.org/10.18653/v1/2023.emnlp-main.564

Improved Unsupervised Chinese Word Segmentation Using Pre-trained Knowledge and Pseudo-labeling Transfer. / Li, Hsiu Wen; Lin, Ying Jia; Li, Yi Ting et al.
EMNLP 2023 - 2023 Conference on Empirical Methods in Natural Language Processing, Proceedings. ed. / Houda Bouamor; Juan Pino; Kalika Bali. Association for Computational Linguistics (ACL), 2023. p. 9109-9118 (EMNLP 2023 - 2023 Conference on Empirical Methods in Natural Language Processing, Proceedings).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Improved Unsupervised Chinese Word Segmentation Using Pre-trained Knowledge and Pseudo-labeling Transfer

AU - Li, Hsiu Wen

AU - Lin, Ying Jia

AU - Li, Yi Ting

AU - Lin, Chun Yi

AU - Kao, Hung Yu

PY - 2023

Y1 - 2023

N2 - Unsupervised Chinese word segmentation (UCWS) has made progress by incorporating linguistic knowledge from pre-trained language models using parameter-free probing techniques. However, such approaches suffer from increased training time due to the need for multiple inferences using a pre-trained language model to perform word segmentation. This work introduces a novel way to enhance UCWS performance while maintaining training efficiency. Our proposed method integrates the segmentation signal from the unsupervised segmental language model to the pre-trained BERT classifier under a pseudo-labeling framework. Experimental results demonstrate that our approach achieves state-of-the-art performance on the seven out of eight UCWS tasks while considerably reducing the training time compared to previous approaches.

AB - Unsupervised Chinese word segmentation (UCWS) has made progress by incorporating linguistic knowledge from pre-trained language models using parameter-free probing techniques. However, such approaches suffer from increased training time due to the need for multiple inferences using a pre-trained language model to perform word segmentation. This work introduces a novel way to enhance UCWS performance while maintaining training efficiency. Our proposed method integrates the segmentation signal from the unsupervised segmental language model to the pre-trained BERT classifier under a pseudo-labeling framework. Experimental results demonstrate that our approach achieves state-of-the-art performance on the seven out of eight UCWS tasks while considerably reducing the training time compared to previous approaches.

UR - https://www.scopus.com/pages/publications/85184798482

U2 - 10.18653/v1/2023.emnlp-main.564

DO - 10.18653/v1/2023.emnlp-main.564

M3 - 会议稿件

AN - SCOPUS:85184798482

T3 - EMNLP 2023 - 2023 Conference on Empirical Methods in Natural Language Processing, Proceedings

SP - 9109

EP - 9118

BT - EMNLP 2023 - 2023 Conference on Empirical Methods in Natural Language Processing, Proceedings

A2 - Bouamor, Houda

A2 - Pino, Juan

A2 - Bali, Kalika

PB - Association for Computational Linguistics (ACL)

T2 - 2023 Conference on Empirical Methods in Natural Language Processing, EMNLP 2023

Y2 - 6 December 2023 through 10 December 2023

ER -

Li HW, Lin YJ, Li YT, Lin CY, Kao HY. Improved Unsupervised Chinese Word Segmentation Using Pre-trained Knowledge and Pseudo-labeling Transfer. In Bouamor H, Pino J, Bali K, editors, EMNLP 2023 - 2023 Conference on Empirical Methods in Natural Language Processing, Proceedings. Association for Computational Linguistics (ACL). 2023. p. 9109-9118. (EMNLP 2023 - 2023 Conference on Empirical Methods in Natural Language Processing, Proceedings). doi: 10.18653/v1/2023.emnlp-main.564

Improved Unsupervised Chinese Word Segmentation Using Pre-trained Knowledge and Pseudo-labeling Transfer

Abstract

Publication series

Conference

Bibliographical note

Access to Document

Other files and links

Fingerprint

Cite this