Improving Multi-Criteria Chinese Word Segmentation through Learning Sentence Representation

Chun Yi Lin; Ying Jia Lin; Yi Ting Li; Chia Jen Yeh; Ching Wen Yang; Hung Yu Kao

doi:10.18653/v1/2023.findings-emnlp.850

Improving Multi-Criteria Chinese Word Segmentation through Learning Sentence Representation

Chun Yi Lin
, Ying Jia Lin
, Yi Ting Li
, Chia Jen Yeh
, Ching Wen Yang
, Hung Yu Kao

National Cheng Kung University

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

2 Scopus citations

Abstract

Recent Chinese word segmentation (CWS) models have shown competitive performance with pre-trained language models' knowledge. However, these models tend to learn the segmentation knowledge through in-vocabulary words rather than understanding the meaning of the entire context. To address this issue, we introduce a context-aware approach that incorporates unsupervised sentence representation learning over different dropout masks into the multi-criteria training framework. We demonstrate that our approach reaches state-of-the-art (SoTA) performance on F1 scores for six of the nine CWS benchmark datasets and out-of-vocabulary (OOV) recalls for eight of nine. Further experiments discover that substantial improvements can be brought with various sentence representation objectives.

Original language	English
Title of host publication	Findings of the Association for Computational Linguistics
Subtitle of host publication	EMNLP 2023
Publisher	Association for Computational Linguistics (ACL)
Pages	12756-12763
Number of pages	8
ISBN (Electronic)	9798891760615
DOIs	https://doi.org/10.18653/v1/2023.findings-emnlp.850
State	Published - 2023
Externally published	Yes
Event	2023 Findings of the Association for Computational Linguistics: EMNLP 2023 - Hybrid, Singapore Duration: 06 12 2023 → 10 12 2023

Publication series

Name	Findings of the Association for Computational Linguistics: EMNLP 2023

Conference

Conference	2023 Findings of the Association for Computational Linguistics: EMNLP 2023
Country/Territory	Singapore
City	Hybrid
Period	06/12/23 → 10/12/23

Bibliographical note

Publisher Copyright:
© 2023 Association for Computational Linguistics.

Access to Document

10.18653/v1/2023.findings-emnlp.850

Cite this

Lin, C. Y., Lin, Y. J., Li, Y. T., Yeh, C. J., Yang, C. W., & Kao, H. Y. (2023). Improving Multi-Criteria Chinese Word Segmentation through Learning Sentence Representation. In Findings of the Association for Computational Linguistics: EMNLP 2023 (pp. 12756-12763). (Findings of the Association for Computational Linguistics: EMNLP 2023). Association for Computational Linguistics (ACL). https://doi.org/10.18653/v1/2023.findings-emnlp.850

@inproceedings{32abd15c313a4cd28151eeb8df379f0c,

title = "Improving Multi-Criteria Chinese Word Segmentation through Learning Sentence Representation",

abstract = "Recent Chinese word segmentation (CWS) models have shown competitive performance with pre-trained language models' knowledge. However, these models tend to learn the segmentation knowledge through in-vocabulary words rather than understanding the meaning of the entire context. To address this issue, we introduce a context-aware approach that incorporates unsupervised sentence representation learning over different dropout masks into the multi-criteria training framework. We demonstrate that our approach reaches state-of-the-art (SoTA) performance on F1 scores for six of the nine CWS benchmark datasets and out-of-vocabulary (OOV) recalls for eight of nine. Further experiments discover that substantial improvements can be brought with various sentence representation objectives.",

author = "Lin, \{Chun Yi\} and Lin, \{Ying Jia\} and Li, \{Yi Ting\} and Yeh, \{Chia Jen\} and Yang, \{Ching Wen\} and Kao, \{Hung Yu\}",

note = "Publisher Copyright: {\textcopyright} 2023 Association for Computational Linguistics.; 2023 Findings of the Association for Computational Linguistics: EMNLP 2023 ; Conference date: 06-12-2023 Through 10-12-2023",

year = "2023",

doi = "10.18653/v1/2023.findings-emnlp.850",

language = "英语",

series = "Findings of the Association for Computational Linguistics: EMNLP 2023",

publisher = "Association for Computational Linguistics (ACL)",

pages = "12756--12763",

booktitle = "Findings of the Association for Computational Linguistics",

address = "澳大利亚",

}

Lin, CY, Lin, YJ, Li, YT, Yeh, CJ, Yang, CW & Kao, HY 2023, Improving Multi-Criteria Chinese Word Segmentation through Learning Sentence Representation. in Findings of the Association for Computational Linguistics: EMNLP 2023. Findings of the Association for Computational Linguistics: EMNLP 2023, Association for Computational Linguistics (ACL), pp. 12756-12763, 2023 Findings of the Association for Computational Linguistics: EMNLP 2023, Hybrid, Singapore, 06/12/23. https://doi.org/10.18653/v1/2023.findings-emnlp.850

Improving Multi-Criteria Chinese Word Segmentation through Learning Sentence Representation. / Lin, Chun Yi; Lin, Ying Jia; Li, Yi Ting et al.
Findings of the Association for Computational Linguistics: EMNLP 2023. Association for Computational Linguistics (ACL), 2023. p. 12756-12763 (Findings of the Association for Computational Linguistics: EMNLP 2023).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Improving Multi-Criteria Chinese Word Segmentation through Learning Sentence Representation

AU - Lin, Chun Yi

AU - Lin, Ying Jia

AU - Li, Yi Ting

AU - Yeh, Chia Jen

AU - Yang, Ching Wen

AU - Kao, Hung Yu

PY - 2023

Y1 - 2023

N2 - Recent Chinese word segmentation (CWS) models have shown competitive performance with pre-trained language models' knowledge. However, these models tend to learn the segmentation knowledge through in-vocabulary words rather than understanding the meaning of the entire context. To address this issue, we introduce a context-aware approach that incorporates unsupervised sentence representation learning over different dropout masks into the multi-criteria training framework. We demonstrate that our approach reaches state-of-the-art (SoTA) performance on F1 scores for six of the nine CWS benchmark datasets and out-of-vocabulary (OOV) recalls for eight of nine. Further experiments discover that substantial improvements can be brought with various sentence representation objectives.

AB - Recent Chinese word segmentation (CWS) models have shown competitive performance with pre-trained language models' knowledge. However, these models tend to learn the segmentation knowledge through in-vocabulary words rather than understanding the meaning of the entire context. To address this issue, we introduce a context-aware approach that incorporates unsupervised sentence representation learning over different dropout masks into the multi-criteria training framework. We demonstrate that our approach reaches state-of-the-art (SoTA) performance on F1 scores for six of the nine CWS benchmark datasets and out-of-vocabulary (OOV) recalls for eight of nine. Further experiments discover that substantial improvements can be brought with various sentence representation objectives.

UR - https://www.scopus.com/pages/publications/85183295437

U2 - 10.18653/v1/2023.findings-emnlp.850

DO - 10.18653/v1/2023.findings-emnlp.850

M3 - 会议稿件

AN - SCOPUS:85183295437

T3 - Findings of the Association for Computational Linguistics: EMNLP 2023

SP - 12756

EP - 12763

BT - Findings of the Association for Computational Linguistics

PB - Association for Computational Linguistics (ACL)

T2 - 2023 Findings of the Association for Computational Linguistics: EMNLP 2023

Y2 - 6 December 2023 through 10 December 2023

ER -

Lin CY, Lin YJ, Li YT, Yeh CJ, Yang CW, Kao HY. Improving Multi-Criteria Chinese Word Segmentation through Learning Sentence Representation. In Findings of the Association for Computational Linguistics: EMNLP 2023. Association for Computational Linguistics (ACL). 2023. p. 12756-12763. (Findings of the Association for Computational Linguistics: EMNLP 2023). doi: 10.18653/v1/2023.findings-emnlp.850

Improving Multi-Criteria Chinese Word Segmentation through Learning Sentence Representation

Abstract

Publication series

Conference

Bibliographical note

Access to Document

Other files and links

Fingerprint

Cite this