Development of nonverbal communication behavior model for nursing students based on deep learning facial expression recognition technology

Nonverbal communication plays a key role in conveying emotions during clinical interactions, but assessing it can be challenging. This study aims to develop a model for nonverbal communication using facial expression recognition and deep learning. A hybrid approach combining convolutional neural network (CNN)-based face detection and Extra Trees classifier was used to leverage CNN’s feature extraction and Extra Trees’ classification abilities. The study involved 88 nursing students participating in communication exercises with standardized patients, with their facial expressions scored by senior nurses and instructors, resulting in 3,318 labeled scores. The model, built using Extra Trees Regressor, found that students displayed adequate receptiveness, politeness, friendliness, and nonverbal empathy, with scores primarily between 60 and 70. By integrating clinical simulations with artificial intelligence, subjective bias was minimized, improving the stability of nonverbal behavior simulations. This model will be integrated into a nonverbal behavior learning platform to help nursing students develop communication skills and reduce anxiety during practice. The platform offers objective feedback, potentially enhancing healthcare education and telemedicine by providing advanced tools for evaluating nonverbal communication.