Research on the Construction of Personalized Learning State Recognition Model Integrating Multimodal Data

DOI: https://doi.org/10.62517/jhet.202515444

Author(s)

Ziqi Meng1, Xiaoping Huang2,*

Affiliation(s)

1Agricultural University Of Hunan, Hunan, China 2Nanyang Technological University, Singapore *Corresponding Author

Abstract

This study addresses the limitations of traditional learning state recognition methods in complex educational scenarios by proposing a multimodal deep learning model that integrates visual, acoustic, physiological, and behavioral data. The hierarchical feature fusion architecture combines heterogeneous data sources, while a dynamic attention mechanism enables weighted feature selection. A lightweight design ensures real-time performance. Experimental results demonstrate significant improvements over baseline methods in both recognition accuracy and F1 scores, with enhanced environmental noise resistance and cross-scenario generalization capabilities. The research confirms that multimodal integration comprehensively captures learners' cognitive and emotional states, providing a reliable personalized analysis tool for intelligent education systems. Future work should focus on optimizing cross-cultural adaptability and addressing long-term concept drift issues.

Keywords

Multimodal Fusion; Learning State Recognition; Attention Mechanism; Personalized leaRning; Educational Artificial Intelligence

References

[1] Xie Dingfeng, Zhou Anzhong, Li Jieqin. Research on Personalized Education Evaluation Empowered by Multimodal Data [J]. Journal of Hubei Open Vocational College, 2025,38(10):149-151. [2] Xue Yaofeng, Qiu Yisheng, Chen Zhan. System Framework for Multimodal Data Fusion and Its Educational Applications [J]. Basic Education, 2024,21(05):62-70. [3] Xie Dingfeng, Zhou Anzhong, Li Jieqin, et al. Research on Precision Intervention for Personalized Learning Based on Multimodal Data [J]. Computer Knowledge and Technology, 2024,20(16):98-100+104. [4] Jiang Jie, Yu Wenting, Wang Haiyan. Research on Student Learning Behaviors in Smart Classrooms Based on Multimodal Data [J]. China Education Informatization, 2024,30(04):107-117. [5] Zhang Lele and Gu Xiaoqing. A Classroom Teaching Behavior Analysis Model and Practical Framework Supported by Multimodal Data [J]. Open Education Research, 2022,28(06):101-110. [6] Hu Wenting. Research on Learning and Analysis Applications for Multimodal Data [J]. China New Communications, 2022,24(17):107-109. [7] Zhang Lizhao. Analysis of Learning Investment in Multimodal Data-Driven Contexts [D]. Central China Normal University, 2022.