Research on a Sign Language Assisted Learning System Based on Unity Platform and Augmented Reality Technology

DOI: https://doi.org/10.62517/jike.202604206

Author(s)

Hongwei Wang1,2,*, Xin Lin1, Qiankun Peng1, Jinlong Yang3, Huoquan Chen4

Affiliation(s)

1Faculty of Mathematics and Computer Science, Quanzhou Normal University, Quanzhou, China; 2Fujian Provincial Key Laboratory of Data-Intensive Computing, Quanzhou, China 3School of Education, Quanzhou Normal University, Quanzhou, China 4School of Innovation and Entrepreneurship of Quanzhou Normal University, Quanzhou, China *Corresponding Author

Abstract

With the development of technology, augmented reality technology, as an effective auxiliary tool, is gradually being applied in the field of education. This study develops an interactive sign language learning system based on the Unity3D platform, aimed at ordinary people who want to learn sign language, exploring how to use modern technology to improve the efficiency and fun of sign language learning. The system adopts augmented reality technology and uses the Vuforia plugin to recognize sign language images and dynamically display corresponding sign language actions, enhancing the intuitiveness and interactivity of the learning process. Combining Blender modeling and animation functions, the system presents high-precision 3D sign language action models. Users can rotate and zoom the model through the touch screen to observe the details of the actions. The augmented reality sign language assisted learning system developed in this study provides ordinary users with a simple and convenient way of learning sign language, enhances the fun of learning, and provides new technical support for the promotion and popularization of sign language. It also makes a beneficial exploration for the application of augmented reality technology in sign language education.

Keywords

Augmented Reality; Sign Language Simulation; Vuforia; Unity3D; Educational Technology

References

[1]Yu T. Y., Xiong S. P., & Zhang M. Z. (2023). Research on the dissemination effect of AR technology in preschool education. Education and Teaching Forum, (35), 45-48. [2]Li X. M. (2017). Research on Chinese sign language recognition for small-to-medium vocabulary based on neural networks (pp. 1-10). University of Science and Technology of China, Hefei. (Doctoral dissertation) [3]Zhang, Yanxiao, Yuecong Min, and Xilin Chen. "Teaching chinese sign language with a smartphone." Virtual Reality & Intelligent Hardware 3.3 (2021): 248-260. [4]Zhu P. F. (2019). Research on augmented reality (AR) technology promoting high school students’ learning of chemical microstructure. Chemistry Teaching, (9), 34-38. [5]Cai S., Li Y. T., Zhao Q., et al. (2018). Exploring microscopic cells with augmented reality technology — The 11th empirical case of AR in K-12 education. Information Technology in Primary and Secondary Education, (12), 66-68. [6] Hou D., & Song H. Z. (2018). Application research of 3D virtual augmented reality technology in middle school physics teaching. Journal of Tianjin Normal University (Elementary Education Edition), 19(3). [7]Tong Q., & Hong Y. (2023). Design and development of junior high school geography teaching software based on augmented reality technology. Journal of Hubei Normal University (Natural Science Edition), 43(4), 62-67. [8]Al-Ansi, Abdullah M., et al. "Analyzing augmented reality (AR) and virtual reality (VR) recent development in education." Social Sciences & Humanities Open 8.1 (2023): 100532. [9]Aditama P W, Putra P S U, Yusa I M M, et al. Designing augmented reality sibi sign language as a learning media//Journal of Physics: Conference Series. IOP Publishing, 2021, 1810(1): 012038. [10]Cheng H. F. (2018). Application research of mobile augmented reality technology in sign language learning teaching. Journal of Wuhu Vocational and Technical College, 20(2), 81-83.