Design of an Intelligent Dining Recommendation System for Nutritional Intake Based on Multimodal Recognition

DOI: https://doi.org/10.62517/jike.202504422

Author(s)

Hao Chen

Affiliation(s)

Nanjing University of Posts and Telecommunication College of Integrated Circuit Science and Engineering (College of Industry-Education Integration Nanjing), China

Abstract

Within the broader context of rising chronic diseases linked to dietary imbalances in China and the rapid advancement of artificial intelligence, this research focuses on developing an intelligent dining recommendation system based on multimodal nutrient intake recognition. Existing approaches are constrained by significant limitations, including the absence of robust multimodal nutritional quantification models, substantial spatio-temporal data fusion challenges, and considerable estimation errors caused by factors like ingredient occlusion, cooking transformations, and individual absorption variability. To address these deficiencies, this study employs a multimodal recognition framework integrating visual (RGB-D), depth, and skeleton data, combined with knowledge graph embedding and dynamic user interest modeling using attention mechanisms. Key findings demonstrate that the proposed system, through the construction of a Chinese Nutrient Map (CNM-NutriMap), achieves markedly lower error rates in calorie, protein, and fat estimation compared to traditional methods. Furthermore, it shows potential for enhancing dietary compliance in hypertension management and generating significant health economic savings. The manuscript's unique value lies in its theoretical contribution of a cross-domain "behavior-nutrition-physiology" correlation model and its practical innovation in providing a scalable, precision nutrition intervention tool for public health.

Keywords

Multimodal; Intelligent; Nutritional

References

[1] Jia Chen, Liu Huaping, Xu Xinyi, et al. Multimodal Information Fusion Based on Width Learning Method [J]. Journal of Intelligent Systems, 2019, 14(1): 150–157. [2] Geng, H. C., Liang, H. T., & Liu, G. Z. (2021). Dietary recommendation algorithm based on knowledge graph and collaborative filtering. Computer & Modernization, (8): 24-29. [3] Wang Cailing, Yan Jingjing, Zhang Zhidong. Research Review on Human Behavior Recognition Methods Based on Multimodal Data [J]. Computer Engineering and Applications, 2024, 60(9): 1-18 [4] Wu Hang. Research and Implementation of a Visual Perception-Based Dietary Health Management System [D]. Jinan: Shandong University, 2024 [5] Gu Jingfan. Interpretation of the "Report on the Status of Nutrition and Chronic Diseases among Chinese Residents (2015)" [J]. Chinese Journal of Nutrition, 2016, 38(6): 525-529 [6] Li L, Rao KQ, Kong LZ, et al. China National Nutrition and Health Survey 2002 [J]. Chinese Journal of Epidemiology, 2005, 26(7): 478-484 [7] Xu Yatao. Study on Physical Development Status and Influencing Factors Among Chinese Children and Adolescents [D]. Shanghai: East China Normal University, 2019 [8] Shao Bangli, Zhu Yin, Zhu Run, et al. A multimodal human-machine intelligent interaction method for smart home device control [J]. Journal of Forest Engineering, 2021, 6(4): 190-196 [9] Yin Yifan. Research on Robot Tactile Sensing and Multimodal Perception Technology [D]. Nanchang: Nanchang University, 2024 [10] Du Xiang, Qian Jiahe, Nie Wennan, et al. Research Progress of Modern Image Analysis Methods in Traditional Chinese Medicine Identification [J]. Chinese Journal of Pharmacy, 2025, 60(14): 1479-1485. [11] Zhang Xinyu. Research and Development of a Knowledge Graph-Based Healthy Diet Recommendation System [D]. Tianjin: Tianjin University of Science and Technology, 2023. [12] Sun Ting. Research on Exercise and Dietary Interventions for Hypertension and Methods to Improve Compliance [D]. Hefei: University of Science and Technology of China, 2025. [13] Gong Weiyan, Yuan Fan, Ding Caicui, et al. Design and Development of a National Nutrition and Health Assessment System [J]. Chinese Journal of Nutrition, 2025, 47(2): 108-112 [14] Li Nan, Jian Yuxuan, Bi Zusong, Gong Lei, Liu Zihao, Fan Jiale. Design of an Intelligent Refrigerator Recommendation System [J]. Internet of Things Technology, 2022, (12): 125-126 [15] Tang Hongtao, Liu Rui, Xia Rui, et al. Current Status of Food Education Practices Domestically and Internationally [J]. Chinese Food and Nutrition, 2020, 26(1): 5-8. [16] Mou Zhijia, Fu Yaru. Review of Multimodal Learning Analysis Research [J]. Journal of Distance Education, 2021, 31(6): 23-31. [17] Sheng Shiwang. Development of a Personalized Intelligent Diet Recommendation System [D]. Hangzhou: Zhejiang University of Science and Technology, 2015 [18] Cui Xiaohui, Li Wei, Gu Chengchun. Big Data and Artificial Intelligence Technologies in Food Science [J]. Chinese Journal of Food Science, 2021, 21(2): 1-8 [19] Liu Rui, Xu Luhui. Research on Intelligent Food Recommendation System Based on Flink [J]. Information Technology and Informatization, 2022, (10): 204-207 [20] Zhan Jianhao, Wu Hongwei, Zhou Chengzu, Chen Xiaoqiu, Li Xiaochao. A Review of Deep Learning-Based Multimodal Fusion Methods for Behavior Recognition [J]. Computer Systems Applications, 2023, 32(1): 41–49. [21] Li Hongliang, Liu Yuliang, Liao Wenhui, Huang Mingxin, Zhang Shuo, Jin Lianwen. Optical Character Recognition in the Era of Large Models: Current Status and Outlook [J]. Chinese Journal of Image and Graphics, 2025, 30(6): 2023-2050 [22] Hu Yong. Research on Information Fusion Applications in Pattern Recognition [D]. Hefei: Hefei University of Technology, 2007. [23] Liu Yongtao. Research on Human Behavior Recognition Based on Deep Learning [D]. Nanjing: Nanjing University of Posts and Telecommunications, 2022.