Research on Behavioural Pattern Mining in Computational Social Science Based on Multimodal Data

DOI: https://doi.org/10.62517/jike.202604406

Author(s)

Yuxi Liu

Affiliation(s)

College of St Hild & St Bede, Durham University, Durham, United Kingdom

Abstract

With the accumulation of multimodal data, computational social science has gained a new analytical tool; however, a semantic gap among various data sources and the dynamic, non-linear changes in social behaviour have remained obstacles to precise and in-depth mining of behaviour patterns. To address the above problems, this paper puts forward a three-stage progressive research framework of "semantic alignment - dynamic modelling - scenario validation". At the level of data, a contrastive learning-based cross-modal semantic bridging mechanism is constructed to eliminate representational bias across modalities. A behavior-mining architecture incorporating causal temporal attention and heterogeneous graph neural networks has been proposed at the model level to address the problem of fine-grained observation of changing group behaviour. At the application layer, online public opinion sentiment polarisation and abnormal group behaviour in emergency situations are used as validation cases. Based on the experiments, the behaviour recognition F1 score is 0.873, and cross-platform transfer accuracy has improved by 11.4%. The results show that the pathway of multimodal causal modelling is suitable for application in social behaviour analysis.

Keywords

Multimodal Data Fusion; Computational Social Science; Behavioral Pattern Mining; Causal Temporal Modeling; Heterogeneous Graph Neural Network

References

[1]Cheng, Z., & Li, Y. (2024). Like, Comment, and Share on TikTok: Exploring the Effect of Sentiment and Second-Person View on the User Engagement with TikTok News Videos.Social Science Computer Review, 42(1), 201–223. [2]Chen, X. K., & Na, J.-C. (2025). A Theory-Driven Approach to Fake News/Information Disorder Analysis and Explanation via Target-Based Emotion–Stance Analysis (TESA) and Interpretive Graph Generation (IGG).Social Science Computer Review, 44(4), 648–677. [3]Stromer-Galley, J., McKernan, B., Zaman, S., Maganur, C., & Regmi, S. (2025). The Efficacy of Large Language Models and Crowd Annotation for Accurate Content Analysis of Political Social Media Messages.Social Science Computer Review, 44(3), 466–487. [4]Zhang, H., & Leung, R. (2025/2026). Joint Text-and-Image Clustering for Social Science Research.Sociological Methods & Research, 56(1). [5]Thapa, S., Shiwakoti, S., Shah, S. B., et al. (2025). Large Language Models (LLM) in Computational Social Science: Prospects, Current State, and Challenges.Social Network Analysis and Mining, 15, Article 4. [6]Vonkova, H., Papajoanu, O., & Bosko, M. (2025). Response Times and Self-Reporting: Response Patterns Across Countries and World Regions Using Data From a Large Scale Computer-Based Assessment.Social Science Computer Review, 44(4), 617–647. [7]Ma, L., Han, W., & Li, J. (2025). Machine Learning Based Learning User Behavior Classification and Recognition Algorithm Under Multimodal Data.Discover Artificial Intelligence, 5, Article 413. [8]Weisskirch, R. S. (2025). Social Media Made Me Do It: Perceptions of Social Media Influence, Risky Behaviors, and Mental Health Among Adolescents.Social Science Computer Review, 44(4), 779–789