The Application of Transformer in Financial Management

DOI: https://doi.org/10.62517/jbdc.202501410

Author(s)

Zeying Yu

Affiliation(s)

Department of CS & IT, Beijing University of Technology, Beijing, China

Abstract

This paper focuses on internet finance scenarios and sorts out the application of Transformer models in financial risk management. Internet finance has developed rapidly due to its characteristics of convenience and real-time performance, but it also faces various risks such as credit, operation, liquidity, and compliance. Traditional risk management tools have obvious shortcomings in integrating big data, achieving data sharing, and making models easy to understand. However, the Transformer model, with its unique self-attention mechanism, shows advantages in dealing with associated risks, systemic risks, and extreme risks in financial markets. It functions through methods such as hierarchical encoding, event-aware modeling, and multi-period feature fusion. Comparative studies have found that in tasks such as stock price prediction, exchange rate prediction, and financial report risk grading, Transformer performs better than traditional models such as ARIMA and GARCH, as well as other deep learning models such as LSTM and GRU, which can reduce errors and improve prediction accuracy. To solve the problem of high computing cost, Transformer is optimized through lightweight designs such as streamlined architecture and reconstructed attention; to cope with unstable data, it enhances its adaptability to different scenarios by means of event embedding and cycle fusion; to meet regulatory requirements, it makes its decision-making process more understandable through mechanisms such as semantic visualization and feature attribution. However, there are some contradictions in current research: simplifying the model structure may affect the expression effect; enhancing adaptability to dynamic changes may bring the risk of overfitting; deeper interpretation of the model may affect computational efficiency.

Keywords

Transformer; Financial Risk Management; Lightweight Design; Interpretability; Scenario Adaptation

References

[1] Aldridge, I., & Kim, D. (2024). Quantitative financial models with scenarios from llm: Temporal fusion transformers as alternative monte-carlo. Available at SSRN 4999492. [2] Dong, Z., & Xu, L. (2025). Deep learning for financial forecasting and strategic business optimisation in enterprises. International Journal of Information and Communication Technology, 26(19), 79-101. [3] Foss, N. J., & Saebi, T. (2017). Fifteen years of research on business model innovation: How far have we come, and where should we go?. Journal of management, 43(1), 200-227. [4] Li, X., Bian, C., Li, X., Yu, S., & Jiang, B. (2025). Lamformer: LSTM-enhanced agent attention and mixture-of-experts transformer for efficient stock price prediction. International Journal of Machine Learning and Cybernetics, 1-14. [5] Liang, C. Y. (2025). Research and Application of Stock Index Trend Prediction Based on Transformer (Master's Thesis, Beijing University of Chemical Technology). Master's https://link.cnki.net/doi/10.26939/d.cnki.gbhgu.2025.000541 doi:10.26939/d.cnki.gbhgu.2025.000541. [6] Lyu, S. (2025, April). BILSTM Seq-to-Seq with Multi-Head Attention for Consumer Price Index Forecasting. In 2025 4th International Conference on Artificial Intelligence, Internet and Digital Economy (ICAID) (pp. 298-301). IEEE. [7] Pradhan, R., Alazzam, M. B., Keswani, S., Bhasin, N. K., Jaff, N. A., & Muthuperumal, S. (2025, February). A Hybrid GRU-Transformer Model for Financial Forecasting and Risk Management. In 2025 3rd International Conference on Integrated Circuits and Communication Systems (ICICACS) (pp. 1-5). IEEE. [8] Ruan, J. H., Wang, K., Feng, T. Y., & Wang, Z. C. (2025). Stock Prediction Method Based on Transformer. Computer and Digital Engineering, 53(05), 1375-1380+1433. [9] Shi, D. J. (2025). Research on Stock Price Prediction Based on TCN-Transformer with Multi-Factor Fusion (Master's Thesis, Jiangxi University of Finance and Economics). Master's https://kns.cnki.net/kcms2/article/abstract?v=IKKGlZ0AkeVpF1-euB3WRFZkORiNRFRI7E6CYGTSeAzLTItGbonRVv2zlXF1NPZWZHA8Bx0qaFH-FF_lpp3X4fQqicF5pkOFDS5qI9-WK63AeelKmhIzFHGuepU3IwwNvch6xvsM9GcpXmabOWy_f9Zw-sEvNnsiYo07B3VXQzdfxrH7V2__JQ==&uniplatform=NZKPT&language=CHS [10] Tan, X. W., & Kok, S. (2025). Explainable AI for Comprehensive Risk Assessment for Financial Reports: A Lightweight Hierarchical Transformer Network Approach. arXiv preprint arXiv:2506.23767. [11] Tian, X., Tian, Z., Khatib, S. F., & Wang, Y. (2024). Machine learning in internet financial risk management: A systematic literature review. Plos one, 19(4), e0300195. [12] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., ... & Polosukhin, I. (2017). Attention is all you need. Advances in neural information processing systems, 30. [13] Wang, L. S. (2024). Research on Enterprise Financial Risk Management in the Era of Big Data. Business Exhibition Economy, (17), 185-188. [14] Xie, C., & Wang, J. (2015). Research on credit risk of P2P network lending platform. Finance, 5(1), 1-5. [15] Zeng, Z. F., & You, Y. (2020). Financial Risk Warning Based on Neural Network Quantile Regression. Statistics and Decision, 36(14), 137-140. [16] Zhang, S., Che, T., Zhu, Z., Luo, G., & Feng, P. (2025). Forecasting of exchange rate time series based on event-aware transformer mode. Soft Computing, 1-11.