Intelligent Hyperparameter Optimization of Convolutional Neural Networks for Robust Multimodal Classification

DOI: https://doi.org/10.62517/jike.202504309

Author(s)

Jing Zhao1, Hui Xing1, Qinwei Fan1,2

Affiliation(s)

1School of Science, Xi'an Polytechnic University, Xi'an, Shaanxi, China. 2School of Mathematics and Information Science, Guangzhou University, Guangzhou, China

Abstract

In this study, a novel hyperparameter optimisation algorithm for Convolutional Neural Networks (CNNs), based on a Multi-Strategy Improved Whale Optimisation Algorithm (IWOA), is proposed to enhance performance across diverse tasks. Recognising the critical impact of hyperparameters on CNN efficacy, the algorithm is designed to autonomously identify optimal parameter settings. To improve population diversity and uniformity during initialisation, the Singer chaotic mapping strategy is employed. Additionally, a nonlinear dynamic speed regulation mechanism is introduced to refine the spiral update control parameters in WOA, thereby enhancing the optimisation process. To further address premature convergence and avoid local optima, Gaussian mutation is utilised, enabling the algorithm to achieve faster convergence toward the global optimum. The enhanced IWOA is integrated with CNNs and evaluated on multiple benchmark functions to validate its optimisation capability. Moreover, extensive image classification experiments on various datasets demonstrate the algorithm's effectiveness in improving CNN recall and accuracy while showcasing strong generalisation ability. The results highlight that the proposed approach significantly outperforms traditional methods in searching and optimising CNN hyperparameters, delivering superior performance and robustness across multiple tasks.

Keywords

Whale Optimisation Algorithm; Convolutional Neural Network; Hyperparameters Optimization; Image Classification

References

[1] Chakraborty A, Kar A K. Swarm intelligence: A review of algorithms[J]. Nature-inspired Computing and Optimization: Theory and Applications, 2017: 475-494. [2] Abdollahzadeh B, Khodadadi N, Barshandeh S, et al. Puma Optimizer (PO): a novel metaheuristic optimization algorithm and its application in machine learning[J]. Cluster Computing, 2024, 27(4): 5235-5283. [3] Wang J, Wang W, Hu X, et al. Black-winged kite algorithm: a nature-inspired meta-heuristic for solving benchmark functions and engineering problems[J]. Artificial Intelligence Review, 2024, 57(4): 98. [4] Zhao W G, Wang L Y, Zhang Z X, et al. Electric eel foraging optimization: A new bio-inspired optimizer for engineering applications[J]. Expert Systems with Applications, 2024, 238: 122200. [5] Passino K M. Biomimicry of bacterial foraging for distributed optimization and control[J]. IEEE Control Systems Magazine, 2002, 22(3): 52-67. [6] Eusuff M, Lansey K, Pasha F. Shuffled frog-leaping algorithm: a memetic meta-heuristic for discrete optimization[J]. Engineering Optimization, 2006, 38(2): 129-154. [7] Mirjalili S, Lewis A. The whale optimization algorithm[J]. Advances in Engineering Software, 2016, 95: 51-67. [8] Tanyildizi E, Cgali T. Whale Optimization Algorithms With Chaotic Mapping[J]. Frat Universitesi Muhendislik Bilimleri Dergisi, 2017, (1): 307-317. [9] Asghari K, Masdari M, Gharehchopogh F S, et al. A chaotic and hybrid gray wolf-whale algorithm for solving continuous optimization problems[J]. Progress in Artificial Intelligence, 2021, 10(3): 349-374. [10] Nam C. Convolutional neural network-based prediction of hardness in bulk metallic glasses with small data[J]. Journal of Non-Crystalline Solids, 2025, 654123451-123451. [11] Hubel D H, Wiesel T N. Receptive fields, binocular interaction and functional architecture in the cat's visual cortex[J]. The Journal of Physiology, 1962, 160(1): 106. [12] LeCun Y, Boser B, Denker J S, et al. Backpropagation applied to handwritten zip code recognition[J]. Neural computation, 1989, 1(4): 541-551. [13] Ha I ,Kim H ,Park S , et al.Image retrieval using BIM and features from pretrained VGG network for indoor localization[J]. Building and Environment, 2018, 14023-31. [14] Yang H, Wenjun K, Jinqiang L. ResNet Combined with Attention Mechanism for Genomic Deletion Variant Prediction[J]. Automatic Control and Computer Sciences, 2024, 58(3): 252-264. [15] Yao X, Wang X, Karaca Y, et al. Glomerulus classification via an improved GoogLeNet[J]. IEEE Access, 2020, 8: 176916-176923. [16] Wang L, Yang Y, Min R, et al. Accelerating deep neural network training with inconsistent stochastic gradient descent[J]. Neural Networks, 2017, 93: 219-229. [17] Raiaan M A K, Sakib S, Fahad N M, et al. A systematic review of hyperparameter optimization techniques in Convolutional Neural Networks[J]. Decision Analytics Journal, 2024: 100470. [18] Bergstra J, Bengio Y. Random search for hyper-parameter optimization[J]. The Journal of Machine Learning Research, 2012, 13(1): 281-305. [19] Tuerxun W, Chang X, Hongyu G, et al. Fault diagnosis of wind turbines based on a support vector machine optimized by the sparrow search algorithm[J]. IEEE Access, 2021, 9: 69307-69315. [20] Gorkemli B, Kaya E, Karaboga D, et al. A review on the versions of artificial bee colony algorithm for scheduling problems[J]. Journal of Combinatorial Optimization, 2025, 49(4): 1-46. [21] Suriyan K, Nagarajan R. Particle swarm optimization in biomedical technologies: innovations, challenges, and opportunities[J]. Emerging Technologies for Health Literacy and Medical Practice, 2024: 220-238. [22] Mirjalili S. Genetic algorithm[J]. Evolutionary Algorithms and Neural Networks: Theory and Applications, 2019: 43-55. [23] Liu Z, Yeh W C. Simplified swarm optimisation for CNN hyperparameters: a sound classification approach[J]. International Journal of Web and Grid Services, 2024, 20(1): 93-113. [24] Munsarif M, Sam'an M, Fahrezi A. Convolution neural network hyperparameter optimization using modified particle swarm optimization[J]. Bulletin of Electrical Engineering and Informatics, 2024, 13(2): 1268-1275. [25] Aghabeigi F, Nazari S, Osati Eraghi N. An efficient facial emotion recognition using convolutional neural network with local sorting binary pattern and whale optimization algorithm[J]. International Journal of Data Science and Analytics, 2024: 1-16. [26] Seyedali M, Andrew L. The Whale Optimization Algorithm[J]. Advances in Engineering Software, 2016 (1): 95-101． [27] Luan F, Cai Z, Wu S, et al. Improved Whale Algorithm for Solving the Flexible Job Shop Scheduling Problem[J]. Mathematics, 2019, 7(5): 384-384. [28] Hiba A, Mohammed A. Software fault prediction using Whale algorithm with genetics algorithm[J]. Software: Practice and Experience, 2020, 51(5): 1121-1146. [29] Taye M M. Theoretical understanding of convolutional neural network: Concepts, architectures, applications, future directions[J]. Computation, 2023, 11(3): 52. [30] Wang Z J, Turko R, Shaikh O, et al. CNN explainer: learning convolutional neural networks with interactive visualization[J]. IEEE Transactions on Visualization and Computer Graphics, 2020, 27(2): 1396-1406. [31] Cheng N, Chen Y, Gao W, et al. An improved deep learning model: S-TextBLCNN for traditional Chinese medicine formula classification[J]. Frontiers in Genetics, 2021, 12: 807825. [32] Ewees A A, El Aziz M A, Hassanien A E. Chaotic multi-verse optimizer-based feature selection[J]. Neural Computing and Applications, 2019, 31: 991-1006. [33] Wei F , Shi X , Feng Y, et al. Improved Harris hawk algorithm based on multi-strategy synergy mechanism for global optimization [J]. Soft Computing, 2024, 28(21): 1-46. [34] Qian P, Pu C, Liu L. Ultra-high-precision pneumatic force servo system based on a novel improved particle swarm optimization algorithm integrating Gaussian mutation and fuzzy theory[J]. ISA Transactions, 2024, 152: 453-466. [35] Zhang J, Yu X, Lei X, et al. A novel deep LeNet-5 convolutional neural network model for image recognition[J]. Computer Science and Information Systems, 2022, 19(3): 1463-1480. [36] Tulbure A A, Tulbure A A, Dulf E H. A review on modern defect detection models using DCNNs–Deep convolutional neural networks[J]. Journal of Advanced Research, 2022, 35: 33-48. [37] Wu X, Li S, Jiang X, et al. Information acquisition optimizer: a new efficient algorithm for solving numerical and constrained engineering optimization problems[J]. The Journal of Supercomputing, 2024, 80(18): 25736-25791. [38] Jamil M, Yang X S. A literature survey of benchmark functions for global optimisation problems[J]. International Journal of Mathematical Modelling and Numerical Optimisation, 2013, 4(2): 150-194. [39] Tanveer M, Gautam C, Suganthan P N. Comprehensive evaluation of twin SVM based classifiers on UCI datasets[J]. Applied Soft Computing, 2019, 83: 105617. [40] Wang C X, Shi T T, Han D N. Adaptive dimensional gaussian mutation of PSO-optimized convolutional neural network hyperparameters [J]. Applied Sciences, 2023, 13(7):4254.