Enhancing Cybersecurity with Machine Learning: A Hybrid Approach for Anomaly Detection and Threat Prediction

Main Article Content

Adil M. Salman
Bashar Talib Al-Nuaimi
Alhumaima Ali Subhi
Hussein Alkattan
Raed H. C. Alfilh

Abstract

In today's digital era, cybersecurity has become a principal concern because of the increasing frequency and advancement of cyber threats. This study explores machine learning models for detecting and predicting anomalies in cybersecurity datasets. The research evaluates models such as linear regression, decision tree, RF, gradient boosting, KNN, SVR, LSTM, and neural networks utilizing performance metrics such as accuracy, MAE and MSE. A hybrid model that integrates different learning strategies is additionally proposed to improve the predictive accuracy and strength. The results highlight the superiority of ensemble approaches, especially the hybrid model, in improving peculiarity detection capabilities. The comparative analysis demonstrates that traditional models struggle with nonlinear patterns, whereas hybrid approaches successfully relieve this limitation. Moreover, this study emphasizes the importance of temporal data analysis for proactive threat detection and response. By leveraging diverse machine learning methods, this research contributes to strengthening cybersecurity infrastructures, enabling early threat detection, and minimizing security breaches. These discoveries emphasize the importance of adopting a comprehensive machine learning system to support cybersecurity resilience.

Article Details

Section

Articles

How to Cite

Enhancing Cybersecurity with Machine Learning: A Hybrid Approach for Anomaly Detection and Threat Prediction (A. M. . Salman, B. T. . Al-Nuaimi, A. . Ali Subhi, H. . Alkattan, & R. H. C. . Alfilh , Trans.). (2025). Mesopotamian Journal of CyberSecurity, 5(1), 202-215. https://doi.org/10.58496/MJCS/2025/014

References

[1] J. P. Anderson, “Technical Report,” in Computer Security Threat Monitoring and Surveillance, P. James, Ed. Washington, DC, USA: Anderson Company, 1980.

[2] Y. Liao and V. R. Vemuri, “Use of k-nearest neighbor classifier for intrusion detection,” Comput. Secur., vol. 21, pp. 439–448, 2002.

[3] G. Kim, S. Lee, and S. Kim, “A novel hybrid intrusion detection method integrating anomaly detection with misuse detection,” Expert Syst. Appl., vol. 41, pp. 1690–1700, 2013.

[4] E. Dela Hoz, E. De La Hoz, A. Ortiz, J. Ortega, and B. Prieto, “PCA filtering and probabilistic SOM for network intrusion detection,” Neurocomputing, vol. 164, pp. 71–81, 2015.

[5] R. Sen, M. Chattopadhyay, and N. Sen, “An efficient approach to develop an intrusion detection system based on multi layer backpropagation neural network algorithm: IDS using BPNN algorithm,” in Proc. 2015 ACM SIGMIS Conf. Comput. People Res., Newport Beach, CA, USA, Jun. 4–6, 2015, pp. 105–108.

[6] L. Koc, T. A. Mazzuchi, and S. Sarkani, “A network intrusion detection system based on a hidden Naïve Bayes multiclass classifier,” Expert Syst. Appl., vol. 39, pp. 13492–13500, 2012.

[7] M. A. Khan and Y. Kim, “Deep learning-based hybrid intelligent intrusion detection system,” Comput. Mater. Contin., vol. 68, pp. 671–687, 2021.

[8] B. T. Devi, S. S. Thirumaleshwari, and M. A. Jabbar, “An appraisal over intrusion detection systems in cloud computing security attacks,” in Proc. 2020 2nd Int. Conf. Innov. Mech. Ind. Appl. (ICIMIA), Bangalore, India, Mar. 5–7, 2020, pp. 722–727.

[9] I. S. Thaseen, B. Poorva, and P. S. Ushasree, “Network intrusion detection using machine learning techniques,” in Proc. 2020 Int. Conf. Emerg. Trends Inf. Technol. Eng. (ic-ETITE), Tamil Nadu, India, Feb. 24–25, 2020, pp. 1–7.

[10] C. Yin, Y. Zhu, J. Fei, and X. He, “A deep learning approach for intrusion detection using recurrent neural networks,” IEEE Access, vol. 5, pp. 21954–21961, 2017.

[11] S. Soheily Khah, P.-F. Marteau, and N. Bechet, “Intrusion detection in network systems through hybrid supervised and unsupervised machine learning process: A case study on the ISCX dataset,” in Proc. 2018 1st Int. Conf. Data Intell. Security (ICDIS), South Padre Island, TX, USA, Apr. 8–10, 2018, pp. 219–226.

[12] F. Folino, G. Folino, M. Guarascio, F. Pisani, and L. Pontieri, “On learning effective ensembles of deep neural networks for intrusion detection,” Inf. Fusion, vol. 72, pp. 48–69, 2021.

[13] B. A. Tama and S. Lim, “Ensemble learning for intrusion detection systems: A systematic mapping study and cross-benchmark evaluation,” Comput. Sci. Rev., vol. 39, p. 100357, 2021.

[14] K. Kim, M. E. Aminanto, and H. C. Tanuwidjaja, Network Intrusion Detection Using Deep Learning: A Feature Learning Approach. Berlin, Germany: Springer, 2018.

[15] O. Avci, O. Abdeljaber, S. Kiranyaz, M. Hussein, M. Gabbouj, and D. J. Inman, “A review of vibration-based damage detection in civil structures: From traditional methods to machine learning and deep learning applications,” Mech. Syst. Signal Process., vol. 147, p. 107077, 2021.

[16] K. P. M. Kumar, M. Saravanan, M. Thenmozhi, and K. Vijayakumar, “Intrusion detection system based on GA-fuzzy classifier for detecting malicious attacks,” Concurr. Comput. Pract. Exp., vol. 33, p. 5242, 2021.

[17] M. H. Khan, “HCRNNIDS: Hybrid convolutional recurrent neural network-based network intrusion detection system,” Processes, vol. 9, p. 834, 2021.

[18] H. Zhang, L. Huang, C. Q. Wu, and Z. Li, “An effective convolutional neural network based on SMOTE and Gaussian mixture model for intrusion detection in imbalanced dataset,” Comput. Netw., vol. 177, p. 107315, 2020.

[19] M. K. Siddiqui and S. Naahid, “Analysis of KDD CUP 99 dataset using clustering based data mining,” Int. J. Database Theory Appl., vol. 6, pp. 23–34, 2013.

[20] A. Binbusayyis and T. Vaiyapuri, “Identifying and benchmarking key features for cyber intrusion detection: An ensemble approach,” IEEE Access, vol. 7, pp. 106495–106513, 2019.

[21] T. T. Bhavani, M. K. Rao, and A. M. Reddy, “Network intrusion detection system using random forest and decision tree machine learning techniques,” in Proc. 13th Int. Conf. Distributed Comput. Artif. Intell., Sevilla, Spain, Jun. 1–3, 2016, pp. 637–643, Springer, Berlin/Heidelberg, Germany, 2019.

[22] G. Karatas, O. Demir, and O. K. Sahingoz, “Increasing the performance of machine learning-based IDSs on an imbalanced and up-to-date dataset,” IEEE Access, vol. 8, pp. 32150–32162, 2020.

[23] H. Xu, K. Przystupa, C. Fang, A. Marciniak, O. Kochan, and M. Beshley, “A combination strategy of feature selection based on an integrated optimization algorithm and weighted k-nearest neighbor to improve the performance of network intrusion detection,” Electronics, vol. 9, p. 1206, 2020.

[24] B. S. Bhati and C. S. Rai, “Analysis of support vector machine-based intrusion detection techniques,” Arab. J. Sci. Eng., vol. 45, pp. 2371–2383, 2019.

[25] I. S. Thaseen, J. S. Banu, K. Lavanya, M. R. Ghalib, and K. Abhishek, “An integrated intrusion detection system using correlation-based attribute selection and artificial neural network,” Trans. Emerg. Telecommun. Technol., vol. 32, p. 4014, 2021.

[26] S. Waskle, L. Parashar, and U. Singh, “Intrusion detection system using PCA with random forest approach,” in Proc. 2020 Int. Conf. Electronics Sustainable Commun. Syst. (ICESC), Coimbatore, India, Jul. 2–4, 2020, pp. 803–808.

[27] R. R. N. AlOgaili, O. A. Raheem, M. H. G. Abdkhaleq, Z. A. A. Alyasseri, S. A. A. A. Alsaidi, A. H. Alsaeedi, Y. R. Muhsen, and S. Manickam, "AntDroidNet Cybersecurity Model: A Hybrid Integration of Ant Colony Optimization and Deep Neural Networks for Android Malware Detection," Mesopotamian Journal of Cybersecurity, vol. 5, no. 1, pp. 104–120, 2025. DOI: https://doi.org/10.58496/MJCS/2025/008.

[28] M. M. Mijwil, M. Aljanabi, M. Abotaleb, B. S. Shukur, A. S. A. A. Sailawi, I. Bala, K. K. Hiran, R. Doshi, and K. Dhoska, "Exploring the Impact of Blockchain Revolution on the Healthcare Ecosystem: A Critical Review," Mesopotamian Journal of Cybersecurity, vol. 5, no. 1, pp. 78–89, 2025. DOI: https://doi.org/10.58496/MJCS/2025/006.

[29] M. A. Habeeb and Y. L. Khaleel, "Enhanced Android Malware Detection through Artificial Neural Networks Technique," Mesopotamian Journal of Cybersecurity, vol. 5, no. 1, pp. 62–77, 2025. DOI: https://doi.org/10.58496/MJCS/2025/005.

[30] S. D. Shamsi, A. S. H. M. Ali, and W. D. Shamsi, "Hybrid Cooperative Spectrum Structured (HCSS) Approach for Adaptive Routing in Cognitive Radio Ad Hoc Networks," Mesopotamian Journal of Cybersecurity, vol. 5, no. 1, pp. 23–38, 2025. DOI: https://doi.org/10.58496/MJCS/2025/003.

[31] J. Sharma, S. Sonia, K. Kumar, P. Jain, R. H. C. Alfilh, and H. Alkattan, "Enhancing Intrusion Detection Systems with Adaptive Neuro-Fuzzy Inference Systems," Mesopotamian Journal of Cybersecurity, vol. 5, no. 1, pp. 1–10, 2025. DOI: https://doi.org/10.58496/MJCS/2025/001.J.

[32] A. S. . Abdulbaqi, A. M. . Salman, and S. B. . Tambe, “Privacy-Preserving Data Mining Techniques in Big Data: Balancing Security and Usability”, SHIFRA, vol. 2023, pp. 1–9, Jan. 2023, doi: 10.70470/SHIFRA/2023/001

[33] A. Khalilian, A. Nourazar, M. Vahidi-Asl, and H. Haghighi, "G3MD: Mining frequent opcode subgraphs for metamorphic malware detection of existing families," Expert Systems with Applications, vol. 112, pp. 15–33, 2018.

[34] R. Lu, "Malware Detection with LSTM using Opcode Language," *arXiv preprint*, 2019.

[35] S. Choudhary and M. D. Vidyarthi, "A Simple Method for Detection of Metamorphic Malware using Dynamic Analysis and Text Mining," Procedia Computer Science, vol. 54, pp. 265–270, 2015.

[36] H. S. Galal, Y. B. Mahdy, and M. A. Atiea, "Behavior-based features model for malware detection," Journal of Computer Virology and Hacking Techniques, vol. 12, pp. 59–67, 2016.

[37] D. Thakur, S. Sah, P. Ailsinghani and V. Ranga, "A Comparative Study of Machine Learning Models for Network Traffic Classification in the IoT Landscape," 2024 3rd International Conference on Applied Artificial Intelligence and Computing (ICAAIC), Salem, India, 2024, pp. 670-676, doi: 10.1109/ICAAIC60222.2024.10575268.

[38] K. Alakkari, A. A. Subhi, H. Alkattan, A. Kadi, A. Malinin, I. Potoroko, M. Abotaleb, and E.-S. M. El-kenawy, "A Comprehensive Approach to Cyberattack Detection in Edge Computing Environments," Journal of Cybersecurity and Information Management (JCIM), vol. 13, no. 01, pp. 69–75, 2024. DOI: https://doi.org/10.54216/JCIM.130107

[39] K. Omari, "Phishing Detection using Gradient Boosting Classifier," Procedia Computer Science, vol. 230, pp. 120–127, 2023. DOI: 10.1016/j.procs.2023.12.067

[40] U. Yokkampon, S. Chumkamon, A. Mowshowitz, R. Fujisawa, and E. Hayashi, "Anomaly Detection Using Support Vector Machines for Time Series Data," Journal of Robotics, Networking and Artificial Life, vol. 8, no. 1, pp. 41–46, June 2021. DOI: 10.2991/jrnal.k.210521.010.

[41] T. Tsigkritis, G. Groumas, and M. Schneider, "On the Use of k-NN in Anomaly Detection," Journal of Information Security, vol. 9, pp. 70–84, 2018. DOI: 10.4236/jis.2018.91006

[42] H. Zare Moayedi and M. A. Masnadi-Shirazi, "Arima model for network traffic prediction and anomaly detection," 2008 International Symposium on Information Technology, Kuala Lumpur, Malaysia, 2008, pp. 1-6, doi: 10.1109/ITSIM.2008.4631947

[43] Z. T. Nayyef, M. M. Abdulrahman, and N. A. Kurdi, “Optimizing Energy Efficiency in Smart Grids Using Machine Learning Algorithms: A Case Study in Electrical Engineering”, SHIFRA, vol. 2024, pp. 46–54, Apr. 2024, doi: 10.70470/SHIFRA/2024/006

[44] G. Ghanshyam and P. Pandey, "A KNN-Based Intrusion Detection System for Enhanced Anomaly Detection in Industrial IoT Networks," International Journal of Innovative Research in Technology and Science, vol. 12, no. 6, pp. 1-7, Nov. 2024.

[45] M. Landauer, F. Skopik, B. Stojanović, et al., "A review of time-series analysis for cyber security analytics: from intrusion detection to attack prediction," International Journal of Information Security, vol. 24, no. 3, 2025. DOI: 10.1007/s10207-024-00921-0.

[46] I. Tareq, B. M. Elbagoury, S. A. El-Regaily, and E.-S. M. El-Horbaty, “Deep Reinforcement Learning Approach for Cyberattack Detection”, Int. J. Onl. Eng., vol. 20, no. 05, pp. pp. 15–30, Mar. 2024.

[47] Y. Xin et al., "Machine Learning and Deep Learning Methods for Cybersecurity," in IEEE Access, vol. 6, pp. 35365-35381, 2018, doi: 10.1109/ACCESS.2018.2836950.

[48] V. Tripathi, A. Dubey, K. Sathvik, and N. Subhashini, "A Comparative Study of Machine Learning Algorithms for Anomaly Based Network Intrusion Detection System," in Topical Drifts in Intelligent Computing. ICCTA 2021. Lecture Notes in Networks and Systems, vol. 426, J. K. Mandal, P. A. Hsiung, and R. Sankar Dhar, Eds. Singapore: Springer, 2022. DOI: 10.1007/978-981-19-0745-6_2

[49] K. Shaukat, S. Luo, V. Varadharajan, I. A. Hameed, S. Chen, D. Liu, and J. Li, "Performance Comparison and Current Challenges of Using Machine Learning Techniques in Cybersecurity," Energies, vol. 13, no. 10, p. 2509, 2020. DOI: 10.3390/en13102509

[50] H. N. Chethan, "Network Intrusion Dataset," Kaggle, 2023. [Online]. Available: https://www.kaggle.com/datasets/chethuhn/network-intrusion-dataset. [Accessed: Augast. 23, 2024].

[51] G. B. Mensah, “The Era of AI: The Impact of Artificial Intelligence (AI) and Machine Learning (ML) on Financial Stability in the Banking Sector”, EDRAAK, vol. 2024, pp. 43–48, Jun. 2024, doi: 10.70470/EDRAAK/2024/007

[52] G.Belding, “Top 7 malware sample databases and datasets for research and training,” infosecinstitute, 2021. [Online]. Available: https://www.infosecinstitute.com/resources/malware-analysis/top-7-malware-sample-databases-and-datasets-for-research-and-training/. [Accessed: Julay. 3, 2024].

[53] D. Gupta and R. Rani, "Improving malware detection using big data and ensemble learning," Computers & Electrical Engineering, vol. 86, p. 106729, 2020. DOI: 10.1016/j.compeleceng.2020.106729.

[54] Muchamad Bachram Shidiq, W. Gata, S. Kurniawan, D. D. Saputra, and S. Panggabean, “Time Effort Prediction Of Agile Software Development Using Machine Learning Techniques”, j. inspir., vol. 13, no. 2, pp. 39–48, Dec. 2023.

[55] Y. A. Ahmed, B. Koçer, S. Huda, B. A. S. Al-Rimy, and M. M. Hassan, "A system call refinement-based enhanced Minimum Redundancy Maximum Relevance method for ransomware early detection," Journal of Network and Computer Applications, vol. 167, p. 102753, 2020.

[56] J. Singh and J. Singh, "Detection of malicious software by analysing the behavioral artifacts using machine learning algorithms," Information and Software Technology, vol. 121, p. 106273, 2020.

[57] M. Norouzi, A. Souri, and M. S. Zamini, "A Data Mining Classification Approach for Behavioral Malware Detection," Journal of Computer Networks and Communications, vol. 2016, pp. 1–9, 2016.

[58] A. Arabo, R. Dijoux, T. Poulain, and G. Chevalier, "Detecting Ransomware Using Process Behavior Analysis," Procedia Computer Science, vol. 168, pp. 289–296, 2020.

[59] M. Belaoued, A. Boukellal, M. A. Koalal, A. Derhab, S. Mazouzi, and F. A. Khan, "Combined dynamic multifeature and rule-based behavior for accurate malware detection," International Journal of Distributed Sensor Networks, vol. 15, p. 155014771988990, 2019.

[60] J. B. Fraley and M. Figueroa, "Polymorphic malware detection using topological feature extraction with data mining," in Proceedings of the SoutheastCon 2016, Norfolk, VA, USA, 2016, pp. 1–7..

[61] I. Bala, I. A. Pindoo, M. M. Mijwil, M. Abotaleb, and W. Yundong, “Ensuring Security and Privacy in Healthcare Systems: A Review Exploring Challenges, Solutions, Future Trends, and the Practical Applications of Artificial Intelligence,” Jordan Medical Journal, vol.58, no.2, pp.250-270, July 2024.

[62] A. K. Abed , Tran., “Utilizing Artificial Intelligence in Cybersecurity: A Study of Neural Networks and Support Vector Machines”, BJN, vol. 2025, pp. 14–24, Feb. 2025, doi: 10.58496/BJN/2025/002.

[63] L. Hussain, “Fortifying AI Against Cyber Threats Advancing Resilient Systems to Combat Adversarial Attacks”, EDRAAK, vol. 2024, pp. 26–31, Mar. 2024, doi: 10.70470/EDRAAK/2024/004.

Similar Articles

You may also start an advanced similarity search for this article.