Enhanced Key Generation Method using Deep Q-Networks Algorithm with Chaotic Maps
Article Sidebar
Main Article Content
Abstract
In contemporary digital environments, exponential cyber threat growth has made cryptographic key generation a critical security challenge. Traditional Pseudo-Random Number Generators (PRNGs) and existing chaos-based methods often exhibit insufficient entropy, limited randomness quality, and inadequate resistance to statistical attacks. Current implementations frequently produce suboptimal entropy values and fail to meet modern cryptographic security standards and rigorous randomness testing protocols. This paper aims to design and implement an advanced cryptographic key generation system that combines Deep Q-Networks (DQN) algorithms with multiple chaotic maps to produce cryptographically secure stream key bits with high randomness and strong resistance to cryptanalytic attacks. The proposed DRLKG-Chaotic (Deep Reinforcement Learning Key Generation with Chaotic maps) system implements six distinct experimental scenarios utilizing five chaotic maps: Tent, Ikeda, Chua's, Rössler, and Double Pendulum. The first five scenarios individually integrate each chaotic map with a DQN algorithm, whereas the sixth scenario implements a novel fusion approach that incorporates all five maps simultaneously. Each scenario generates key streams of three different lengths (128-bit, 192-bit, and 256-bit) to accommodate varying security requirements. A comprehensive evaluation using the National Institute of Standards and Technology (NIST) statistical test suite, brute-force attack resistance analysis, Auto Correlation (AC), Cross Correlation (CC), and Discrete Fourier Transform (DFT) analysis demonstrates the significant improvements over standard chaotic implementations. The results indicate that the DQN scenarios achieve entropy values ranging from 0.9097--0.9999, whereas the standard chaotic maps achieve values ranging from only 0.3627--0.5463. All NIST test P values consistently exceed 0.90 across all the parameters, indicating superior randomness characteristics. In addition, reliable results are obtained when various types of attacks, such as brute-force attacks, side‑channel attacks, and timing attacks, are applied.
Article Details
Issue
Section
This work is licensed under a Creative Commons Attribution 4.0 International License.
How to Cite
References
[1] M. S. Rathore et al., “A novel trust-based security and privacy model for Internet of Vehicles using encryption and steganography,” Computers and Electrical Engineering, vol. 102, Sep. 2022, doi: 10.1016/j.compeleceng.2022.108205.
[2] A. A. Salih, Z. A. Abdulrazaq, and H. G. Ayoub, “Design and Enhancing Security Performance of Image Cryptography System Based on Fixed Point Chaotic Maps Stream Ciphers in FPGA,” Baghdad Science Journal, vol. 21, no. 5 SI, pp. 1754–1764, 2024, doi: 10.21123/bsj.2024.10521.
[3] R. Naik and U. Singh, “Secured 6-Digit OTP Generation using B-Exponential Chaotic Map,” 2021. [Online]. Available: www.ijacsa.thesai.org
[4] R. B. Prajapati and S. D. Panchal, “Enhanced Approach To Generate One Time Password (OTP) Using Quantum True Random Number Generator (QTRNG),” International Journal of Computing and Digital Systems, vol. 15, no. 1, pp. 279–292, 2024, doi: 10.12785/ijcds/150122.
[5] U. Zia, M. McCartney, B. Scotney, J. Martinez, and A. Sajjad, “A novel pseudo-random number generator for IoT based on a coupled map lattice system using the generalised symmetric map,” SN Appl Sci, vol. 4, no. 2, Feb. 2022, doi: 10.1007/s42452-021-04919-4.
[6] L. Baldanzi et al., “Cryptographically secure pseudo-random number generator IP-core based on SHA2 algorithm,” Sensors (Switzerland), vol. 20, no. 7, Apr. 2020, doi: 10.3390/s20071869.
[7] M. Farajallah, M. Abutaha, M. Abu Joodeh, O. Salhab, and N. Jweihan, “PSEUDO RANDOM NUMBER GENERATOR BASED ON LOOK-UP TABLE AND CHAOTIC MAPS,” J Theor Appl Inf Technol, vol. 31, p. 20, 2020, [Online]. Available: www.jatit.org
[8] M. D. Al-Hassani, “A Novel Technique for Secure Data Cryptosystem Based on Chaotic Key Image Generation,” Baghdad Science Journal, vol. 19, no. 4, pp. 905–913, 2022, doi: 10.21123/bsj.2022.19.4.0905.
[9] S. A. S. Hussien, B. N. Al Din Abed, and K. A. Ibrahim, “Encrypting Text Messages via Iris Recognition and Gaze Tracking Technology,” Mesopotamian Journal of CyberSecurity, vol. 5, no. 1, pp. 90–103, Jan. 2025, doi: 10.58496/MJCS/2025/007.
[10] M. M. Hoobi, “Multilevel Cryptography Model using RC5, Twofish, and Modified Serpent Algorithms,” Iraqi Journal of Science, vol. 65, no. 6, pp. 3434–3450, 2024, doi: 10.24996/ijs.2024.65.6.37.
[11] N. H. M. Ali, M. M. Hoobi, and D. F. Saffo, “Development of Robust and Efficient Symmetric Random Keys Model based on the Latin Square Matrix,” Mesopotamian Journal of CyberSecurity, vol. 4, no. 3, pp. 203–215, 2024, doi: 10.58496/MJCS/2024/023.
[12] I. A. Abdulmunem and M. M. Hoobi, “Enhanced DES Algorithm Using Efficient Classical Algorithm,” Iraqi Journal of Science, vol. 65, no. 12, pp. 7251–7275, 2024, doi: 10.24996/ijs.2024.65.12.37.
[13] N. E. El-Meligy, T. O. Diab, A. S. Mohra, A. Y. Hassan, and W. I. El-Sobky, “A Novel Dynamic Mathematical Model Applied in Hash Function Based on DNA Algorithm and Chaotic Maps,” Mathematics, vol. 10, no. 8, Apr. 2022, doi: 10.3390/math10081333.
[14] A. Zellagui, N. Hadj-Said, and A. Ali-Pacha, “A new hash function inspired by sponge construction using chaotic maps,” Journal of Discrete Mathematical Sciences and Cryptography, vol. 26, no. 2, pp. 529–559, 2023, doi: 10.1080/09720529.2021.1961900.
[15] A. T. Maolood, E. K. Gbashi, and E. S. Mahmood, “Novel lightweight video encryption method based on ChaCha20 stream cipher and hybrid chaotic map,” International Journal of Electrical and Computer Engineering, vol. 12, no. 5, pp. 4988–5000, Oct. 2022, doi: 10.11591/ijece.v12i5.pp4988-5000.
[16] M. Alawida, J. Sen Teh, A. Mehmood, A. Shoufan, and W. H. Alshoura, “A chaos-based block cipher based on an enhanced logistic map and simultaneous confusion-diffusion operations,” Journal of King Saud University - Computer and Information Sciences, vol. 34, no. 10, pp. 8136–8151, 2022, doi: 10.1016/j.jksuci.2022.07.025.
[17] J. Liu, Y. Wang, Q. Han, and J. Gao, “A Sensitive Image Encryption Algorithm Based on a Higher-Dimensional Chaotic Map and Steganography,” International Journal of Bifurcation and Chaos, vol. 32, no. 01, p. 2250004, 2022, doi: 10.1142/S0218127422500043.
[18] M. Bhandari, S. Panday, C. P. Bhatta, and S. P. Panday, “Image Steganography Approach Based Ant Colony Optimization with Triangular Chaotic Map,” in Proceedings of 2nd International Conference on Innovative Practices in Technology and Management, ICIPTM 2022, Institute of Electrical and Electronics Engineers Inc., 2022, pp. 429–434. doi: 10.1109/ICIPTM54933.2022.9753917.
[19] K. Wang, T. Gao, D. You, X. Wu, and H. Kan, “A secure dual-color image watermarking scheme based 2D DWT, SVD and Chaotic map,” Multimed Tools Appl, vol. 81, no. 5, pp. 6159–6190, 2022, doi: 10.1007/s11042-021-11725-y.
[20] M. Irfan and M. A. Khan, “Cryptographically Secure Pseudo-Random Number Generation (CS-PRNG) Design using Robust Chaotic Tent Map (RCTM),” Aug. 2024, [Online]. Available: http://arxiv.org/abs/2408.05580
[21] R. B. Naik and U. Singh, “A Review on Applications of Chaotic Maps in Pseudo-Random Number Generators and Encryption,” Annals of Data Science, vol. 11, no. 1, pp. 25–50, 2022, doi: 10.1007/s40745-021-00364-7.
[22] B. V Nair, V. V S, S. S. Muni, and A. Durdu, “Deep Learning and Chaos: A combined Approach To Image Encryption and Decryption,” Jun. 2024, [Online]. Available: http://arxiv.org/abs/2406.16792
[23] E. Kopets, V. Rybin, O. Vasilchenko, D. Butusov, P. Fedoseev, and A. Karimov, “Fractal Tent Map with Application to Surrogate Testing,” Fractal and Fractional, vol. 8, no. 6, Jun. 2024, doi: 10.3390/fractalfract8060344.
[24] D. F. M. Oliveira, “Mapping Chaos: Bifurcation Patterns and Shrimp Structures in the Ikeda Map,” Aug. 2024, [Online]. Available: http://arxiv.org/abs/2408.11254
[25] W. Zhao, Z. Chang, C. Ma, and Z. Shen, “A Pseudorandom Number Generator Based on the Chaotic Map and Quantum Random Walks,” Entropy, vol. 25, no. 1, Jan. 2023, doi: 10.3390/e25010166.
[26] S. Subathra and V. Thanikaiselvan, “Enhanced security for medical images using a new 5D hyper chaotic map and deep learning based segmentation,” Sci Rep, vol. 15, no. 1, Dec. 2025, doi: 10.1038/s41598-025-04906-4.
[27] C. S. Devi and R. Amirtharajan, “A novel 2D MTMHM based key generation for enhanced security in medical image communication,” Sci Rep, vol. 15, no. 1, Dec. 2025, doi: 10.1038/s41598-025-10485-1.
[28] S. B. N. Premakumari, G. Sundaram, M. Rivera, P. Wheeler, and R. E. P. Guzmán, “Reinforcement Q-Learning-Based Adaptive Encryption Model for Cyberthreat Mitigation in Wireless Sensor Networks,” Sensors, vol. 25, no. 7, Apr. 2025, doi: 10.3390/s25072056.
[29] J. Ding, K. Chen, Y. Wang, N. Zhao, W. Zhang, and N. Yu, “Discop: Provably Secure Steganography in Practice Based on ‘Distribution Copies,’” 2023, doi: 10.1109/SP46215.2023.00155.
[30] A. Daoui, M. Yamni, S. A. Chelloug, M. A. Wani, and A. A. A. El-Latif, “Efficient Image Encryption Scheme Using Novel 1D Multiparametric Dynamical Tent Map and Parallel Computing,” Mathematics, vol. 11, no. 7, Apr. 2023, doi: 10.3390/math11071589.
[31] N. F. Hassan, A. Al-Adhami, and M. S. Mahdi, “Digital Speech Files Encryption based on Hénon and Gingerbread Chaotic Maps,” Baghdad Journal of Science, vol. 63, no. 2, pp. 830–842, 2022, doi: 10.24996/ijs.2022.63.2.36.
[32] A. Al-Daraiseh, Y. Sanjalawe, S. Al-E’mari, S. Fraihat, M. Bany Taha, and M. Al-Muhammed, “Cryptographic Grade Chaotic Random Number Generator Based on Tent-Map,” Journal of Sensor and Actuator Networks, vol. 12, no. 5, Oct. 2023, doi: 10.3390/jsan12050073.
[33] N. Kuznetsov, T. Mokaev, V. Ponomarenko, E. Seleznev, N. Stankevich, and L. Chua, “Hidden attractors in Chua circuit: mathematical theory meets physical experiments,” Nonlinear Dyn, vol. 111, no. 6, pp. 5859–5887, Mar. 2023, doi: 10.1007/s11071-022-08078-y.
[34] R. Rocha and R. O. Medrano-T, “Chua Circuit based on the Exponential Characteristics of Semiconductor Devices,” Dec. 2021, doi: 10.1016/j.chaos.2021.111761.
[35] B. Arpacı, E. Kurt, and K. Çelik, “A new algorithm for the colored image encryption via the modified Chua’s circuit,” Engineering Science and Technology, an International Journal, vol. 23, no. 3, pp. 595–604, Jun. 2020, doi: 10.1016/j.jestch.2019.09.001.
[36] Z. Galias, “Continuation-based method to find periodic windows in bifurcation diagrams with applications to the Chua’s circuit with a cubic nonlinearity,” IEEE Transactions on Circuits and Systems I: Regular Papers, vol. 68, no. 9, pp. 3784–3793, Sep. 2021, doi: 10.1109/TCSI.2021.3089420.
[37] B. Emin and Z. Musayev, “Chaos-based Image Encryption in Embedded Systems using Lorenz-Rossler System,” Chaos Theory and Applications, vol. 5, no. 3, pp. 153–159, 2023, doi: 10.51537/chaos.1246581.
[38] B. Kharabian and H. Mirinejad, “Synchronization of Rossler chaotic systems via hybrid adaptive backstepping/sliding mode control,” Results in Control and Optimization, vol. 4, no. May, p. 100020, 2021, doi: 10.1016/j.rico.2021.100020.
[39] J. P. Parker, D. Goluskin, and G. M. Vasil, “A study of the double pendulum using polynomial optimization,” Jun. 2021, doi: 10.1063/5.0061316.
[40] S. Cabrera, E. D. Leonel, and A. C. Marti, “Regular and chaotic phase space fraction in the double pendulum,” Dec. 2023, [Online]. Available: http://arxiv.org/abs/2312.13436
[41] S. R. de Oliveira, “Deterministic chaos: A pedagogical review of the double pendulum case,” Revista Brasileira de Ensino de Fisica, vol. 46, 2024, doi: 10.1590/1806-9126-RBEF-2024-0060.
[42] J. J. López and V. J. García-Garrido, “Chaos and Regularity in the Double Pendulum with Lagrangian Descriptors,” Mar. 2024, [Online]. Available: http://arxiv.org/abs/2403.07000
[43] R. S. Abdulaali and R. K. Jamal, “A Comprehensive Study and Analysis of the Chaotic Chua Circuit,” Iraqi Journal of Science, vol. 63, no. 2, pp. 556–570, 2022, doi: 10.24996/ijs.2022.63.2.13.
[44] N. Sanghi, “Deep Q-Learning,” in Deep Reinforcement Learning with Python: With PyTorch, TensorFlow and OpenAI Gym, Berkeley, CA: Apress, 2021, pp. 155–206. doi: 10.1007/978-1-4842-6809-4_6.
[45] L. Graesser and W. Loon Keng, “Foundations of Deep Reinforcement Learning _ Theory and Practice in Python,” Nov. 2021.
[46] A. Plaat, Deep Reinforcement Learning. Springer Nature, 2022. doi: 10.1007/978-981-19-0638-1.
[47] N. Ketkar and J. Moolayil, Deep Learning with Python. 2021. doi: 10.1007/978-1-4842-5364-9.
[48] T. Xu, Y. Liu, Z. Ma, Y. Huang, and P. Liu, “A DQN-Based Multi-Objective Participant Selection for Efficient Federated Learning,” Future Internet, vol. 15, no. 6, Jun. 2023, doi: 10.3390/fi15060209.
[49] F. Li, J. Yang, K. Y. Lam, B. Shen, and G. Wei, “Dynamic spectrum access for Internet-of-Things with joint GNN and DQN,” Ad Hoc Networks, vol. 163, Oct. 2024, doi: 10.1016/j.adhoc.2024.103596.
[50] L. E. Bassham et al., “A statistical test suite for random and pseudorandom number generators for cryptographic applications,” Gaithersburg, MD, 2022. doi: 10.6028/NIST.SP.800-22r1a.
[51] Y. Zhang, L. Zhang, Z. Zhong, L. Yu, M. Shan, and Y. Zhao, “Hyperchaotic image encryption using phase-truncated fractional Fourier transform and DNA-level operation,” Opt Lasers Eng, vol. 143, p. 106626, 2021, doi: https://doi.org/10.1016/j.optlaseng.2021.106626.
[52] Y. A. Liu et al., “A dynamic AES cryptosystem based on memristive neural network,” Sci Rep, vol. 12, no. 1, Dec. 2022, doi: 10.1038/s41598-022-13286-y.
[53] E. Barker, “Recommendation for key management:,” Gaithersburg, MD, May 2020. doi: 10.6028/NIST.SP.800-57pt1r5.
[54] Entropy Method for Assessing the Strength of Encryption Algorithms. IEEE, 2024.
[55] I. Buhan, L. Batina, Y. Yarom, and P. Schaumont, “SoK: Design Tools for Side-Channel-Aware Implementations,” Jun. 2021, [Online]. Available: http://arxiv.org/abs/2104.08593
[56] X. Lou, T. Zhang, J. Jiang, and Y. Zhang, “A Survey of Microarchitectural Side-channel Vulnerabilities, Attacks and Defenses in Cryptography,” Mar. 2021, [Online]. Available: http://arxiv.org/abs/2103.14244
[57] D. Ojha and S. Dwarkadas, “Timing Cache Accesses to Eliminate Side Channels in Shared Software,” Dec. 2021, doi: 10.1109/ISCA52012.2021.00037.
[58] F. Mahmud, S. Kim, H. S. Chawla, C.-C. Tsai, E. J. Kim, and A. Muzahid, “Attack of the Knights: A Non Uniform Cache Side-Channel Attack,” May 2023, doi: 10.1145/3627106.3627199.
[59] R. L. Schröder, S. Gast, and Q. Guo, Divide and Surrender: Exploiting Variable Division Instruction Timing in HQC Key Recovery Attacks. [Online]. Available: https://www.usenix.org/conference/usenixsecurity24/presentation/schr
[60] A. Tsuneda, “Auto-Correlation Functions of Chaotic Binary Sequences Obtained by Alternating Two Binary Functions,” Dynamics, vol. 4, no. 2, pp. 272–286, Jun. 2024, doi: 10.3390/dynamics4020016.
[61] F. Ye, S. Zhang, P. Wang, and C.-Y. Chan, “A Survey of Deep Reinforcement Learning Algorithms for Motion Planning and Control of Autonomous Vehicles,” May 2021, [Online]. Available: http://arxiv.org/abs/2105.14218