Blockchain-Based Metadata Management in Distributed File Systems
Article Sidebar
Main Article Content
Abstract
Managing large-scale data in distributed environments is essential for developing the distributed file system (DFS) concept, which ensures reliable, scalable, and fault-tolerant data storage across multiple nodes. In a DFS, DataNodes divides large datasets into blocks, assigning replicas to enhance data redundancy. The NameNode is a central control unit that manages metadata that governs data storage and retrieval. However, the NameNode presents a potential single point of failure, creating challenges in ensuring metadata, integrity, trustworthiness, and overall system reliability in distributed environments. This study proposes a new method to address these challenges by designing and implementing a new distributed file system architecture using blockchain as a repository to store the metadata of the NameNode. The proposed system achieves several significant improvements by integrating the blockchain into the DFS architecture. The tamper-proof and immutable nature of blockchain ensures metadata integrity. The metadata recorded on the blockchain can be accessible and recoverable at any time because multiple replicas are maintained over the network, and it becomes resistant to unauthorized modifications, enhancing trust and data reliability. The proposed architecture simulated the DFS via Python and integrated it with Ganache as an Ethereum platform via the Web3 library. The results show that the proposed system achieves the best time to upload files in the DFS compared with the traditional Hadoop distributed file system; the metadata stored in the blockchain enhance the overall system performance by improving metadata trustworthiness, management, and data integrity. The performance metrics for the proposed system are memory utilization and the file execution time for files ranging from 1 MB to 100,000 MB. The results show that even as the file size increase and the number of executions increases, the system retains efficient memory utilization, requiring less RAM for larger files. Although the system's ability to handle large datasets is demonstrated by its scalability in memory usage, adjustments are required to counteract the longer processing times linked to larger files. This paper examines the trade-offs and limitations of integrating blockchain with DFS and issues with scalability, latency, and storage costs. Despite these obstacles, the proposed approach shows that blockchain offers a workable option for safe and dependable metadata management in that DFS. This makes room for additional research to increase productivity and reduce resource use.
Article Details
Issue
Section
This work is licensed under a Creative Commons Attribution 4.0 International License.
How to Cite
References
[1] J. Blomer, “A survey on distributed file system technology,” J. Phys. Conf. Ser., vol. 608, no. 1, 2015, doi: 10.1088/1742-6596/608/1/012039.
[2] R. Kumar and R. Tripathi, “Implementation of Distributed File Storage and Access Framework using IPFS and Blockchain,” Proc. IEEE Int. Conf. Image Inf. Process., vol. 2019-Novem, pp. 246–251, 2019, doi: 10.1109/ICIIP47207.2019.8985677.
[3] P. M. Dhulavvagol and S. G. Totad, “Performance Enhancement of Distributed System Using HDFS Federation and Sharding,” Procedia Comput. Sci., vol. 218, pp. 2830–2841, 2022, doi: 10.1016/j.procs.2023.01.254.
[4] X. Pan, Z. Luo, and L. Zhou, “Navigating the Landscape of Distributed File Systems: Architectures, Implementations, and Considerations,” Innov. Appl. Eng. Technol., pp. 1–12, 2023, doi: 10.62836/iaet.v2i1.157.
[5] V. Mothukuri, S. S. Cheerla, R. M. Parizi, Q. Zhang, and K. K. R. Choo, “BlockHDFS: Blockchain-integrated Hadoop distributed file system for secure provenance traceability,” Blockchain Res. Appl., vol. 2, no. 4, p. 100032, 2021, doi: 10.1016/j.bcra.2021.100032.
[6] H. Guo and X. Yu, “A survey on blockchain technology and its security,” Blockchain Res. Appl., vol. 3, no. 2, p. 100067, 2022, doi: 10.1016/j.bcra.2022.100067.
[7] X. Zhang and Y. Wang, “Research on intelligent medical big data system based on Hadoop and blockchain,” Eurasip J. Wirel. Commun. Netw., vol. 2021, no. 1, 2021, doi: 10.1186/s13638-020-01858-3.
[8] A. M. Badr, L. C. Fourati, and S. Ayed, “A Novel System for Confidential Medical Data Storage Using Chaskey Encryption and Blockchain Technology,” Baghdad Sci. J., vol. 20, pp. 2651–2671, 2023, doi: 10.21123/bsj.2023.9203.
[9] M. Al-Zubaidie and W. Jebbar, “Transaction Security and Management of Blockchain-Based Smart Contracts in E-Banking-Employing Microsegmentation and Yellow Saddle Goatfish,” Mesopotamian J. CyberSecurity, vol. 4, no. 2, pp. 71–89, 2024, doi: 10.58496/mjcs/2024/005.
[10] H. Honar Pajooh, M. A. Rashid, F. Alam, and S. Demidenko, “IoT Big Data provenance scheme using blockchain on Hadoop ecosystem,” J. Big Data, vol. 8, no. 1, 2021, doi: 10.1186/s40537-021-00505-y.
[11] H. H. Abdul-Sada and Furkan Rabee, “The Genetic Algorithm Implementation in Smart Contract for the Blockchain Technology,” Al-Salam J. Eng. Technol., vol. 2, no. 2, pp. 37–47, 2023, doi: 10.55145/ajest.2023.02.02.005.
[12] A. K. Tyagi, S. Dananjayan, D. Agarwal, and H. F. T. Ahmed, “Blockchain — Internet of Things Applications : Opportunities,” Multidiscip. Digit. Publ. Inst., 2023.
[13] D. H. Tahayur and M. Al-Zubaidie, “Enhancing Electronic Agriculture Data Security with a Blockchain-Based Search Method and E-Signatures,” Mesopotamian J. CyberSecurity, vol. 4, no. 3, pp. 129–149, 2024, doi: 10.58496/mjcs/2024/012.
[14] “Computational Intelligence and Neuroscience - 2022 - Nair - Blockchain‐Based Decentralized Cloud Solutions for Data.pdf.”
[15] M. K. Gupta and R. K. Dwivedi, “Blockchain Enabled Hadoop Distributed File System Framework for Secure and Reliable Traceability,” Adv. Distrib. Comput. Artif. Intell. J., vol. 12, pp. 1–19, 2023, doi: 10.14201/adcaij.31478.
[16] M. S. Mohammed and A. N. Hashim, “Blockchain technology, methodology behind it, and its most extensively used encryption techniques.,” Al-Salam J. Eng. Technol., vol. 2, no. 2, pp. 140–151, 2023, doi: 10.55145/ajest.2023.02.02.017.
[17] R. F. Ghani, A. A. S. Al-Karkhi, and S. M. Mahdi, “Proposed Framework for Official Document Sharing and Verification in E-government Environment Based on Blockchain Technology,” Baghdad Sci. J., vol. 19, no. 6, pp. 1592–1602, 2022, doi: 10.21123/bsj.2022.7513.
[18] M. Ben Farah et al., “A survey on blockchain technology in the maritime industry: Challenges and future perspectives,” Futur. Gener. Comput. Syst., vol. 157, no. April, pp. 618–637, 2024, doi: 10.1016/j.future.2024.03.046.
[19] A. F. Mahdi and F. Rabee, “A Blockchain Mining Proof of Work Approach Based on Fog Computing Virtualization for Mobile CrowdSensing,” 2024 3rd Int. Conf. Distrib. Comput. High Perform. Comput. DCHPC 2024, 2024, doi: 10.1109/DCHPC60845.2024.10454071.
[20] M. Alessi, A. Camillo, E. Giangreco, M. Matera, S. Pino, and D. Storelli, “Make Users Own Their Data: A Decentralized Personal Data Store Prototype Based on Ethereum and IPFS,” 2018 3rd Int. Conf. Smart Sustain. Technol. Split. 2018, 2018.
[21] S. S. Kushwaha, S. Joshi, and A. K. Gupta, “An efficient approach to secure smart contract of Ethereum blockchain using hybrid security analysis approach,” J. Discret. Math. Sci. Cryptogr., vol. 26, no. 5, pp. 1499–1517, 2023, doi: 10.47974/JDMSC-1815.
[22] M. Bez, G. Fornari, and T. Vardanega, “The scalability challenge of ethereum: An initial quantitative analysis,” Proc. - 13th IEEE Int. Conf. Serv. Syst. Eng. SOSE 2019, 10th Int. Work. Jt. Cloud Comput. JCC 2019 2019 IEEE Int. Work. Cloud Comput. Robot. Syst. CCRS 2019, pp. 167–176, 2019, doi: 10.1109/SOSE.2019.00031.
[23] K. Adel, A. Elhakeem, and M. Marzouk, “Decentralized System for Construction Projects Data Management Using Blockchain and Ipfs,” J. Civ. Eng. Manag., vol. 29, no. 4, pp. 342–359, 2023, doi: 10.3846/jcem.2023.18646.
[24] Z. Wenhua, F. Qamar, T. A. N. Abdali, R. Hassan, S. T. A. Jafri, and Q. N. Nguyen, “Blockchain Technology: Security Issues, Healthcare Applications, Challenges and Future Trends,” Electron., vol. 12, no. 3, 2023, doi: 10.3390/electronics12030546.
[25] M. Reena and S. More, “Integrated HDFS for secure traceability With Blockchain,” 2022.
[26] F. K. Nishi et al., “Electronic Healthcare Data Record Security Using Blockchain and Smart Contract,” J. Sensors, vol. 2022, 2022, doi: 10.1155/2022/7299185.
[27] G. Wan et al., “Decentralized Data Dominion : Unraveling the Power and Promise of Distributed File Systems To cite this version : HAL Id : hal-04670349 Decentralized Data Dominion : Unraveling the Power and Promise of Distributed File Systems,” 2024.
[28] H. Huang, J. Lin, B. Zheng, Z. Zheng, and J. Bian, “When Blockchain Meets Distributed File Systems: An Overview, Challenges, and Open Issues,” IEEE Access, vol. 8, pp. 50574–50586, 2020, doi: 10.1109/ACCESS.2020.2979881.
[29] G. Liao and D. J. Abadi, “FileScale: Fast and Elastic Metadata Management for Distributed File Systems,” SoCC 2023 - Proc. 2023 ACM Symp. Cloud Comput., pp. 459–474, 2023, doi: 10.1145/3620678.3624784.
[30] O. Ali, A. Jaradat, A. Kulakli, and A. Abuhalimeh, “A Comparative Study: Blockchain Technology Utilization Benefits, Challenges and Functionalities,” IEEE Access, vol. 9, pp. 12730–12749, 2021, doi: 10.1109/ACCESS.2021.3050241.
[31] Saihood, Ahmed, Mohammed Adel Al-Shaher, and Mohammed A. Fadhel. "A New Tiger Beetle Algorithm for Cybersecurity, Medical Image Segmentation and Other Global Problems Optimization." Mesopotamian Journal of CyberSecurity 4.1 (2024): 17-46.
[32] A. A. Almuqren, "Cybersecurity threats, countermeasures and mitigation techniques on the IoT: Future research directions," Journal of Cyber Security and Risk Auditing, vol. 1, no. 1, pp. 1–11, 2025.
[33] R. Almanasir, D. Al-solomon, S. Indrawes, M. A. Almaiah, U. Islam, and M. Alshar’e, "Classification of threats and countermeasures of cloud computing," Journal of Cyber Security and Risk Auditing, vol. 2025, no. 2, pp. 27–42, 2025. [Online]. Available: https://doi.org/10.63180/jcsra.thestap.2025.2.3