Scalable Database Solutions in the Cloud Era: Challenges and Best Practices
Downloads
Cloud computing has revolutionized data management, creating demand for highly scalable and adaptive database systems. Traditional architecture has given way to cloud-native databases that offer elasticity, modularity, and real-time responsiveness. This paper reviews modern approaches to building scalable cloud databases, highlighting critical challenges and emerging solutions. Key advancements include microservices-based architecture and intelligent tuning systems like CDBTune and HUNTER, which use AI to optimize performance under dynamic workloads. Security is addressed through homomorphic encryption, blockchain, and oblivious data structures, ensuring data protection in untrusted environments. The study also examines migration strategies and middleware for cross-platform interoperability between SQL and NoSQL systems. Application use cases across agriculture, IoT, healthcare, and industrial domains illustrate practical impacts. Benchmarking approaches focus on latency, throughput, and cost-efficiency. By analyzing over 40 research sources, this paper provides actionable best practices for designing resilient, secure, and efficient cloud database solutions that meet the demands of today’s data-intensive world.
J. Zhang et al., ‘An end-to-end automatic cloud database tuning system using deep reinforcement learning’, in Proceedings of the ACM SIGMOD International Conference on Management of Data, Association for Computing Machinery, Jun. 2019, pp. 415–432. doi: 10.1145/3299869.3300085.
J. Tan et al., ‘Choosing a cloud DBMS: Architectures and tradeoffs’, in Proceedings of the VLDB Endowment, VLDB Endowment, 2018, pp. 2170–2182. doi: 10.14778/3352063.3352133.
J. Rao et al., ‘Data migration in the cloud database: A review of vendor solutions and challenges’, ~ 96 ~ International Journal of Computing and Artificial Intelligence, vol. 3, no. 2, 2022,
doi: 10.33545/27076571.2022.v.
R. Guo et al., ‘Manu: A Cloud Native Vector Database Management System’, Jun. 2022, [Online]. Available:
http://arxiv.org/abs/2206.13843
W. Cao et al., ‘PolarDB Serverless: A Cloud Native Database for Disaggregated Data Centers’, in Proceedings of the ACM SIGMOD International Conference on Management of Data, Association for Computing Machinery, 2021, pp. 2477–2489. doi: 10.1145/3448016.3457560.
R. Awadallah and R. Awadallah, ‘Using Blockchain in Cloud Computing to Enhance Relational Database Security’, IEEE Access, vol. 9, pp. 137353–137366, 2021,
doi: 10.1109/ACCESS.2021.3117733.
L. Wang, Z. Yang, and X. Song, ‘SHAMC: A Secure and highly available database system in multi-cloud environment’, Future Generation Computer Systems, vol. 105, pp. 873–883, Apr. 2020, doi: 10.1016/j.future.2017.07.011.
Y. Zhang et al., ‘Towards cost-effective and elastic cloud database deployment via memory disaggregation’, in Proceedings of the VLDB Endowment, VLDB Endowment, 2021, pp. 1900–1912. doi: 10.14778/3467861.3467877.
A. Bin Mohiuddin et al., ‘Block chain-Based Solutions for Improved Cloud Data Integrity and Security’.
X. Yang, Y. Zhang, H. Chen, C. Sun, F. Li, and W. Zhou, ‘PolarDB-SCC: A Cloud-Native Database Ensuring Low Latency for Strongly Consistent Reads’, in Proceedings of the VLDB Endowment, VLDB Endowment, 2023, pp. 3754–3767.
doi: 10.14778/3611540.3611562.
D. Campi, N. Mounet, M. Gibertini, G. Pizzi, and N. Marzari, ‘Expansion of the Materials Cloud 2D Database’, ACS Nano, vol. 17, no. 12, pp. 11268–11278, Jun. 2023, doi: 10.1021/acsnano.2c11510.
C. Wang, Z. Arani, and L. Gruenwald, ‘Adaptive Time, Monetary Cost Aware Query Optimization on Cloud Database Systems’, 2018. [Online]. Available: https://hal.science/hal-01962218v1
J. J. Yan, Q. Q. Jin, S. S. Jain, S. D. D. S. Viglas, and A. A. Lee, ‘Snowtrail: Testing with production queries on a cloud database’, in Proceedings of the Workshop on Testing Database Systems, DBTest 2018, Association for Computing Machinery, Inc, Jun. 2018. doi: 10.1145/3209950.3209958.
D. Vinayagamurthy, A. Gribov, and S. Gorbunov, ‘StealthDB: a Scalable Encrypted Database with Full SQL Query Support’, Proceedings on Privacy Enhancing Technologies, vol. 2019, no. 3, pp. 370–388, Jul. 2019, doi: 10.2478/popets-2019-0052.
C. Li and J. Gu, ‘An integration approach of hybrid databases based on SQL in cloud computing environment’, Softw Pract Exp, vol. 49, no. 3, pp. 401–422, Mar. 2019, doi: 10.1002/spe.2666.
X. Shi, W. Lu, Y. Zhao, and P. Qin, ‘Prediction of Indoor Temperature and Relative Humidity Based on Cloud Database by Using an Improved BP Neural Network in Chongqing’, IEEE Access, vol. 6, pp. 30559–30566, 2018,
doi: 10.1109/ACCESS.2018.2844299.
C. Curinoo et al., ‘massachusetts institute of technology Relational Cloud: The Case for a Database Service Relational Cloud: The Case for a Database Service’, 2010. [Online]. Available: www.csail.mit.edu
T. H. Shareef, K. H. Shareef, and B. N. Rashid, ‘A Survey of Comparing Different Cloud Database Performance: SQL and NoSQL’, Passer Journal Passer, vol. 4, pp. 45–57, 2022,
doi: 10.24271/psr.40.
S. Henning and W. Hasselbring, ‘A configurable method for benchmarking scalability of cloud-native applications’, Empir Softw Eng, vol. 27, no. 6, Nov. 2022, doi: 10.1007/s10664-022-10162-1.
O. V Korotun, ‘Model of using cloud-based environment in training databases of future IT specialists’, 2020.
C. J. Stubenrauch et al., ‘Lessons Learned from the Updated GEWEX Cloud Assessment Database’, Surv Geophys, Dec. 2024,
doi: 10.1007/s10712-024-09824-0.
A. Al-Said Ahmad and P. Andras, ‘Scalability analysis comparisons of cloud-based software services’, Journal of Cloud Computing, vol. 8, no. 1, Dec. 2019, doi: 10.1186/s13677-019-0134-y.
E. Tzavidas, P. Enevoldsen, and G. Xydis, ‘A University-industry knowledge transfer online education approach via a cloud-based database global solution’, Smart Learning Environments, vol. 7, no. 1, Dec. 2020, doi: 10.1186/s40561-020-00128-5.
W. Xie, S. Dong, Y. Sang, X. Zhao, X. Zhang, and K. Liu, ‘RACBase: A cloud-based database of recycled aggregate concrete durability’, Case Studies in Construction Materials, vol. 18, Jul. 2023, doi: 10.1016/j.cscm.2023.e02004.
O. Claudius Oyeniran, A. Okechukwu Adewusi, A. Gbolahan Adeleke, L. Anthony Akwawa, and C. Francisca Azubuko, ‘Microservices Architecture in Cloud-Native Applications: Design Patterns and Scalability’, International Journal of Advanced Research and Interdisciplinary Scientific Endeavours, vol. 1, no. 2, p. 2024,
doi: 10.61359/11.2206-2409.
M. S. Hosen et al., ‘Data-Driven Decision Making: Advanced Database Systems for Business Intelligence’, Nanotechnol Percept, vol. 20, no. S3, pp. 687–704, 2024,
doi: 10.62441/nano-ntp.v20iS3.51.
T. Hoang, C. D. Ozkaptan, G. Hackebeil, and A. A. Yavuz, ‘Efficient Oblivious Data Structures for Database Services on the Cloud *’.
J. Zhou, C. Chen, C. Wei, and K. Du, ‘An Improved Connection Cloud Model of an Updated Database: A Multicriteria Uncertainty Model for Coal Burst Liability Evaluation’, Natural Resources Research, vol. 31, no. 3, pp. 1687–1704, Jun. 2022,
doi: 10.1007/s11053-022-10042-x.
A. Depoutovitch et al., ‘Taurus Database: How to be Fast, Available, and Frugal in the Cloud’, in Proceedings of the ACM SIGMOD International Conference on Management of Data, Association for Computing Machinery, Jun. 2020, pp. 1463–1478. doi: 10.1145/3318464.3386129.
B. Cai et al., ‘HUNTER: An Online Cloud Database Hybrid Tuning System for Personalized Requirements’, in Proceedings of the ACM SIGMOD International Conference on Management of Data, Association for Computing Machinery, Jun. 2022, pp. 646–659. doi: 10.1145/3514221.3517882.
A. Petrova, ‘Cloud Computing in the Age of Big Data: Storage, Analytics, and Scalability’, 2023.
S. Chinamanagonda, ‘Cloud-native Databases: Performance and Scalability-Adoption of cloud-native databases for improved performance’, 2023.
H. Gadde, ‘REVISTA DE INTELIGENCIA ARTIFICIAL EN MEDICINA AI-Enhanced Adaptive Resource Allocation in Cloud-Native Databases’. [Online]. Available: https://redcrevistas.com/index.php/Revista
H. Zheng, T. Zhang, C. Fang, J. Zeng, and X. Yang, ‘Design and implementation of poultry farming information management system based on cloud database’, Animals, vol. 11, no. 3, pp. 1–15, Mar. 2021, doi: 10.3390/ani11030900.
Navendu Mishra, Jayant Sawarkar, and Suraj Dubey, ‘Database Performance Management in Cloud’, International Journal of Engineering and Management Research, vol. 10, no. 5, pp. 46–51, Oct. 2020, doi: 10.31033/ijemr.10.5.11.
P. Ferrari et al., ‘On the performance of cloud services and databases for industrial IoT scalable applications’, Electronics (Switzerland), vol. 9, no. 9, pp. 1–17, Sep. 2020,
doi: 10.3390/electronics9091435.
M. M. Eyada, W. Saber, M. M. El Genidy, and F. Amer, ‘Performance Evaluation of IoT Data Management Using MongoDB Versus MySQL Databases in Different Cloud Environments’, IEEE Access, vol. 8, pp. 110656–110668, 2020,
doi: 10.1109/ACCESS.2020.3002164.
S. Bharany, K. Kaur, S. Eltayeb Mohamed Eltaher, A. Osman Ibrahim, S. Sharma, and M. Merghany Mohammed Abd Elsalam, ‘A Comparative Study of Cloud Data Portability Frameworks for Analyzing Object to NoSQL Database Mapping from ONDM’s Perspective’. [Online]. Available: www.ijacsa.thesai.org
[39] M. Okada, T. Suzuki, N. Nishio, H. M. Waidyasooriya, and M. Hariyama, ‘FPGA-Accelerated Searchable Encrypted Database Management Systems for Cloud Services’, IEEE Transactions on Cloud Computing, vol. 10, no. 2, pp. 1373–1385, 2022,
doi: 10.1109/TCC.2020.2969655.
N. Santos and G. L. Masala, ‘Big Data Security on Cloud Servers Using Data Fragmentation Technique and NoSQL database’.
W. Liu, L. Xiang, T. Zheng, J. Jin, and G. Zhang, ‘TranslatomeDB: A comprehensive database and cloud-based analysis platform for translatome sequencing data’, Nucleic Acids Res, vol. 46, no. D1, pp. D206–D212, Jan. 2018,
doi: 10.1093/nar/gkx1034.
Kishan Patel, ‘A review on cloud computing-based quality assurance: Challenges, opportunities, and best practices’, International Journal of Science and Research Archive, vol. 13, no. 1, pp. 796–805, Sep. 2024, doi: 10.30574/ijsra.2024.13.1.1693.
P. Kochovski, R. Sakellariou, M. Bajec, P. Drobintsev, and V. Stankovski, ‘An architecture and stochastic method for database container placement in the edge-fog-cloud continuum’, in Proceedings - 2019 IEEE 33rd International Parallel and Distributed Processing Symposium, IPDPS 2019, Institute of Electrical and Electronics Engineers Inc., May 2019, pp. 396–405.
doi: 10.1109/IPDPS.2019.00050.
Z. Feng et al., ‘A Global High-Resolution Mesoscale Convective System Database Using Satellite-Derived Cloud Tops, Surface Precipitation, and Tracking’, Journal of Geophysical Research: Atmospheres, vol. 126, no. 8, Apr. 2021,
doi: 10.1029/2020JD034202.
J. Das and S. Healthineers, ‘Leveraging Cloud Computing for Medical AI: Scalable Infrastructure and Data Security for Advanced Healthcare Solutions’, IJRAR19S1826 International Journal of Research and Analytical Reviews, p. 504, 2020, [Online].
Available: https://www.researchgate.net/publication/389782177
C. Li et al., ‘FluteDB: An efficient and scalable in-memory time series database for sensor-cloud’, J Parallel Distrib Comput, vol. 122, pp. 95–108, Dec. 2018, doi: 10.1016/j.jpdc.2018.07.021.
S. Malhotra, M. N. Doja, B. Alam, and M. Alam, ‘Generalized query processing mechanism in cloud database management system’, in Advances in Intelligent Systems and Computing, Springer Verlag, 2018, pp. 641–648. doi: 10.1007/978-981-10-6620-7_61.
S. Kumar, R. Thumburu, S. E. Analyist, A. Asea, B. Boveri, and S. Corresponding, ‘MZ Journals Performance Analysis of Data Exchange Protocols in Cloud Environments’. [Online]. Available: https://mzjournal.com/index.php/MZCJ
K. Reddy Gade, ‘MZ Journals Data Quality in the Age of Cloud Migration: Challenges and Best Practices’. [Online].
Available: https://mzjournal.com/index.php/MZJAI
S. Wang et al., ‘Natural language semantic construction based on cloud database’, Computers, Materials and Continua, vol. 57, no. 3, pp. 603–619, 2018, doi: 10.32604/cmc.2018.03884.
