CHALLENGES OF USING BIG DATA IN DISTRIBUTED EXASCALE SYSTEMS

Details

Hits: 1715

Volume 3 (2), December 2020, Pages 245-254

Firuza Tahmazli-Khaligova

Department of General and Applied Mathematics, Azerbaijan State Oil and Industry University, Baku, Azerbaijan, This email address is being protected from spambots. You need JavaScript enabled to view it.

Abstract

In a traditional High Performance Computing system, it is possible to process a huge data volume.  The nature of events in classic High Performance computing is static. However, distributed exascale system has a different nature. The processing big data in a distributed exascale system evokes a new challenge. The dynamic and interactive character of a distributed exascale system changes process’s status and system elements. This paper discusses the challenge of the big data attributes: volume, velocity, variety; how they influence distributed exascale system dynamic and interactive nature. While investigating the effect of the dynamic and interactive nature of exascale systems in computing big data, this research suggests the Markov chains model. This model constructs the transition matrix, which identifies system status and memory sharing. It lets us analyze convergence of the two systems. As a result both systems are explored by the influence of each other.

Keywords:

High Performance Computing, Distributed Exascale System, Dynamic and Interactive, Big Data.

DOI: https://doi.org/10.32010/26166127.2020.3.2.245.254

Reference 

Cheng, P., Lu, Y., Du, Y., & Chen, Z. (2018, March). Experiences of converging big data analytics frameworks with high performance computing systems. In Asian Conference on Supercomputing Frontiers (pp. 90-106). Springer, Cham.

Fox, G. C., Qiu, J., Kamburugamuve, S., Jha, S., & Luckow, A. (2015, May). Hpc-abds high performance computing enhanced apache big data stack. In 2015 15^th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (pp. 1057-1066). IEEE.

Jackson, A., Weiland, M., Parsons, M., & Homoelle, B. (2019, June). An architecture for high performance computing and data systems using byte-addressable persistent memory. In International Conference on High Performance Computing (pp. 258-274). Springer, Cham.

Khaneghah, E. M., & Sharifi, M. (2014). AMRC: an algebraic model for reconfiguration of high performance cluster computing systems at runtime. The Journal of Supercomputing, 67(1), 1-30.

Kim, Y., Park, S., Wang, F., Sun, G., & Wang, S. (2018). HPC software and programming environments for big data applications. Scientific Programming, 1-2.

Kumar, M.P., Kumar, S.S., Ramya, I. G. (2018) Big data analytics: A brief survey. International Journal of Trend in Scientific Research and Development (ijtsrd), 2(4), 2264-2268.

Mercier, M., Glesser, D., Georgiou, Y., & Richard, O. (2017, December). Big data and HPC collocation: Using HPC idle resources for big data analytics. In 2017 IEEE International Conference on Big Data (Big Data) (pp. 347-352). IEEE.

Pektürk, M. K., & Ünal, M. (2018, August). Performance-aware high-performance computing for remote sensing big data analytics. In Data Mining (p. 69). BoD–Books on Demand.

Reed, D. A., & Dongarra, J. (2015). Exascale computing and big data. Communications of the ACM, 58(7), 56-68.

Tulasi, B., Wagh, R. S., & Balaji, S. (2015). High performance computing and big data analytics - paradigms and challenges. International Journal of Computer Applications, 116(2), 28-33.

Xie, J., Song, Z., Li, Y., et al. (2018). A survey on machine learning-based mobile big data analysis: Challenges and applications. Wireless Communications and Mobile Computing, 1-19.

Xuan, P., Denton, J., Srimani, P. K., Ge, R., & Luo, F. (2015, November). Big data analytics on traditional HPC infrastructure using two-level storage. In Proceedings of the 2015 International Workshop on Data-Intensive Scalable Computing Systems (pp. 1-8).

Xuan, P., Denton, J., Srimani, P. K., Ge, R., & Luo, F. (2015, November). Big data analytics on traditional HPC infrastructure using two-level storage. In Proceedings of the 2015 International Workshop on Data-Intensive Scalable Computing Systems (pp. 1-8).