A cloud-based architecture for explainable Big Data analytics using self-structuring Artificial Intelligence

被引:2
作者
Mills N. [1 ]
Issadeen Z. [1 ]
Matharaarachchi A. [1 ]
Bandaragoda T. [1 ]
De Silva D. [1 ]
Jennings A. [1 ]
Manic M. [2 ]
机构
[1] Centre for Data Analytics and Cognition, La Trobe University, Melbourne, 3086, VIC
[2] Department of Computer Science, Virginia Commonwealth University, Richmond, 23284, VA
来源
Discover Artificial Intelligence | 2024年 / 4卷 / 01期
关键词
Artificial Intelligence; Big Data analytics; Cloud platforms; Explainable AI; Self structuring Artificial Intelligence;
D O I
10.1007/s44163-024-00123-6
中图分类号
学科分类号
摘要
Big Data is steadily expanding beyond the boundaries of its foundational constructs of three primary Vs, Volume, Velocity and Variety, and two secondary Vs, Veracity and Value. The advent of 5G networks, Edge computing and IoT technologies has transformed Big Data into this modern context. With these new manifestations of Big Data, the focus is not only on the data itself but on the context that it applies to its immediate environment as well as the human and societal perception of this context. It is increasingly challenging for conventional AI algorithms to process and transform this data, analyse and visualise a broad spectrum of insights, and then formulate the explainability of such insights in terms of bias, transparency, safety, ethics, and causality. Self-structuring Artificial Intelligence (SSAI) addresses the limitations of conventional AI by adapting to the inherent structure of the data, incrementally learning and abstracting from this structure. SSAI has not been investigated in a cloud-based setting for generating explainable insights from these new types of Big Data. In this paper we propose a cloud-based architecture for explainable Big Data analytics using SSAI in highly-connected 5G and Edge computing environments. The proposed architecture is empirically evaluated on a commercial scale Big Data use case of Smart Grid for Smart Cities. The results of these experiments confirm the functionality and effectiveness of the proposed architecture. © The Author(s) 2024.
引用
收藏
相关论文
共 39 条
[1]  
Hoffmann J., Borgeaud S., Mensch A., Buchatskaya E., Cai T., Rutherford E., Casas D., Hendricks L.A., Welbl J., Clark A., Et al., Training compute-optimal large language models, (2022)
[2]  
Biderman S., Schoelkopf H., Anthony Q.G., Bradley H., O'Brien K., Hallahan E., Khan M.A., Purohit S., Prashanth U.S., Raff E., Et al., Pythia: A suite for analyzing large language models across training and scaling, International Conference on Machine Learning, pp. 2397-2430, (2023)
[3]  
De Silva D., Burstein F., Jelinek H.F., Stranieri A., Et al., Addressing the complexities of Big Data analytics in healthcare: the diabetes screening case, Aust J Inf Syst, (2015)
[4]  
Nawaratne R., Bandaragoda T., Adikari A., Et al., Incremental knowledge acquisition and self-learning for autonomous video surveillance, IECON 2017-43Rd Annual Conference of the IEEE Industrial Electronics Society, pp. 4790-4795, (2017)
[5]  
Hashem I.A.T., Yaqoob I., Anuar N.B., Mokhtar S., Gani A., Khan S.U., The rise of “Big Data” on cloud computing: review and open research issues, Inf Syst, 47, pp. 98-115, (2015)
[6]  
Ji C., Li Y., Qiu W., Awada U., Li K., Big Data processing in cloud computing environments, 2012 12Th International Symposium on Pervasive Systems, Algorithms and Networks, pp. 17-23, (2012)
[7]  
Chen M., Mao S., Liu Y., Big Data: a survey, Mobile Netw Appl, 19, 2, pp. 171-209, (2014)
[8]  
Al-Jarrah O.Y., Yoo P.D., Muhaidat S., Karagiannidis G.K., Taha K., Efficient machine learning for Big Data: a review. Big Data, analytics, and high-performance computing, Big Data Res, 2, 3, pp. 87-93, (2015)
[9]  
Katal A., Wazid M., Goudar R.H., Big Data: Issues, challenges, tools and good practices, 2013 Sixth International Conference on Contemporary Computing (IC3), pp. 404-409, (2013)
[10]  
Evans D., The internet of things: how the next evolution of the internet is changing everything, CISCO White Paper, 1, 2011, pp. 1-11, (2011)