Optimization of artificial intelligence in localized big data real-time query processing task scheduling algorithm

被引:0
|
作者
Sun, Maojin [1 ]
Sun, Luyi [1 ]
机构
[1] CEICloud Data Storage Technol Beijing Co Ltd, Beijing, Peoples R China
来源
FRONTIERS IN PHYSICS | 2024年 / 12卷
关键词
task scheduling algorithm; artificial intelligence (AI); support vector machines (SVM); big data; optimization model; REVOLUTION;
D O I
10.3389/fphy.2024.1484115
中图分类号
O4 [物理学];
学科分类号
0702 ;
摘要
Introduction The development of science and technology has driven rapid changes in the social environment, especially the rise of the big data environment, which has greatly increased the speed at which people obtain information. However, in the process of big data processing, the allocation of information resources is often unreasonable, leading to a decrease in efficiency. Therefore, optimizing task scheduling algorithms has become an urgent problem to be solved.Methods The study optimized task scheduling algorithms using artificial intelligence (AI) methods. A task scheduling algorithm optimization model was designed using support vector machine (SVM) and K-nearest neighbor (KNN) combined with fuzzy comprehensive evaluation. In this process, the performance differences of different nodes were considered to improve the rationality of resource allocation.Results and Discussion By comparing the task processing time before and after optimization with the total cost, the results showed that the optimized model significantly reduced task processing time and total cost. The maximum reduction in task processing time is 2935 milliseconds. In addition, the analysis of query time before and after optimization shows that the query time of the optimized model has also been reduced. The experimental results demonstrate that the proposed optimization model is practical in handling task scheduling problems and provides an effective solution for resource management in big data environments. This research not only improves the efficiency of task processing, but also provides new ideas for optimizing future scheduling algorithms.
引用
收藏
页数:12
相关论文
共 50 条
  • [41] Research on Real-Time Task Scheduling Algorithm for Hybrid Reconfigurable System
    Ge, Jiao
    Hui, Liu
    2011 7TH INTERNATIONAL CONFERENCE ON WIRELESS COMMUNICATIONS, NETWORKING AND MOBILE COMPUTING (WICOM), 2011,
  • [42] Energy-Aware Real-Time Task Scheduling for Heterogeneous Multiprocessors with Particle Swarm Optimization Algorithm
    Zhang, Weizhe
    Xie, Hucheng
    Cao, Boran
    Cheng, Albert M. K.
    MATHEMATICAL PROBLEMS IN ENGINEERING, 2014, 2014
  • [43] Real-time big data processing for anomaly detection: A Survey
    Habeeb, Riyaz Ahamed Ariyaluran
    Nasaruddin, Fariza
    Gani, Abdullah
    Hashem, Ibrahim Abaker Targio
    Ahmed, Ejaz
    Imran, Muhammad
    INTERNATIONAL JOURNAL OF INFORMATION MANAGEMENT, 2019, 45 : 289 - 307
  • [44] Railway Big Data Real-time Processing Based on Storm
    Guo, Shihang
    Zhang, Lichen
    PROCEEDINGS OF THE 2016 2ND WORKSHOP ON ADVANCED RESEARCH AND TECHNOLOGY IN INDUSTRY APPLICATIONS, 2016, 81 : 536 - 539
  • [45] Real-time prediction and optimization of NOx emissions using artificial intelligence and online combustion data
    Wang, Cong
    Xu, Jun
    Xu, Kai
    Jiang, Long
    Wang, Yi
    Su, Sheng
    Hu, Song
    Xiang, Jun
    FUEL, 2025, 391
  • [46] Design and development of real-time query platform for big data based on hadoop
    刘小利
    Xu Pandeng
    Liu Mingliang
    Zhu Guobin
    High Technology Letters, 2015, 21 (02) : 231 - 238
  • [47] Design and development of real-time query platform for big data based on hadoop
    Liu, Xiaoli
    Xu, Pandeng
    Liu, Mingliang
    Zhu, Guobin
    High Technology Letters, 2015, 21 (02) : 231 - 238
  • [48] Artificial Intelligence for Real Time Cluster Tool Scheduling EO: Equipment Optimization
    Suerich, Doug
    McIlroy, Trevor
    2022 33RD ANNUAL SEMI ADVANCED SEMICONDUCTOR MANUFACTURING CONFERENCE (ASMC), 2022,
  • [49] A Real-Time Autonomous Highway Accident Detection Model Based on Big Data Processing and Computational Intelligence
    Ozhayoglu, Mural
    Kucukayan, Gokhan
    Dogdu, Erdogan
    2016 IEEE INTERNATIONAL CONFERENCE ON BIG DATA (BIG DATA), 2016, : 1807 - 1813
  • [50] ALGORITHM OF AIRCRAFT FLIGHT DATA PROCESSING IN REAL-TIME
    Sabziev, Elkhan
    SCIENTIFIC JOURNAL OF SILESIAN UNIVERSITY OF TECHNOLOGY-SERIES TRANSPORT, 2020, 108 : 213 - 221