Reinforcement Learning for Intelligent Sensor Virtualization and Provisioning in Internet of Vehicles (IoV)

被引:3
作者
Abbes, Slim [1 ]
Rekhis, Slim [2 ]
机构
[1] Univ Carthage, Higher Sch Commun Tunis SUPCOM, Commun Networks & Secur CNAS Res Lab, LR11TIC04, Tunis 1054, Tunisia
[2] Univ Carthage, Green & Smart Commun Syst GRESCOM Res Lab, LR11TIC02, Tunis 1054, Tunisia
关键词
Internet of Vehicles; reinforcement learning; sensor virtualization; intelligent sensor provisioning; RESOURCE-ALLOCATION;
D O I
10.1109/ACCESS.2024.3387295
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
The Internet of Vehicles (IoV) is a powerful application of the Internet of Things (IoT) in the Intelligent Transportation System (ITS). It enables device connectivity, and interaction with the environment, and improves efficiency in utilizing sensor data. Leveraging the capabilities of vehicle sensors to provide virtual sensor service presents an opportunity to make the most of underutilized sensor resources and offer on-demand sensor services. However, challenges remain in ensuring spatio-temporal sensor availability due to the high mobility of vehicles and frequent changes in the IoV network topology. Furthermore, providing sensor services in the dynamic IoV-Cloud market, while balancing cost-effectiveness for consumers and profitability for service providers, poses a significant challenge. To address these challenges, we develop an IoV-Cloud architecture tailored for vehicle virtual sensor provisioning, integrating interactive functional layers and components for a comprehensive solution. This architecture supports vehicle sensor virtualization and management, facilitating the intelligent provisioning of on-demand, elastic, and scalable vehicle virtual sensors from existing physical sensors within mobile vehicles. To address vehicle mobility and enhance sensor availability, our design permits virtual sensors to maintain data provision even when switched to different physical sensors. Further optimizing sensor utilization, our system allows multiple virtual vehicle sensors to share a single physical sensor by employing a configuration mapping mechanism. At the heart of our strategy is a reinforcement learning model that dynamically selects the most suitable vehicle physical sensor for each time slot throughout the service period, considering both physical availability and cost-effectiveness for the Sensor Cloud Service Provider (SCSP). We simulate the proposed intelligent sensor selection model using both Q-learning and SARSA algorithms, demonstrating its effectiveness in intelligent and dynamic sensor provisioning.
引用
收藏
页码:54352 / 54370
页数:19
相关论文
共 42 条
[1]  
Abbes Slim, 2022, 2022 IEEE 21st International Symposium on Network Computing and Applications (NCA), P217, DOI 10.1109/NCA57778.2022.10013541
[2]  
Abbes Slim, 2021, 2021 International Wireless Communications and Mobile Computing (IWCMC), P1129, DOI 10.1109/IWCMC51323.2021.9498940
[3]   Sensor Virtualization and Provision in Internet of Vehicles [J].
Abbes, Slim ;
Rekhis, Slim .
ADVANCED INFORMATION NETWORKING AND APPLICATIONS, AINA-2022, VOL 2, 2022, 450 :386-397
[4]   Energy Efficient IoT Virtualization Framework With Peer to Peer Networking and Processing [J].
Al-Azez, Zaineb T. ;
Lawey, Ahmed Q. ;
El-Gorashi, Taisir E. H. ;
Elmirghan, Jaafar M. H. .
IEEE ACCESS, 2019, 7 :50697-50709
[5]  
Alam S., 2010, PROC IEEE INT C NETW, P1
[6]   Cloud-based virtualization environment for IoT-based WSN: solutions, approaches and challenges [J].
Almurisi, Nasr ;
Tadisetty, Srinivasulu .
JOURNAL OF AMBIENT INTELLIGENCE AND HUMANIZED COMPUTING, 2022, 13 (10) :4681-4703
[7]   A cluster-based vehicular cloud architecture with learning-based resource management [J].
Arkian, Hamid Reza ;
Atani, Reza Ebrahimi ;
Diyanat, Abolfazl ;
Pourkhalili, Atefe .
JOURNAL OF SUPERCOMPUTING, 2015, 71 (04) :1401-1426
[8]  
Baek S, 2017, 2017 IEEE SYMPOSIUM SERIES ON COMPUTATIONAL INTELLIGENCE (SSCI), P3058
[9]  
Chenaru O, 2019, 2019 8TH INTERNATIONAL CONFERENCE ON SYSTEMS AND CONTROL (ICSC'19), P388, DOI [10.1109/ICSC47195.2019.8950640, 10.1109/icsc47195.2019.8950640]
[10]   In-situ observation virtual sensor in building systems toward virtual sensing-enabled digital twins [J].
Choi, Youngwoong ;
Yoon, Sungmin .
ENERGY AND BUILDINGS, 2023, 281