Performance Evaluation System Based on Multi-Indicators for Signal Recognition

被引:1
作者
Li, Qingming [1 ]
Yang, Songlin [2 ]
Chen, Hu [3 ]
机构
[1] VES Technol Corp Ltd, Beijing 100192, Peoples R China
[2] Harbin Engn Univ, Coll Informat & Commun Engn, Harbin 150001, Peoples R China
[3] Wuhan Second Ship Design & Res Inst, Wuhan 430064, Peoples R China
关键词
Electromagnetics; Machine learning algorithms; Classification algorithms; Predictive models; Indexes; Data models; Robustness; Evaluation systems; machine learning; ensemble learning; signal recognition; MODULATION CLASSIFICATION; NETWORKS;
D O I
10.1109/ACCESS.2022.3228641
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
Nowadays, the electromagnetic environment is becoming more and more complex, and it is increasingly difficult to accurately identify electromagnetic signals. At the same time, the quality of models and algorithms largely determines the final result of electromagnetic signal recognition, and a comprehensive and holistic evaluation system is needed to assess the quality of model or algorithm performance in the field of electromagnetic signal recognition. To address this phenomenon, this paper uses different machine learning models and AdaBoost algorithms to identify 100 classes of WiFi radiation source electromagnetic signals. In this paper, the classification performance, complexity and robustness of the models are evaluated comprehensively in three aspects respectively, and a multi-dimensional evaluation system for signal recognition is constructed, and the differences between different model algorithms in signal recognition effects are compared and analyzed, and the correlations between each evaluation index in different dimensions are explored. The experimental results show that different machine learning models and algorithms have different recognition effects on electromagnetic signals, among which the recognition accuracy of ResNet can reach more than 90%, but its computational complexity is high and easily affected by noise. DNN has poor recognition effect and the highest computational complexity, but it is not easily affected by noise. The AdaBoost algorithm does not necessarily improve the recognition classification accuracy of the underlying classifier. The evaluation system established in this paper is meaningful for assessing the performance of models and algorithms.
引用
收藏
页码:2820 / 2830
页数:11
相关论文
共 30 条
  • [1] Threat of Adversarial Attacks on DL-Based IoT Device Identification
    Bao, Zhida
    Lin, Yun
    Zhang, Sicheng
    Li, Zixin
    Mao, Shiwen
    [J]. IEEE INTERNET OF THINGS JOURNAL, 2022, 9 (11) : 9012 - 9024
  • [2] Biggio B., 2011, PROC INT JOINT C BIO, P1, DOI DOI 10.1109/IJCB.2011.6117474
  • [3] Security Evaluation of Pattern Classifiers under Attack
    Biggio, Battista
    Fumera, Giorgio
    Roli, Fabio
    [J]. IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING, 2014, 26 (04) : 984 - 996
  • [4] The Performance Evaluation of Big Data-Driven Modulation Classification in Complex Environment
    Cai, Zhuoran
    Wang, Jidong
    Ma, Minghuan
    [J]. IEEE ACCESS, 2021, 9 : 26313 - 26322
  • [5] SR2CNN: Zero-Shot Learning for Signal Recognition
    Dong, Yihong
    Jiang, Xiaohan
    Zhou, Huaji
    Lin, Yun
    Shi, Qingjiang
    [J]. IEEE TRANSACTIONS ON SIGNAL PROCESSING, 2021, 69 : 2316 - 2329
  • [6] A decision-theoretic generalization of on-line learning and an application to boosting
    Freund, Y
    Schapire, RE
    [J]. JOURNAL OF COMPUTER AND SYSTEM SCIENCES, 1997, 55 (01) : 119 - 139
  • [7] Guo J, 2018, CHIN CONTR CONF, P4185, DOI 10.23919/ChiCC.2018.8483351
  • [8] Hao M., 2021, 2021 13 INT S ANTENN, P1, DOI 10.1109/ISAPE54070.2021.9752950
  • [9] Generalization Characteristics of Complex-Valued Feedforward Neural Networks in Relation to Signal Coherence
    Hirose, Akira
    Yoshida, Shotaro
    [J]. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, 2012, 23 (04) : 541 - 551
  • [10] Multisignal Modulation Classification Using Sliding Window Detection and Complex Convolutional Network in Frequency Domain
    Hou, Changbo
    Liu, Guowei
    Tian, Qiao
    Zhou, Zhichao
    Hua, Lijie
    Lin, Yun
    [J]. IEEE INTERNET OF THINGS JOURNAL, 2022, 9 (19) : 19438 - 19449