Sign Language Recognition Using Multiple Kernel Learning: A Case Study of Pakistan Sign Language

被引:17
|
作者
Shah, Farman [1 ]
Shah, Muhammad Saqlain [1 ]
Akram, Waseem [2 ]
Manzoor, Awais [2 ]
Mahmoud, Rasha Orban [3 ]
Abdelminaam, Diaa Salama [3 ,4 ]
机构
[1] Int Islamic Univ Islamabad, Dept Comp Sci & Software Engn, Islamabad 44000, Pakistan
[2] COMSATS Univ Islamabad CUI, Dept Comp Sci, Islamabad 45550, Pakistan
[3] Benha Univ, Fac Comp & Artificial Intelligence, Banha 13518, Egypt
[4] Misr Int Univ, Fac Comp Sci, Cairo 611310, Egypt
关键词
Assistive technology; Gesture recognition; Feature extraction; Kernel; Support vector machines; Image edge detection; Histograms; Sign language; image recognition; machine learning; features extraction;
D O I
10.1109/ACCESS.2021.3077386
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
All over the world, deaf people use sign language as the only reliable source of communication with each other as well as with normal people. These communicating signs are made up of the shape of the hand and movement. In Pakistan, deaf people use Pakistan sign language (PSL) as a means of communication with people. In scientific literature, many studies have been done on PSL recognition and classification. Most of these work focused on colored-based hands while some others are sensors and Kinect-based approaches. These techniques are costly and also avoid user-friendliness. In this paper, a technique is proposed for the recognition of thirty-six static alphabets of PSL using bare hands. The dataset is obtained from the sign language videos. At a later step, four vision-based features are extracted i.e. local binary patterns, a histogram of oriented gradients, edge-oriented histogram, and speeded up robust features. The extracted features are individually classified using Multiple kernel learning (MKL) in support vector machine (SVM). We employed a one-to-all approach for the implementation of basic binary SVM into the multi-class SVM. A voting scheme is adopted for the final recognition of PSL. The performance of the proposed technique is measured in terms of accuracy, precision, recall, and F-score. The simulation results are promising as compared with existing approaches.
引用
收藏
页码:67548 / 67558
页数:11
相关论文
共 50 条
  • [31] Sign language comprehension:: The case of Spanish sign language
    Rodriguez-Ortiz, I. R.
    JOURNAL OF DEAF STUDIES AND DEAF EDUCATION, 2008, 13 (03): : 378 - 390
  • [32] On the role of multimodal learning in the recognition of sign language
    Pedro M. Ferreira
    Jaime S. Cardoso
    Ana Rebelo
    Multimedia Tools and Applications, 2019, 78 : 10035 - 10056
  • [33] On the role of multimodal learning in the recognition of sign language
    Ferreira, Pedro M.
    Cardoso, Jaime S.
    Rebelo, Ana
    MULTIMEDIA TOOLS AND APPLICATIONS, 2019, 78 (08) : 10035 - 10056
  • [34] Contrastive Learning for Sign Language Recognition and Translation
    Gan, Shiwei
    Yin, Yafeng
    Jiang, Zhiwei
    Xia, Kang
    Xie, Lei
    Lu, Sanglu
    PROCEEDINGS OF THE THIRTY-SECOND INTERNATIONAL JOINT CONFERENCE ON ARTIFICIAL INTELLIGENCE, IJCAI 2023, 2023, : 763 - 772
  • [35] News 'with' or 'in' sign language? Case study on the comprehensibility of sign language in news broadcasts
    Alexander, Dhoest
    Rijckaert, Jorn
    PERSPECTIVES-STUDIES IN TRANSLATION THEORY AND PRACTICE, 2022, 30 (04): : 627 - 642
  • [36] Pakistan Sign Language Recognition and Translation System using Leap Motion Device
    Raziq, Nosheela
    Latif, Seemab
    ADVANCES ON P2P, PARALLEL, GRID, CLOUD AND INTERNET COMPUTING, 2017, 1 : 895 - 902
  • [37] Collection and Preprocessing of Czech Sign Language Corpus for Sign Language Recognition
    Campr, Pavel
    Hruz, Marek
    Trojanova, Jana
    SIXTH INTERNATIONAL CONFERENCE ON LANGUAGE RESOURCES AND EVALUATION, LREC 2008, 2008, : 3175 - 3178
  • [38] A recognition method of sign language sentences for Japanese sign language translation
    Sagawa, H
    Takeuchi, M
    PROCEEDINGS OF THE FIFTH JOINT CONFERENCE ON INFORMATION SCIENCES, VOLS 1 AND 2, 2000, : A858 - A861
  • [39] Indian sign language recognition using SVM
    Raheja J.L.
    Mishra A.
    Chaudhary A.
    Pattern Recognition and Image Analysis, 2016, 26 (2) : 434 - 441
  • [40] Recognition of Sign Language using Capsule Networks
    Beser, Fuat
    Kizrak, Merve Ayyuce
    Bolat, Bulent
    Yildirim, Tulay
    2018 26TH SIGNAL PROCESSING AND COMMUNICATIONS APPLICATIONS CONFERENCE (SIU), 2018,