Context awareness based Sketch-DeepNet architecture for hand-drawn sketches classification and recognition in AIoT

被引：0

作者：

Ali S. ^{[1
]}

Aslam N. ^{[1
]}

Kim D. ^{[2
]}

Abbas A. ^{[3
]}

Tufail S. ^{[1
]}

Azhar B. ^{[1
]}

机构：

[1] Department of Software Engineering, University of Lahore, Punjab, Lahore

[2] Department of Computer Engineering, Jeju National University, Jeju, Jeju

[3] Department of Computer Science, University of Central Punjab, Punjab, Lahore

来源：

PeerJ Computer Science | 2023年 / 9卷

关键词：

Convolutional neural networks (CNNs); Deep neural networks (DNNs); Sketch recognition; TU-Berlin;

D O I：

10.7717/PEERJ-CS.1186

中图分类号：

学科分类号：

摘要：

A sketch is a black-and-white, 2-D graphical representation of an object and contains fewer visual details as compared to a colored image. Despite fewer details, humans can recognize a sketch and its context very efficiently and consistently across languages, cultures, and age groups, but it is a difficult task for computers to recognize such low-detail sketches and get context out of them. With the tremendous increase in popularity of IoT devices such as smartphones and smart cameras, etc., it has become more critical to recognize free hand-drawn sketches in computer vision and human-computer interaction in order to build a successful artificial intelligence of things (AIoT) system that can first recognize the sketches and then understand the context of multiple drawings. Earlier models which addressed this problem are scaleinvariant feature transform (SIFT) and bag-of-words (BoW). Both SIFT and BoW used hand-crafted features and scale-invariant algorithms to address this issue. But these models are complex and time-consuming due to the manual process of features setup. The deep neural networks (DNNs) performed well with object recognition on many large-scale datasets such as ImageNet and CIFAR-10. However, the DDN approach cannot be carried out for hand-drawn sketches problems. The reason is that the data source is images, and all sketches in the images are, for example, ‘birds’ instead of their specific category (e.g., ‘sparrow’). Some deep learning approaches for sketch recognition problems exist in the literature, but the results are not promising because there is still room for improvement. This article proposed a convolutional neural network (CNN) architecture called Sketch-DeepNet for the sketch recognition task. The proposed Sketch-DeepNet architecture used the TU-Berlin dataset for classification. The experimental results show that the proposed method beats the performance of the state-of-the-art sketch classification methods. The proposed model achieved 95.05% accuracy as compared to existing models DeformNet (62.6%), Sketch-DNN (72.2%), Sketch-a-Net (77.95%), SketchNet (80.42%), Thinning-DNN (74.3%), CNN-PCA-SVM (72.5%), Hybrid-CNN (84.42%), and human recognition accuracy of 73% on the TU-Berlin dataset © Copyright 2023 Ali et al

引用

共 40 条

[11]

Hu R, James S, Wang T, Collomosse J., Markov random fields for sketch based video retrieval, Proceedings of the 3rd ACM Conference on International Conference on Multimedia Retrieval, pp. 279-286, (2013)

[12]

Joachims T., Text categorization with support vector machines: learning with many relevant features, European Conference on Machine Learning, pp. 137-142, (1998)

[13]

Kabakus AT., A novel sketch recognition model based on convolutional neural networks, 2020 International Congress on Human-Computer Interaction, Optimization and Robotic Applications (HORA), pp. 1-6, (2020)

[14]

Kingma DP, Ba J., Adam: a method for stochastic optimization, (2014)

[15]

Klare B, Li Z, Jain AK., Matching forensic sketches to mug shot photos, IEEE Transactions on Pattern Analysis and Machine Intelligence, 33, 3, pp. 639-646, (2010)

[16]

Krizhevsky A, Hinton G., Learning multiple layers of features from tiny images, (2009)

[17]

Krizhevsky A, Sutskever I, Hinton GE., ImageNet classification with deep convolutional neural networks, Advances in Neural Information Processing Systems 25 (NIPS 2012), 25, pp. 1097-1105, (2012)

[18]

LeCun Y, Bottou L, Bengio Y, Haffner P., Gradient-based learning applied to document recognition, Proceedings of the IEEE, 86, 11, pp. 2278-2324, (1998)

[19]

Li Q, Cai W, Wang X, Zhou Y, Feng DD, Chen M., Medical image classification with convolutional neural network, 2014 13th International Conference on Control Automation Robotics & Vision (ICARCV), pp. 844-848, (2014)

[20]

Li Y, Hospedales TM, Song Y-Z, Gong S., Free-hand sketch recognition by multi-kernel feature learning, Computer Vision and Image Understanding, 137, 4, pp. 1-11, (2015)

← 1 2 3 4 →