Characters as graphs: Interpretable handwritten Chinese character recognition via Pyramid Graph Transformer

It is meaningful but challenging to teach machines to recognize handwritten Chinese characters. However, conventional approaches typically view handwritten Chinese characters as either static images or tempo-ral trajectories, which may ignore the inherent geometric semantics of characters. Instead, here we first propose to represent handwritten characters as skeleton graphs, explicitly considering the natural charac-teristics of characters (i.e., characters as graphs). Furthermore, we propose a novel Pyramid Graph Trans-former (PyGT) to specifically process the graph-structured characters, which fully integrates the advan-tages of Transformers and graph convolutional networks. Specifically, our PyGT can learn better graph fea-tures through (i) capturing the global information from all nodes with graph attention mechanism and (ii) modelling the explicit local adjacency structures of nodes with graph convolutions. Furthermore, the PyGT learns the multi-resolution features by constructing a progressive shrinking pyramid. Compared with ex-isting approaches, it is more interpretable to recognize characters as geometric graphs. Moreover, the pro-posed method is generic for both online and offline handwritten Chinese character recognition (HCCR), and it also can be feasibly extended to handwritten text recognition. Extensive experiments empirically demonstrate the superiority of PyGT over the prevalent approaches including 2D-CNN, RNN/1D-CNN, and Vision Transformer (ViT) for HCCR. The code is available at https://github.com/ganji15/PyGT-HCCR .& COPY; 2023 Elsevier Ltd. All rights reserved.