Feature Descriptors for Spotting 3D Characters on Triangular Meshes

Increased use of 3D digitization methods leads to large numbers of 3D models with script like cuneiform tablets, temple wall inscriptions from the Mayan empire and Minoan and Mycenaean seal imprints. Analyzing and in particular comparing those objects in our projects is a core challenge for robust symbol-spotting and transcription of ancient scripts. The discrete manifolds using irregular grids describe the objects surface and contain complex shapes that cannot be wholly captured by a single rendering to a two dimensional plane in an image format. As the plethora of standard image descriptor extraction methods is not available on those 3D meshes we propose a novel approach adapting relevant image descriptor methods to process irregular grids. We show preliminary evaluations of our approach by performing a symbol spotting task on cuneiform tablets. Our approach proceeds in three stages: (i) We compute a local surface feature with Multi-scale Integral Invariants (MSII), (ii) for each vertex we extract radial geodesic patches which are embedded locally into a polar coordinate system, (iii) a rectangular sampling scheme converts these patches into raster image patches that are accessible to 2D image descriptors. For the symbol spotting task we compute Histogram of Oriented Gradients (HOG) and Oriented FAST and Rotated BRIEF (ORB) descriptors on the query mesh and document mesh for each vertex. Similar symbols are retrieved by sliding a geodesic patch along the document mesh. Like in many other Cultural Heritage applications we have to consider the lack of any ground-truth. Therefore we provide a qualitative analysis of the spotting results showing that a query mesh is reliably found on the document mesh.