CNN based spatial classification features for clustering offline handwritten mathematical expressions

Published in Pattern Recognition Letters, 2020 (WoS-Q2, IF-3.255 (2019))

Abstract: To help human markers mark a large number of answers of handwritten mathematical expressions (HMEs), clustering them makes marking more efficient and reliable. Clustering HMEs, however, faces the problem of extracting both localization and classification representation of mathematical symbols for an HME image and defining the distance between two HME images. First, we propose a method based on Convolutional Neural Networks (CNN) to extract the representations for an HME. Symbols in various scales are located and classified by a combination of features from a multi-scale CNN. We use weakly supervised training combined with symbols attention to enhance localization and classification predictions. Second, we propose a multi-level spatial distance between two representations for clustering HMEs. Experiments on CROHME 2016 and CROHME 2019 dataset show the promising results of 0.99 and 0.96 in purity, respectively.

Recommended citation: Cuong Nguyen, Vu Khuong, Hung Nguyen, Masaki Nakagawa, "CNN based spatial classification features for clustering offline handwritten mathematical expressions." Pattern Recognition Letters, 2020. https://www.sciencedirect.com/science/article/abs/pii/S0167865519303782