Local feature with bag-of-words (BOW) representation has become one of the most popular approaches in object classification and image retrieval applications in the computer vision community. The recent efforts in the remote sensing community have demonstrated that the BOW approach can also effectively apply to geographic images for the applications of classification and retrieval. However, the BOW representation discards spatial information, which is critical for the remotely sensed land use classification. Several algorithms have incorporated spatial information into the BOWrepresentation by hard encoding coordinates of local features. Such rigid spatial encoding is not robust to translation and rotation variations, which are common characteristics of geographic images. To effectively incorporate spatial information into the BOW model for the land use classification, we propose a pyramid-of-spatial-relatons (PSR) model to capture both absolute and relative spatial relationships of local features. Unlike the conventional cooccurrence approach to describe pairwise spatial relationships between local features, the PSR model employs a novel concept of spatial relation to describe relative spatial relationship of a group of local features. As the result, the storage cost of the PSR model only linearly increases with the visual word codebook size instead of the quadratic relationship as in the cooccurrence approach. The PSR model is robust to translation and rotation variations and demonstrates excellent performance for the application of remotely sensed land use classification. On the Land Use and Land Cover image database, the PSR achieves 8% higher in the classification accuracy than the state of the art. If using only gray images, it outperforms the state of the art by more than 11%.
