A User-Centered Approach for Recognizing Convenience Images in Personal Photo Collections

Personal photo collections today usually include a huge number of convenience images (i.e. images with a practical character, such as handwritten notes, flipchart drawings, WI-FI logins or car parking positions). This paper proposes an iterative approach for the automatic recognition of convenience photos combining the methods from user-centered design and deep learning. In the first stage, we conducted a study to specify the categories of photos typically perceived by users as convenience photos. We then employed the specified categories to generate semi-automatically a dataset for training and testing a convolutional neural network model. After an evaluation of this model using from owners self-rated personal photo collections, we were able to improve both the quality of the dataset and our model. This paper therefore demonstrates the importance of integrating users at different stages of the development of deep learning solutions.