The semantic desktop is an extension of the semantic web technologies to the field of desktops for the purpose of intelligent processing and share of data, in the PC. Although most current technologies of the semantic desktop focus only on the virtual world in the PC, the field of user's activity is not limited to it; in a typical office environment, the user also interacts with various physical objects such as printed documents and office goods during the activity. The aim of this paper is to show some important concepts and a prototype of the interface necessary to extend the field of semantic desktop to the physical worlds are technologies for bridging the gap between virtual and physical worlds are fast and robust object recognition and document image retrieval.