Building environments for visual programming of robots by demonstration

The primary advantage of visual programming languages is that they directly represent the structure of algorithms and data, thereby enhancing the programmer's ability to build and comprehend programs. Recently, there has been considerable interest in applying visual programming languages to the problem of controlling robots. An important characteristic of this domain is that a robot and its environment has a physical existence and therefore have an obvious visual representation. Also, the actions a robot performs are most naturally represented in terms of changes in this representation. Although general-purpose visual programming languages are as useful for programming a robot as they are for any other programming task, they take no advantage of this natural representation. In achieving a visual robot programming language in which the robot and its environment are concretely represented and programmed by direct manipulation, we face an interesting dilemma. Clearly, we want such a language not only to support concrete visualisations of a specific robot, but also to be general purpose. As a solution, we propose that a robot programming system should consist of two parts; a definition module with which to describe the structure, function and visual representation of a specific robot, and a programming module that uses this description to enable programming by direct manipulation. We describe the visual editors that constitute the first of these modules, discuss the underlying structure generated by it, and briefly show how this structure can be used in the second module. This paper is an extended version of one which was presented at the 1998 IEEE Symposium on Visual Languages [1]. (C) 2000 Academic Press.