This paper describes architecture and implementation of a multimedia authoring environment which features content-based management of video. The goal of our work is to provide multimedia application developers with a useful video database, where they gather video materials from various sources, store them in a organized format, and use them in multimedia applications. The principal design concepts are that (1) it should have a Flexible means to store video materials in a organized format so that video clips can be retrieved easily, and that (2) it should have a view to each application so that various applications can share the video materials. To meet these requirements we designed a multimedia authoring system based on a video object model. The video object model features (1) a method to attach annotations not only to scenes but also to things shot in the video, and (2) a method to define video clips independently from physical video data. Our experiences in using the system to develop a multimedia CAI application are also described to show the advantages of our system.