A framework for video event representation and annotation that is based on the definition of an ontology suitable for video content is presented. An ontology consists of a specific vocabulary for describing a certain reality and a set of explicit assumptions regarding the vocabulary's intended meaning. The Video Event Representation Language (VERL) describes an event ontology, and the Video Event Markup Language (VEML) allows the annotation of the instances of the events described in VERL. The framework can be integrated with MPEG-7 standards.