STATE-OF-THE-ART IN AUTOMATED VALIDATION OF KNOWLEDGE-BASED SYSTEMS

Validation of Knowledge-Based Systems (KBS) is an important aspect of the overall KBS development process, which aims to assure the system's ability to reach correct conclusions. The objective of this paper is to discuss the desirable functionality of an automated validation tool and to provide a survey of existing methods and tools supporting that functionality. The scope of our discussion is limited to validating the level of performance of the KBS as a problem solver, since this is the aspect in which KBS differ most from conventional software; more conventional aspects of system evaluation, such as assessing the ''usability'' of the system, are not covered. Automated validation tools are considered in two categories: dynamic and static. Dynamic validation tools are those that measure and, in some cases, refine the level of performance of a KBS using a suite of test cases. Use of such tools assumes that an adequate set of real test cases is available. Static validation tools are used to create test cases by making use of domain knowledge already embodied in the KBS or meta-knowledge. Such tools are used when an inadequate set of test cases is available.