A case study of duplications detection for educational domain thorough ad hoc search and identification NLP-based method

During the organization and planning of lecture courses for a discipline, its content may be overlapped and partially delivered in more than one course. Sometimes this action causes time loss through unnecessary repeating. This paper introduces an automated tool for duplications detections adapting methods of natural language processing used for Web search. The experiment for unstructured electronic document repositories clustering for thematic duplicate identification in different documents in the case of educational domain is presented. A prototype of this Web service-based software search engine is being designed and discussed. The experiment aimed to identify thematic duplicates of various courses within one of the teaching disciplines is also presented.