Open Information Extraction from the Web

Open Information Extraction (IE), where the identities of the relations to be extracted are unknown and the billions of documents found on the web necessitate highly scalable processing, is a reliable way of extracting information from the Internet. The first IE systems relied on some form of pattern-matching rules that were manually crafted for each domain. Modern IE automatically learns an extractor from a training set in which domain-specific examples are tagged. The development of suitable training data for IE requires substantial effort and expertise. The Know-ItAll web IE system automates IE by learning to label its own training examples using only a small set of domain-independent extraction patterns. TextRunner is a fully implemented Open IE system that utilizes the two-phase architecture. It's first phase uses a general model of language, which trains a graphical model called a conditional random field (CRF). Open IE also supports aggregating, fusing information across a large number of web pages.