ALGEBRAIC LEARNING OF STATISTICAL ASSOCIATIONS FOR LANGUAGE-ACQUISITION

Statistical association measures play an important role in many areas of natural language research. An impediment to reliable estimation of such associations is the problem of small-sample statistics, since in any natural corpus there will be many infrequently occurring words. This paper proposes an alternative method for estimating such associations, which circumvents the small-sample issues. The idea is to view sentence/meaning pairs as algebraic equations, rather than as observations of a pattern in some class. Associations are estimated via solving these equations rather than via relative frequency estimates of mutual information. We develop a theoretical foundation for the dual algebraic/statistical nature of associations, proving two uniqueness theorems. We then exploit this theory to provide an algorithmic solution to the estimation problem. This algorithm is experimentally evaluated on 1494 natural languages messages from a rudimentary Data Retrieval experiment. One striking result is that the algebraic algorithm can often provide reliable estimates even for words which occur only once in a corpus.