System Entropy and Its Application in Feature Selection

Feature selection is always an important issue in the research on data mining technologies. However, the problem of optimal feature selection is NP hard. Therefore, heuristic approaches are more practical to actual learning systems. Usually, that kind of algorithm selects features with the help of a heuristic metric compactum to measure the relative importance of features in a learning system. Here a new notion of ‘system entropy’ is described in terms of rough set theory, and then some of its algebraic characteristics are studied. After its intrinsic value biase is effectively counteracted, the system entropy is applied in BSE, a new heuristic algorithm for feature selection. BSE is efficient, whose time complexity is lower than that of analogous algorithms; BSE is also effective, which can produce the optimal results in the mini-feature biased sense from varieties of learning systems. Besides, BSE is tolerant and also flexible to the inconsistency of a learning system, consequently able to elegantly handle data noise in the learning system.