On Convergence of the Class Membership Estimator in Fuzzy k-Nearest Neighbor Classifier

The fuzzy k-nearest neighbor classifier (FkNN) improves upon the flexibility of the k-nearest neighbor classifier by considering each class as a fuzzy set and estimating the membership of an unlabeled data instance for each of the classes. However, the question of validating the quality of the class memberships estimated by FkNN for a regular multiclass classification problem still remains mostly unanswered. In this paper, we attempt to address this issue by first proposing a novel direction of evaluating a fuzzy classifier by highlighting the importance of focusing on the class memberships estimated by FkNN instead of its misclassification error. This leads us to finding novel theoretical upper bounds, respectively, on the bias and the mean squared error of the class memberships estimated by FkNN. Additionally the proposed upper bounds are shown to converge toward zero with increasing availability of the labeled data points, under some elementary assumptions on the class distribution and membership function. The major advantages of this analysis are its simplicity, capability of a direct extension for multiclass problems, parameter independence, and practical implication in explaining the behavior of FkNN in diverse situations (such as in presence of class imbalance). Furthermore, we provide a detailed simulation study on artificial and real data sets to empirically support our claims.