Estimation of probability density functions from limited training samples

In this paper, we analyze probability density functions when the number of training samples is limited, assuming normal distributions. As the dimension of data increases significantly, the performance of a classifier suffers when the number of training samples is not adequate. This problem becomes worse as high dimensional data such as hyperspectral images are widely available. The key factor in designing a classifier is estimation of probability density functions, which are completely determined by covariance matrices and mean vectors in case of the Gaussian ML classifier. In this paper, we provide in-depth analyses of estimation of probability density functions in terms of the number of training samples assuming normal distributions and provide a guideline in choosing the dimensionality of data for a given set of training samples.