Enhancing Software Requirements Classification with Machine Learning and Feature Selection Techniques

Requirements engineers have the responsibility for classifying software requirements into functional and nonfunctional variants. As software architects need quality requirements to be known to get their job done, machine learning is employed to speed up and add consistency to the process of identifying and categorizing requirements so that effort may be spent more effectively. We experimented with the effects of different machine learning algorithms, as well as different pre-processing and feature selection techniques. It was determined that, for this application, stop words should not be removed and that performing lemmatization on words provides the most effective features for classification. Furthermore, after finalizing our choices of pre-processing techniques and algorithm to use, we proposed a modification to the Extensive Feature Selector by gathering the most distinctive words in each category and using a list of those as our main features. By using a threshold of 0.013, we obtained an F1 score of 0.787, which is an improvement on the base Enhanced Feature Selector's F1 score of 0.761 with the same number of word features.