Background: Masses are one of the most important indicators of breast cancer in mammograms, and their classification into two groups as benign and malignant is highly necessary. Computer Aided Diagnosis (CADx) helps radiologists enhance the accuracy of their decision. Hence, the system is required to support and assess with radiologist's interaction as an expert. Methods: In this research, classification of breast masses using mammography in the two main views which include MLO and CC, is evaluated with respect to the shape, texture and asymmetry aspect. Additionally, a method was developed and proposed using the classification of breast tissue density based on the decision tree. Discussion: This study therefore, aims to provide a method based on the human decision-making model that will help in designing the perfect tool for radiologists, regardless of the complexity of computing, costly procedures and also reducing the diagnosis error. Conclusion: Results show that the proposed system for entirely fat, scattered fibroglandular densities, heterogeneously dense, and extremely dense breast achieved 100, 99, 99 and 98% true malignant rate, respectively with cross-validation procedure.