Mobile health applications (mHealth) have revolutionized healthcare sector by leveraging mobile technology to provide personalized services. As a rapidly growing industry, mHealth aligns with the World Health Organization's goal of empowering patients to take control of their healthcare journey. In the realm of mHealth, ensuring user satisfaction always remains a key concern. Therefore, recognizing crucial factors that determine customer satisfaction levels can help mHealth applications improve their quality of service. Leveraging machine learning techniques for this task can prove to be highly beneficial. However, the existing machine learning methods in this domain are 'black-box' and possess several limitations, such as less accuracy, lack of explainability, and many more. To resolve these challenges, the current research introduces a novel approach based on deep transformers and explainable artificial intelligence (EAI). This approach aims to analyze user-generated content to determine mHealth ratings and the factors influencing user satisfaction. The proposed pipeline encompasses several steps, namely, data preprocessing and anonymization, feature extraction, feature selection, transformer architecture building, and evaluating performance using a dataset containing reviews of several different mHealth applications. The sensitivity analysis of the proposed approach is performed by utilizing several feature selection techniques and comparing the prediction performance with existing benchmark solutions available in the literature. From the comparative evaluation, it is observed that the proposed approach outperforms existing techniques by providing 98% accuracy and 99% F1-score, with a 3-5% relative improvement over benchmark solutions. In addition, the proposed method incorporates EAI to determine several crucial factors that affect user satisfaction or app rating scores. This information will be beneficial for the stakeholders in devising better platforms and strategies for enhancing user satisfaction and experience in the mHealth domain.