Heart disease has become very common nowadays. Machine learning-based heart disease prediction has significant potential in clinical applications, enhancing early diagnosis and treatment. Detection of the disease at early stage can save human lives. The accurate predictive models help in early identification of the condition & help to avail appropriate treatment. In this work, we conduct a thorough exploratory data analysis (EDA) on a dataset that includes a range of clinical characteristics associated with heart health. Features in the dataset include, age, gender, kind of chest discomfort, resting bp, cholesterol level, and ECG readings. Our main goal is to find out how well the methods for logistic regression, decision tree classifier, random forest, and support vector machine can predict the occurrence of heart disease. Through EDA, this study analyses the distribution, correlation, and significance of features, gaining insights into potential risk factors associated with heart disease. Subsequently, we train and evaluate each machine learning model on the dataset, employing appropriate performance metrics to assess their prediction. In addition, this study has developed a hybrid model that merges the strengths of the Random Forest and Support Vector Machine (SVM) algorithms, resulting in an improved accuracy. This innovative approach highlights the advantages of combining advanced machine learning techniques to boost predictive reliability and consistency. This study highlights the significant potential of machine learning to improve early disease detection and treatment.