Mitochondrial toxicityis a significant concern in the drug discoveryprocess, as compounds that disrupt the function of these organellescan lead to serious side effects, including liver injury and cardiotoxicity.Different in vitro assays exist to detect mitochondrial toxicity atvarying mechanistic levels: disruption of the respiratory chain, disruptionof the membrane potential, or general mitochondrial dysfunction. Inparallel, whole cell imaging assays like Cell Painting provide a phenotypicoverview of the cellular system upon treatment and enable the assessmentof mitochondrial health from cell profiling features. In this study,we aim to establish machine learning models for the prediction ofmitochondrial toxicity, making the best use of the available data.For this purpose, we first derived highly curated datasets of mitochondrialtoxicity, including subsets for different mechanisms of action. Dueto the limited amount of labeled data often associated with toxicologicalendpoints, we investigated the potential of using morphological featuresfrom a large Cell Painting screen to label additional compounds andenrich our dataset. Our results suggest that models incorporatingmorphological profiles perform better in predicting mitochondrialtoxicity than those trained on chemical structures alone (up to +0.08and +0.09 mean MCC in random and cluster cross-validation, respectively).Toxicity labels derived from Cell Painting images improved the predictionson an external test set up to +0.08 MCC. However, we also found thatfurther research is needed to improve the reliability of Cell Paintingimage labeling. Overall, our study provides insights into the importanceof considering different mechanisms of action when predicting a complexendpoint like mitochondrial disruption as well as into the challengesand opportunities of using Cell Painting data for toxicity prediction.