Random Forests for Heteroscedastic Data

Random forests are a popular machine learning technique that are effective across a range of scientific problems. We extend the standard algorithm to incorporate the uncertainty information that arises in heteroscedastic data - datasets where the amount of noise in the target value varies between datapoints. We consider datasets where the relative amount of measurement noise in different datapoints is known. This is not the standard scenario, but does commonly exist in real data, as we illustrate on 10 drug design datasets. Utilising this uncertainty information can lead to significantly better predictive performance. We introduce three random forest variations to learn from heteroscedastic data: parametric bootstrapping, weighted random forests and variable output smearing. All three can improve model performance, demonstrating the adaptability of random forests to heteroscedastic data and thus expanding their applicability. Additionally, variations in the relative performance of the three methods across datasets provides insight into the mechanisms of random forests and the purpose of the different random elements within the model.