Highly adaptive regression trees

The development of machine learning methods that are both accurate and interpretable is of paramount importance in healthcare and many other fields. The Highly Adaptive Lasso (HAL) has been shown to have predictive performance on par with state-of-the art algorithms. HAL involves performing regularized regression of the outcome on a tensor product of indicator basis functions. In this paper we show that this basis can be represented as a non-recursive partitioning of the feature space and propose a method for mapping this partitioning implied by HAL to a recursive partitioning. Such a mapping then allows for the representation of HAL as a decision tree, thereby providing interpretability of predictions made by the algorithm. We refer to this post-hoc method for interpretability as Highly Adaptive Regression Trees (HART). We provide a set of algorithms to construct this mapping and conveniently visualize the resulting tree. Using real data, we show that HAL's predictive performance is on par with state-of-the-art methods, and we demonstrate the construction and interpretation of HARTs.