A Comparative Study on Machine Learning algorithms for Knowledge Discovery

For centuries, the process of formulating new knowledge from observations has driven scientific discoveries. With rapid advancements in machine learning, it is natural to question the possibility of automating knowledge discovery in the scientific field. A benchmark task for automated knowledge discovery is called symbolic regression. The task aims to predict a mathematical equation that best describes the observational data. The advancements in symbolic regression have significant potential to aid research in understanding unexplored systems' dynamics and governing properties. However, the combinatorial nature of the problem makes it an expensive and challenging problem to solve efficiently. Several types of symbolic regression algorithms exist, from genetic programming and sparse regression to deep generative models. However, no survey collates these prominent algorithms. Therefore, this paper aims to summarize key research works in symbolic regression and perform a comparative study to understand the strength and limitations of each method. Finally, we highlight the challenges in the current methods and future research directions in the application of machine learning in knowledge discovery.