In the modern world, the ubiquity and critical importance of metallic materials are evident in everything from infrastructure and transportation to electronics and aerospace. Additive manufacturing (AM) of metals has revolutionized traditional production methods by enabling the creation of high-value components with topologically optimized complex geometries and functionalities. This review addresses the critical need for sophisticated physics-based models to investigate and optimize the AM processes of metals. We explore both melt-based and solid-state AM techniques, highlighting the current state-of-the-art modeling approaches. The purpose of this review is to evaluate existing models, identify their strengths and limitations, and suggest areas for future research to enhance the predictability and optimization of AM processes. By summarizing and comparing various modeling techniques, this review aims to provide a comprehensive understanding of the current research landscape. We focus on the pros and cons of different models, including their applicability to key elements and processes common to both melt-based and solid-state AM methods. Where multiple models exist for a single technique, a comparison is drawn to highlight their relative pros and cons. Concluding this review, we contemplate prospective advancements in sophisticated physics-based process modeling and strategies for their integration with models for structure-properties relations.