Finite Element (FE) analysis is widely used for process simulation of advanced composites in aerospace applications. However, as the fidelity of FE models improves, the cost and time to calibrate, setup and perform simulations also increases significantly. To speed-up process simulation, reduced-order FE can be used. Reduced-order FE and Machine Learning (ML) can be combined to further speed-up the process simulation and enable process optimization. However, theory-agnostic ML usually requires large datasets which may not be feasible in industry. This can be mitigated by integrating the underlying physics into ML to develop Theory-Guided Machine Learning (TGML) models. In this study, for process simulation and optimization of a composite stringer on a wing skin, three modeling approaches are compared: 1) high-fidelity FE modeling, 2) surrogate ML modeling based on high-fidelity FE results, and 3) surrogate TGML modeling based on both high-fidelity and low-fidelity FE results. It is shown that TGML, with the guidance of underlying physics of composites curing, can improve prediction accuracy and significantly reduce the amount of data needed for training compared to theory-agnostic ML.