Our paper investigates the relationship between physical behavior and affective states, employing machine learning to analyze wearable sensor data. We specifically focus on the psychological impacts of sedentary behavior, integrating sports science, psychology, and data science. A significant contribution of our work is the development of a personalized machine learning model that predicts Energetic Arousal, Valence, and Calmness variations using Ecological Momentary Assessment (EMA) and physical behavior data from wearable sensors. The study compares multiple tree-based models, Recurrent Neural Network (RNN) approaches and an end-to-end implementation. Our best-performing model, an LSTM-based RNN, achieved an RMSE of 18.74, 14.17, and 15.61; and a trend prediction accuracy of 73.3%, 71.1%, and 71.6% in Energetic Arousal, Valence, and Calmness prediction respectively.