Enhancing News Recommendation with Transformers and Ensemble Learning

News recommendation is an important task in the digital media landscape, challenged by the rapid decline of article relevance and the need for personalized content delivery. The RecSys Challenge 2024, organized by Ekstra Bladet, focuses on this problem using the EB-NeRD dataset. This paper presents the solution developed by team ":D", which secured the first place in the challenge. Our approach combines Transformers, Gradient Boosting Decision Trees (GBDT), and ensemble techniques in a three-stage recommendation pipeline. We introduce time-aware feature engineering methods and effective data-splitting strategies to address the temporal nature of news articles and improve model generalization. Through extensive experiments and ablation studies, we evaluated our system's performance, achieving an Area Under the ROC Curve (AUC) of 0.8924. Our analysis also examines the effects of data leakage, offering considerations for the practical implementation of news recommendation systems.