Detecting defects on steel surfaces is crucial for ensuring product quality and production safety in industrial settings. Object detection using deep learning, particularly the YOLOv5 model, has become a widely adopted method for this purpose. However, the complex shapes of current steel surface defects pose challenges for precise detection, especially when using low-cost recognition devices with small resolution images. To address these challenges, we integrated the RepBi-PAN fusion network into YOLOv5, enhancing the detection capability for large targets in complex backgrounds. To mitigate issues related to the premature introduction of shallow features and decrease in Precision, we optimized the model structure by incorporating the DenseNet structure into the backbone for improved feature extraction. Additionally, we introduced the Normalized Attention Module (NAM) to enhance the detection capability for small targets. Experimental results demonstrate the effectiveness of the enhanced model, showing a 4.1% increase in mean average precision (mAP), a 3.2% improvement in precision, and a 2.4% enhancement in recall. The improved algorithm outperforms the original in complex backgrounds and recognizing small targets, addressing limitations of the Rep-Bi network. Compared to other YOLO algorithms, our approach achieves optimal values for recall and mAP while maintaining a smaller model size. When compared to YOLOv9, which is the best-performing algorithm in the YOLO series on the dataset used in this study, our model demonstrates several advantages. Specifically, it maintains superior overall performance with fewer parameters and lower computational requirements compared to deeper YOLOv9 variants. Furthermore, when compared to YOLOv9s, our model exhibits better performance in terms of precision, recall, and mAP, while also having fewer GFLOPs, a smaller parameter count, and a reduced model size.