Deep-learning-based object detection has been increasingly attractive in various intelligent edge applications, including remote sensing and autonomous driving. However, achieving an optimal tradeoff between computing efficiency and detection accuracy is challenging. YOLO-series networks provide fast and lightweight detection but often sacrifice accuracy. In this work, we propose the multibranch cascading aggregation YOLO (MCA-YOLO) model, designed for multiscenario object detection tasks on edge devices. MCA-YOLO enhances detection accuracy by learning comprehensive feature information while maintaining computational efficiency through three key components: multibranch spatial pyramid pooling (MSPP), Ghost-convolution (GC)-based efficient layer aggregation network (G-ELAN), and hierarchical aggregation neck (HAN) that integrates MSPP and G-ELAN with GC. We train and validate MCA-YOLO using four benchmark datasets: VOC, COCO, SIMD, and VisDrone. The experimental results demonstrate significant improvements in detection accuracy and inference speed. Besides, we deploy the MCA-YOLO on a Jetson Xavier NX edge device embedded in an unmanned aerial vehicle (UAV), creating a remote sensing system capable of real-time, high-accuracy object detection. Our code is publicly available at https://github.com/lawlawCodes/MCA-YOLO.