Based on the experiment of a one-bay, two-story semi-rigid composite steel frame model with cross-stiffened steel plate shear wall under lateral cyclic loads, the interaction between the joint stiffness and cross-stiffened steel plate shear wall, the failure mode and energy dissipation mechanics of the structure were analyzed with regard to the load-carrying capacity, ductility, stiffness and energy dissipation capacity. The results showed that the specimen exhibited excellent ductility, energy dissipation capacity and safety margin. The stiffness degradation of the joints was not serious, and the requirement for ductility was lowered by the infill panels and the collaboration between the frame and the steel plate shear wall. The stiffeners improved the work condition of the steel panels, increased the stiffness and load-carrying capacity of the panels, and reduced the shrinkage of the hysteresis loop and the noise and tremor of the panels. The failure mode of the structure might be induced by buckling of the stiffeners, local buckling and interactive buckling of the infill panels, plastic hinges were formed at the bottom semi-rigid connection of the column, and the in-plane failure of the specimen was bending failure.