The primary instability of the flow past rectangular cylinders is studied to comprehensively describe the influence of the aspect ratio AR and of rounding the leading- and/or trailing-edge corners. Aspect ratios ranging between 0.25 and 30 are considered. We show that the critical Reynolds number (Re-c) corresponding to the primary instability increases with the aspect ratio, starting from Re-c approximate to 34.8 for AR = 0.25 to a value of Re-c approximate to 140 for AR = 30. The unstable mode and its dependence on the aspect ratio are described. We find that positioning a small circular cylinder in the flow modifies the instability in a way strongly depending on the aspect ratio. The rounded corners affect the primary instability in a way that depends on both the aspect ratio and the curvature radius. For small AR, rounding the leading-edge corners has always a stabilising effect, whereas rounding the trailing-edge corners is destabilising, although for large curvature radii only. For intermediate AR, instead, rounding the leading-edge corners has a stabilising effect limited to small curvature radii only, while for AR >= 5 it has always a destabilising effect. In contrast, for AR >= 2 rounding the trailing-edge corners consistently increases Re-c. Interestingly, when all the corners are rounded, the flow becomes more stable, at all aspect ratios. An explanation for the stabilising and destabilising effect of the rounded corners is provided.