In recent years, increased emphasis has been placed on the self-centering performance of buildings due to the elimination of plastic hinges and a significant reduction in residual deformations, which lowers the cost of post-earthquake repairs and thus allows the building to operate continuously. Previous studies primarily focused on self-centering low-rise reinforced concrete (RC) frames because the effects of higher modes and flexural deformations are more pronounced in medium- and high-rise frames. These effects result in non-uniform height deformation of the frame; in other words, the deformation pattern in buildings' height due to higher modes effect is curved rather than linear. This phenomenon causes problems with self-centering performance and compromises the structure's stability. On the other hand, the reduction of residual drift in mid-rise and high-rise structures is more important than in low-rise structures due to construction's difficulties in rehabilitation after an earthquake. With these interpretations, the need for self-centering action in medium- and high-rise structures is high. This study introduced columns with a high flexural stiffness as semi-rigid elements in numerical models of self-centering RC frames with 5 and 8 stories. Steel angles are used in the beam-column joints as an energy-dissipating device in self-centering frames, and the column-base connection and post-tensioning tendons are also passed through the cross-section of the beam and column for self-centering performance. For the first time, the behavior of self-centering RC frames with and without semi-rigid columns was investigated with incremental dynamic analysis (IDA). According to IDA results of numerical models in OpenSees software, using semi-rigid columns in self-centering RC frames without increasing the frame's overall rigidity reduces the negative effects of higher modes and flexural deformations. In addition, this study shows that using semi-rigid columns in the self-centering RC frames leads to a more uniform story drift ratio. This effect is more significant in the 8-story frame than in the 5-story frame.