Inertial navigation system (INS) based on Micro-Electro-Mechanical System (MEMS) has been used in the both civil and military fields. However, large errors such as bias errors restrict them from being applied widely. Rotating modulation technique is a mature method which can modulate the constant errors of MEMS. In this way, all the devices' errors can be modulated if INS has two axes. Dual-axis INS has different forms of organizational structures which will affect the error propagation characteristics. Traditional dual-axis RINS cannot modulate all the devices' errors at the same time, in this paper, dual-unit RINS is studied and an improved rotation scheme is proposed. Theoretical analysis proves that the performance of the reported scheme is better than the traditional dual-axis RINS. Simulation results based on Matlab show that the performance of the dual-unit RINS with proposed rotation scheme is much better than the traditional dual-axis RINS with an eight-position scheme, whose position accuracy is better than the strap-down mode.What is more, the dual-unit RINS can improve the fault-tolerant performance of the system.