Rectangular loop is one of the most popular transient electromagnetic (TEM) devices, which is widely used in engineering, hydro-geological and ore deposit exploration. However, in theory, circular loop is used to approximately simulate the rectangular loop. Traditionally, this gives rise to a series of problems. Meanwhile, in practice, the rectangular loop is not easy to layout on field especially in complex terrains or mountainous areas. Therefore, we have modified the rectangular loop. The geometry of the loop may now be arbitrarily named, modified-rectangular loop. However, some bias will be caused if standard rectangular-loop parameters are used to invert the field data from modified rectangular loop. And this bias lowers the detection precision of the modified rectangular loop. Therefore, the bias caused by the approximation of circular and modified rectangular loop is analyzed in this paper. Firstly, the response distribution and bias between circular loop and rectangular loop are compared using induced voltage and primary field. Then, the forward method of vertical magnetic field from modified rectangular loop based on coordinate transformation is given. The bias between rectangular loop and modified rectangular loop are then analyzed using vertical magnetic field. From theoretical modeling that there is an indication that changing the loop geometry will result in some error and lower the detection precision.