As ultrasounds are incident upon a rough surface of an object, ultrasonic speckles with random phase and amplitude are formed in the backscattering space. The speckles carry the deformation information of the surface. Moreover, they not only exist in the air, but also in liquid and solid compared to laser speckles. Based on these features, ultrasonic speckle correlation method was previously employed to detect the displacement of an object in a non-contact way, however, in this method, the displacement to be measured must be greater than the average size of the speckles. To break through this limitation, the ultrasonic speckle phase-shifting technique is proposed in this paper. In this approach, a transmitting probe radiates ultrasound to the inner surface of an object with a polished outer surface in water. The speckle signal on the inner surface is received by a focus probe. Then it is digitized and added to a digital reference signal. Their added signal intensity is recorded. Next, the phase of the reference signal is shifted two times by using a computer program. After each phase shifting, the reference signal interferes with the speckle signal. Based on the intensities of three added signals, the phase of the speckle signal can be calculated. When the inner surface has minor displacement, the speckle amplitude can be considered to be unchanged, but its phase is changed. With the same proposed phase-shifting and interfering process, the phase variation of the speckle signal due to the object displacement can be obtained. There is a relationship between the speckle phase variation and its displacement. Therefore, according to the working mode of the probes and their angles facing to the object inner surface, the minor out-of-plane and in-plane displacement of the inner surface of an object can be measured.