Although deep learning models have demonstrated impressive performance in various biomedical image segmentation tasks, their effectiveness heavily relies on a large amount of annotated training data, which can be costly to acquire. Semi-supervised learning (SSL) methods have emerged as a potential solution to mitigate this challenge by leveraging the abundance of unlabeled data. In this paper, we propose a highly effective SSL method for 3D biomedical image segmentation, called Pyramid Pseudo-Labeling Supervision (PPS). The PPS comprises three segmentation networks, forming a pyramid-like network structure. To enforce consistency in the outputs of the unlabeled data, we introduce a novel rectified pyramid consistency (RPC) loss. The PPS learns from the plentiful unlabeled data by minimizing the RPC loss, which ensures consistency between the pyramid predictions and the cycled pseudo-labeling knowledge among the three segmentation networks. Additionally, weak data augmentation is applied to perturb the inputs, further enhancing the consistency of the unlabeled data outputs. Experimental results demonstrate that our method achieves state-of-the-art performance on two publicly available 3D biomedical image datasets.