Psychophysical experiments have demonstrated that it is possible to simultaneously perceive binocular depth and rivalry in plaids (A. Buckthought & H. R. Wilson, 2007). Here, we used fMRI at 3T to image activity in the visual cortex while human subjects perceived depth and rivalry from plaids. Six subjects performed either a rivalry or depth task. The spatial frequencies of the near-vertical and diagonal components were, respectively: 2.5, 6.4 cpd; 6.4, 2.5 cpd; or 6.4, 6.4 cpd. The network of activated cortical areas was very similar for the depth compared to the rivalry task. Nevertheless, regions of superior and inferior parietal cortices (including intraparietal sulcus) were activated more during the depth than the rivalry task, independent of spatial frequency, whereas a bias toward rivalry was seen in a lateral occipital region, superior temporal sulcus, and retrosplenial and ventral temporal cortices. Several retinotopic areas in the visual cortex showed a preference for the task with the higher (V1, V2, V3) or lower spatial frequency component (MT+), regardless of the depth or rivalry condition. Our results indicate that depth and rivalry are processed in a similar network of cortical areas and are perceived simultaneously by coexisting in different spatial channels. These results place constraints on binocular vision models.