Mesangial cells share features with contractile smooth muscle cells and mechanically support the capillary wall. The role of vitamin D compounds and the transforming growth factor-beta (TGF-beta) type II receptor in modulating the smooth muscle phenotype of cultured mesangial cells was examined. Cell proliferation was significantly inhibited by the vitamin D analog 22-oxa-1,25-dihydroxyvitamin D-3 (22-oxacalcitriol; OCT) rather than by 1,25-dihydroxyvitamin D-3 (1,25(OH)(2)D-3) in a dose-dependent manner. OCT-treated early passage mesangial cells (MC-E cells) had increased expression levels of type IV collagen and smooth muscle alpha actin mRNA, but 1,25(OH)(2)D-3-treated MC-E cells did not. The addition of a TGF-beta(1)-neutralizing antibody to the OCT-treated MC-E cells blocked this inhibitory effect for cell proliferation and attenuated the up-regulated mRNA levels. However, after exposure to 1,25(OH),D, or OCT, there was no significant difference in the secretion of active TGF-beta. We next investigated whether TGF-beta type II receptor (RII) was involved in this regulation. OCT treatment significantly increased the expression of the RII mRNA in MC-E cells. These results suggest that the vitamin D analog OCT induces smooth muscle phenotypic alterations and that this phenomenon was mediated through the induction of RII in cultured mesangial cells.