Four new transition metal carboxyphosphonates with 2D and 3D supramolecular structures, namely, Fe-2[(HL)(H2O)] (1), Fe(H4L)(2) (2), Zn(H3L) (3) and Zn-2(HL) (4) (H5L = 4-HO2C-C6H4-CH2N(CH2PO3H2)(2)), have been synthesized under hydrothermal conditions. For compound 1, {FeO5N} and {CPO3} polyhedra are interconnected into a 2D layer in the bc-plane via corner-sharing. Then the adjacent layers are further assembled into a 3D supramolecular structure through pi-pi stacking interactions. Compound 2 shows a 3D supramolecular structure. {FeO6} and {CPO3} polyhedra are interconnected into a 2D layer in the bc-plane via corner-sharing, which is further linked through pi-pi stacking interactions to form a 3D supramolecular structure. The overall structure of compound 3 can be described as a 2D supramolecular structure. The {ZnO4} polyhedra are interconnected by {CPO3} tetrahedra via corner-sharing to form a 1D chain. These neighboring metal phosphonate chains are connected through hydrogen bonding interactions to give rise to a 2D supramolecular structure in the ac-plane. In compound 4, {ZnO3N} and {ZnO4} polyhedra are interconnected by {CPO3} tetrahedra via corner-sharing and edge-sharing to form a 2D inorganic layer in the bc-plane, which is further linked through p-p stacking interactions to form a 3D supramolecular structure. The surface photovoltage properties of compounds 1-2 and luminescence properties of compounds 3-4 have been investigated. More interestingly, compound 4 is selective for sensing DMF and acetone.