Isostructural alloying has been regarded as an important approach to boosting thermoelectric performance of narrow gap semiconductors through thermal conductivity reduction. However, it does harm to carrier mobilities as well because of atomic disorders. Here, we extend the alloying concept from "isostructural" to "quasi-isostructural", which means that the two alloying components are no longer strictly isostructural but structurally closely related. As a proof-of-concept, a series of cubic Cu2SnSe3-tetragonal Cu3SbSe4 quasi-isostructural alloys (Cu2+xSn1-xSbxSe3+x) were synthesized, and their thermoelectric transport properties were systematically studied. No significant loss of carrier mobilities was found during quasi-isostructural alloying, probably due to the structural similarity of the two end members as well as the negligible mass contrasts between Sn and Sb. Surprisingly, a remarkable reduction of lattice thermal conductivity from 1.2 W m(-1) K-1 for Cu2SnSe3/Cu3SbSe4 to 0.4 W m(-1) K-1 for Cu5SnSbSe7 at 673 K was observed, which can hardly be explained from a solid solution perspective. Our X-ray photoelectron spectroscopy analysis unambiguously demonstrate the coexistence of Sn2+ and Sn4+ in the alloying systems, whereas Sn simply exhibits Sn4+ in Cu2SnSe3. We deduce that the minute structural changes of Cu2SnSe3 during quasi-isostructural alloying make Sn discordant, leading to its valence state splitting. The mixed-valence scattering therefore strongly impedes phonon propagation beyond alloying. Altogether, a considerably enhanced ZT approximate to 0.4 is realized in the sample with x = 0.2, which is 150% improvement over the x = 0 sample. By selecting the x = 0.5 sample as a representative, a preliminary doping study was performed but seems unsuccessful because of the varying Sn2+/Sn4+ ratio. We expect higher thermoelectric performance in those quasi-isostructural alloys if their carrier concentration can be fully optimized.