We have compared the metallicity (represented by oxygen abundance), X-o, and the dust-to-gas ratio, D, in a sample of dwarf galaxies. For dwarf irregulars (dIrrs) we find a good correlation between the two quantities, with a power-law index of 0.52 +/- 0.25. Blue compact dwarf (BCD) galaxies do not show such a good correlation; in addition, both the dust-to-gas ratio and the metallicity tend to be higher than for dIrrs. We have then developed a simple but physical analytical model for the above relation. Comparing the model results with the data, we conclude that (i) for low values of D, the D - X-o relation is quasilinear, whereas for higher values, the curve strongly deviates from the linear behavior, implying that the commonly used power-law approximation is very poor; (ii) the deviation from the linear behavior depends critically on the parameter chi, the "differential" mass outflow rate from the galaxy in units of the star formation rate, psi; (iii) the shape of the D-X-o curve does not depend on psi, but only on chi; however, the specific location of a given galaxy on the curve does depend on psi; and (iv) the BCD metallicity segregation is the result of a higher psi, together with a significant differential mass outflow. Thus, the lack of correlation can be produced by largely different star formation rates and values of chi in these objects.