CaBaCo4O7 represents a class of ferrimagnets whose structure is built up of CoO4 tetrahedra only, similarly to other members LnBaCo(4)O(7) of the "114" series, forming an alternate stacking of kagome and triangular layers. Neutron powder diffraction reveals, that this compound exhibits the largest distortion within the "114" series, characterized by a strong buckling of the kagome layers. Differently from all other members it shows charge ordering, with Co2+ sitting on two sites (Co2, Co3) and "mixed valent" cobalt "Co3+/Co2+(L) under bar" sitting on two other sites (Co1, Co4). The unique ferrimagnetic structure of this cobaltite at 4 K can be described as the assemblage of ferrimagnetic triple chains (Co1 Co2 Co3) running perpendicular to the kagome layers, ferromagnetically coupled within the layers, and antiferromagnetically coupled with a fourth cobalt species Co4. The lifting of the geometrical frustration toward ferrimagnetism, which appears in spite of the triangular topology of the cobalt lattice, is explained by the very large structural distortion, charge ordering phenomena and large cobalt valence compared to other LnBaCo(4)O(7) oxides.