The aim of this paper is to discuss algebraic rough set and its relationships with convex space, rough set and generalized neighborhood space. Specifically, the notion of algebraic relations is introduced and a pair of lower approximation operator and upper approximation operator are presented. Then, several conditions of algebraic relations such as seriality, reflexivity, (resp., weak, primitive) symmetry and (resp., strong) transitivity are characterized by algebraic approximation operators. Based on this, relationships among algebraic rough sets, convex structures and generalized neighborhood systems are investigated. It is proved that the category of reflexive and transitive algebraic rough spaces is isomorphic to the category of convex spaces. In particular, the category of reflexive, weakly symmetric and transitive algebraic rough spaces is isomorphic to the category of convex matroids and the category of reflexive, weakly symmetric and transitive algebraic generalized neighborhood spaces. © The Author(s), under exclusive licence to Springer-Verlag GmbH Germany, part of Springer Nature 2024.