This paper is devoted to the convergence analysis of the generalized Robin-Neumann schemes introduced in Fernandez et al. (2015, Generalized Robin-Neumann explicit coupling schemes for incompressible fluid-structure interaction: stability analysis and numerics. Internat. J. Numer. Methods Engrg., 101, 199-229) for the coupling of a viscous incompressible fluid with a thick-walled elastic or viscoelastic structure. To this purpose, a representative linearized setting is considered. The methods are formulated within a class of operator splitting schemes which treat implicitly the coupling between the fluid and the solid inertia contributions. This guarantees energy stability. A priori error estimates are derived for all the explicit and semi-implicit variants. The analysis predicts a nonuniformity in space of the splitting error, hence confirming the numerical evidence of Fernandez et al. (2015, Generalized Robin-Neumann explicit coupling schemes for incompressible fluid-structure interaction: stability analysis and numerics. Internat. J. Numer. Methods Engrg., 101, 199-229) for the explicit variants. Besides, the analysis demonstrates that the genesis of this accuracy loss is the spatial nonuniformity of the discrete elastic or viscoelastic solid operator. The theoretical findings are illustrated via a numerical study which shows, in particular, that alternative splitting schemes recently reported in the literature also suffer from these accuracy issues.