We study quantum isometry groups, denoted by Q(Gamma, S), of spectral triples on C-r*(Gamma) for a finitely generated discrete group Gamma coming from the word-length metric with respect to a symmetric generating set S. We first prove a few general results about Q(Gamma, S) including: For a group Gamma with polynomial growth property, the dual of Q(Gamma, S) has polynomial growth property provided the action of Q(Gamma, S) on C-r* (Gamma) has full spectrum. Q(Gamma, S) congruent to QISO((Gamma) over cap, d) for any discrete abelian group Gamma, where d is a suitable metric on the dual compact abelian group (Gamma) over cap. We then carry out explicit computations of Q(Gamma, S) for several classes of examples including free and direct product of cyclic groups, Baumslag-Solitar group, Coxeter groups etc. In particular, we have computed quantum isometry groups of all finitely generated abelian groups which do not have factors of the form Z(2)(k) or Z(4)(l) for some k, l in the direct product decomposition into cyclic subgroups.