In this paper, we introduce a method which can generate a family of growing symmetrical tree networks. The networks are constructed by replacing each edge with a reduced-scale of the initial graph. Repeating this procedure, we obtain the fractal networks. In this paper, we define the average geodesic distance of fractal tree in terms of some integral, and calculate its accurate value. We find that the limit of the average geodesic distance of the finite networks tends to the average geodesic distance of the fractal tree. This result generalizes the paper [Z. Zhang, S. Zhou, L. Chen, M. Yin and J. Guan, Exact solution of mean geodesic distance for Vicsek fractals, J. Phys. A: Math. Gen. 41(48) (2008) 7199-7200] for which the mean geodesic distance of Vicsek fractals was considered.