In a variety of threading methods, often poorly ranked (low z-score) templates have good alignments. Here, a new method, TASSER_low-zsc that identifies these low z-score ranked templates to improve protein structure prediction accuracy, is described. The approach consists of clustering of threading templates by affinity propagation on the basis of structural similarity (thread_cluster) followed by TASSER modeling, with final models selected by using a TASSER_QA variant. To establish the generality of the approach, templates provided by two threading methods, SP3 and SPARKS(2), are examined. The SP3 and SPARKS(2) benchmark datasets consist of 351 and 357 medium/hard proteins (those with moderate to poor quality templates and/or alignments) of length <= 250 residues, respectively. For SP3 medium and hard targets, using thread cluster, the TM-scores of the best template improve by similar to 4 and 9% over the original set (without low z-score templates) respectively; after TASSER modeling/refinement and ranking, the best model improves by similar to 7 and 9% over the best model generated with the original template set. Moreover, TASSER_low-zsc generates 22% (43%) more foldable medium (hard) targets. Similar improvements are observed with low-ranked templates from SPARKS(2). The template clustering approach could be applied to other modeling methods that utilize multiple templates to improve structure prediction.