In many applications of fuzzy modeling, not only high accuracy but also high interpretability is needed. It is usually difficult to determine the number of fuzzy rules. This paper seeks to address these issues and decrease the computational cost. Considering that pyramid fuzzy systems (PFSs) have been developed to improve the accuracy while simplifying the structure of fuzzy systems. On this basis, a hierarchical cluster-based optimization algorithm of hierarchical pyramid fuzzy system (HPFS) modeling is proposed, and a T-S-type PFS is considered as its subsystem. First, a new clustering method based on the K-means algorithm is applied to partition the input universe in the learning process of the initial structure. Unlike a traditional fuzzy system with a predefined rule number, the number of fuzzy rules (i.e., the cluster size) can be defined automatically. Secondly, a novel hierarchical genetic algorithm (HGA) that produces a tree-like chromosome structure, i.e., a natural structure identical to HPFS that combines several low-dimensional T-S-type PFSs, is proposed. To verify and validate the effectiveness of the proposed method, four experiments are performed, and detailed comparisons with other methods are given.