Multi-view subspace clustering have attracted more attention recently due to their promising capabilities to reveal the underlying structure between data points. Nonetheless, most current methods endure high time computational complexity, that results in the inapplicability to medium and large-scale datasets. In addition, attributing to the existence of heterogeneous noise, it is tremendously arduous to study an effective low-dimensional subspace structure directly from the raw data points, leading to underperforming clustering results. To tackle these obstacles, we propose Flexible Anchor-based Multi-view Clustering with Low-rank Decomposition (FAMCL) method that combines the anchor learning with the learnable low-rank matrix factorization strategy. Specifically, the anchor point learning and anchor graph construction are fused into a joint optimization framework, which provides a solid foundation to boost the specific representations within different views. To delve deeper into the underlying structure, a low-rank decomposition strategy is applied, decomposing the anchor graph matrix into two components: an orthogonal matrix and a latent representation. Furthermore, an effective alternating direction iterative method with augmented Lagrangian multiplier is introduced to optimize our model. Extensive experiments on seven standard multi-view datasets demonstrate the advantages of FAMCL over other progressive methods.