Driven by the mass personalization model, online meta-learning has garnered significant attention from resource-constrained agents due to its wide adaptability, continuous learning, and lightweight characteristics. However, as cutting-edge artificial intelligence advances, the intelligence and autonomy of agents are increasingly improving, posing challenges to data synchronization and decision-making consistency in collaborative processes. To this end, this paper proposes a distributed online meta-learning multi-agent collaboration framework based on hybrid parallelism, which meets the needs of synchronous collaboration and asynchronous collaboration in different stages of personalization. To implement this framework, we designed two key algorithms. First, an agent clustering algorithm based on graph theory groups similar agents. Synchronous collaboration within the group satisfies the manufacturing time constraint, while asynchronous collaboration among groups ensures decision consistency. Second, a multi-agent online meta-learning algorithm with gradient tracking monitors global gradients through limited communications, accelerating rapid adaptation to personalization tasks. Finally, we validated our approach through experimental testing on a personalized production platform. The results underscore the effectiveness of the proposed multi-agent collaboration mechanism and implementation algorithms, providing a new solution for multi-agent collaboration based on artificial intelligence in mass personalization environments.