The complexity of design and optimization tasks of modern products which cannot be carried out by a single expert or by a single design team motivated the development of the field of decomposition-based optimization. In general, the process of decomposition-based optimization consists of two procedures: (1) Partitioning the original problem into sub-problems according to the design disciplines, components or other aspects; and (2) Coordinating these sub-problems to guarantee that the aggregation of their optimal solutions results in a feasible and optimal solution for the original problem. Much current work focuses on alternative approaches for these two procedures. For a decomposed problem with a hierarchical structure, the above two procedures work very well and the Analytical Target Cascading method tailored for this type of problems can be used as the coordination method. However, for a more generally decomposed problem with a non-hierarchical structure, there are several factors that might affect the performance of the optimization by decomposition besides the traditional two procedures. In this paper, these factors are identified as: (1) the number of Lagrangian multipliers; (2) the number of decomposition levels (3) the existence and the selection of the master sub-problem. These factors further characterize the structure to solve the decomposed problem: the Solving Structure for decomposition-based optimization. Both mathematical and engineering test problems are used to explore the role of the solving structure. The results show that under the same partition and using the same coordination method, the Augmented Lagrangian Coordination, the performance of the decomposition-based optimization may be largely different in terms of efficiency, accuracy and resource cost. The results highlight the importance of choosing an optimal solving structure after deciding on the procedures of partitioning and coordination. Based on these test results, several suggestions for guidelines on the selection of an optimal solving structure selection are proposed.