We present a new approach to shape-based segmentation and tracking of multiple, deformable anatomical structures in cardiac MR images. We propose to use an energy-minimizing geometrically deformable template (GDT) which can deform into similar shapes under the influence of image forces. The degree of deformation of the template from its equilibrium shape is measured by a penalty function associated with mapping between the two shapes. In 2D, this term corresponds to the bending energy of an idealized thin-plate of metal. By minimizing this term along with the image energy terms of the classic deformable model, the deformable template is attracted towards objects in the image whose shape is similar to its equilibrium shape. This framework allows for the simultaneous segmentation of multiple deformable objects using intra- as well as inter-shape information. The energy minimization problem of the deformable template is formulated in a Bayesian framework and solved using relaxation techniques: Simulated Annealing (SA), a stochastic relaxation technique is used for segmentation while Iterated Conditional Modes (ICM), a deterministic relaxation technique is used for tracking. We present results of the algorithm applied to the reconstruction of the left and right ventricle of the human heart in 4D MR images.