Hydronic heating and cooling systems are growing in complexity as the buildings and other infrastructure systems they serve demand greater efficiency and become larger. Increasing the scale and interconnectivity of these systems yields higher probabilities of component failure along with computational tractability issues. This paper presents a distributed hydronic control architecture featuring scalable computations and resiliency to component failure in both the cyber and physical domains. In this agent-based control system, an agent is defined as a set of colocated components in the cyber and physical domains, which together have sensations, actions, and/or goals. The proposed architecture consists of three types of agents: pumps, valves, and loads. Respectively, these agents solve a convex constraint satisfaction problem, a steady-state hybrid control problem, and a scalar bilinear model predictive control problem. The network on which these agents transfer thermal energy and on which they communicate are codesigned to share the same graph representation. The resulting coordinated behavior achieves the global control objective: maintaining the thermal loads at safe temperatures while minimizing pump power. Analytical analysis, simulations, and laboratory experiments are employed to investigate the controller. The results demonstrate steady-state robustness to component failures in the cyber-physical domains, superiority to benchmarks, and are suitable although not guaranteed to achieve optimal real-time performance.