Large-scale coordination and control problems in cyber-physical systems are often expressed within the networked optimization model. While significant advances have taken place in optimization techniques, their widespread adoption in practical implementations has been impeded by the complexity of internode coordination and lack of programming support for the same. Currently, application developers build their own elaborate coordination mechanisms for synchronized execution and coherent access to shared resources via distributed and concurrent controller processes. However, they typically tend to be error prone and inefficient due to tight constraints on application development time and cost. This is unacceptable in many CPS applications, as it can result in expensive and often irreversible side-effects in the environment due to inaccurate or delayed reaction of the control system. This article explores the design of a distributed shared memory (DSM) architecture that abstracts the details of internode coordination. It simplifies application design by transparently managing routing, messaging, and discovery of nodes for coherent access to shared resources. Our key contribution is the design of provably correct locality-sensitive synchronization mechanisms that exploit the spatial locality inherent in actuation to drive faster and scalable application execution through opportunistic data parallel operation. As a result, applications encoded in the proposed Hotline Application Programming Framework are error free, and in many scenarios, exhibit faster reactions to environmental events over conventional implementations. Relative to our prior work, this article extends Hotline with a new locality-sensitive coordination mechanism for improved reaction times and two tunable iteration control schemes for lower message costs. Our extensive evaluation demonstrates that realistic performance and cost of applications are highly sensitive to the prevalent deployment, network, and environmental characteristics. This highlights the importance of Hotline, which provides user-configurable options to trivially tune these metrics and thus affords time to the developers for implementing, evaluating, and comparing multiple algorithms.
