Modeling Human Behavior in Cyber-Physical-Social Infrastructure Systems

The concept of cyber-physical-social infrastructure systems (CPSISs) has been introduced to offer a paradigm shift necessary for including human-centered, or "social," objectives in traditional cyber-physical systems (CPSs). In this paper we focus on one of the greatest challenges of developing CPSISs: modeling and predicting human behavior. Traditional approaches like rule-based models (e.g., agent-based models) and stochastic methods fall short in capturing the complex, multi-objective, and dynamic nature of human behavior. We propose the novel use of Hierarchical Imitation and Reinforcement Learning to combine the strengths of imitation learning and reinforcement learning for this purpose. This approach offers a cost-effective and efficient way of modeling human behavior, reducing reliance on extensive community surveys and expert opinions and instead supporting the integration of in-situ data. This model imitates high-level human activities through imitation learning using real-world observed data and optimizes low-level actions via reinforcement learning, offering a dynamic and adaptable solution that can generalize to new, unseen environments. We demonstrate this approach through a case study, illustrating its performance in modeling complex human behaviors and decision-making processes within a conference room. This research advances the understanding of human behavior modeling within a CPSIS framework and paves the way to explore new avenues of infrastructure design and management for social benefit.