Retrospective surprise: A computational component for active inference

In the free energy principle (FEP) proposed by Friston, it is supposed that agents seek to minimize the "surprise'' - the negative log (marginal) likelihood of observations (i.e., sensory stimuli) - given the agents' current belief. This is achieved by minimizing the free energy, which provides an upper bound on the surprise. The FEP has been applied to action selection in a framework called "active inference,'' where agents are supposed to select an action so that they minimize the "expected free energy'' (EFE). While EFE can be decomposed into interpretable components such as epistemic value and extrinsic value, it is difficult to understand intuitively how EFE itself is directly related to "surprise'' and what psychological construct is related to EFE itself (as a single quantity). To facilitate the discussion and interpretation of psychological processes underlying active inference, we introduce a computational component termed the "retrospective surprise,'' which is the surprise of an observation after updating the belief given the observation itself. The predicted retrospective surprise (PRS) is mathematically derived from a special case of EFE and provides a lower bound on EFE. We illustrate the properties of EFE and PRS using examples of inference for a binary hidden cause given a binary observation. We discuss how information-seeking behavior is accounted for by EFE and PRS. Our results highlight the role of prior distribution of future observation in EFE. By setting this prior distribution to be fixed irrespective of which action is selected, the epistemic value can exert an influence on EFE, leading to information-seeking behavior. (C) 2020 Elsevier Inc. All rights reserved.