Predictability limit of partially observed systems

被引:6
作者
Abeliuk, Andres [1 ,3 ]
Huang, Zhishen [2 ]
Ferrara, Emilio [1 ]
Lerman, Kristina [1 ]
机构
[1] Univ Southern Calif, Informat Sci Inst, Marina Del Rey, CA 90292 USA
[2] Univ Colorado, Boulder, CO 80302 USA
[3] Univ Chile, Dept Comp Sci, Santiago, Chile
关键词
INFORMATION-THEORY; BIG DATA; PREDICTION; MODEL;
D O I
10.1038/s41598-020-77091-1
中图分类号
O [数理科学和化学]; P [天文学、地球科学]; Q [生物科学]; N [自然科学总论];
学科分类号
07 ; 0710 ; 09 ;
摘要
Applications from finance to epidemiology and cyber-security require accurate forecasts of dynamic phenomena, which are often only partially observed. We demonstrate that a system's predictability degrades as a function of temporal sampling, regardless of the adopted forecasting model. We quantify the loss of predictability due to sampling, and show that it cannot be recovered by using external signals. We validate the generality of our theoretical findings in real-world partially observed systems representing infectious disease outbreaks, online discussions, and software development projects. On a variety of prediction tasks-forecasting new infections, the popularity of topics in online discussions, or interest in cryptocurrency projects-predictability irrecoverably decays as a function of sampling, unveiling predictability limits in partially observed systems.
引用
收藏
页数:10
相关论文
共 55 条
[1]   Beyond prediction: Using big data for policy problems [J].
Athey, Susan .
SCIENCE, 2017, 355 (6324) :483-485
[2]   Exposure to opposing views on social media can increase political polarization [J].
Bail, Christopher A. ;
Argyle, Lisa P. ;
Brown, Taylor W. ;
Bumpus, John P. ;
Chen, Haohan ;
Hunzaker, M. B. Fallin ;
Lee, Jaemin ;
Mann, Marcus ;
Merhout, Friedolin ;
Volfovsky, Alexander .
PROCEEDINGS OF THE NATIONAL ACADEMY OF SCIENCES OF THE UNITED STATES OF AMERICA, 2018, 115 (37) :9216-9221
[3]   Permutation entropy: A natural complexity measure for time series [J].
Bandt, C ;
Pompe, B .
PHYSICAL REVIEW LETTERS, 2002, 88 (17) :4
[4]   Entropy of interval maps via permutations [J].
Bandt, C ;
Keller, G ;
Pompe, B .
NONLINEARITY, 2002, 15 (05) :1595-1602
[5]   Interrupted time series regression for the evaluation of public health interventions: a tutorial [J].
Bernal, James Lopez ;
Cummins, Steven ;
Gasparrini, Antonio .
INTERNATIONAL JOURNAL OF EPIDEMIOLOGY, 2017, 46 (01) :348-355
[6]   Predicting poverty and wealth from mobile phone metadata [J].
Blumenstock, Joshua ;
Cadamuro, Gabriel ;
On, Robert .
SCIENCE, 2015, 350 (6264) :1073-1076
[7]   INTERVENTION ANALYSIS WITH APPLICATIONS TO ECONOMIC AND ENVIRONMENTAL PROBLEMS [J].
BOX, GEP ;
TIAO, GC .
JOURNAL OF THE AMERICAN STATISTICAL ASSOCIATION, 1975, 70 (349) :70-79
[8]  
Chatfield C., 2000, Time-Series Forecasting, DOI [10.1201/9781420036206, DOI 10.1201/9781420036206]
[9]   Natural Experiments: An Overview of Methods, Approaches, and Contributions to Public Health Intervention Research [J].
Craig, Peter ;
Katikireddi, Srinivasa Vittal ;
Leyland, Alastair ;
Popham, Frank .
ANNUAL REVIEW OF PUBLIC HEALTH, VOL 38, 2017, 38 :39-56
[10]  
DelSole T, 2004, J ATMOS SCI, V61, P2425, DOI 10.1175/1520-0469(2004)061<2425:PAITPI>2.0.CO