Various statistical procedures related to linear prediction and optimal filtering are developed for general, irregularly sampled, data sets. The data set may be a function of time, a spatial sample, or an unordered set. In the case of time series, the underlying process may be low-frequency divergent (weakly nonstationary). Explicit formulas are given for (i) maximum likelihood reconstruction (interpolation) with estimation of uncertainties, (ii) reconstruction by unbiased estimators (Gauss-Markov), (iii) unconstrained Monte Carlo realization of the underlying process, (iv) Monte Carlo realizations constrained by measured data, and (v) simultaneous reconstruction and determination of unknown linear parameters.