Recently, we proposed the so-call RPS (rough polyharmonic splines) basis, which has the optimal accuracy and localization property for the numerical homogenization of divergence form elliptic equation with rough (L^/infty) coefficients. The construction is found by the compactness of solution space. Surprisingly, this basis can be obtained by the reformulation of the numerical homogenization problem as a Bayesian inference problem: given a finite number of observations, the basis is the conditional expectation when the right hand side of the PDE is replaced by a Gaussian random field. This formulation can be applied to general linear integro-differential equations, and can be further extended to finite temperature systems.