Adaptive importance sampling methods for Bayesian inference in complex systems – PISCES

Many problems in different scientific domains can be described through statistical models that relate observed data to a set of hidden parameters of interest. This kind of statistical models can be found in a broad range of applications such as biology, medicine, econometrics, computer science, artificial intelligence, astronomy, physics, chemistry, communications, earth science, among many others. In the Bayesian framework, the probabilistic estimation of the unknowns is represented by the posterior distribution of these parameters. The posterior allows to deal with the uncertainty of the estimation in a systematic way, compacting the data with the available prior knowledge of the parameters. Bayesian inference has been successfully applied in all the aforementioned disciplines, and there is clear tendency for a wider adoption. However, in most of the realistic models, the posterior is intractable and must be approximated. Monte Carlo methods are computational tools that allow for approximating intractable posteriors by drawing random samples. The problem there is to solve very challenging inferential problems by drawing samples from certain simple distributions and based on them and appropriate computations, conduct estimation, filtering, prediction, model assessment, or model selection, among other statistical tasks. Importance Sampling (IS) is a Monte Carlo methodology that has shown a satisfactory performance in many problems of Bayesian inference. Compared to other Monte Carlo methods, IS methods have sound theoretical properties. However, its use has been mostly restricted to low-dimensional spaces. The reason is that the performance of the IS methods is poor when the proposal distributions used for drawing the samples are not adequately selected. This problem worsens as the dimensionality is increased, due to the so-called curse of dimensionality. In this project, we will research novel adaptive IS-based methods for Bayesian inference in complex systems. We will push the adaptive IS (AIS) methodology so it can be applied to intricate realistic complex systems, achieving a high performance in non-linear high-dimensional models, adjusting automatically the required computational complexity, and still attaining solid theoretical guarantees that we will also analyze. We will test the novel AIS algorithms in three complicated real-world applications with real data in the context of wireless sensor networks, cell biology, and demand forecast in the supply chain. AIS is a flexible and promising methodology for Bayesian inference. By identifying and addressing its current limitations, we will enable its widespread use in complex problems.