On sampling from multivariate distributions

Abstract

Let X 1, X 2,..., X n be a set of random variables. Suppose that in addition to the prior distributions of these random variables we are also given linear constraints relating them. We ask for necessary and sufficient conditions under which we can efficiently sample the constrained distributions, find constrained marginal distributions for each of the random variables, etc. We give a tight characterization of the conditions under which this is possible. The problem is motivated by a number of scenarios where we have separate probabilistic inferences in some domain, but domain knowledge allows us to relate these inferences. When the joint prior distribution is a product distribution, the linear constraints have to be carefully chosen and are crucial in creating the lower bound instances. No such constraints are necessary if arbitrary priors are allowed. © 2011 Springer-Verlag.