Abstract
When there is a distributional shift between data used to train a predictive algorithm and current data, performance can suffer. This is known as the domain adaptation problem. Bootstrap aggregating, or bagging, is a popular method for improving the stability of predictive algorithms, while reducing variance and protecting against overfitting. This article proposes a domain adaptive bagging method coupled with a new iterative nearest neighbor sampler. The key idea is to draw bootstrap samples from the training data in such a manner that their distribution equals that of the new testing data. The proposed approach provides a general ensemble framework that can be applied to arbitrary classifiers in complex domains, including manifolds. We further modify the method to allow for anomalous samples in the test data corresponding to outliers in the training data. Theoretical support is provided and the approach is compared to alternatives in simulations and real-data applications.