Abstract
Estimates of the locations and effects of quantitative trait loci (QTL) can be obtained by regressing phenotype on marker genotype. Under certain basic conditions, the signs of regression coefficients flanking QTL must be the same. There is no guarantee, however, that the signs of the regression coefficient estimates will be the same. We use sign inconsistency to describe the situation in which there is disagreement between the signs of the estimated regression coefficients flanking QTL. The presence of sign inconsistency can undermine the effectiveness of QTL mapping strategies that presume intervals whose markers have regression coefficient estimates of differing sign to be devoid of QTL. This article investigates the likelihood of sign inconsistency under various conditions. We derive an analytic expression for the approximate probability of sign inconsistency in the single-QTL case. We also examine sign inconsistency probabilities when multiple QTL are present through simulation. We have discovered that the probability of sign inconsistency can be unacceptably high, even when the conditions for QTL detection are otherwise quite favorable.