GENETIC ARCHITECTURE AND POSTZYGOTIC REPRODUCTIVE ISOLATION: EVOLUTION OF BATESON–DOBZHANSKY–MULLER INCOMPATIBILITIES IN A POLYGENIC MODEL

The Bateson–Dobzhansky–Muller model predicts that postzygotic isolation evolves due to the accumulation of incompatible epistatic interactions, but few studies have quantified the relationship between genetic architecture and patterns of reproductive divergence. We examined how the direction and magnitude of epistatic interactions in a polygenic trait under stabilizing selection influenced the evolution of hybrid incompatibilities. We found that populations evolving independently under stabilizing selection experienced suites of compensatory allelic changes that resulted in genetic divergence between populations despite the maintenance of a stable, high‐fitness phenotype. A small number of loci were then incompatible with multiple alleles in the genetic background of the hybrid and the identity of these incompatibility loci changed over the evolution of the populations. For F₁ hybrids, reduced fitness evolved in a window of intermediate strengths of epistatic interactions, but F₂ and backcross hybrids evolved reduced fitness across weak and moderate strengths of epistasis due to segregation variance. Strong epistatic interactions constrained the allelic divergence of parental populations and prevented the development of reproductive isolation. Because many traits with varying genetic architectures must be under stabilizing selection, our results indicate that polygenetic drift is a plausible hypothesis for the evolution of postzygotic reproductive isolation.