Effects on phenotypic variability of directional selection arising through genetic differences in residual variability

In standard models of quantitative traits, genotypes are assumed to differ in mean but not variance of the trait. Here we consider directional selection for a quantitative trait for which genotypes also confer differences in variability, viewed either as differences in residual phenotypic variance when individual loci are concerned or as differences in environmental variability when the whole genome is considered. At an individual locus with additive effects, the selective value of the increasing allele is given by ia/sigma + 1/2 ixb/sigma2, where i is the selection intensity, x is the standardized truncation point, sigma2 is the phenotypic variance, and a/sigma and b/sigma2 are the standardized differences in mean and variance respectively between genotypes at the locus. Assuming additive effects on mean and variance across loci, the response to selection on phenotype in mean is isigma2(Am)/sigma + 1/2 ixcov(Amv)/sigma2 and in variance is icov(Amv)/sigma + 1/2 ixsigma2(Av)/sigma2, where sigma2(Am) is the (usual) additive genetic variance of effects of genes on the mean, sigma2(Av) is the corresponding additive genetic variance of their effects on the variance, and cov(Amv) is the additive genetic covariance of their effects. Changes in variance also have to be corrected for any changes due to gene frequency change and for the Bulmer effect, and relevant formulae are given. It is shown that effects on variance are likely to be greatest when selection is intense and when selection is on individual phenotype or within family deviation rather than on family mean performance. The evidence for and implications of such variability in variance are discussed.