The correlation of substitution effects across populations and generations in the presence of nonadditive functional gene action

Abstract Allele substitution effects at quantitative trait loci (QTL) are part of the basis of quantitative genetics theory and applications such as association analysis and genomic prediction. In the presence of nonadditive functional gene action, substitution effects are not constant across populations. We develop an original approach to model the difference in substitution effects across populations as a first order Taylor series expansion from a “focal” population. This expansion involves the difference in allele frequencies and second-order statistical effects (additive by additive and dominance). The change in allele frequencies is a function of relationships (or genetic distances) across populations. As a result, it is possible to estimate the correlation of substitution effects across two populations using three elements: magnitudes of additive, dominance, and additive by additive variances; relationships (Nei’s minimum distances or Fst indexes); and assumed heterozygosities. Similarly, the theory applies as well to distinct generations in a population, in which case the distance across generations is a function of increase of inbreeding. Simulation results confirmed our derivations. Slight biases were observed, depending on the nonadditive mechanism and the reference allele. Our derivations are useful to understand and forecast the possibility of prediction across populations and the similarity of GWAS effects. In presence of functional non-additive gene action, substitution effects at quantitative trait loci change across genetic backgrounds. This is relevant for genomic prediction and assessing the role of epistasis in evolution. Legarra et al. analytically derive differences of substitution effects across two populations, showing that these differences are functions of: the magnitude of additive, dominance and additive by additive variance; the genetic distance of the populations; and their heterozygosity. They illustrate these differences with simulation and real-life examples from literature.