Looking for the right mate in diploid species: how dominance relationships affect population differentiation in sexual trait?

Divergence between populations for a given trait can be driven by sexual selection, interacting with migration behaviour. Mating preference for different phenotypes may indeed lead to specific migration behaviour, with departures from populations where the preferred trait is rare. Such preference can then trigger the emergence and persistence of differentiated populations, even without any local adaptation. However the genetic architecture underlying the trait targeted by mating preference may have a profound impact on population divergence. In particular, dominance between alleles encoding for divergent phenotypes can interfere in such differentiation process. Using a diploid model of trait determining both mating success and migration rate, we explored differentiation between two connected populations, assuming either co-dominance or strict dominance between alleles. The model assumes that individuals prefer mating with partners displaying the same phenotype and therefore tend to move to the other population when their own phenotype is rare. We show that the emergence of differentiated populations in this diploid moded is limited as compared to results obtained with the same model assuming haploidy. When assuming co-dominance, differentiation arises only when migration is limited as compared to preference. Such differentiation is less dependent on migration when assuming strict dominance between haplotypes. Dominant alleles frequently invade populations because their phenotype is more frequently expressed, resulting in higher local mating success and rapid decrease in migration. However, depending on the initial distribution of alleles, this advantage associated with dominance (i.e. Haldane's sieve) may lead to fixation of the dominant allele throughout both populations. Depending on the initial distribution of heterozygotes in the two populations, persistence of polymorphisms within populations can also occur because heterozygotes displaying the predominant phenotype benefit from high mating success. Altogether, our results highlight that heterozygotes' behaviour has a strong impact on population differentiation and stress out the need of diploid models of differentiation and speciation driven by sexual selection.