Marker‐based estimates of relatedness and inbreeding coefficients: an assessment of current methods

Inbreeding (F) of and relatedness (r) between individuals are now routinely calculated from marker data in studies in the fields of quantitative genetics, conservation genetics, forensics, evolution and ecology. Although definable in terms of either correlation coefficient or probability of identity by descent (IBD) relative to a reference, they are better interpreted as correlations in marker‐based analyses because the reference in practice is frequently the current sample or population whose F and r are being estimated. In such situations, negative estimates have a biological meaning, a substantial proportion of the estimates are expected to be negative, and the average estimates are close to zero for r and equivalent to FIS for F. I show that although current r estimators were developed from the IBD‐based concept of relatedness, some of them conform to the correlation‐based concept of relatedness and some do not. The latter estimators can be modified, however, so that they estimate r as a correlation coefficient. I also show that F and r estimates can be misleading and become biased and marker dependent when a sample containing a high proportion of highly inbred and/or closely related individuals is used as reference. In analyses depending on the comparison between r (or F) estimates and a priori values expected under ideal conditions (e.g. for identifying genealogical relationship), the estimators should be used with caution.