Effects of Social Structure on Effective Population Size Change Estimates

Most methods currently used to infer the "demographic history of species" interpret this expression as a history of population size changes. The detection, quantification, and dating of demographic changes often rely on the assumption that population structure can be neglected. However, most vertebrates are typically organized in populations subdivided into social groups that are usually ignored in the interpretation of genetic data. This could be problematic since an increasing number of studies have shown that population structure can generate spurious signatures of population size change. Here, we simulate microsatellite data from a species subdivided into social groups where reproduction occurs according to different mating systems (monogamy, polygynandry, and polygyny). We estimate the effective population size (Ne) and quantify the effect of social structure on estimates of changes in Ne. We analyze the simulated data with two widely used methods for demographic inference. The first approach, BOTTLENECK, tests whether the samples are at mutation-drift equilibrium and thus whether a single Ne can be estimated. The second approach, msvar, aims at quantifying and dating changes in Ne. We find that social structure may lead to signals of departure from mutation-drift equilibrium including signals of expansion and bottlenecks. We also find that expansion signals may be observed under simple stationary Wright-Fisher models with low diversity. Since small populations tend to characterize many endangered species, we stress that methods trying to infer Ne should be interpreted with care and validated with simulated data incorporating information about structure. Spurious expansion signals due to social structure can mask critical population size changes. These can obscure true bottleneck events and be particularly problematic in endangered species.