Divergent ecological selection maintains species boundaries despite gene flow in a rare endemic tree, Quercus acerifolia (maple-leaf oak)

Strong gene flow from outcrossing relatives tends to blur species boundaries, while divergent ecological selection can counteract gene flow. To better understand how these two forces affect the maintenance of species boundaries, we focused on a species complex including a rare species, maple-leaf oak (Quercus acerifolia), which is found in only four disjunct ridges in Arkansas. Its limited range and geographic proximity to co-occurring close relatives create the possibility for genetic swamping. In this study, we gathered genome-wide single nucleotide polymorphisms (SNPs) using restriction-site-associated DNA sequencing (RADseq) from 190 samples of Q. acerifolia and three of its close relatives, Q. shumardii, Q. buckleyi, and Q. rubra. We found that Q. shumardii and Q. acerifolia are reciprocally monophyletic with low support, suggesting incomplete lineage sorting, introgression between Q. shumardii and Q. acerifolia, or both. Analyses that model allele distributions demonstrate that admixture contributes strongly to this pattern. Populations of Q. acerifolia experience gene flow from Q. shumardii and Q. rubra, but we found evidence that divergent selection is likely maintaining species boundaries: 1) ex situ collections of Q. acerifolia have a higher proportion of hybrids compared to the mature trees of the wild populations, suggesting ecological selection against hybrids at the seed/seedling stage; 2) ecological traits co-vary with genomic composition; and 3) Q. acerifolia shows genetic differentiation at loci hypothesized to influence tolerance of radiation, drought, and high temperature. Our findings strongly suggest that in maple-leaf oak, selection results in higher divergence at regions of the genome despite gene flow from close relatives.