Modelling spatially autocorrelated detection probabilities in spatial capture-recapture using random effects

Spatial capture-recapture (SCR) models are now widely used for estimating density from repeated individual spatial encounters. SCR accounts for the inherent spatial autocorrelation in individual detections by modelling detection probabilities as a function of distance between the detectors and individual activity centres. However, additional spatial heterogeneity in detection probability may still creep in due to environmental or sampling characteristics. if unaccounted for, such variation can lead to pronounced bias in population size estimates.In this paper, we address this issue by describing three Bayesian SCR models that use generalized linear mixed modelling (GLMM) approach to account for latent heterogeneity in baseline detection probability across detectors with: independent random effects (RE), spatially autocorrelated random effects (SARE) with components of prior covariance matrix modelled as a decreasing function of inter-detector distance, and a two-group finite mixture model (FM) to identify latent detectability classes of each detector. We test these models using a simulation study and an empirical application to a non-invasive genetic monitoring data set of female brown bears (Ursus arctos) in central Sweden.In the simulation study, all three models largely succeeded in mitigating the biasing effect of spatially heterogeneous detection probability on population size estimates. Overall, SARE provided the least biased population size estimates (median RB:-9% - 6%). When spatial autocorrelation in detection probability was high, SARE also performed best at predicting the spatial pattern of heterogeneity in detection probability. At intermediate levels of autocorrelation, spatially-explicit estimates of detection probability obtained with FM were more accurate than those generated by SARE and RE. The empirical example revealed patterns consistent with the results from the simulation study. We found that ignoring spatial heterogeneity in detection probability led to at least 22% lower estimate of bear population size compared to models that accounted for it (i.e., SARE and RE models). When the number of detections per detector is low (<= 1), the GLMMs considered here may require dimension reduction of the random effects by pooling baseline detection probability parameters across neighbouring detectors ("aggregation") to avoid over-parameterization.The added complexity and computational overhead associated with SCR-GLMMs may only be justified in extreme cases of spatial heterogeneity, e.g., large clusters of inactive detectors unbeknownst to the investigator. However, even in less extreme cases, detecting and estimating spatially heterogeneous detection probability may assist in planning or adjusting monitoring schemes.