Estimating party positions on immigration: Assessing the reliability and validity of different methods

We provide a systematic assessment of various methods to position political parties on immigration, a policy domain that does not necessarily overlap with left-right and is characterized by varying salience and issue complexity. Manual and automated coding methods drawing on 283 party manifestos are compared - manual sentence-by-sentence coding using a conventional codebook, manual coding using checklists, automated coding using Wordscores, Wordfish and keywords. We also use expert surveys and the Comparative Manifesto Project (CMP), covering the main parties in Austria, Belgium, France, Ireland, the Netherlands, Spain, Switzerland and the United Kingdom, between 1993 and 2013. We find high levels of consistency between expert positioning, manual sentence-by-sentence coding and manual checklist coding and poor or inconsistent results with the CMP, Wordscores, Wordfish and the dictionary approach. An often-neglected method - manual coding using checklists - offers resource efficiency with no loss in validity or reliability.