Statistical analysis of ERA and the quality of research in Australian universities

Purpose - The purpose of this paper is to explore the impact of the Excellence Research in Australia (ERA) process in boosting research quality at Australian universities, this paper presents an analysis of a policy initiative, ERA, and compares the results of its measures as calculated in 2018 with those observed in previous implementation, namely, 2015 and 2012. Design/methodology/approach - Two approaches are implemented in this study; Excellence Index (EI) scores for both cited and peer-reviewed 4 digits FoR codes and citation per paper (CPP) approach for the cited 4 digits FoR codes. Findings - The authors show that the higher education providers' (HEPs') performance in the cited FoRs in ERA in 2018 was improved by 27% compared to that in 2015, and that HEPs' performance in the cited FoR codes in ERA 2015 was improved by 80% compared to that in 2012. A reason for this visibility of research improvement may be due to the universities are simply getting better at reporting outcomes using ERA-driven criteria. Moreover, even though EI scores steadily increased in ERA rounds, there is no significant statistical evidence available of improvement in research quality between two consecutive ERA rounds. Originality/value - These findings underpin the importance of more future research and deep analysis using the other complementary variables, like Relative Citation Impact (RCI), citation centiles and distribution of papers based on the centiles and RCI classes and more transparency and data availability from the Australian Research Council (ARC) site. Given the introduction of the Engagement and Impact Assessment by the ARC to accompany the ERA exercise in 2018, the authors expect that the results of these findings will be useful as well as prompting further debate and scholarship to the relevance and value of the ERA process.