creators_name: Smith, Andy T
creators_name: Eysenck, Michael
creators_id: 3788
creators_id: 
type: techreport
datestamp: 2003-01-30
lastmod: 2011-03-11 08:55:08
metadata_visibility: show
title: The correlation between RAE ratings and citation counts in psychology
ispublished: unpub
subjects: comp-sci-stat-model
subjects: peer-review
full_text_status: public
keywords: research assessment exercise, rae, united kingdom, impact factor, citation analysis, scientometrics
note: Westney, Lynn C. Hattendorf (1998) Historical Rankings of Science and Technology: A Citationist Perspective. The Journal of the Association for History and Computing,
Vol. I, No. 1., June 1998
http://mcel.pacificu.edu/history/jahcI1/Westney/Westney.htm

 
abstract: We counted the citations received in one year (1998) by each staff member in each of 38 university psychology departments in the United Kingdom. We then averaged these counts across individuals within each department and correlated the averages with the Research Assessment Exercise (RAE) grades awarded to the same departments in 1996 and 2001. The correlations were extremely high (up to +0.91). This suggests that whatever the merits and demerits of the RAE process and citation counting as methods of evaluating research quality, the two approaches measure broadly the same thing. Since citation counting is both more cost-effective and more transparent than the present system and gives similar results, there is a prima facie case for incorporating citation counts into the process, either alone or in conjunction with other measures. Some of the limitations of citation counting are discussed and some methods for minimising these are proposed. Many of the factors that dictate caution in judging individuals by their citations tend to average out when whole departments are compared.
date: 2002-06
date_type: published
institution: University of London, Royal Holloway
department: Psychology
refereed: FALSE
referencetext: Chapman, A. J. (1989). Assessing research: citation-count
shortcomings. The Psychologist, 8, 336-344.   

Chubin, D. E. & Hackett, E. J. (1990) Augmenting peer review: The place of research evaluation. Chapter 6 of: Peerless Science: Peer Review and U. S. Science Policy, Albany, NY: State University of New York Press 165-190 

Cicchetti, D.  V. (1991). The reliability of peer review for manuscript and grant submissions: A cross-disciplinary investigation. Behavioral and Brain Sciences, 14, 119-186.

Rushton, J. P. (1984). Evaluating research eminence in
psychology: The construct validity of citation counts. Bulletin of The British Psychological Society, 37, 33-36

citation:   Smith, Dr Andy T and Eysenck, Prof Michael  (2002) The correlation between RAE ratings and citation counts in psychology.  [Departmental Technical Report]    (Unpublished)  
document_url: http://cogprints.org/2749/1/citations.pdf