From Mice to Men – 24 years of Evaluation in CHI

This paper analyzes trends in the approach to evaluation taken by CHI papers in the last 24 years. A set of papers was analyzed according to our schema for classifying type of evaluation. Our analysis traces papers’ trend in type and scope of evaluation. Findings include an increase in the proportion of papers that include evaluation, and a decrease in the median number of subjects in quantitative studies. We also critique the types of subjects, in particular an over reliance on students, and lack of appropriately gender balanced samples. We contextualize these findings in historical trends as we move from machines intended for the technical elite in laboratories to computers integrated into the daily life of everyone.