Ledo, D; Houben, S; Vermeulen, J; Marquardt, N; Oehlberg, L; Greenberg, S; (2018) Evaluation strategies for HCI Toolkit research. In: Mandryk, R and Hancock, M and Perry, M and Cox, A, (eds.) CHI '18, Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. (pp. 36:1-36:17). ACM: New York, USA.

Preview

Text pn1203-ledo-2.pdf - Accepted Version Download (1MB) | Preview

Abstract

Toolkit research plays an important role in the field of HCI, as it can heavily influence both the design and implementa-tion of interactive systems. For publication, the HCI commu-nity typically expects toolkit research to include an evalua-tion component. The problem is that toolkit evaluation is challenging, as it is often unclear what 'evaluating' a toolkit means and what methods are appropriate. To address this problem, we analyzed 68 published toolkit papers. From our analysis, we provide an overview of, reflection on, and dis-cussion of evaluation methods for toolkit contributions. We identify and discuss the value of four toolkit evaluation strat-egies, including the associated techniques that each employs. We offer a categorization of evaluation strategies for toolkit researchers, along with a discussion of the value, potential limitations, and trade-offs associated with each strategy.

Download activity - last month

Export as XMLCSVJSON

Download activity - last 12 months

Export as JSONXMLCSV

Downloads by country - last 12 months

Export as CSVXMLJSON

Archive Staff Only

View Item