Constructing and Evaluating Visualisation Task Classifications: Process and Considerations

Abstract

Categorising tasks is a common pursuit in the visualisation research community, with a wide variety of taxonomies, typologies, design spaces, and frameworks having been developed over the last three decades. While these classifications are universally purported to be useful in both the design and evaluation processes and in guiding future research, remarkably little attention has been paid to how these frameworks have-and can be-constructed and evaluated. In this paper we review the task classification literature and report on current practices in construction and evaluation. We consider the stages of task classification construction and identify the associated threats to validity arising at each stage and in response to the different methods employed. We provide guidance on suitable validation approaches in order to mitigate these threats. We also consider the appropriateness of evaluation strategies according to the different aspects of the classification which they evaluate. In so doing, we seek to provide guidance for developers of classifications in determining appropriate construction and evaluation strategies when developing a classification, and also for those selecting between competing classifications for use in the design and evaluation processes.