ConceptGraph: A Formal Model for Interpretation and Reasoning During Visual Analysis

Abstract

In order to discuss the kinds of reasoning a visualization supports and the conclusions that can be drawn within the analysis context, a theoretical framework is needed that enables a formal treatment of the reasoning process. Such a model needs to encompass three stages of the visualization pipeline: encoding, decoding and interpretation. The encoding details how data are transformed into a visualization and what can be seen in the visualization. The decoding explains how humans construct graphical contexts inside the depicted visualization and how they interpret them assigning meaning to displayed structures according to a formal reasoning strategy. In the presented model, we adapt and combine theories for the different steps into a unified formal framework such that the analysis process is modelled as an assignment of meaning to displayed structures according to a formal reasoning strategy. Additionally, we propose the ConceptGraph, a combined graph‐based representation of the finite‐state transducers resulting from the three stages, that can be used to formalize and understand the reasoning process. We apply the new model to several visualization types and investigate reasoning strategies for various tasks.