What Would a Theory of Data Visualization Look Like?

Wrestling with the idea of having more theory in visualization

Lately, I have been thinking quite a lot about the way we do research in visualization. Most of us seem to operate under a pretty fixed model: build different versions of a visualization by varying some of its properties and test a hypothesis about which one “works best”.

One problem I see with this approach is that this kind of experimental approach almost never seems to originate from the desire to test a theory. The experiment does not aim at validating elements of a theory, but more often than not at establishing whether technique A is better than technique B (or variations of this template) without any opportunity to go beyond the result of the specific test. This is not to say that this is necessarily bad or wrong, but one problem I see with this model is that we end up having a lot of piecemeal results that never seem to build a unifying and coherent picture. Take the endless debate on whether one should use pie charts or bar charts. A large number of papers have been written on this very specific question and the debate keeps going. But even assuming we will finally get to the bottom of this issue, how is this question going to help us design better visualizations beyond the narrow scope of bars and pies? Again, don’t get me wrong, this kind of research is interesting and probably necessary but what is missing is a complementary approach where people come up with theories and develop experiments to test those theories.

Examples of theories

Let me clarify what I mean by “theory” here. The best way is to provide a couple of examples.

A first example is the ranking of visual variables. The theory goes a little like this (I am simplifying):

• Every visualization uses a set of visual channels to communicate quantitative information (position, length, area size, color intensity, etc.);

• Some channels are better than others in terms of how precisely people can extract quantitative information out of them;

• Visualizations that use better channels are more effective.

In one of my previous posts, I have criticized this theory and I do think it has some relevant limitations but at the same time, it’s a great example of how powerful theories can be. What I like about this theory is that it can be applied to many different visualization techniques (all visualizations where quantitative information is displayed) and it is also very practical (check which channels you are using and see if you can use more of the better ones). But even more than these two properties, what I like is that the theory is potentially predictive and verifiable. Following the theory, you can produce conjectures on whether visualization A is better than visualization B according to which set of channels they use. And then, after producing the prediction, you can build experiments to verify it. One can systematically manipulate the channels, make predictions according to the theory, and then verify whether the results are consistent with what the theory predicts. Unfortunately, I don’t think this specific theory has ever been tested this way. As far as I know, the existing experimental work focuses on verifying which channels work best and not necessarily on verifying that visualizations that use the best channels are more effective (and one big problem here is to define what “more effective” means). But in principle, it could be tested using the procedure I outlined above.

The second example of theory in visualization is the one we (implicitly) use to decide how to communicate data with color. The theory goes like that (again, I am simplifying a lot here):

• Humans perceive colors according to three properties, color hue (the name of the color), color intensity (the brightness of the color), and color saturation (the vividness of the color);

• Color hue is perceived as different qualities or categories, whereas color intensity and saturation are perceived as quantities or amounts;

• Visualizations that use hue for categorical information and intensity or saturation for quantitative information are more effective than those that “break these rules”.

This is quite nice. It applies to any visualization that uses color as one of the channels, it provides very specific guidance and it is predictive and verifiable. Let me stress this again, my characterization above is a gross oversimplification of how color works but it’s useful to get a better sense of what I mean by theory in this context.

Theories and experiments

Now contrast the way we usually do research in visualization and the way theory works. In visualization, we typically have two situations: validating a new technique or comparing existing techniques. The first case happens when we develop a new technique: in order to demonstrate its validity, we compare it with the state of the art (e.g., horizon charts vs. standard line charts). The second case happens when we want to put some order and investigate which solution, among the existing ones, works best for a given data visualization problem (bars vs. pies). In both cases what you produce as output is just the results. You can mostly only state whether A is better than B, and that’s it. It’s very hard to extrapolate beyond the specific results of the experiment.

I suspect that what is potentially deceiving is the fact that experiments aimed at testing a theory look very similar to those we use to perform more narrow A/B comparisons. This is because when you test a theory you necessarily have to come up with an experiment that compares instantiations of the theory that vary systematically along with factors the theory employs. So, at a superficial level, they look the same, but in practice, they are very different.

If you look back at the examples of theories I described above you will recognize a few relevant properties. Theories apply to a very large class of visualizations and are predictive. Let me state this differently because this is a crucial difference: theories allow you to predict the effectiveness of a visualization without having to test it again every time you have a new solution. Theories are predictive and provide very useful guidance and shortcuts. If a theory works well, you do not need to build an experiment to decide which solution works best for every new situation, you can just follow its precepts. Of course, no theory is ever perfect and verification may still be needed, but the beauty of a theory is that it can guide designers in making informed decisions that are good first approximations for a given problem.

Taking the two examples I described above, with the ranking of channels theory you can decide which representation to use according to which channels it employs, and with the color theory you can decide which colors to use according to what type of information you need to communicate.

What kind of theory for visualization?

Now, after describing what I mean by theory in visualization, one interesting question is: what kind of theories could we build in visualization? And what would be the major challenges in building such theories?

Let me try to draw the contours of a theory that I would like to see. Just as a speculative exercise (at the risk of embarrassing myself). Please do not take this too seriously.

The main purpose of my theory would be to predict the effectiveness of a visualization and to provide guidance in designing effective visualizations (I am not defining effectiveness on purpose yet).

The theory would identify the main elements that affect the outcome and the relationships between these elements and the outcome. More precisely, as a first approximation, we will need elements that describe properties of the visualization, properties of the reader, and properties of the medium.

This is a good time to stop for a moment and go back to the question of effectiveness. If we want to be able to define (and hopefully measure) effectiveness we have to have some notion of “purpose”. If we do not agree on what is the purpose of visualization, there is simply no way to agree on what is effective. And this is already an area where we stumble on a huge problem because there is very little agreement on what visualization is for. If you ask ten different experts, you will probably receive ten different answers, each meaningful but also most probably idiosyncratic. In the rest of this post, I focus on one possible definition of purpose, but my larger point is not that this definition is better than others but that purpose must be defined and agreed upon before we embark on the task of building a coherent theory of data visualization.

One way to define the purpose of visualization is this: the purpose of visualization is to communicate information and, as a corollary, the effectiveness of a visualization can be expressed in terms of how much information it communicates and the quality (accuracy, completeness, etc.) of such information. Note that my statement above contains a relevant ambiguity: communication is about the information the receiver acquires, not an intrinsic property of a representation. So, when I write that effectiveness depends on how much information is communicated and its quality, I am describing properties that exist in the receiver’s mind, not in the visual representation. This (admittedly restrictive) definition assumes that there is some information that somebody wants to communicate to somebody else and as such it restricts its applicability to a subset of the goals that visualization covers in existing applications (for example, visualization can be used for data analysis, where a specific message to communicate does not exist yet).

If we focus on this specific purpose and definition we can trace a few more lines of this theory. We have information a sender wants to communicate, a receiver, a visual representation, and a medium (this starts getting similar to Shannon’s information theory but I’ll not go there in this post - Min Chen from Oxford has a fantastic account of his speculations on the use of information theory in visualization). Each of these elements has properties that can be used to characterize the theory. We can describe properties of the information (note that in visualization pedagogy we tend to talk about properties of data rather than properties of information), properties of visual representations, properties of the receiver, and properties of the medium. Each of these properties can have an effect on effectiveness and properties can interact in relevant ways. So for example, you may have that the relationship that ties a given visual property to effectiveness is mediated by the properties of the receiver so that different solutions are needed for people with different characteristics (not a crazy assumption to have).

While causal relationships can be derived from specific properties of the elements I laid out above, it’s more common (and reasonable) to characterize the relationship that ties information with visual representation rather than the individual properties of the information and the individual properties of the visualization. In other words, it is not only important which visual properties are used but more whether these properties are appropriate for the information that one wants to convey.

Of course, one common problem with building theories is that the more elements we intend to capture the harder it is to use. So even if in principle one could build a theory with all of the elements I mentioned above, it may be more practical and useful to focus on a subset of these elements. Traditionally, visualization research focuses on the mapping between properties of information and properties of the visual representation, and as a first approximation, I think it’s a reasonable focus. That said, it’s amazing how little we know about how visualization effectiveness is modulated by individual differences and other contextual factors such as the medium, the audience, cultural aspects, etc.

Conclusion

As you can see I did not really provide a theory of visualization, but I hope I have sketched an outline able to provide some inspiration to the reader. Who knows ... maybe you will be tempted to build a theory after reading this post!

This is all I have to share about theory so far. Before concluding I just want to mention I have been highly influenced in writing this post by Robert Dubin’s book “Theory Building”, a hidden gem I discovered a while ago thanks to Kasper Hornbæk. I had a few conversations with Kasper last year and he helped me a lot to think more systematically about what a theory is and how new theories could be built in human-computer interaction and in visualization.

I am looking forward to hearing from your what you think about theories in visualization. I am sure I have probably missed a lot. If you have anything to share please let me know!

—

P.s. I want to thank Jessica Hullman for reading a preview of this post right before I published it. Interestingly, she published a blog post titled “Taking theory more seriously in psychological science” right when I was finalizing my post. She has a number of great observations there and links stemming from a workshop on theory she recently attended. Make sure to also check out her own paper on the need for theory in EDA.