What Makes a Data Visualization Confusing?

Why are some visualizations so easy to understand at first sight while others seem to be some sort of graphical puzzle? Why do some visualizations need to be relearned every time you see them again, whereas others need only one simple initial exposure?

The more I think about these questions, the more I realize we don’t have a good understanding of what makes a visualization confusing. Without pretending to give a complete account of what makes some visualizations more confusing than others, I believe there are two distinct phenomena in this space that often get conflated: novelty and something I’ll tentatively call cognitive incongruence.

Whenever I talk about this problem, there is always someone getting back to me with something to the effect of: “you know bar charts used to be confusing, right?” implying that the source of confusion with certain representations stems exclusively from a lack of sufficient exposure. While, for sure many representations are confusing only at first sight and have a learning curve, it is also clear to me that some representations are confusing no matter how many times you use them. In other words, confusion can stem both from novelty (one needs to learn how to read a new representation) and visual mappings that are inherently hard to make sense of (regardless of how often one gets exposed to them).

Judging from my personal experience, every time I see a new connected scatter plot, I have a hard time figuring out what information I can extract out of it. It doesn’t matter how many connected scatter plots I have seen before; for me, they always lead to confusion.

Here is an example of connected scatter plot (took from PolicyViz, thanks Jon):

Can you make sense of it? I personally have to go through a lot of effort to figure out how to read it and what it’s telling me.

Another visualization I have to relearn every time I see it is the famous IPCC burning embers. I have seen this visualization hundreds of times, yet every time I see it again, I have to rediscover the proper way to read it.

Here is how the burning embers looks like:

Can you figure out how to read it? Believe it or not, as I am writing this I still have problems figuring out how I am supposed to read it.

I have heard similar reactions regarding visualizations like parallel coordinates, chord diagrams, and multidimensional projection plots, even though I am not sure to what extent confusion with these representations depends more on lack of exposure, clutter, or inherent limitations of the visual metaphors they employ.

If we set aside the clear case of visual representations that are confusing only at first because they are novel, it would be interesting to understand better what makes some graphs confusing even after multiple exposures. It seems to me there could be interesting insights to gain from such an exploration.

If I had to come up with a framework to start thinking about the problem, I would look into mental models. Mental models are simplified mental representations of how things work (I posted something about the role of mental models in visualization a while back here). We build mental models of everything we interact with, and for sure, we build mental models of how data visualizations work. The interesting thing about mental model formation is that, as with any other type of knowledge we acquire, mental models stem both from the information we receive and the information we already have stored in our heads.

If we look at this problem under the lens of mental models, we can hypothesize that confusing graphs are visual representations for which it is hard to build an accurate mental model; that is, the model we make does not match how the graph works.

One hypothesis I have is that they evoke the wrong analogies, either because there is a mismatch between the visual representation and the information they represent or because there is a clash with mental models of similar representations we have acquired in the past.

An example of the first case is visualizing quantities with the area of a symbol and representing large values with small areas and small values with large areas. While such mapping is perfectly possible from the technical standpoint, it breaks the metaphor we derive from our experience with objects in the world, where bigger objects normally correspond to “more” of something (weight, count, etc.)

An example of the second case is the burning embers I mentioned above. To me, they look a lot like bar charts, so when I see them, I try to read them as bar charts, which is the wrong mental model for them. A similar problem exists with connected scatter plots. I want to read them as line charts, but this is the wrong way to read them, so I get confused.

There is so much more to say about this problem, and I feel the way I am addressing this issue here is still quite primitive. It would be nice to have more theories about how chart interpretation works. Maybe some of those theories are already out there, and I am unaware of them. Keep in mind that I am not a cognitive psychologist, so there is always the risk I am just exposing my ignorance here. Please let me know if you know of an existing psychological framework to look at this problem.

I am vaguely familiar with Pinker’s theory of graph comprehension, but to me, it seems a little overly complex. In his theory, Pinker introduces the concept of “graph schema,” which is related to the mental model of the graph idea I have mentioned above. Barbara Tversky also has done a lot of work in this space which I’ll need to review. The highly cited paper on animation that Tversky wrote with co-authors Julie Bauer Morrison and Mireille Betrancourt introduces the “congruence principle,” which states that “the structure and content of the external representation should correspond to the desired structure and content of the internal representation.” This is similar to what I am describing above in that confusion may stem from a mismatch between the mental model one has developed internally and the way the visual representation looks. Unfortunately, it is not immediately clear how to use the principle in practice because it is not evident what such structures and content we have internally are. That said, I find the idea of explicitly considering mental models a useful lens to think about the problem of confusing graphs. I wish we could tell more and be more specific, but I am not sure if there is any existing work that covers these specific gaps.

That’s all I have for now. I would be happy to know more if you have useful information to share about this topic.

Thanks for reading 🙏.

P.s. As I finish writing this I think I did not discuss the role of clutter in perceiving a graph as confusing. Clutter seems to be a problem of a different nature and I am still not sure how to think about it in the context of mental models.