Simulation Visualization
What could we achieve if we had more visualizations based on simulated data?
I wrote a tweet a while back regarding an interesting synergy between data visualization, simulation and education.
I was surprised by the enthusiastic response, so I thought I would expand a bit more on it here.
Why is simulation visualization interesting?
In more than 15 years of activity in visualization research and practice, I have rarely worked on visualizations that are based on simulated data. And when I look at the work of many of my colleagues I rarely see anything that is based on simulations. The only exception is the community of academics working on Scientific Visualization, where visualization is used to investigate complex scientific phenomena such as climate change, weather forecast, and fluid dynamics.
The work done in this space is great, but it tends to be highly technical and it typically targets scientists working with high-end equipment and costly infrastructures. Furthermore, scientific visualization tends to focus on spatial phenomena, where the visual representation is basically given. But simulations do not necessarily need to be about spatial phenomena and much more can be explored in domains where space does not play a prominent role.
When I think about data visualization I can’t help noticing that a vast majority of visualizations represent data sampled from real-world phenomena, and very few depict data sampled from simulations. Yet, the more I think about visualization for simulations, the more I think it has some specific properties that venture beyond what we normally address in visualization.
Two interesting ones are:
The dynamic nature of simulations
The key role of parameter exploration
Most data visualizations are static. The interactive ones tend to filter existing data but do not allow users to explore new relationships within the data. So, while interactive on a visual level, these visualizations still represent static relationships of data. The beauty of simulations is that they depend on parameters and when new parameters are set the dynamics of a process can change accordingly. In other words, simulations allow users to “replay” reality multiple times (or in parallel universes) and generate new data every time the parameters are changed.
From a research and design standpoint simulations are interesting because they can push people to figure out how to visualize and systematically explore a less-defined design space.
But even more interesting is the fact that entirely new data is generated as soon as a user changes the parameters of a simulation. This is what I find really exciting: new parameter settings → new data. How do we handle that in visualization environments? What kind of interfaces and interactivity do we need to make parameter exploration effective?
This is an area where the interplay between computation and interaction becomes really interesting. It’s not only a matter of how to visualize the simulated data, but also how to tie together user actions with dynamic changes in the visualization. While it’s been studied a bit within the context of techniques like “dynamic queries”, I think this visualization challenge deserves way more attention.
Another interesting aspect of simulations is their trade-off between precision and cost. Simulations may require intensive computational procedures that prevent systems from generating results interactively. Imagine changing the value of a parameter and having to wait hours or days before seeing a result. But maybe if we accept visualizing results at a coarser level, then simulations could be sped up considerably. In fact, we may even find that some degrees of precision are useless because more precision does not fundamentally change what information or insights one can derive from the simulation.
I remember talking with a researcher from a major national lab a few years back who told me that it is easy to speed up simulations if your goal is exclusively to visualize the results (as opposed to producing precise statistics and numbers). If I remember the conversation correctly, the researcher just applied some really brutal and unsophisticated methods of simplification and was stunned by what they gained in terms of insights, despite having less reliable information.
Simulation in education (and personal decision making).
I do not know to what extent simulation in education has been studied so far (I need to do more research), but its relevant application seems like such a clear no-brainer to me. We know that people learn better when they are able to play with something (see Constructionism and the work of Seymour Papert). And isn’t simulation just a kind of playful activity? Change this and that and see what happens. Pure exploration.
I was quite impressed by a YouTube video published by Peter Attia (popular MD and web personality) early on in the COVID-19 pandemic (unfortunately I was not able to find it again). In the video he explains that a simulation does not necessarily need to be accurate or predictive in order to be useful. A more interesting role of simulation is to enable people to explore and play with parameters in order to understand the nature of a problem better.
In fact, COVID-19 simulations have been quite popular in the last two years. Amanda Makulec has an excellent article titled “Move over, data visualization. The era of ‘data simulation’ is here” in Fast Company, describing the role of simulations and specifically showcasing examples from the pandemic. But there is no reason to focus only on virus spread and pandemics! There is no lack of abstract and yet critical dynamic phenomena that could be described through interactive visualizations of simulations.
In this respect, I wonder if simulations could also play a role in what I’ll tentatively call “personal decision making”; all those situations where I have to make complex decisions in “spaces” I don’t understand well. Take the choice of which insurance plan to use or how to invest in index funds. Wouldn’t it be great to have simple simulators that allow us to finally feel in control of our decisions? These are all decisions that require: a) understanding choices; b) understanding consequences; and c) calibration based on the personal level of risk one is willing to bear. Isn’t it amazing how little support exists in this space and, at the same time, how useful interactive visualization systems could be? A good example in this area is the great New York Times interactive “Rent or Buy?”, which guides people in deciding whether they should rent or buy their house. Another example is the work of my former colleague at NYU, Oded Nov, who works on interfaces for decision making. He has done some really interesting research with his student Junius Gunaratne on how different user interface designs may lead to better or worse decisions in personal investments.
What about “explainers”?
Of course the idea of explainers is adjacent to the idea of simulations. An explainer is typically an interactive article that explains a complex concept or phenomenon through a mix of data visualization, text, and media of various types. If you want to see some really fantastic ones I suggest you take a look at the work of Nicky Case (who we interviewed a few years ago in our Data Stories Podcast) and the amazing math-oriented YouTube videos of 3Blue1Brown. There are amazing visual explainers out there and I think explainers should also be studied more systematically. The main difference between an explainer and a simulation is that in simulations the learner has much more control and the visualization tends to be more interactive.
That said, I recognize the distinction between the two is a bit blurry, though I am not sure it matters that much. What is exciting is the idea that visualizations can be used to represent simulated data to improve understanding of complex phenomena, and that this type of knowledge is potentially accessible to a large segment of the population, including young kids and students of any age.
My ideas in this space are still very blurry but I would love to see more people developing and studying simulations in a more rigorous way. I believe that this is a very exciting topic for visualization research.