Log Scales, Exponential Growth, and the Importance of User Testing
Reproduction from the Journal of the American Medical Association, Jan 11, 1919. Image source
All-cause mortality in Major Cities, 1918-1919 Image source
Excess Mortality and Temperature, 1840s London. From Report on the mortality of cholera in England, 1848-49, by William Farr (1952). Image Source
Cholera & Plague in London (Created 1852) Image source
May 6 2020: Three graphs that show a global slowdown in COVID-19 deaths, Image Source
Active Cases vs. Total Deaths, Reddit, May 9 2020 Image source
Tests vs. Cases, Our World in Data (May 19 2020)
Rate of Death Change, March 28, 2020. Romain Vuillemot (@romsson) Image source
91-DIVOC Diagonal Reference Lines. May 11, 2020.
To help people predict what is likely to happen?
(It doesn’t do that very well)
To help people predict what is likely to happen?
Humans are awful at predicting exponential growth
Wagenaar and Sagaria (1975; Timmers and Wagenaar 1977)
Log or linear scales?
Reference lines?
Aligning x-axis to date? Days since ___ cases?
“Pace” of the case counts vs. raw counts?
Comparing across populations – size, policies, susceptibility…
To inform
To aid individual decision-making
To aid policy development
Different goals = different charts
The question is not What is the best chart?
but… What is the best chart for this purpose?
For an answer, we need to subject charts to a full spectrum of user tests.
3 different ways of engaging with the data
Can we
graphs of exponential growth with log and linear scales?
300 participants completed all 3 experiments
Conclusion: It’s easier to spot a curve among lines than it is to spot a line among curves
Robinson, Howard, and Vanderplas (2023a)
Frederick Mosteller et al. (1981) Eye Fitting Straight Lines. The American Statistician
New York Times’ ‘You Draw It’ features:
Replicate Eye Fitting Straight Lines using the you-draw-it tool (4 charts) Robinson, Howard, and Vanderplas (2022)
Explore exponential growth predictions on log and linear scale (8 charts)
12 total graphs to complete
Next level of engagement is estimating quantities from a graph
This is a much harder experiment to set up
How to make it generalizable?
Free response: Between \(t_1\) and \(t_2\), how does the population of \(X\) change?
Estimating Population given a year
Process Sketch
Estimating Population given a year
From Year1
to Year2
, the population increases by ____ individuals
Process Sketch
From Year1
to Year2
, the population increases by ____ individuals
How many times more creatures are there in Year2
than Year1
?
Process Sketch
How many times more creatures are there in Year2
than Year1
?
How many times more creatures are there in Year2
than Year1
?
How many times more creatures are there in Year2
than Year1
?
How long does it take for the population in Year 1
to double?
Process Sketch
How long does it take for the population in Year 1
to double?
Conflicting results can be hard to reconcile
Conducting multiple studies is multiple times the work
(multiple times the payoff?)
Greater insight into the tradeoffs of design decisions
Testing method needs to match level of engagement
Examine graphical choices across engagement levels
A “Visual Hypothesis Test”
Embed the question in array of charts
Can people identify the different plot?
Null model can be tricky to create
Test statistic is the visual evaluation
Infer cognitive processes from directed (conscious) attention
May be accompanied by direct estimation or other protocols
Stream of consciousness narration Guan et al. (2006; Cooke 2010)
Reasoning to justify a decision
Have participants visually fit statistics
Compare visual statistics to numerical calculations
Differences tell us about our implicit perception of data
e.g. visual regression is more robust to outliers
Also useful as a teaching tool
Testing method needs to be matched to level of engagement
Need to examine graphical choices across levels of engagement