Graphics are one of the primary tools for science or data communication, and are powerful because they make use of our visual system in a way that off-loads much of the work of processing the data, freeing cognitive resources up to consider the content rather than the representation. Unfortunately, not everyone can leverage their visual systems in this way, due to differences in the visual system or in information processing systems within the brain. There are a wide range of issues which may impact how effectively people can use graphics, including colorblindness, poor visual acuity, blindness, dyslexia, dyscalculia, and even differences in data literacy and numerical literacy. In some sense, it can be useful to go through the introspection process when looking at a graph, and consider what perceptual and cognitive resources are required to complete each step when looking at a chart. Ultimately, as scientists and people working with data, we need to work to make our data representations accessible. The specifics of which populations we focus on, and how we adapt existing representations (or create new ones) are based on the target audience(s), and will differ across different disciplines. For instance, if you are designing graphics to be used in Air Traffic Controller trainings, you likely do not need to accommodate Blind and Low Vision (BLV) individuals or even consider color-blindness. However, if you are creating graphics to be consumed by a general web audience, it is important to consider a range of visual impairments and accommodations.
This is an active area of research in data science, information visualization, and design. It is always useful to see what new solutions have come out recently, because there are new developments in this area on a regular basis.
Adapting Existing Graphics
One of the simplest ways to increase accessibility of graphics is to ensure that they meet basic guidelines for discoverability and distinguishability. Discoverability ensures that screen-reader users know that the graphic exists; this requires designing the entire web page with these users in mind, but is critical to ensuring equal access to information for blind and low-vision users. Creating graphics which adhere to contrast, color selection, font size, and other distinguishability guidelines helps low-vision people, those with sensory processing issues, and people with colorblindness use existing data representations effectively.
Discoverability
Alt Text
When retrofitting an existing page for accessibility, it may not be possible to make charts and graphics fully accessible to individuals who are blind or using screen-readers. In these cases, it is important to write good alt-text for each graph that is intended to convey information (it is fine to skip alt-text for purely decorative images).
Good alt-text is:
concise
accurate
relevant
context-dependent - the same image may require different alt-text depending on the broader context of the web page.
Graphs are some of the hardest images to fully describe in alt-text, in part because providing the same information that the image provides may require thousands of words, full data tables, or other accommodations. The alt text field in HTML does not allow for paragraphs, line breaks, and other structural elements; as a result, it is often better to include a short description in the alt-text field and a longer description (or table) as part of the web page source. Linking the alt-text and the longer description together may facilitate keyboard navigation between the two, making the navigation process less cognitively intensive for screen reader users.
The BrailleR R package integrates with many common R plotting functions (including base graphics and ggplot2) and can generate some functional alt-text automatically.
BrailleR alt-text demo
library(BrailleR)data <-read.csv("../data/penguins.csv")library(ggplot2)scatterplot <-ggplot(data, aes(x = bill_length_mm, y = bill_depth_mm, color = species)) +geom_point()scatterplot
Warning: Removed 2 rows containing missing values or values outside the scale range
(`geom_point()`).
This is an untitled chart with no subtitle or caption.
It has x-axis 'bill_length_mm' with labels 40, 50 and 60.
It has y-axis 'bill_depth_mm' with labels 15.0, 17.5 and 20.0.
There is a legend indicating colour is used to show species, with 3 levels:
Adelie shown as strong reddish orange colour,
Chinstrap shown as vivid yellowish green colour and
Gentoo shown as brilliant blue colour.
The chart is a set of 342 big solid circle points of which about 97% can be seen.
Page Structure
The entire web-page structure is important to consider when designing to include screen-reader users. Sighted users may be able to take in the web page structure visually and determine which elements to focus on; screen reader users have to take in the page structure audibly, and in sequence. Some users describe it as “viewing a web page through a straw”.
It is important to provide a contextual overview first, and then provide specific data and details in a structured hierarchy. We can update the “information seeking mantra” of overview, zoom and filter, details on demand to “gist”, “supporting methods” (contextual information), and “details” (actual data content). The structure of the page needs to support understanding when navigated hierarchically. Information embedded in visual design themes (font sizes, colors, nesting, space) need to be explicitly accessible via aria- fields to be accessible to screen reader users.
Navigating with a Screen Reader
Discriminability
In addition to the information below, which includes some web references, there is a lovely series of Observable posts on accessibility, contrast, and color choice for data visualization. Check it out!
red### Color selection
We’ll first approach color selection with color impairment (aka “colorblindness”, though most color impaired people can see some colors) in mind, though many of the considerations here factor into contrast considerations later. There are several approaches to accommodating color impairment:
Avoid red and green combinations. This helps but is not sufficient, particularly for those who have trouble with red and blue, rather than green.
Use palettes designed to be “colorblind-friendly”, such as David Nichol’s, Okabe-Ito’s, Paul Tol’s. Colorbrewer’s colorblind friendly palettes are less useful than these options.
Design your graphic so that it is functional in greyscale. This will make it safe for all types of color impairment.
Dual-encode colors with other attributes, such as shape or linetype.
It can be difficult to fully accommodate those with color impairment, particularly when working with graphics that use many different hues. Keep in mind that even people with full color vision cannot keep more than \(7 \pm 2\) items in working memory - so using many different colors is problematic for everyone, not just for those with impaired color vision.
Contrast
It can be hard to see content that does not have much contrast against the background. People with low vision rely on contrast even more than the rest of the population; in addition, individuals with color impairment tend to rely on contrast cues to determine whether ambiguous colors are, in fact, different.
W3C (World Wide Web Consortium) creates Web Content Accessibility Guidelines (WCAG) to provide a standard of accessible online content. These guidelines have recommendations for creating alt-text, how to ensure accessibility of different types of media, and standards for how to make content distinguishable.
WCAG guidelines are provided on a scale from A (basic accessibility) to AAA (most accessible).
Large Text: Large-scale text and images of large-scale text have a contrast ratio of at least 3:1;
Incidental: Text or images of text that are part of an inactive user interface component, that are pure decoration, that are not visible to anyone, or that are part of a picture that contains significant other visual content, have no contrast requirement.
Logo: Text that is part of a logo or brand name has no contrast requirement.
Customizable: The image of text can be visually customized to the user’s requirements;
Essential: A particular presentation of text is essential to the information being conveyed. Logotypes (text that is part of a logo or brand name) are considered essential.
User Interface Components: Visual information required to identify user interface components and states, except for inactive components or where the appearance of the component is determined by the user agent and not modified by the author;
Graphical Objects: Parts of graphics required to understand the content, except when a particular presentation of graphics is essential to the information being conveyed.
Line height (line spacing) to at least 1.5 times the font size;
Spacing following paragraphs to at least 2 times the font size;
Letter spacing (tracking) to at least 0.12 times the font size;
Word spacing to at least 0.16 times the font size.
Exception: Human languages and scripts that do not make use of one or more of these text style properties in written text can conform using only the properties that exist for that combination of language and script.
Dismissible: A mechanism is available to dismiss the additional content without moving pointer hover or keyboard focus, unless the additional content communicates an input error or does not obscure or replace other content;
Hoverable: If pointer hover can trigger the additional content, then the pointer can be moved over the additional content without the additional content disappearing;
Persistent: The additional content remains visible until the hover or focus trigger is removed, the user dismisses it, or its information is no longer valid.
Exception: The visual presentation of the additional content is controlled by the user agent and is not modified by the author.
Large Text: Large-scale text and images of large-scale text have a contrast ratio of at least 4.5:1;
Incidental: Text or images of text that are part of an inactive user interface component, that are pure decoration, that are not visible to anyone, or that are part of a picture that contains significant other visual content, have no contrast requirement.
Logotypes: Text that is part of a logo or brand name has no contrast requirement.
No Background: The audio does not contain background sounds.
Turn Off: The background sounds can be turned off.
20 dB: The background sounds are at least 20 decibels lower than the foreground speech content, with the exception of occasional sounds that last for only one or two seconds.
Foreground and background colors can be selected by the user.
Width is no more than 80 characters or glyphs (40 if CJK).
Text is not justified (aligned to both the left and the right margins).
Line spacing (leading) is at least space-and-a-half within paragraphs, and paragraph spacing is at least 1.5 times larger than the line spacing.
Text can be resized without assistive technology up to 200 percent in a way that does not require the user to scroll horizontally to read a line of text on a full-screen window.
Chartability is a set of heuristics for ensuring accessibility of data visualizations (and the pages that contain them). It’s created by BLV designers and is designed to help you locate accessibility barriers in data visualizations. They maintain an audit workbook that has tests that help identify design failures.
Creating More Accessible Graphics
In general, charts created as images, which are the default in many systems such as ggplot, matplotlib, and SAS, require alt-text, inclusion of data tables, and other modifications that still do not produce full accessibility. By contrast, d3, Observable.js, Highcharts.js, and other svg-based web graphics allow for some navigation within the chart by screen reader users. However, these tools still require some extra planning to design charts that are accessible and well-formatted for screen-reader users.
The OLLi project works within Observable and Vega/VegaLite visualizations to create a navigable, hierarchical tree for keyboard navigation and descriptions.
Let’s consider three different pages produced using quarto to showcase the penguins data. Explore the graphics yourself first, then play the video below to see how my screen reader handled each one. Note the level of detail available to the user.
Warning: Removed 2 rows containing missing values or values outside the scale range
(`geom_point()`).
This is an untitled chart with no subtitle or caption.
It has x-axis 'bill_length_mm' with labels 40, 50 and 60.
It has y-axis 'bill_depth_mm' with labels 15.0, 17.5 and 20.0.
There is a legend indicating colour is used to show species, with 3 levels:
Adelie shown as strong reddish orange colour,
Chinstrap shown as vivid yellowish green colour and
Gentoo shown as brilliant blue colour.
The chart is a set of 342 big solid circle points of which about 97% can be seen.
Figure 1: A plot of the palmer penguins data, showing bill length in mm on the x axis and bill depth in mm on the y axis. Points are colored by species. For each species, there is a positive relationship between bill depth and bill length, with each species occupying a different region of the space. Adelie penguins have shorter, deeper bills. Chinstrap penguins have longer, deeper bills. Gentoo penguins have longer, shallower bills.
Figure 2: A plot of the palmer penguins data, showing bill length in mm on the x axis and bill depth in mm on the y axis. Points are colored by species. For each species, there is a positive relationship between bill depth and bill length, with each species occupying a different region of the space. Adelie penguins have shorter, deeper bills. Chinstrap penguins have longer, deeper bills. Gentoo penguins have longer, shallower bills.
// Create plot with specificationPlot.plot(penguinChart);
Figure 3: A plot of the palmer penguins data, showing bill length in mm on the x axis and bill depth in mm on the y axis. Points are colored by species. For each species, there is a positive relationship between bill depth and bill length, with each species occupying a different region of the space. Adelie penguins have shorter, deeper bills. Chinstrap penguins have longer, deeper bills. Gentoo penguins have longer, shallower bills.
Olli is a fantastic project, but support for Observable is limited; there is better support for Vega charts, but even then, not all chart types are supported.
Sonification
There are other methods of communicating data without relying primarily on visual methods and adapting those representations to remove reliance on vision. Zong et al. (2024) developed Umwelt, which allows for editing of multimodal data representations, providing some support for sonification and non-visual data communication. There are also methods for adapting existing visualizations to produce sonified equivalents, using R tools like sonify or Python tools like Strauss, miditools(Russo 2024).
Sonifying Data With Python
Here’s an example of how to create a data sonification using the penguins data, adapted from Russo (2024).
# Code adapted from https://hub.ovh2.mybinder.org/user/systemsounds-so-ation-tutorials-vr3cdobo/doc/tree/data2midi-part1.ipynbimport pandas as pdpenguins = pd.read_csv("../data/penguins.csv").dropna()# Define a general mapping functiondef map_value(value, min_value, max_value, min_result, max_result):'''maps value (or array of values) from one range to another''' result = min_result + (value - min_value)/(max_value - min_value)*(max_result - min_result)return result# Set desired duration: 15 seconds/beatspenguins.bill_length_mm.describe # get info on penguin bill lengths
penguins = penguins.sort_values(by=['species', 'bill_length_mm'], ascending =True) # sort data by bill lengthduration_beats =15bpm =60duration_sec = duration_beats*60/bpm# Scale x axispenguins["t_data"] = map_value(penguins.bill_length_mm, min(penguins.bill_length_mm), max(penguins.bill_length_mm), 0, duration_beats)# Scale y axispenguins["y_data"] = map_value(penguins.bill_depth_mm, min(penguins.bill_depth_mm), max(penguins.bill_depth_mm), 0, 1)# May want to transform data a bit - example uses sqrt# y_scale = 0.5# # penguins["y_data"] = penguins.y_data**y_scaleimport matplotlib.pyplot as pltplt.scatter(penguins.t_data, penguins.y_data)plt.xlabel('time (bill length, mm, normalized)')plt.ylabel('bill depth, mm, normalized')plt.show()
plt.clf()# Scale penguin speciespenguins["track"] = penguins.species.replace({"Adelie":0,"Gentoo":1,"Chinstrap":2})for track_i inrange(3): dat = penguins.query("track==@track_i") plt.scatter(dat.t_data, dat.y_data)plt.xlabel('time (bill length, mm, normalized)')plt.ylabel('bill depth, mm, normalized')plt.show()
Now that we’ve transformed the data, we can map data values to tones. MIDI tones and velocities are integers, so we must transform and then round our values to match the requirements of the medium we’re using.
# In this case, I'm happy with just 2 octaves of chromatic notes, from C3 to C5. # These correspond to MIDI notes 48:72penguins["midi_y"] =round(map_value(penguins.y_data, 0, 1, 48, 72))penguins["midi_y"] = penguins.midi_y.convert_dtypes()for track_i inrange(3): dat = penguins.query("track==@track_i") plt.scatter(dat.t_data, dat.midi_y)plt.xlabel('time (bill length, mm, normalized)')plt.ylabel('MIDI note number')plt.show()# Map data to note velocity - velocity is a combination of volume and intensity# We dual-encode pitch and velocity# # vel_min, vel_max = 35, 127# penguins["midi_vel"] = round(map_value(penguins.y_data, 0, 1, vel_max, vel_min))# penguins["midi_vel"] = penguins.midi_vel.convert_dtypes()
Finally, we create the midi file. We add 3 tracks, one for each species - this would allow us to change the “program” (instrument) to correspond to each species group. For the moment, I’ve commented the program changes out because the addition of instruments makes the end result sound like elementary school band warm-up time (complete chaos).
from midiutil.MidiFile import MIDIFile #import library to make midi file, https://midiutil.readthedocs.io/en/1.2.1/#create midi file object, add tempomy_midi_file = MIDIFile(3, deinterleave=False) #three tracks, one for each species my_midi_file.addTempo(track=0, time=0, tempo=bpm) my_midi_file.addTempo(track=1, time=0, tempo=bpm) my_midi_file.addTempo(track=2, time=0, tempo=bpm) # # # .addProgramChange(track, channel, time, program)# my_midi_file.addProgramChange(0, 0, 0, 71) # first set of penguins as clarinet# my_midi_file.addProgramChange(1, 0, 0, 75) # second set of penguins as pan flute# my_midi_file.addProgramChange(2, 0, 0, 59) # third set of penguins as muted trumpets#add midi notesfor i in penguins.index: my_midi_file.addNote(track=penguins.track[i], channel=0, pitch=penguins.midi_y[i], time=penguins.t_data[i], duration=0.25, volume=35)#create and save the midi file itselffilename ='penguins_sonification.mid'withopen(filename, "wb") as f: my_midi_file.writeFile(f) # Listen# import pygame #import library for playing midi files, https://pypi.org/project/pygame/# pygame.init()# pygame.mixer.music.load(filename)# pygame.mixer.music.play()
Results of sonification of Figure 1 using python and MIDI audio encoding.
Physicalization
There are a number of ways to create accessible tactile charts using embossing machines, capsule paper (Brauner 2023), or 3D printers. Tactile graphics have higher performance than tactile tables or electronic tables accessed via screen reader (Watanabe and Mizukami 2018), in addition, tactile bar charts presented either alone or with auditory information have higher performance than audio-only presentation (Goncu, Marriott, and Hurst 2010).
R packages like rayshader can be used to convert ggplot2 plots into 3D-printable STL files (Morgan-Wall 2024). This produces a STL file that has some tactile information, without requiring too much specialized software; it could be made more accessible by using a Braille font. One downside is that the height of the plot object is mapped to color/fill, and does not accommodate categorical mappings.
Rayshader demo
data <-read.csv("../data/penguins.csv")library(ggplot2)plot <-ggplot(data) +stat_density_2d(aes(x = bill_length_mm, y = bill_depth_mm, fill =after_stat(!!str2lang("density"))),contour = F, geom ="raster") +scale_x_continuous(expand=c(0,0)) +scale_y_continuous(expand=c(0,0))library(rayshader)plot_gg(plot, emboss_text = .05)
Warning: Removed 2 rows containing non-finite outside the scale range
(`stat_density2d()`).
Removed 2 rows containing non-finite outside the scale range
(`stat_density2d()`).
This is an untitled chart with no subtitle or caption.
It has x-axis 'bill_length_mm' with labels 40 and 50.
It has y-axis 'bill_depth_mm' with labels 14, 16, 18 and 20.
There is a legend indicating fill is used to show density, ranging from 5.50919365996529e-09 represented by fill dark purplish blue to 0.0160915333509369 shown as fill brilliant blue.
The chart is a raster graph that VI cannot process.
Warning: Removed 2 rows containing non-finite outside the scale range
(`stat_density2d()`).
