TalkBack is Google’s screen reader for Android devices. It’s hard to understand Android’s accessibility issues without experiencing them yourself. Take 5 minutes to read this article, download this cheatsheet, and then go explore your TalkBack app for yourself. You might be surprised by what you find.

What is a screen reader?

What it sounds like: It reads out what’s on the screen.

It mediates your interaction with the screen

When a screen reader is active, touches to the screen activate its responses. It acts like a go-between to explain what you’re pointing at. It also provides a gesture language to tell it how to interact with the thing you last pointed at. There are also TalkBack gestures for controlling the device in general, like triggering the Back button.

It tells you what you touch

You can touch anywhere on the screen, listen to what the screen reader says, and if you’ve touched a button or something else you can interact with, ask the screen reader to click it for you by double-tapping.

It maintains a current focus

Imagine you have finished typing an email. Now you need to click the Send button. It could take a long time to find the button just by probing the screen and listening to what is at each touch point.

So there’s an alternative. The screen reader keeps an item in focus. Touching the screen places the focus on the touched item. But from there, you can “look around” that point by swiping left and right. This works like using Tab and Shift-Tab to navigate a form in your browser.

This notion of “focus” also lets you act on the current focus: click a button, start editing in a text field, or nudge a slider. Unlike the normal touch gestures used to do these things, TalkBack’s gestures are addressed to the screen as a whole. You can double-tap anywhere on the home screen to click a focused button.

Getting Started with TalkBack

How do I turn TalkBack on?

• Open the “Settings” application.
• Select “Accessibility” from the “System” section, way down at the bottom.
• Select “TalkBack” from the “Services” section.
• Toggle it to “On”.

To make this easier in future, you may want to configure a volume key shortcut.

How do I turn TalkBack off?

Head back into Settings and turn off TalkBack Android.

To make toggling TalkBack on and off easier, you can enable the suspend and resume shortcut in the “Miscellaneous” section of TalkBack Settings.

How do I use TalkBack?

TalkBack Android is controlled entirely by one finger.

Gestures with two or more fingers will not be handled by TalkBack. They’ll be sent directly to the underlying view. Two or more fingers will “pierce the veil”, so you can pinch-to-zoom or scroll the same as ever.

How do I look around?

• Forward: Swipe right with one finger anywhere on the home screen to move the focus forward. This is “forward” in the sense of text, so right/down.
• Back: Swipe left with one finger anywhere on the screen to move the focus back.

How do I explore what’s on the screen by touch?

Touch, listen. Touch somewhere else, listen again. You can also touch-and-drag to more rapidly explore the screen.

How is this useful?

• Jump to known location: If you know the widget you want to interact with is somewhere in the middle of the screen, this can get you close to it, and you can swipe the focus around from there.
• Hack around busted accessibility: Not everyone tests their apps using TalkBack. Being able to aim the focus “by hand” can break you out of a focus loop or let you aim focus at something you can’t otherwise tab to.
• Skimming: Touch-and-drag can be faster than pushing the focus around with separate swipes.

How do I type?

Google’s keyboard supports a variant of explore-by-touch:

• Slide your finger around till you hear the letter you want.
• Lift your finger. This types the letter.

This combines the “find” and “activate” gestures to speed up typing.

Some third-party keyboards follow Google’s example.
Others do not – sometimes, by choice; other times, seemingly, out of ignorance.

How do I interact with the focus?

• Click: Double-tap anywhere on the screen with one finger.
• Long Click: Double-tap and hold anywhere on the screen with one finger.

How do I navigate a long webpage?

Maybe you’re wondering what swiping up and down does now. This lets you tweak what left and right swipes do by changing the navigation settings. Instead of moving element to element, they can navigate a more specific list of things, like “all headings” or “all links”.

• Next Navigation Setting: Swipe down
• Previous Navigation Setting: Swipe up

Swipes are also how you scroll:

• Scroll Forward: Swipe right then left
• Scroll Backward: Swipe left then right

And they provide a way to reliably jump focus around the screen:

• Focus First Item on Screen: Swipe up then down
• Focus Last Item on Screen: Swipe down then up

As a bonus, you can use the local context menu (more on this below) to ask TalkBack to read all the links in a block of text, without you having to cursor through the list yourself.

What about hardware buttons?

For this, you’ll use angle gestures. These go one direction, then 90 degrees in another direction.

These also let you trigger some other system-level actions, like showing the notifications.

Gesture Cheatsheets

The Basic D-Pad: Moves

• Simple swipes affect focus.
  Swipe left and right to move focus between items.
  Swipe up and down to change the kind of item to focus on.

For example, you may want to only focus on headings or links.
(If you’ve used iOS VoiceOver, this is kind of like some of the Rotor options.)

Back-And-Forth: Jumps

Quickly swiping out and then back to where you started in a continous motion either jumps focus or scrolls the screen.
(Though if a slider is focused, its thumb “scrolls” rather than the screen.)

• Swipe up then back to focus the first item on the screen.
  Swipe down then back to focus the last item on the screen.

If you know the item you want to focus is near the top or bottom of the screen, these gestures can help you focus that item faster.
You can also build muscle memory for the controls in an app relative to these anchor points.

• Swipe left then back to scroll up or to move a slider left.
  Swipe right then back to scroll down or to move a slider right.

You can also use two fingers to scroll like always, because two-finger touches
are ignored by TalkBack.

Angle: System Buttons

Actions like Back, Home, and Overview once had hardware buttons.

They still occupy a privileged place in the UI.

TalkBack also gives them pride of place: they have their own dedicated gestures.

The angle gestures equivalent to the hardware buttons involve swiping to the left:

• Home: Swipe up then left
• Back: Swipe down then left
• Overview: Swipe left then up

Angle gestures that involve swiping to the right are more peculiar to TalkBack:

• Notification Drawer: Swipe right then down
  • This lets you pull the drawer down without worrying about feeling for the top of the physical screen.s
• Local Context Menu: Swipe up, then right
  • The contents of this menu depend in part on the current focus.
  • For example, if you’ve focused on an unlabeled image, you’ll see an option to give it a custom label.
  • But they also provide another way to change the kind of item focused.
    Changing the target item using the menu can be faster than making repeated single swipes up or down.
• Global Context Menu: Swipe down then right
  • If you want to tell TalkBack to read a chunk of content, or reread something, or even copy the last thing it read to the clipboard, this is the menu for you.
  • It also has convenient shortcuts to TalkBack Settings (where you can change gesture behavior, amongst other things) and an option to Dim Screen.

Try this at home!

• Configure your own actions for currently-unassigned gestures
• Use the local context menu to set a custom label for an unlabeled widget
• Use the global context menu to “Dim screen”. This will black out the whole screen, so you’re forced to lean on TalkBack to interact with your device.
• Use Google’s Accessibility Scanner app on the home page of your app. What issues does it find?
• Explore the keyboard shortcuts available with a hardware keyboard.

Other Assistive Tech

TalkBack isn’t the only assistive tech available on Android. Here are several other unique ways people might be interacting with your app:

• BrailleBack: like TalkBack, but for Braille displays
• Switch Access: could be one switch, or two like in a sip-puff straw device
• Voice Access: hands-free control

For the More Curious: How does TalkBack work?

It navigates a virtual tree of accessibility nodes. Luckily, SDK classes take care of building these nodes in most cases. Tweaking the tree can improve the experience, though. And if you’re building a custom view, or abusing a stock one, you’ll need to work a bit to make it accessible.

TalkBack will send performClick() and performLongClick() as needed.

For more, dig into the android.view.accessibility documentation and follow the links from there.

For yet more, Google has published the TalkBack and Switch Access source code. Included is a test app that exercises the functionality of both. Playing with this test app would be a great way to see everything these tools can do.