Do Apple’s accessibility efforts point at its AI plans?

You can usually measure a society by the way it treats its most vulnerable populations, and technology often can help people live better, more autonomous lives. Apple firmly believes that, and this year’s raft of accessibility announcements introduced to mark Global Accessibility Awareness Day shine a light on that belief. 

The company has won a string of awards that recognize its work, including praise from the National Federation of the Blind, the American Foundation for the Blind, the Cerebral Palsy Foundation, and the National Association of the Deaf. These tools matter to everyone, of course; as we age and our faculties decline, the accessibility solutions Apple creates today promise better tomorrows.

AI + Accessibility + Apple 

With WWDC just weeks away, Apple’s latest accessibility features promise powerful technologies for all. Most will arrive with the 27 series of Apple operating systems — and many of the most powerful tools lean deeply into AI and Apple Intelligence.

“The accessibility features our users rely on every day become even more powerful with Apple Intelligence,” said Sarah Herrlinger, Apple’s senior director of Global Accessibility Policy and Initiatives. “With these updates, we’re bringing new, intuitive options for input, exploration, and personalization, designed to protect users’ privacy at every step.”

They also give us a glimpse at what Apple has planned across its operating systems in terms of improved contextual intelligence in Siri. For example, a new tool called Image Explorer in VoiceOver lets you use Apple Intelligence to generate detailed descriptions of images held across your system, and those images can be documents, bills, receipts.

The idea is that you can hold up your iPhone, point your camera at the item and ask Apple Intelligence to describe what it is, read it to you, ask questions about what’s there, and even ask follow-up questions about what it sees. This will make a huge difference for disabled people who use voice to control their iPhone or iPad. (It remains unknown whether these services will also be available on Mac.)

All the same, the fact that these new accessibility improvements work on device mean you can use them in complete privacy, which makes them even more compelling.

See it, say it, do it

While Apple hasn’t said anything specific, it’s hard to ignore that this feature could be of use in a more context-savvy Siri. If you think about it, what you see on your iPhone display is also an image; it seems plausible you’ll be able to use Siri to get things done on your device pretty soon.

That’s certainly true of a second accessibility improvement Apple introduced — Voice Control, which will let you navigate your device using natural speech. This is great for those of us who cannot easily use touch to navigate a device, and in combination with Image Explorer suggests deep use cases for all of us. After all, if Siri can open files with a voice command, why not with a text prompt? And if that file happens to be a workflow or agentic action, this could utterly transform the iPhone UI.

AI that solves real problems

Apple is also putting more intelligence into Accessibility Reader, an invaluable tool for users with low vision or dyslexia that reads text to them. Now boosted by AI, this can handle far more complex source materials, including tables and multi-column layouts.

Accessibility Reader also takes a leap beyond just simply reading such material; thanks to AI, it can now generate on-demand summaries and even live translation of the text you choose to read. There’s intelligence in FaceTime conversations, too. Apple intends to introduce a new API for sign language interpretation app developers that lets users add human interpreters to ongoing calls.

AI is also available in video, meaning your device will automatically generate subtitles for spoken dialog for any content, including videos shared by family and friends. Apple’s on-device speech recognition means subtitles can be generated privately and appear automatically for uncaptioned videos on iPhone, iPad, Mac, Apple TV, and Apple Vision Pro.

Mobility transformed with Vision Pro

Apple introduced a version of the Apple Watch Activity app for wheelchair users  in 2016. This was a new first, as there had never been an accurate fitness tracker for wheelchair users before. The team building the solution had to create brand new algorithms and engage in massive tests to ensure it got this right.

Ten years later, and Apple has introduced something new: the capacity to control compatible power wheelchairs using the calibration-free eye-tracking capabilities of Apple Vision Pro. While this is interesting from a technical point of view, for some wheelchair users — particularly for those who cannot use a joystick to control their system — it’s a major benefit.

Pat Dolan is the founder of GeoALS, which works to improve care, accelerate research, and advocate for the Amyotrophic lateral sclerosis (ALS) community. ALS is a progressive neurodegenerative disease that affects nerve cells in the brain and spinal cord. Dolan, who has lived with ALS for a decade said: “The option to control my power wheelchair on my own is gold to me; Apple is developing life-enhancing technology for the people who need it most.”

For the many

In many ways, these features open up new opportunities for people who are customarily denied at least some of the chances many of us take for granted.

Apple CEO Tim Cook in 2018 explained why Apple places so much focus on accessibility within its platforms. “It’s a basic core value of Apple,” he said. “We don’t make products for a particular group of people; we make products for everybody. We feel very strongly that everyone deserves an equal opportunity and equal access.”

Ultimately, that’s the point with Apple’s approach to accessibility. The company builds all these features into its basic devices, which means people who need them aren’t forced into paying an accessibility tax in order to access the features they need. 

“Apple’s approach to accessibility is unlike any other,” said Cook in a statement. “Now, with Apple Intelligence, we are bringing powerful new capabilities into our accessibility features while maintaining our foundational commitment to privacy by design.”

You can follow me on social media! Join me on BlueSky,  LinkedIn, and Mastodon.EU moves forward on $5.8B scale-up fund to keep startups from leaving – ComputerworldRead More