Apple also previewed the ability to mirror Apple Watch displays on iPhones, and Live Captions, which will add automatic subtitles to any audio content from video calls to streaming video, although only in the US and Canada. This is something Google already offers globally on its Android phones, and both companies have added accessibility features over the years that can read out notifications or describe on-screen elements, let you control your phone by talking or allow users to plug in a switch to navigate with external equipment.
Google previously rolled out Android features that transcribe real-world audio into captions, and describe objects and text seen by the phone’s camera, and is currently testing a system called Project Relate that can adapt its voice recognition to people with conditions that can make their speech difficult to understand.
“Speech recognition technology can potentially change people’s lives around the world, but if you have difficulty speaking, then those voice-activated technologies may not always work well,” said Google Australia’s Disability Alliance lead Lucinda Longcroft.
“We’ve been able to engage with the community, with the help of partner organisations, which enabled us to gather over 1000 hours of speech samples to make Project Relate possible. We want to continue researching and building helpful products for all people, including those of us who have speech impairments, and as a community we can accomplish that goal.”
But accessibility isn’t all about software. A team within Microsoft established a hardware lab in 2017 which created the Adaptive Controller; a gadget that connects to Xbox consoles and allows all standard buttons and sticks to be replaced with any preferred input devices. Last year the company produced the Surface Adaptive Kit; an inexpensive collection of labels, stickers and tools to make opening portable PCs and identifying the various ports and buttons easier.
Microsoft is producing a range of adaptive PC accessories.
Earlier this month Microsoft announced a new line of Adaptive Accessories for PC including buttons, hubs and a customisable mouse, and said its Inclusive Tech Lab was being expanded to become an incubator of new designs in collaboration with disability communities.
And development tools mean solutions can come not only from big tech in collaboration with disability communities, but independent app developers as well.
Aaron Stephenson, a software engineer for an Australian fintech company who also makes apps in his spare time, has developed program that can scan Braille with an iPhone’s camera and transcribe it into English text. At first, it was just a test using Apple’s machine learning tools to see if he could check his own attempts to learn Braille, but when he posted his findings on Twitter he realised there was a demand.
App developer Aaron Stephenson.
“I got like 13,000 views, I got 100 retweets. I just got so much feedback, and good feedback. I had people coming to me saying ‘this would be amazing’,” he said.
“I guess the expectation when people saw the video was that it was an app and it worked, but it was purely a proof of concept.”
Two years on and Braille Scanner officially launched on the App Store last month. Stephenson said he’s been contacted by many people with a huge range of use cases, including someone who wanted to help their nephew with their homework, someone from a theme park who wanted to check their Braille signage was the right way around and readable, an American library which wants to reproduce its collection of deteriorating Braille documents, and an aged care worker who said Braille documents and letters are often thrown out when a person passes away if their family can’t afford to have them transcribed.
The biggest challenge he faces is making the app work reliably for vision impaired people in all situations, as it was only initially designed with flat paper in a well-lit room.
Loading
“I’ve had people from Germany send me photos of medication with Braille on it, and I didn’t even think about there being Braille medication,” he said, adding that the app is built for privacy, so he relies on users sending him problem images rather than letting the app collect them automatically.
“The more data I get from the users, obviously the more accurate I can make it.”
Get news and reviews on technology, gadgets and gaming in our Technology newsletter every Friday. Sign up here.