Home > Emerging Tech Taobao Using AI to Help Sight-Impaired Users Shop By Christine Chou | Published on Jan. 22, 2019 Taobao, China’s largest online marketplace, is taking steps to make shopping more accessible to blind and partially sighted users, launching technologies to help navigate the site designed specifically for this demographic. For one, Taobao last October added Optical Character Recognition technology to its pages, which is an artificial intelligence-driven feature that reads text written on images. Before adopting OCR, Taobao’s 300,000 daily active users who are blind or have reduced vision would have used screen-reading software that simply announced “image” as it scanned the page. By early December, OCR was being used to read close to 100 million images per day, Taobao said. “Images are becoming ever more important in the shopping experience,” said Wang Yongpan, the algorithm specialist that led the OCR upgrade. “A typical product page on the site contains about 40 images, and most product specifications and descriptions are often found within images, rather than typed out in plain text.” Wang noted that while Alibaba has been using OCR for many years in various capacities, the technology’s accuracy in reading images has grown exponentially thanks to advances in machine learning. Now, that accuracy is being put to work for the disabled. Taobao’s Optical Character Recognition technology helps visually impaired consumers read both the text and images on each page. Independent Living For 31-year-old Zheng Rui, who is among the 285 million people worldwide recognized by the World Health Organization as living with some form of visual impairment, accessibility technology brings more than mere convenience. “Barrier-free shopping helps us live more independently, with dignity,” he said, adding that shopping at brick-and-mortar stores can be an awkward and troublesome experience for the visually impaired. He gave the example of women buying feminine hygiene products, who might feel embarrassed about seeking help from others, such as male family members or passersby, to pick the right product. Today, Zheng is his family’s chief shopper, procuring everything from household necessities to toys for his 1-year-old child. He can complete all the steps in the buyer’s journey, from searching for the product to payment, in less than a minute. Making the platform more inclusive, user-friendly and a home for creativity is part of the larger strategy outlined by Taobao President Jiang Fan last May. “If I had to do one thing this year, that would be to make Taobao simpler and bring [us] back to our original purpose,” Jiang said. Alibaba is famously known by its motto, “To make it easy to do business anywhere.” Differently Abled: Voices From Alibaba’s Frontline of Customer Experience Barrier-Free Technology This wasn’t the first time Alibaba introduced accessibility technology. The OCR launch was driven by the company’s “Barrier-Free Lab,” which started with a handful of employees in 2011 and has since grown to hundreds of volunteers, ranging from programmers to user-experience designers. Now, similar tools can be seen across Alibaba’s ecosystem, expanding from Taobao to B2C e-commerce site Tmall, payments affiliate Alipay, online delivery platform Ele.me, enterprise chat app Dingtalk, navigation firm Amap, music streaming app Xiami and internet browser UC Web, from desktop to mobile. A thin, textured film added to the touchscreen helps the visually impaired navigate through features on their smartphones. Last November, DAMO Academy, the e-commerce giant’s $15 billion global research program, revealed that it has been working with China’s Tsinghua University to improve the smartphone experience for the blind and partially sighted. Smart Touch, one of their latest innovations, is set to launch later this year. Users pair a specially designed mobile app with a silicone film on top of their phone screen for a new, intelligent interface. The thin film has multiple textured touchscreen buttons that users can feel along the edges of the screen, representing actions such as “Confirm” and “Return to Homepage.” Each button can lead to different destinations depending on the app, such as “My Shopping Cart,” “Tmall Global,” and “Tmall Supermarket” for Mobile Taobao. Smart Touch also supports an “ear touch” feature, which gives blind and visually impaired users a simple way to listen to text clearly and privately in public, without the need for headphones. It uses a shape-finding algorithm to sense when users hold the phone to their ear and automatically routes the sound output from the loudspeaker to the earpiece speaker. Globally, many tech companies have been devoting resources to improve digital accessibility. Facebook has been working on technology to help the blind hear text descriptions of photos shared across its platforms, including Instagram. Microsoft has been developing 3D soundscape technology to help visually impaired people navigate the world around them, a similar goal that companies such as IBM and Google have been working towards as well. Barrier-FreeDAMO Academy Reuse this content Never Miss a StorySubscribe to Our Newsletter
