FreshSight

An AI-powered application for the visually impaired that assists in navigating the store, selecting quality of products, and managing purchased items.
Overview
Problem

Visually impaired individuals face significant challenges during their grocery shopping experience. They struggle with navigating stores, selecting quality of products, and managing purchased items, which can lead to a lack of independence, difficulty in ensuring food quality, and increased food spoilage. These difficulties can significantly impact their overall shopping experience and everyday life.

Goal

Enhancing the grocery shopping experience for visually impaired individuals, empowering their independence and increasing their satisfaction.

Team

team of 3

Role

UX Researcher
UI designer

duration

16 weeks

outcome

Final application design & solution

All features are designed with a voice assistant to aid visually impaired users.

01
Navigate the store

Visually impaired users can tell the application what items to buy, and it will navigate the store based on the store layout.

02
Tells the surroundings

During navigation mode, the application will also alert visually impaired users to their surroundings and read out relevant information aloud.y, and it will navigate the store based on the store layout.

03
Analyze products

This feature analyzes the quality of products, then reads out the condition, expiration date, and other relevant information to help visually impaired users decide whether to pick the item or not.

04
Remind the best before date

This feature can remind visually impaired users of the items they have and alert them to the best-before dates to prevent forgotten and expired foods.

So...why did we design FreshSight ?

Simple tasks like grocery shopping can pose significant challenges for visually impaired individuals. While sighted shoppers may take for granted the ability to find specific products, navigate aisles, and read labels, these tasks often require assistance for those who are visually impaired. However, they too deserve the opportunity to shop without relying solely on sight. By providing the right tools and support, we can empower visually impaired individuals to navigate grocery stores on their own, enhancing their independence, confidence, and overall satisfaction.

70% of visually impaired shoppers rely on assistance from family, friends, or store employees when grocery shopping.

42% of visually impaired individuals report difficulties with grocery shopping. Challenges include identifying products, reading labels, and navigating stores.

empathize

Defining Target user: legal blindness

85% of legally blind still have some remaining vision and perceive light.

This residual sight often allows them to perceive shapes, colors, or light, providing some assistance in navigating and recognizing objects. This capability makes them willing to go grocery shopping to maintain a sense of independence in their daily lives.

secondary research

How do visually impaired do grocery shopping?

✔︎ Use a smartphone application to scan each item.
✔︎ Uncertainty about which section an item is located in.

market research

✔︎ None of the existing products cater to their specific daily life needs.

✔︎ AI provides real-time assistance, consistency, and accuracy without the need for human intervention.

This process allows us to understand the current landscape of available products, identify gaps in the market, and uncover unaddressed needs of visually impaired individuals. By analyzing market, we can position our product more effectively, ensuring it stands out and provides unique value to visually impaired users.

40% of visually impaired individuals use assistive technology to identify products while shopping.

However, these apps still fail to meet the needs of daily enjoyment and overall life satisfaction.

Most existing applications designed to aid visually impaired individuals use AI-powered technology.

User journey map
User research

Pain Points:
✔︎ Navigating the store
✔︎ Selecting quality products
✔︎ Managing purchased items

Through secondary research, field observations, and user & expert interviews, we gained a deeper understanding of our target users. This helped us identify pain points and three main needs and wants of visually impaired individuals.

Navigating

Navigating the store to find the needed items.

Selecting

Selecting the ripeness and quality of produce, as well as distinguishing similar-looking produce or packages.

managing

Managing purchased groceries to receive alerts about expiration dates.

User journey map

Problem Statement

How might we enhance the grocery shopping experience for the visually impaired by developing an assistive application that empowers their independence and satisfaction?

Ideation

Sound, voice, gesture control, and physical feedback are essential in designing for the visually impaired.

01
Simplified & Accessible

Implement clear and consistent navigation with large, easy-to-tap buttons to prevent misclicks and enhance intuitiveness. Use high-contrast color schemes to ensure readability for the visually impaired.

02
Voice access

Incorporate voice commands to allow users to navigate and perform actions hands-free, and implement Text-to-Speech functionality to read out content, notifications, and instructions, enhancing accessibility.

03
Gesture Control

Use simple gestures like swiping and double-tapping, complemented by haptic feedback to confirm actions and navigation, ensuring a seamless user experience for visually impaired users.

design

Initial design

Introduction video

User testing

Conducted user testing with visually impaired individuals and simulated real conditions with sighted people, given the challenge of finding visually impaired testers. This approach allowed us to ensure the app could be operated without relying on vision and met the needs of visually impaired users.

Test items

• Voice Command Navigation
• Indication Clarity
• Task Completion Intuitiveness
• Overall user experience

Restriction

Not sure if the prototype works well with the phone’s accessibility features and cannot add haptic feedback and real-time camera functionality.

Results

1. The content contained word redundancy and inaccuracies, making it less accessible for visually impaired users.
2. The buttons were found to be too narrow, making them difficult to click.
3. Not colorblind-friendly, lacking accessible color contrast and symbols.

Iteration

Through the user testing phase, we gained a deeper understanding of the needs of visually impaired users. By putting ourselves in their shoes, we enhanced the product to be more user-friendly for the visually impaired.

Reflection

Where to Improve

Borden user pool, testing and iterating

During this project, we encountered time constraints and limited resources that restricted our primary research and user testing to a small group. I wish to conduct more usability testing with visually impaired individuals, as this would provide more feedback and allow us to better iterate and improve the product.

What I’ve learned

Importance of accessibility consideration!

This was my first project designing for a minority group, and it profoundly influenced my perspective on design. Understanding the importance of accessibility has taught me to consider the diverse needs of all users, ensuring that everyone can benefit from and enjoy the products I design. This approach not only enhances usability but also promotes equality and inclusion, making accessibility a fundamental aspect of my design philosophy.