Hebby
AI voice assistant for H-E-B
Timeline
Sept 2023 - Dec 2023
Ritprabha Suryavanshi
Rachel Sung
Nowshin Mirza
Figma
FigJam
Google Docs
User Research
Design System
Prototyping
Team
Tools
My Role
Overview
H-E-B, a Texas-based supermarket chain, offering a wide range of products, from groceries to household essentials. Their app, MyHEB, is a go-to choice for Texans seeking quick and convenient shopping! For our Inclusive Design class, we developed a concept to integrate assistive technology for individuals with mild to moderate visual impairments.
Problem
How might we provide an experience that assists people with Visual Impairment to do their grocery shopping with ease?
Background
Empathizing with users
Ideating
Testing
Skimming through usable technology
Design System
Final Designs
Existing
First Proposal
Designing an accessible product utilizing voice assistant
Understanding H-E-B
We took a field trip to better understand the experience of people with visual impairment
Focusing on features that can enhance the shopping experience
We tested the prototype in a ‘stable setting’
We also took a trip to H-E-B to find groceries
Utilizing Voice Assistant & Navigation
We utilized color system and motions for easy interpretations
Setting location
Shopping from home
Setting delivery
Shopping at H-E-B
We came up with a new assisting color system with haptic interaction for people with different levels of perception.
After completing onboarding, we designed the process to set the location immediately for users to place orders for delivery without any unnecessary steps. Choosing a location also allows users to familiarize themselves with how to use the voice assistant initially.
For shopping at home, we designed the system to use the voice assistant for product recommendations and comparisons. The assistant reads out a sequence of options present for the user to choose from
In contrast to the traditional H-E-B cart summary, we redesigned it with larger text and voice-over summary. Users can easily set the order date and time during the ordering process.
To facilitate easy navigation we designed a system using the store's layout, allowing users to move easily to different aisles. Additionally, we incorporated a feature that makes it easier to call a personal helper in situations where the user can’t find their desired product.
Once the required option is chosen, the user can ask the assistant to read out nutritional facts, ingredients and dietary restrictions.
Shopper’s Compass: Creating Archetypes
Post-COVID-19, we've seen a surge in technological advancements, like kiosks and virtual consultation services. This shift has paved the way for AI-based products designed to help people achieve their goals more quickly and effortlessly.
After reviewing H-E-B and MyHEB services, it was clear that their focus on customer experience and inclusivity stood out, inspiring us to respect their design process while enhancing it with our ideas.
The product search categories are visually appealing and efficient, especially with the barcode feature. However, the arrangement of categories could be more user-friendly. Additionally, H-E-B currently provides only basic contact information, which is why we thought of integrating a feature for inquiries or assistance especially for people with disabilities.
By visiting the H-E-B at Lake Austin, the team was able to assess the navigation within the store by understanding its layout, categorization, shelves and checkout counters.
Additionally, we verified whether H-E-B still provides assistive features to enhance accessibility in person, while also trying out the H-E-B app with an external assistive feature using voice over commands in-store and through the guidance of a worker.
After getting as many insights as we could, we decided to focus on revamping the interface keeping two personas in mind: slight to moderate visual impairment and color blindness.
Since a majority of the HEB shoppers consist of the elderly (also diagnosed with Glaucoma, Retinopathy and other peripheral field loss), we wanted to make the design as simple and effective as possible.
For ordering from home, the team took turns in ordering 'ground beef' for delivery. From the prototype testing, we noticed a few flaws in the design. We went through multiple iterations for the voice assistant feature and removed unnecessary buttons and features, primarily focusing on haptic feedback.
We took a trip to the store to locate ground beef. We figured that if the navigation worked seamlessly at the store, this feature would be a great support not only for people with visual impairment but for other users with disabilities and for the elderly as well.
However, one important pain point was the difficulty in navigation with a full cart or in a crowded area which had to be considered in future iterations.
This project helped me gain empathy and a deeper understanding of inclusive design principles, and how it makes a product easier and more universal.
We looked into how voice assistant technology works and operates, and shortlisted a few references to see how other technologies function compared to each other.
Sometimes a combination of colors can deliver more information than words. With the target audience having visual impairments, we decided to put emphasis on colors which were color-blind friendly (blue/orange hues) for conveying messages.
Following comprehensive research and interviews, we developed an affinity map to reveal potential users and archetypes. These archetypes played a pivotal role in shaping our user-centered design strategy.
Partial visual impairment
Shops in-store
Home delivery
Color blindness
Learnings
I learned how important it is to think about accessibility early on and how small design changes, can make a big difference. Researching, learning and implementing inclusive design knowledge has made it a cornerstone of my design ethos!
Solution Preview
Option to start accessibility mode during onboarding
Done
Do you need an accessible feature?
I don’t need an accessible feature
Yes, I have visual impairment
Yes, I have color blindness
Generate a code for your buddy
Voice assistant feature provided
Color correction provided
My H-E-B provides accessible feature for visually impaired. Select from below
9:41
Try the Voice Assistant with saying ‘Hey Hebby!’
9:41
Hey Hebby!
I’m here! What can I help you with?
You can also press Hebby to start talking!
Hold the button while speaking to the assistant!
9:41
Pay Close Attention
to the Color!
The Pop-up will show whether the task has been completed or not!
You’ll also get suggestions.
Suggestion
This is our suggestion
Error
There is an error occured.
Complete!
Your task is completed!
Get Started
9:41
Key findings
Opportunities
Difficulty with locating products in store
Interactive navigation with voice assistant
Hard time comparing product details
Bar code scanning & voice assistant for comparison
Voice Assistant:
Technology utilizing Al services with algorithms
Navigator:
System indicating specific products on location.
Hard to locate personal shoppers
Easy-call personal shopper option through app
Lots of information and categorization
Clear categorization through voice assistant
Complex process for setting-up delivery location
Setting up location easily on launch
Prefers online shopping and uses assistive technology like screen readers and voice assistant
Needs some accessibility tools, such as magnifiers or larger text labels
Has difficulty differentiating between products based on color. Relies on other cues like text, shapes, and labels
Prefers online shopping due to color blindness. Finds it easier to compare products online where text descriptions are clear
Home-Dependent Shopper
Independent In-Store Shopper
Confident In-Store Shopper
Home-Comfort Shopper
Done!
Write your message here
Suggestion
Write your message here
Error
Write your message here
Reflection
Diving Deeper
Test the voice assistant with diverse users, including those with various visual impairments, for targeted feedback on accessibility features.
Widening Approach
Enable multi-language and accent support to accommodate a wider user demographic.
Mixing-up technology
Explore options like haptic feedback or sound cues to enhance user experience, especially for those who struggle with voice-only interaction.
Personalization
Personalize the voice assistant's guidance according to individual preferences, interaction speed, and desired level of detail.
Dear Diary,
Future possibilities
Peacock
New Blue
Charcoal Grey
Interested in seeing more?
FXCM Onboarding
MyUT Usability Testing
Hebby
AI voice assistant for H-E-B
Overview
H-E-B, a Texas-based supermarket chain, offering a wide range of products, from groceries to household essentials. Their app, MyHEB, is a go-to choice for Texans seeking quick and convenient shopping! For our Inclusive Design class, we developed a concept to integrate assistive technology for individuals with mild to moderate visual impairments.
Problem
How might we provide an experience that assists people with Visual Impairment to do their grocery shopping with ease?
FXCM Onboarding
MyUT
always down to talk all things f1, accessibility and fun. drop me a line
You have reached the
Timeline
Sept 2023 - Dec 2023
User Research
Design System
Prototyping
Tools
Figma
FigJam
Google Docs
Solution Preview
Done
Do you need an accessible feature?
I don’t need an accessible feature
Yes, I have visual impairment
Yes, I have color blindness
Generate a code for your buddy
Voice assistant feature provided
Color correction provided
My H-E-B provides accessible feature for visually impaired. Select from below
9:41
Try the Voice Assistant with saying ‘Hey Hebby!’
9:41
Hey Hebby!
I’m here! What can I help you with?
You can also press Hebby to start talking!
Hold the button while speaking to the assistant!
9:41
Pay Close Attention
to the Color!
The Pop-up will show whether the task has been completed or not!
You’ll also get suggestions.
Suggestion
This is our suggestion
Error
There is an error occured.
Complete!
Your task is completed!
Get Started
9:41
Learnings
I learned how important it is to think about accessibility early on and how small design changes, can make a big difference. Researching, learning and implementing inclusive design knowledge has made it a cornerstone of my design ethos!
Background
Designing an accessible product utilizing voice assistant
Post-COVID-19, we've seen a surge in technological advancements, like kiosks and virtual consultation services. This shift has paved the way for AI-based products designed to help people achieve their goals more quickly and effortlessly.
Understanding H-E-B
After reviewing H-E-B and MyHEB services, it was clear that their focus on customer experience and inclusivity stood out, inspiring us to respect their design process while enhancing it with our ideas.
The product search categories are visually appealing and efficient, especially with the barcode feature. However, the arrangement of categories could be more user-friendly. Additionally, H-E-B currently provides only basic contact information, which is why we thought of integrating a feature for inquiries or assistance especially for people with disabilities.
Empathizing with users
We took a field trip to better understand the experience of people with visual impairment
By visiting the H-E-B at Lake Austin, the team was able to assess the navigation within the store by understanding its layout, categorization, shelves and checkout counters.
Additionally, we verified whether H-E-B still provides assistive features to enhance accessibility in person, while also trying out the H-E-B app with an external assistive feature using voice over commands in-store and through the guidance of a worker.
Key findings
Difficulty with locating products in store
Hard time comparing product details
Hard to locate personal shoppers
Lots of information and categorization
Complex process for setting-up delivery location
Opportunities
Interactive navigation with voice assistant
Bar code scanning/ voice assistant for comparison
Easy-call personal shopper option through app
Clear categorization through voice assistant
Setting up location easily on launch
Shopper’s Compass: Creating Archetypes
Following comprehensive research and interviews, we developed an affinity map to revel potential users and archetypes. These archetypes played a pivotal role in shaping our user-centered design strategy.
Partial visual impairment
Shops in-store
Home delivery
Color blindness
Prefers online shopping and uses assistive technology like screen readers and voice assistant
Needs some accessibility tools, such as magnifiers or larger text labels
Has difficulty differentiating between products based on color. Relies on other cues like text, shapes, and labels
Prefers online shopping due to color blindness. Finds it easier to compare products online where text descriptions are clear
Home-Dependent Shopper
Independent In-Store Shopper
Confident In-Store Shopper
Home-Comfort Shopper
Ideating
Focusing on features that can enhancing the shopping experience
After getting as many insights as we could, we decided to focus on revamping the interface keeping two personas in mind: slight to moderate visual impairment and color blindness.
Since a majority of the HEB shoppers consist of the elderly (also diagnosed with Glaucoma, Retinopathy and other peripheral field loss), we wanted to make the design as simple and effective as possible.
Skimming through usable technology
Utilizing Voice Assistant & Navigation
We looked into how voice assistant technology works and operates, and shortlisted a few references to see how other technologies function compared to each other.
Voice Assistant:
Technology utilizing Al services with algorithms
Navigator:
System indicating specific products on location.
Design System
Peacock
New Blue
Charcoal Grey
We utilized color system and motions for easy interpretations
Sometimes a combination of colors can deliver more information than words. With the target audience having visual impairments, we decided to put emphasis on colors for conveying messages.
We came up with a new assisting color system with haptic interaction for people with different conditions.
Done!
Write your message here
Suggestion
Write your message here
Error
Write your message here
Final Designs
Setting location
After completing onboarding, we designed the process to set the location immediately for users to place orders for delivery without any unnecessary steps. Choosing a location also allows users to familiarize themselves with how to use the voice assistant initially.
Shopping from home
For shopping at home, we designed the system to use the voice assistant for product recommendations and comparisons. The assistant reads out a sequence of options present for the user to choose from
Once the required option is chosen, the user can ask the assistant to read out nutritional facts, ingredients and dietary restrictions.
Setting delivery
In contrast to the traditional H-E-B cart summary, we redesigned it with larger text and voice-over summary. Users can easily set the order date and time during the ordering process.
Shopping at H-E-B
To facilitate easy navigation we designed a system using the store's layout, allowing users to move easily to different aisles. Additionally, we incorporated a feature that makes it easier to call a personal helper in situations where the user can’t find their desired product.
Testing
We tested the prototype in a ‘stable setting’
For ordering from home, the team took turns in ordering 'ground beef' for delivery. From the prototype testing, we noticed a few flaws in the design. We went through multiple iterations for the voice assistant feature and removed unnecessary buttons and features, primarily focusing on haptic feedback.
We also took a trip to
H-E-B to find groceries
We took a trip to the store to locate ground beef. We figured that if the navigation worked seamlessly at the store, this feature would be a great support not only for people with visual impairment but for other users with disabilities and for the elderly as well.
However, one important pain point was the difficulty in navigation with a full cart or in a crowded area.
Reflection
Dear Diary,
This project helped me gain empathy and a deeper understanding of inclusive design principles, and how it makes a product easier and more universal.
Future possibilities
Diving Deeper
Test the voice assistant with diverse users, including those with various visual impairments, for targeted feedback on accessibility features.
Widening Approach
Enable multi-language and accent support to accommodate a wide user demographic.
Mixing-up technology
Explore options like haptic feedback or sound cues to enhance user experience, especially for those who struggle with voice-only interaction.
Personalization
Personalize the voice assistant's guidance according to individual preferences, interaction speed, and desired level of detail.
MyUT Usability Testing
Interested in seeing more?
My Role
Ritprabha Suryavanshi
Rachel Sung
Nowshin Mirza
Team