Thank you for being here. For a better browsing experience, please use computer to open this website:)

Home Depot

AR, Retail, A11y

Aug. '18 - Dec '18

Yizhou, Shelly, Taylor, Yannu

A more efficient, flexible and smart way to work as an in-store associate.

Design Problem

As a project in collaboration with Home Depot, we tried to solve real day-to-day challenges for them. Our target users are Home Depot in-store associates, they have many challenges on their daily work.

In-store associates are responsible for tons of shelf checking and arrangement every day.
How could we improve their task efficiency and work experience?

Outcome Overview

We created a phone application for these in-store associates. We optimized their workflows and digital experience, also incorporated AR technology to help users check the shelves.


From overview to detailed to-dos, in one click.

Associates can have a better control their daily work. They are able to manage their progress and be guided to optimize their workflow.


Efficient tasking with AR workspace.

Associates can increase the efficiency by scanning the shelves and directly checking planogram information with AR technology.


Report working progress and problems, easy!

Associates can record their working progress and report problem faced in multiple ways. They have choices to do it in a more easy way.


My Contributions

1. I took part in all user research and worked with teammates to analyze our data and distilled the user requirements.
2. I produced all design outputs including high fidelity interfaces, wireframes and user flow charts. Also, I actively contributed in the ideation, sketch and testing phases. I also worked as facilitator and notetaker in both heuristic evaluation and usability testing.
3. I was the coordinator for our industrial partner. I negotiated and scheduled several meetings, also continuously communicated our needs with Home Depot.

.user research

Expert Interview with Home Depot

At the very beginning, we interviewed our industrial partner: Home Depot. We tried to figure out their business objectives and concerns. Also, with zero background knowledge, we designed questions to grasp a general understanding of the user group.


To familiarize ourselves to our target users’ working flow and working pace, we did an observation in the store. We seperated into 2 groups: one followed the associates closely for detailed tasks, another followed in a distance to know their working pace and normal behavior (because staffs might act differently when feeling that they’re being observed).

Pictures taken during observation

> Research Focus

1. What are their detailed tasks conducted every day.
2. What are their work flow and tricks to finish the tasks.
3. How associates work and communicate with each other.

Task Analysis

After observations and expert interviews, we summarized hierarchical task flows of in-store associates' daily work. An interesting fact is that they do not strictly follow the standard workflow expected by Home Depot.
Besides, we also analyzed the workflow of their main stakeholder: in-store supervisors. We found that the communication channels between them are not so smooth and it's hard for supervisors to track associates' work in an efficient way.


Why did not in-store associates follow the standard workflow?

Contextual Inquiry

To understand why this occurs, we started contextual inquiry to know more about users’ thoughts and motivations in their working process.
We conducted contextual inquiry for 4 associates at 2 Home Depot stores separately. As they conducting tasks, we encouraged them to think aloud. At the same time, we observed their behavior, asked clarifying questions, and took notes.

In this process, we found 2 important reasons behind users' behaviors.


Low efficiency, low flexibility and high communication cost.

Affinity Diagram

After having a thorough investigation about associates and their context, an affinity diagram is made to conclude out the data from all former researches. Click here to check the details of Affinity Map.

Wrap-up: Jobs-to-be-Done

When the project went here, we just learned about a UX technique called Jobs-to-be-done.

We incorporated it to walk through the entire user experience based on our insights from affinity diagram. Building empathy from both task-based and cognitive perspective, we thought more from associates' standpoints and finalized the user needs and expectations.

Design Principles & Success Criterias

Finally, we prioritized user requirements and expectations, then summarized the following design principles to guide the next stage of design. These principles are also used as our success criterias for self-evaluation.


From 100+ ideas to 3 design alternatives, then to the final solution.


After defining the desired outcomes and pain points, we conducted 2 rounds of brainstorming and generate 100+ possible ideas. At first round, we individually come up with ideas and then discussed together to group them into 12 categories. Based on these, we brainstormed more ideas and created a mindmap to wrap it up.
Click here to check the details of Mindmap.

Idea Evaluation

We evaluated our ideas in 2 dimensions: Impact and Feasibility based on our industy partner's expectation. We picked up all the ideas with high impact and feasibility to explore about their potentials - And then, merge them into 3 design directions.

> Design Directions

Design Alternatives

We built 3 design alternatives based on the design directions. For each alternative, we tried to design effective features in an intuitive user flow - then tell a good user story.

1. Tasking with AR+Camera

Associates could use the camera to scan the bay areas and shelves and AI could detect and analysis the content, recognize the potential availability and shoppability problems. Associates could just need to follow the guidance and finish the checking.

2. Tasking with Audio

Associates will wear earphones and use voice to ask about instructions, report the working progress and problems and communicating with supervisors for help. Using NLP, system will generate related options, actions and information to assist users' work.

3. Tasking with Phone Screen

Detailed tasks are categorized and users can choose their own tasking sequence. We named it as 'freestyle' checking. Also, for in-store projects, they can set alarms to remind of their own schedule and do report by uploading photos.

Design Direction Evaluation

We created paper prototypes first and invited 2 experts in Home Depot to evaluate these design directions with us. We created 3 evaluation dimensions and discuss about the pros and cons of each option.

Based on our analysis, we decided our key features.

> Evaluation Dimensions

1. Design Principles
2. Comply to Associates' Habits
3. Business Objective Satisfaction

Paper prototypes

Interfaces and
user stories

We turned the directions and principles into detailed design.


After deciding the design directions, we refined our design alternatives and combine the features. Then, we went to the home depot store and ran a quick feedback session and went through it with 3 associates. The feedback we got helped us gain more insight to make the prototype.

    User Testing Results and Refinements

    Generally speaking, users felt the features and entire flow are effective and usable for them. But we identified several issues that associates are concerned about, and we iterated based on these feedback.

    > Most Important Issue: More Convenient AR Workspace

    Associates' main concern falls in the process of 'AR Planogram Checking': Can AR help more with complex tasks such as layout checking? Therefore, besides one-by-one item info checking, we redesigned the AR workspace and provided a more convenient way.

> Other Feedback and Refinements

Hifi interfaces

I personally created high fidelity interfaces and make modification based on user feedback.

.test and iteration

We made a prototype in Invision. It can be seen here.
We conducted heuristic evaluation and task-based user tests to identify usability problems and collect feedbacks on design concepts and interfaces.

Heuristic Evaluation

The goal for heuristic evaluation is to check if the overall experience of the system is seamless and fluent and whether our design language match their guidelines.To do this, we can quickly get inexpensive feedback from Home Depot UX team and identify important usability issues.
We conducted it with 5 UX designers in Home Depot. Before that, we identified 7 evaluation dimensions based on the pain points we identified for end-users.

Me explaining our design to Alan, Home Depot UX designer.

> Evaluation Matrix

1. Match between system and the real world
2. Flexibility in user flow
3. Efficiency of use
4. Accessibility
5. Error Prevention
6. Consistenct and standards
7. Aesthetic and minimalist design

We categorized our findings in the heurstic evalution in 7 evaluation matrix that we identified before. The details of the usability issues we obtained from them can be viewed here.

Task-based Usability Testing

After heursitic evaluation, we conducted task-basked usability tests with 3 associates. Our goal was to understand if our designs followed user’s mental models and to find out if they perceived our solutions as better than the current solution.
We created 3 main tasks for them to do.

> Tasks for test takers

1.Get an overview of the bays you need to finish today and then finish the tasks in the first bay including AR POG checking and shoppability checking.
2.Complete the tasks using freestyle mode and use voice to report an issue.
3.Get an overview of the projects, complete the “Reset installation” project, and set an alarm.

Testing Session with a participant

> Procedure

1. Introduce the project goal, and put users at ease.
2. Describe the 3 tasks and their scenarios, let users use the prototype while thinking aloud.
3. Ask users to finish ASQ questionnaire everytime they finish a benchmark task.
4.Let them fill up SUS questionnaire and ask follow-up questions.

We got a avergae SUS score of 85.83, which indicates that that our system has adequate usability. Also, we summerized the ASQ scores and insights from follow-up questions, and found out some detailed issues of the interfaces. They can be viewed here.

Then we dealt with the usability issues that identified in the former tests and iterated our design. Below are the modifations for 2 main issues that we identified.

Iteration 1: A more accessible and readable AR workspace

The working contexts are complex in Home Depot stores. When users are using AR workspace, they had the problems of readability and button haptics. Therefore, we tried to redesign how users see the information in AR workspace.

Iteration 2: Learnability Issues

In the tests, one big problem we identified is that the system is not easy for users to learn. Users are confused about what a certain button for or why this notificatin shows up here. Even though the training for APP usage can be a solution, we conducted iteration to reduce users' unnecessary learning cost.

Iteration 3: Scan to start

Users really love the idea of scanning products/shelves and get information. They felt that it's helpful not only when doing shelf checking, but also when helping customers or other scenarios. Therefore, we walked through their working experience and made this function more available at every stage.

.what's next

Future steps

Supervisors' experience

As the most important stakeholder for associates, supervisors' current experience cannot satisfy their needs too. Based on our research, we found that they are difficult to have an overall grasp of his team situation and usually need to spend extra time for communication and allocation. It would be great if we can design something for him.

Better test methods for AR

For AR part in the project, we just used stastic pages and a selected bay area to let associates try the prototype. If we can really prototype something out, users may be able to have a better understanding and we can get a deeper insight about how AR works better for them.


Be practical but always think of something big.

This is the problem that we have always faced. In this project, we came up with the AR assistant to help associates improve their efficiency. Then, after talking to Alan, we started to think about how AR could be applied to the whole retailing area, such as to the customers’ side, to the whole associates’ working flow, etc. But in reality there are many constraints in the store and the cost often outweighs the benefit. So a lot of our innovative ideas have not been prototyped for practical reasons. However, we think these ideas are awesome and we envision that in the future, AR technology could be applied to different interesting scenarios as we thought.

Your ideas, especially the AR one really give me many inspirations about how our products can be improved.

——— Quote from Alan, UX manager of Home Depot

To go further, go together.

I have an amazing team for this project! We collaborated so well and supported each others in the whole process. One important lesson that I learned is that keeping on the same page and trying to achieve consensus are so important for a team, especially for 4 people from different backgrounds like us. Take a look at the portfolios of my talented teammates: Xi Chen, Taylor Stillman, Yannu Lee.




Google Internship