Evaluating an AI Shopping Cart (Acquired by Instacart) for Adoption at Scale
Project Overview
Working with the Kroger Technology team, I was tasked with evaluating the viability and usability of Caper AI. As lead researcher, I planned the study, recruited customers, moderated each shopping trip, synthesized data, and reported findings.
Study Design, Moderation, Insight Synthesis
Stakeholder Interviews
Microsoft Office
Retail
Study Setup & Prep
I began by speaking with Kroger's Customer Technology team, a group of product managers, store managers, and technology specialists, to understand the product and business goals. The product, Caper AI, is an intelligent shopping cart powered by AI. It promised the following improvements to the in-store customer experience:
Convenience
Instant Scanning: Shoppers scan and weigh items as they place them into the cart
Real-Time Payments: Payments are made directly through the cart, eliminating the formal checkout process
Integrated Display: An interactive touchscreen that helps customers find items, suggest recipes, or offer promotions based on what they are buying
Personalized Shopping Experience
Item Recommendations: AI product recommendations based on shopper behavior and purchases
Navigational Assistance: Navigation recommendations based on shopping list
Personalized coupons: Personalized coupons based on location and shopper behavior
From a business standpoint, it promised:
Increased Throughput: By eliminating some of the checkout process, customers shopped more quickly, reducing store traffic and improving flow.
Inventory and Data Insights: Real-time data collection to drive store personalization and optimize inventory management.
Security & Loss Prevention: Built-in scales and sensors help reduce theft and checkout errors.
Since our team could not evaluate every aspect of the cart in a single test, we prioritized our focus areas. The most important area was security and loss prevention. How would the cart handle foreign items? Would it be easy to steal? Or add similar weighted (but more expensive) items? How would the cart handle these errors? How would it help the customer remedy the issue?
Secondly, shopping efficiency. Would this decrease shopping trip time? By how much? How would customers perceive it? Was checkout as seamless and easy as self scan or mobile checkout?
Finally, instant scanning. How would the cart handle scanning and inputting items? How would it deal with items of varying size, weight, packaging, temperature, consistency? How would customers feel? Would it be a "I can't live without this" moment or be met with vague enthusiasm?
The study and discussion guide took shape. I planned for a study involving three parts with customers:
Customer Interview
- Can you describe your typical grocery shopping routine?
- How often do you shop?
- Do you usually shop in-store, online, or both?
- How long do you usually spend grocery shopping?
- Can you describe the types of items you and your family usually buy?
- What are the most important factors for you when grocery shopping? Speed? Convenience? Cost?
- Do you usually make a shopping list or buy as you go? How important is this to you?
- Can you talk to me about how you approach couponing, sales and deals while shopping?
- How do you usually navigate the checkout process? Lane? Self Checkout?
- What do you think about it?
- Can you talk to me about how you use technology (mobile apps, digital coupons, etc.) in your grocery shopping?
- How do you navigate new or large stores?
- Do you often ask store associates for help navigating the store or finding items? If yes, can you walk me through this process. If not, why not? How do you get help on your own?
Mock Shopping Trip
Consulting with stakeholders, we chose 14 items of varying size, weight, packaging and temperature consisting of a wide range of products to accurately represent the many scenarios customers might find themselves in; heavy items, small items, refrigerated items, flexible packaging items etc. The mock shopping trip was a contextual inquiry format in which participants were asked to fulfill the list as if the researcher was not there and rate each item on a scale of 1-5 to quantify difficulty. Time to complete was measured in the background.
Follow Up Interview
Finally, a post-trip interview was conducted to evaluate customer experience and perception.
Test Execution & Moderation
I met with 6 customers at our testing store in Northern Kentucky.
Findings
After conducting the sessions, I synthesized the results into a report including an executive summary of key learnings, both qualitative and quantitative, pain points, benefits, and considerations; as well as raw interview data in the form of observations, and quotes.
Avg. Trip Time: 25.2min
Average Item Rating:
Pain Points
Cart Size
Cart size was a pain point. The majority of users expressed that it was too small and did not have a bottom section for large bulky items such as dog food and water or options for small children.
Getting Started
Customers expressed the need for more onboarding information and details about how the cart works. They didn't fully understand everything they needed to do because it was so new to them and they had never experienced something so different. This left them feeling intimidated.
Interface
Removing/Rearranging items
Customers expressed the need for more information and signifiers around removing and rearranging items in the cart. They weren't always clear on what items were removed or added back because the cart does it automatically. This made them feel less confident they had everything they needed.
Customers said not being able to see a running price total was a pain point.
Customers wanted the option to upload their shopping list and view their list by aisle location.
Security
Items of very similar weight could be substituted for one another without recognition. For example, chips of different brands with the same weight and package size could be freely substituted. This could result in inaccurate inventory throughout the store.
Benefits
Scanning
This was a delight moment. Customers were happily surprised by the ease and quickness of scanning items. Item variety was not a significant factor. Produce, large items, flimsy items, frozen items, the cart had little issue scanning.
Convenience
Customers liked that everything was self-contained in the cart. They could scan, weigh, and check out seamlessly. They really liked not having to interact with a traditional checkout process and, for those who used apps to support their shopping, did not have to hold a second device while shopping.
Auditory Feedback
Customers enjoyed the auditory feedback the cart gave them. They didn't always have to look at the screen when navigating aisles and products.
The full report was shared with the Customer Technology team and subsequent studies were conducted.
The Outcome
The work I began with the Customer Technology team resulted in a multi-store pilot in the Cincinnati and Northern Kentucky area. Since then, Caper AI was acquired by Instacart for $350 million.
