Introduction to Retail AI

Welcome to our series “Ultimate Guide to!” where we’ve gathered all our most useful insights in a bite-sized article to give you the opportunity to get up to speed with everything you need to know about annotation in your industry! Today we’ll be exploring the Retail annotation applications.

AI technology has become a significant part of business success and we have all noticed it used even in our local chain retailers and supermarkets. For example, in recent years we saw tech giants like Amazon launch cashierless partially automated stores like Amazon Go across the UK and the US.

Data annotation services like shelf analysis, product recognition, barcode analysis, smart carts, customer behavior tracking, and item detection play a significant role in developing solutions which help retailers improve both the customer and the employee experience.

Retail AI Dataset

Supermarket Shelves Dataset

Got a product recognition or detection model you’d like to test? Why not try our new Retail dataset – it’s as easy as 1, 2, 3!

Challenges and best practices

Based on our extensive experience annotating data for Retail AI, we are sharing below some of our best practices and tips on how to ensure your AI project will be a success:

Challenge

Best Practice

When performing stock monitoring and shelf analysis, retailers have to deal with an extremely high number of unique SKUs (stock keeping units) which is much higher than in standard annotation projects, frequently in the thousands.

For SKU labeling, we use some of the best tools on the market which support reference databases of SKUs with information about each product (brand, size, etc) as well as a reference image that annotators can refer to. Some other helpful functions include batch labeling of SKU data and searching through the SKU database using product name or category. We start with smaller batches of SKUs and then scale to cover the entire inventory once the labelers have acquired the necessary expertise.

Shelf analysis and stock monitoring is a task that has to be completed on a continuous basis and the training data always needs to be refreshed because of ever changing promotional layouts, seasonal pricing, new products, etc.

This can be solved best by having a continuous labeling process to accompany the deployment of AI models: whenever there is new data available or data drift detected, it needs to be sent to your dedicated labeling team so that they can annotate it and generate the ground truth necessary to verify whether your models are performing well on this new data they are encountering.

When it comes to shopper behavior tracking in physical stores, even if you have a very comprehensive training dataset, humans are quite unpredictable and during deployment AI models may come across unexpected scenarios and actions that they would not know how to analyze.

Whenever there is an alert triggered by an unknown action or unexpected shopper behavior, it’s best to have a 24/7 dedicated team of humans-in-the-loop who are available to handle such types of edge cases and alerts in real time. In this way, the shopper action can be correctly classified and the alert could be either promoted or rejected so that the staff on the ground can react accordingly.

This is why it’s really important to start with the right training data and to adopt a human-in-the-loop approach in order to continuously improve the document classification and data extraction AI models that you are using!

Types of annotations for Retail AI

Below you can find some of the most common types of annotation that are required for Retail purposes, as well as some tips and tricks for making the most of them:

Shelf analysis pre annotation exampleShelf analysis post annotation example

Shelf analysis

Monitoring the stocks on shelves can be very useful for supplier logistics and inventory management. Product positioning can be analyzed using bounding boxes, including for the detection of empty spaces, as well as incorrectly placed or stacked products.

Product recognition

Product recognition is a challenging task due to the enormous number of different products and the fact that many of them look similar. By using special tool add-ons for SKUs, our workforce can make sure each product is matched to its SKU, even if it’s one in a million.

Product recognition post annotation exampleProduct recognition post annotation example
Barcode analysis pre annotation exampleBarcode analysis post annotation example

Barcode analysis

While traditional barcodes have to be scanned once at a time, computer vision models can be trained on labeled data to recognise and process barcodes at scale. We can provide labeling for the entire barcode, as well as separate labels and transcription for each number.

Smart carts

The way people shop has never been easier. Smart carts equipped with cameras can be trained on labeled datasets in order to identify, classify, and record the contents even when there is occlusion – which is where polygonal annotation comes in handy. Again, an SKU database can be used in order to identify each unique object based on the retailer’s own classification.

smart carts pre annotation examplesmart carts post annotation example
Shopper tracking pre-annotation exampleShopper tracking post-annotation example

Shopper tracking

Analyzing the behavior of customers in a store is essential for making cashier-free shops a reality. The labeling can consist of bounding boxes with unique IDs for easy tracking across frames, skeleton keypoints for more precise movement detection, detailed action tracking across video frames, as well as product localization and intention prediction.

E-commerce

In order to classify products for search engines at scale, automatic tagging is needed for e-commerce applications. By tagging each product with the appropriate tags according to the search filters of the website (eg type, color, material, brand, size, etc), our annotators can make sure clients discover exactly what they want. Garments and objects can also be labeled on user generated videos in order to create unique shopping experiences which enable viewers to click on the item they like and buy it instantly.

E-commerce pre segmentation exampleE-commerce post segmentation
Shelf analysis pre annotation exampleShelf analysis post annotation example

Shelf analysis

Monitoring the stocks on shelves can be very useful for supplier logistics and inventory management. Product positioning can be analyzed using bounding boxes, including for the detection of empt3

y spaces, as well as incorrectly placed or stacked products.

Product recognition post annotation exampleProduct recognition post annotation example

Product recognition

Product recognition is a challenging task due to the enormous number of different products and the fact that many of them look similar. By using special tool add-ons for SKUs, our workforce can make sure each product is matched to its SKU, even if it’s one in a million.

Barcode analysis pre annotation exampleBarcode analysis post annotation example

Barcode analysis

While traditional barcodes have to be scanned once at a time, computer vision models can be trained on labeled data to recognise and process barcodes at scale. We can provide labeling for the entire barcode, as well as separate labels and transcription for each number.

smart carts pre annotation examplesmart carts post annotation example

Smart carts

The way people shop has never been easier. Smart carts equipped with cameras can be trained on labeled datasets in order to identify, classify, and record the contents even when there is occlusion – which is where polygonal annotation comes in handy. Again, an SKU database can be used in order to identify each unique object based on the retailer’s own classification.

Shopper tracking pre-annotation exampleShopper tracking post-annotation example

Shopper tracking

Analyzing the behavior of customers in a store is essential for making cashier-free shops a reality. The labeling can consist of bounding boxes with unique IDs for easy tracking across frames, skeleton keypoints for more precise movement detection, detailed action tracking across video frames, as well as product localization and intention prediction.

E-commerce pre segmentation exampleE-commerce post segmentation

E-commerce

In order to classify products for search engines at scale, automatic tagging is needed for e-commerce applications. By tagging each product with the appropriate tags according to the search filters of the website (eg type, color, material, brand, size, etc), our annotators can make sure clients discover exactly what they want. Garments and objects can also be labeled on user generated videos in order to create unique shopping experiences which enable viewers to click on the item they like and buy it instantly.

Tools we love

While data is paramount, the tools we use to process it are of no less importance! Here are some of our tips and recommendations re: tools we’ve used for Retail AI Annotation.

This is our most prominent choice. The feature to track the movement of objects, especially for video annotation, is a life-changer. As an open source tool, it’s the most cost-effective option out there for getting started with your retail AI projects. 

Great tool overall which provides a unique SKU database add-on for your more complex projects. It also provides advanced functionalities for object tracking on videos, as well as neural networks integrations.

Blogs

Best Open Source Tools for 2022

Found your humans – now all you need is the right tool for the job? Here’s our review of the most popular tools for 2022!

How to use a human-in-the-loop for Retail AI

Annotation for Retail AI is a continuous operation within enterprises and in order to keep models up to date and to handle the inevitable data drift, it’s important to use human input on a continuous basis, not just for the initial training of your models. Here are some of the ways in which humans can be plugged into the entire MLOps cycle:

1.Ground truth annotation: in order to train your initial models, we offer full dataset annotation from scratch in batches: anything from classification to bounding box annotation, polygons, video frame-by-frame annotation, and others, even with thousands of classes 

2.Output validation with active learning: once you’ve trained an initial model, we can use it in order to pre-annotate a large part of the dataset, which will both increase the speed of the annotators and the impact of their work, by setting up an active learning workflow and prioritizing instances where your model is least certain

3.Real-time edge case handling: once you have a model in deployment, our humans-in-the-loop are available 24/7 to handle potential edge cases that appear in real time or close to real time, using a simple API request and sending the correct response in seconds in order to ensure a second layer of verification for your model’s most critical responses

Wondering who is annotating your data?

When you are hiring a company to help you with your annotation needs, you frequently never meet the workers who are labeling your data. We want to change this and present to you the inspiring stories of our annotators!

HITL Retail Annotation team quote Abdul

Does this sound like something you’d like to try out? Get in touch with us and we’d be happy to schedule a free trial so as to explore how we can best help you with your retail annotation needs!

Get In Touch

We’re an award winning social enterprise powering the AI solutions of the future