AI Image Tagging: What can it do for Ecommerce?

In modern day ecommerce, businesses have tried to make the shopping experience as convenient as possible with personalized recommendations or even predictive algorithms. With the advent of AI tech, it begs the question of how it can integrate and improve the ecommerce experience.

Hypothetical Scenario

Let’s say a customer is checking out a white shirt with a specific logo printed on it. With AI, a use case would be to scour the internet or social media for photos of people wearing that specific shirt for the customer to get a better insight into how they would look wearing that shirt. Add AI image tagging into the mix and the customer can choose from a variety of options on which photos they want to see. For example, the customer sets short, red hair with this body type wearing eyeglasses as tags and photos of people matching those characteristics would be returned, creating a bespoke customer experience.

All this sounds good so let’s try to implement it. In my research, I mainly explored the use of Cloudinary, a cloud-based image and video management platform widely used in the ecommerce field. They offer their own AI services like Visual Search and AI image tagging. The workflow looks like this:

Visual Search

Cloudinary offers Visual Search as an experimental feature and is only available for their Enterprise pricing. It is an AI feature that takes in a text, image, or an image url input and searches through the Cloudinary assets library for matches according to that input. What’s great about Visual Search is it does not require the images to have any metadata, tags, or titles to work since it visually analyzes the image bypassing the need for manual tagging.

AI Image Tagging

To generate tags for an image, Cloudinary offers three options: Google, Amazon Rekognition, or Imagga. They all provide tags to an image input but which option to choose depends entirely on your needs and I suggest testing each of them to gauge their compatibility to your use case. Cloudinary also offers AI Content Analysis aside from the three models mentioned previously. These other models have other tagging parameters but what's most useful for product tagging would be their openimages and cld-fashion models. Their tagging results are shown below but you can also try them out for yourself here.

openimages Model

cld-fashion Model

Notice how the tags displayed are generic like the footwear tag could be "sandals" or eyewear could be "sunglasses" instead.

The Catch

AI today is definitely more than capable of detecting specific attributes and other personal features but due to privacy concerns, the tags returned from the tagging services (including Imagga, Google, and Amazon) are limited to generic terms. For example, the tags returned cannot give the exact hair color or hair length, color of shirt, type of bag, model of eyeglasses, and etc.

Cloudinary also offers Advanced Facial Attributes Detection which leverages Microsoft Azure facial recognition but the quality of its tags have also been limited due to privacy issues as stated by the warning below from Cloudinary’s website.

This severely limits the use cases for AI image tagging which prevents it from being a viable ecommerce feature.

Other Avenues To Explore

If the privacy issues cannot be helped, then we can turn towards generative AI. It might be more helpful to instead generate a photo of a person wearing a specific product according to the given tags or parameters.

Aside from image tagging, another separate use case I found is leveraging Amazon Rekognition for an in-house face tagging feature where you can index faces and train your own model. Then, you can use the training data and check for face occurrences in other images.