Have you ever seen a product or outfit that you liked but didn't know how to find online? Maybe you want a certain aesthetic, and putting that idea into words isn't really working.

Try typing "green shoulder ties dress flowy bottom" into your favorite apparel store's search bar. If it doesn't return the right results, they're likely not using visual discovery.

To showcase what's possible, we built an app using hundreds of stores: try it out.

Here's an example of it in action:

Shoppers can even combine pictures with words to find exactly what they're looking for. Search likes "I want this product, but in a retro style" are possible.

So let's dive into visual shopping and how it will shape the future of eCommerce.

What is Visual Shopping? Is it Really a Game-Changer?

Starting with the science: Amazon published a paper where they introduced visual search via an A/B test and noticed an entire 2% increase in CTR (clickthrough-rate):

For a company that does 1.6m purchases per day [1], that's an instant increase in 32,000 purchases. Per day. This means every store that isn't using this technology is losing out on sales.

Making the Most of It

There's a lot that goes into these systems, but let's highlight what they offer your shoppers in order to increase your store's conversion rates.

Search by Description

Traditional online shopping supports keywords today, but the challenge is if those keywords aren't associated with the product, they won't turn up. Typically this is your merchandiser that's responsible for product tagging.

Ross, Marshall's, and TJ Maxx can attribute much of their success to the treasure hunt of product discovery. Adding product discovery like this online is entirely possible with the right tooling.

Search by Image

Rather than writing a detailed and possibly incorrect description, just upload a picture. Thankfully innovative companies are starting to adopt this tech:

Search by Image & Description

You could even upload a picture of a trendy red sneaker and get recommendations for similar styles or different colors.

Here's an example, where we uploaded a picture of a dress and combined it with the terms "longer and brown":

Some other popular combinations can be "this picture but a retro style", or "this dress but with a bow".

Adding Precision to Your Search: Refine, and Retry

You can even select the products you like, and refine them with another search. This takes the images you like and combines them with text to deliver results that overlap in similarity.

Shopping your Feeds

Pictures and videos across the internet feature products your shoppers want. What if you could automatically tag them:

You can even tag products across videos down to the timestamp:

Context is King

Let's say you search "polo shirts":

Then search "green shirts":

If the first result is not a green polo shirt, then the search failed to understand the history of the users' activity.

Visual shopping should understand how the user arrived at this search. It should take into account the interaction history of the user. They search a term, click a product then search another term, the results should be re-ordered to optimize for what they're looking for.

Maximize Purchase Size with Bundling

Often you not only want to maximize the volume of purchases, but the size of each purchase. Therein lies the importance of bundling, which these systems understand how to do effectively.

For example, if a user is looking to purchase a yoga mat, you should certainly be presenting other recovery-related products. Amazon tends to do this quite well:

Typically, again you'd rely on the merchandiser to make these associations, but with visual search systems, you can sit back and watch the A.I. make these categories automatically.

Another similar addition is to do a reverse image search on the same product. This gets you similar-looking products:

So How do These Systems Understand my products?

Building a visual search system is complicated, here are the steps:

1. Establish bounding box - This allows our system to understand which parts of the image are the product to look up
2. Categorize products - Different products are stored in different collections, we'll need to use the correct collection
3. Associate keywords - Trends change daily, you need your search to understand that a "dad hat" is different from a "baseball cap"
4. Brand & sizing filtering - One size certainly does not fit all, your users have specific criteria
5. Product search - Finally conduct the search and return results

Here's what a final outcome could look like:

Point and Click Solution, That Just Works

Your sales rely on the quality of your search results. Nailing down that perfect relevance is not easy.

Our eCommerce experts at Mixpeek take time to understand your catalog and your shoppers' habits to build a custom A.I. model that maximizes your stores' conversions.

Don't believe us? We'll put our money where our mouth is and conduct an A/B test for you, completely free of charge:

Have questions on how to build this yourself? We're happy to share insights ☺️  Just ask to speak with an engineer.