Vector Search Primer for Normies
Search UX is more than ‘ease of use’ or task completion while that is certainly part of it. The experience of search is key to a successful search use case (ecommerce, app, etc.) Jakob Nielsen’s Law of Internet User Experience famously says that most users of your site or app spend their time elsewhere. This is brutally true with search.
In the retail ecommerce space, where sales are expected to exceed 8 trillion dollars by 2026 providing a search experience that meets or exceeds a users expectations is (or should be) a competitive advanced. The big retailers are setting expectation for users. The technology stack is vital to providing experience users are expecting. A key technology component to consider in building a better search experience is vector search.
What is vector search?
Traditionally, search experiences rely on keyword-based algorithms, like Frequency/Inverse Document Frequency (TF-IDF), so a searchers’ request is made against an index that might contain the content that they’re looking for based on this algroithm. This approach is limited by the effort in balancing precision and recall (also know as relevancy).
Vector search is a different approach for both information retrieval and relevancy. The experience can be magical. Using a collection of Machine Learning techniques, vector search finds information by meaning. Vector search is faster and more relevant than traditional lexical based (keyword) search. Matching on tokens and their variants vector search uses vector to determine the ‘nearest neighbor’ or similarity.
Why vector search?
Better results drive revenue. Better ‘more helpful’ results leads to more engagement on your website site and apps. Vectors perform well when words can have multiple meanings or are you are searching a large dataset.
- Improve Personalization Improve customer’s search experience by defining vectors based on activities, past purchase history and other attributes and creating more personalized item for item recommendations.
- Better product search Vector search can use existing data like product features, description, price, location, etc. to provide product recommendations or to answer a variety of questions; like, “What other products that are in stock will work?” or “What in stock product will fix this application?”
Summary
Vector search is a DEEP topic. Implementing vector search allows solutions to really explore how to build engaging experiences.