Everyone's talking about ... vector search

Vector search suddenly finds itself the spotlight, with database vendors rushing to add it to their capabilities, and startups jostling for funding to get ahead of the game. But what is vector search, and why now? We spoke to three vendors about the new search kid in town and what it means.

Inevitably, it turns out vector search isn't really new at all. The vector model for information retrieval goes back at least to the 1960s, and software companies like Autonomy were exploiting the technology in the early 2000s. So why the headlines now?

That can be explained in one word, said Ed Anuff, chief product officer at DataStax: "ChatGPT".

"Everybody who's in the database space is looking at the new types of applications that people are trying to build and saying, 'how can I make my database play better with these large language models?' And vector search is probably the most important piece."

This is because describing content - be that audio, images, videos or documents - in term of vectors is a perfect fit for such models, Anuff continued.

"Vector representations are the native language of LLMs."

Vectors - the native language of LLMs

Vectors, or vector embeddings, are simply long arrays of numbers that are produced when content is fed into an embedding model, that give a value to a number of attributes or dimensions (more than 1,000 in the case of ChatGPT).

LLMs deal with concepts rather than keywords. Concepts are collections of diverse elements that depend on context. That context is provided by vectors. Comparing the vectors attached to a chunk of text tells the models how close it is to other chunks within the that particular context.

The vectors can then be queried by algorithms using something like nearest neighbour search and the results used to power recommendation engines

"This enables you to satisfy queries like 'give me movies that feel sad' or 'give me images that look like this image I have'. This feature unlocks a whole new class of capabilities," said Benjamin Flast, lead product manager for vector search at MongoDB.

Information clustered in this way can also be used as input for machine learning models, providing the sort of positive feedback mechanism that can lead to rapid improvement in capabilities. The model might return a set of vectors around a specific concept, which might be an object such as a product, or a relationship or an idea.

Personalisation at the heart of the proposition

Personalisation is at the heart of the proposition. Anuff provided an example of a person working on a DIY project, whose favourite store knows what tools and materials they have purchased before and is thus able to recommend what they might need to complete the project. Smart travel based on known personal preferences is another use case. A third is image-matching where you might upload a photo of a desired product to an ecommerce site which will then return possible matches. In summary, any process that involves individual preferences could potentially be sharpened by vector search, and personalisation is still a retail holy grail.

Then there's search engines themselves. The front doors to the web risk looking decidedly old hat when compared to the new LLMs and their consumer friendly interfaces. Hence, Google and Bing rushing to bolt LLMs onto their search (with mixed results).

"People expect search functionality to improve and become 'smarter' at the same rate as the technology tools they've been using," said Matt Riley, general manager enterprise search at Elastic.

Semantic search

At this point readers may be experiencing a distinct feeling of déjà vu.

Yes, this does sound a lot like semantic search which creates matches on the basis of contextual meaning, looking at the intent behind a query rather than simply matching strings of words. In fact, vector search is a subset of semantic search. User-oriented semantic search will typically combine vector search with lexical forms such as keyword or full-text search.

But it's the underlying vector search element that is growing rapidly. New capabilities offered by the rapidly proliferating open source LLM models on the likes of HuggingFace have captured the imagination of developers, eager to get on board what Anuff says is already the biggest bandwagon since the iPhone, and likely to be much bigger: "It's one of those once-in-a-decade things where everybody suddenly goes, we need a new type of application."

This demand is leading to a focus on improving vector embeddings through better machine learning methods, in order to capture more semantic meaning in the vectors and improving semantic search in a wide range of use cases. After all, this probabilistic, contextual method of searching lives or dies on the relevance of its results as perceived by the individual.

"These models and experiences will get increasingly accurate and more industry-specific, predicts Riley. "We're also likely to see more organisations and entities building their own models and generative search experiences tailored to their specific use cases."

What can vector search do that other models can't?

The key advantage of vector search is its flexibility, explained Riley.

"It can search across different data formats, including text and images, audio, video and others," he said.

"From a user perspective, when compared to textual search, vector understands the meaning and contextual relationships behind the query. It is flexible as to the input, and able to use concepts such as pictures to create a query. And it delivers a single highly relevant answer, reducing effort and time to resolution."

The role of vector databases

Vectors may be the native language of LLMs, but there's still a need for storage and retrieval.

There are specialised vector databases out there, but most vendors these days take the multimodal approach, bundling functionality under one roof. The arrival of user-friendly LLMs has stoked fierce competition between these vendors.

Last month, MongoDB added Atlas Vector Search to its cloud rostrum, and with it a unified database and vector store to enable multiple types of searching. Meanwhile, DataStax, which provides cloud database-as-a-service based on the Apache Cassandra database, launched Astra Vector Search, with an emphasis on speed and scalability. Elastic also provides a vector database as part of its platform, as well as the Elasticsearch Relevance Engine (ESRE) with vector search and transformer models. And there are offerings from Google, Microsoft and Amazon and others besides.

One of the functions of a vector database is to act as the "memory" of an LLM, said Anuff. The models underlying ChatGPT or Bard only get updated once every few months. All other updates, including the iterative responses to queries, come from a vector database that sits alongside the model, giving the appearance that it's learning. "Everything you put into chat is actually going into a separate vector database that they then replay. So that's where the memory comes from."

Anuff continued: "If you have a database that understands how to take those vectors and return the appropriate data, now the database is able to work hand in hand with the model, getting the information into the model and helping the model come up with better responses."

Vector databases also allow organisations to use their own data in machine learning models, said Riley. "Artificial intelligence experiences that rely on proprietary data are being built to integrate with LLMs."

Use cases for vector search

With AI/ML encroaching on almost every aspect of life, business applications for vector search and vector databases are similarly diverse. As well as personalisation of retail "experiences", our interviewees mentioned use cases among customers including as customer service automation, gaming development platforms, financial services, document management, predictive maintenance, question-answering, sentiment analysis, data classification and software development tools.

MongoDB's Flast mentioned a car manufacturer that stores audio data of engine noises alongside other information about different models: "This all-in-one combination of an operational database, a search engine and a vector search system allows their technicians to record the audio from the car in the shop and then quickly match and diagnose it to a problem that exists," he said.

A similar use case was mentioned by Elastic's Riley, but using image search.

"For example, suppose you are looking for a part of a car and don't know what it is called. In that case, you can take a picture and upload it to the search engine, and it will use visual similarity to show you images alike. This will allow you to find out the part's name to go and buy it."

For Datastax, Anuff said that many companies are starting to use it to optimise their own internal operations.

"All our customers are asking what are you doing with AI, what are you doing with LLMs?" he said.

"There's a bunch of interesting use cases that, that you wouldn't immediately expect that are not at the user-facing level."