Back to BlogBuilding the ML Stack for Conversational Advertising
Every major advertising platform has been shaped by the surface it was built on. Search gave us queries, keywords, and auctions. Social gave us profiles, feeds, and interest graphs. Mobile created an ecosystem around apps, installs, and in-app events. Conversational AI introduces a different kind of surface, and with it great challenges and opportunities.
The problem is that those signals rarely arrive in a clean, structured format. They can be spread across several turns, mixed with irrelevant context, and expressed in almost any language.
In a system like this, conversational intents are unusually valuable. The challenge is to find those intents, which can be very specific. The known advertising stack built around keywords, audiences, and predefined placements is not a good fit for this problem.
At Velocity, a lot of our engineering work comes down to one question:
How do we turn a conversation into an advertising decision, accurately and fast enough to sit in the request path?
At a high level, the system still has two familiar jobs:
- Retrieval: find campaigns worth considering.
- Ranking: decide which of those campaigns, if any, is good enough to show.
Neither problem is new. What is new is the input.
The conversation is not the query
Users do not write ad queries. They talk to an assistant about what they are trying to do. For example, let’s consider the following:
“My son outgrew his hiking boots, and we’re going to the Dolomites this summer.”
There is obvious commercial intent in that sentence. But the raw text is not necessarily the representation we want to send directly into an ad retrieval system. A keyword system might focus heavily on boots and retrieve work boots or casual footwear. Another system might miss that the product is for a child, that hiking is the relevant activity, or that the user is preparing for an upcoming trip.
- Work boots
- Casual leather boots
- Men’s hiking boots
- category
- kids’ hiking footwear
- constraint
- child, next size up
- activity
- alpine hiking
- location
- Dolomites
- timing
- this summer
- stage
- ready to buy
- confidence
- 0.92
This is where language models become useful as infrastructure rather than only as an interface. SLMs, classifiers, and other lightweight modeling techniques can run inline and help us transform conversational context into a more structured representation of intent.
Depending on the use case, that representation might contain:
- commercial intents
- product or service category
- relevant constraints
- brands and entities
- location
- stage in the decision process
- extraction confidence
The important part is not simply that a model can summarize the conversation. It is that intent extraction becomes a measurable modeling problem.
We can compare extracted intent against raw conversational text and evaluate different models and measure retrieval quality. Also, we are able to introduce confidence thresholds and handle the right business logic.
Retrieval looks more like search and RAG than classical ad targeting
Once we have a useful representation of intent, the next problem is matching it against live campaign inventory. This starts looking much more like modern information retrieval.
When retrieving, we can use a hybrid approach that takes into consideration both semantic and lexical understanding. Semantic retrieval lets us match on meaning rather than exact wording.
On the other hand, lexical signals remain extremely useful when the conversation contains a specific brand, product, model, destination, or other entity where exact matching matters. Pure vector search can miss commercial signals that are important to catch.
We also do substantial work offline on the campaign side. LLMs and other models can enrich campaigns, creatives, landing pages, product information, and advertiser metadata before they ever enter the online serving path. Those representations can be precomputed and indexed, keeping expensive semantic work out of the request path.
That lets first-stage retrieval stay extremely fast, down to sub-millisecond lookup times in parts of the system, while still benefiting from richer representations produced offline.
or nothing
Once retrieval gives us a small set of intent-matched campaigns, we can afford to use a more accurate model to evaluate each candidate jointly with the conversation.
That lets us ask a more useful question:
How well does this specific ad actually fit this specific conversation? Retrieval and relevance are not the same thing.
A relevance score of 0.71 does not mean “71% relevant.” Score distributions vary across models, verticals, languages, campaign types, and inventory.
So we continuously measure how scores map to actual relevance judgments and downstream behavior. The goal is to build a strong calibrated scoring system that could be trusted.
A value-driven ranking system
Once an advertising system has enough traffic, another familiar challenge emerges: evaluating and predicting conversion value. Rather than simply estimating the probability of a click or relevancy, we aim to measure whether an ad will truly satisfy the user’s underlying need.
A model trained strictly on maximizing raw clicks does not guarantee that the ad resolves the user’s intent or delivers genuine downstream value.
Without a focus on long-term utility, the system might push high-CTR creatives into loosely related conversations, generating clicks at the expense of user experience and advertiser conversion quality.
Instead of treating relevance only as a mandatory filter, we make it one of our most prominent inputs to the model. Features describing the semantic relationship between the conversation and the ad can sit alongside the many other signals available to the ranking system.
That allows us to build a system that returns how closely an ad matches what someone is currently trying to do, which should help predict whether they need it and eventually buy it.
On a conversational surface, relevance is not simply a constraint placed on top of our model. It is part of the model itself.
Experimentation is part of the serving system
Experimentation infrastructure matters almost as much as the models themselves.
As the system evolves, we want to test new intent models, retrieval strategies, campaign representations, thresholds, ranking functions, and combinations of them without turning every experiment into a big engineering project. A new retrieval strategy should be easy to compare against the production baseline.
A new model should be evaluated not only on conversion probability, but also on relevance, yield, latency, and the downstream user experience.
In a new market, many assumptions that sound obviously correct turn out not to be. Fast experimentation is how we find them, and a strong A/B platform allows us to understand where we are heading.
The stack is still being invented
What makes this problem interesting is that none of these components exists independently.
That is why we do not think about conversational advertising as an existing ad stack with an LLM added somewhere in front of it. The interaction between retrieval, relevance, prediction, and experimentation is the system.
Now with our platform we are capable of generating structured signals from previously unstructured data.
At Velocity, we are putting those pieces together in production.
We are still early, but velocity is high!
We're building these formats at Velocity, for teams designing ad experiences inside conversational AI. Reach high-intent audiences or get in touch.
