Meta Andromeda: Next-Gen Personalized Ads Retrieval Engine
How Meta rebuilt its ads retrieval stage on the NVIDIA Grace Hopper Superchip with hierarchical indexing and a 10,000x capacity increase to serve Advantage+ automation at scale.
Retrieval is the part everyone calls “just a filter” — a coarse pass to discard obvious irrelevance before the serious ranking models get involved. But if retrieval can only apply limited personalization, the ranking stage is working with a biased sample no matter how sophisticated it gets. Andromeda treats retrieval as a full ML problem, and the numbers make the case that this was the right call.
Source
By: Meta
Published: Dec 02 2024
Type: Engineering article
This article is unusually specific about hardware co-design and inference architecture, which is rare in ML system posts that usually stay vague about infrastructure. The combination of custom deep neural networks, hierarchical indexing jointly trained with the retrieval model, and deployment on the Grace Hopper Superchip is worth understanding as a system — not just as a list of features.
The Retrieval Scaling Wall
Ads retrieval at Meta operates under two constraints that work against each other. The first is volume: retrieval must score tens of millions of ad candidates and narrow them to a few thousand for downstream ranking — three orders of magnitude more candidates than any subsequent stage ever touches. The second is latency: any delay disrupts the real-time feed experience, and as advertising becomes increasingly dynamic with frequent creative updates, the models need to reflect near-current signal. Trying to add meaningful personalization while holding both of these constraints is where previous systems hit their limit.
Before Andromeda, Meta’s retrieval relied on isolated model stages connected by rule-based heuristics. This made sense as a way to decompose the problem, but it blocked end-to-end optimization. The heuristics couldn’t be trained; they could only be tuned by hand. Worse, the architecture had low GPU utilization because the bottleneck was CPU-to-GPU interconnect bandwidth and memory IO overhead — the classic problem when you have lots of precomputed embeddings that need to move around before inference can happen.
Andromeda addresses this directly by moving all precomputed ad embeddings and features into the local memory of the Grace Hopper Superchip, eliminating the interconnect bottleneck entirely. GPU preprocessing handles feature extraction. The result is highly parallel, computation-intensive retrieval that can run a deep neural network with 10,000x more model capacity than the previous system while still meeting latency requirements.
The network design matters here. A standard two-tower model — one tower for the user, one for the ad, combined via dot product or cosine similarity — is simple to index with approximate nearest neighbor (ANN) search, but the interaction between user and ad features is limited to what fits in that final dot product. Andromeda uses a deep neural network with more complex interaction features to capture higher-order relationships between a person’s interests and ad attributes. The cost of this complexity is managed through sublinear inference — meaning adding more ads to the index doesn’t increase inference cost linearly, which is what makes scaling to Advantage+’s growing creative inventory tractable.
The mechanism behind sublinear cost is hierarchical indexing. Rather than scoring every candidate, Andromeda organizes ads into a multi-layer index and traverses only the most relevant nodes, much like a search tree. The critical detail is that the index and the retrieval model are jointly trained — the index representations are aligned with the neural network during training, not built separately and then attached. This is what separates it from traditional ANN approaches: in ANN, the index is a data structure over fixed embeddings; in Andromeda, the index is part of the learned model. This alignment improves both precision and recall over standard two-tower baselines.
The Advantage+ dimension adds another layer to why this matters. Advantage+ automates audience targeting, budget allocation, placement across Meta surfaces, and creative generation. When advertisers turn on AI-driven targeting, the number of eligible ad creatives grows significantly — Meta reports over a million advertisers using generative AI tools to create more than 15 million ads in a single month. A retrieval system that can’t scale to that creative volume is a hard ceiling on advertiser value. Andromeda’s hierarchical index is specifically designed to handle this growth, with a segment-aware design that routes different request types through different model complexity tiers to maximize infrastructure ROI.
Measured outcomes: +6% recall improvement in the retrieval stage, +8% ads quality improvement on selected segments. Advertisers who enabled Advantage+ creative saw a 22% increase in return on ad spend; those using image generation tools saw +7% conversions. These numbers flow downstream from the retrieval improvement — better retrieval sends better candidates to ranking, and better ranked candidates produce better outcomes.
One underappreciated part is what Meta calls “AI development efficiency.” By minimizing rule-based components, Andromeda makes the system more amenable to end-to-end gradient-based optimization. Rules are brittle and slow to update; a fully differentiable pipeline can absorb new training signal continuously. This isn’t just a maintenance win — it determines how fast future ML improvements can actually reach production.
My take
The jointly trained hierarchical index is the thing to pay attention to. In most retrieval systems, the index is a static structure you build after training — ANN over frozen embeddings. Andromeda trains the index as part of the model, which means the tree structure itself encodes learned relevance, not just proximity in embedding space. That’s a fundamentally different contract with the retrieval problem: instead of “find vectors near this query vector,” it’s “find ads the model has learned to group as relevant together.” The precision and recall gains over two-tower baselines aren’t surprising once you understand this — you’re comparing a learned taxonomy against a geometric approximation.
Where this is useful
- Designing retrieval systems where personalization quality matters as much as throughput
- Reasoning about hardware co-design tradeoffs when the bottleneck is memory bandwidth, not compute
- Understanding why jointly trained indexes outperform post-hoc ANN on high-interaction models
- Not applicable when ad inventories are small enough that exhaustive scoring is feasible — the sublinear cost only pays off at scale
Apply this: Before accepting “approximate nearest neighbor” as the default retrieval mechanism, ask whether your index and your model were trained together or bolted together after the fact.
What I want to remember
Remember: A hierarchical index that is jointly trained with the retrieval model is not the same as ANN over frozen embeddings — the tree encodes learned relevance, not geometric proximity, which is why it beats two-tower baselines on precision and recall.