Building a Recommendation Engine with Two-Tower Models

Modern internet platforms handle millions of items and users, making traditional collaborative filtering and brute-force similarity matching computationally impossible in real time. To achieve sub-millisecond retrieval across massive product or content catalogs, modern recommendation engines rely on Two-Tower Neural Network architectures.

By separating user feature embeddings from item feature embeddings into independent neural network 'towers' that meet at a dot-product similarity layer, Two-Tower models allow you to precompute and index item vectors for lightning-fast vector search.

How Two-Tower Architectures Work

The Two-Tower framework decouples representation learning into two distinct optimization pathways before combining them via vector dot products:

• The User Tower: Ingests user features (demographics, historical interactions, context tags) and passes them through dense layers to output a unified user embedding vector.
• The Item Tower: Ingests item features (metadata, category tags, textual descriptions, pricing) through parallel dense layers to output a corresponding item embedding vector.
• Dot Product & Retrieval: During inference, user and item vectors are compared using cosine similarity or inner product. Because item embeddings remain static, they can be pre-indexed in vector databases like FAISS or ScaNN for instant retrieval.

Collaborative Filtering vs. Two-Tower Deep Embeddings

Python Implementation Blueprint

Using TensorFlow Recommenders (TFRS) or PyTorch, you can construct a clean Two-Tower retrieval model with independent user and item embedding layers:

import tensorflow as tf
import tensorflow_recommenders as tfrs

class TwoTowerModel(tfrs.Model):
    def __init__(self, user_vocab_size, item_vocab_size, embedding_dim):
        super().__init__()
        
        # User Tower
        self.user_model = tf.keras.Sequential([
            tf.keras.layers.StringLookup(vocabulary=user_vocab_size, mask_token=None),
            tf.keras.layers.Embedding(len(user_vocab_size) + 1, embedding_dim)
        ])
        
        # Item Tower
        self.item_model = tf.keras.Sequential([
            tf.keras.layers.StringLookup(vocabulary=item_vocab_size, mask_token=None),
            tf.keras.layers.Embedding(len(item_vocab_size) + 1, embedding_dim)
        ])
        
        # Retrieval Task Loss
        self.task = tfrs.tasks.Retrieval(metrics=tfrs.metrics.FactorizedTopK(
            candidates=tf.data.Dataset.from_tensor_slices(item_vocab_size).batch(128)
        ))

    def compute_loss(self, features, training=False):
        user_embeddings = self.user_model(features["user_id"])
        positive_embeddings = self.item_model(features["item_id"])
        return self.task(user_embeddings, positive_embeddings)

Production Best Practices

Deploying Two-Tower recommendation systems into high-throughput production environments requires strict attention to negative sampling and embedding freshness:

• In-Batch & Hard Negative Sampling: Use in-batch negatives combined with hard negative mining to ensure the model learns subtle distinctions between relevant and irrelevant items.
• Asynchronous Embedding Refresh: Decouple real-time user embedding generation from item embedding indexing, updating item vector indices daily or hourly in your vector database.
• Normalize Output Embeddings: L2-normalize the output vectors of both towers so that the inner product matches cosine similarity, stabilizing retrieval scores.