Recommender Systems — Sr Staff edition

A course-level reference for the 2026 interview loop. Built for an experienced RecSys engineer who needs to articulate the field crisply enough to clear Pinterest / DoorDash / Reddit / TikTok / Snap / Anthropic / Databricks Sr Staff bars. The depth here is what you should be able to say out loud at a whiteboard, not what you read off a slide.

Part 1 — RecSys architecture

1.1 The canonical funnel

Every large-scale recommender — YouTube, Instagram, TikTok, Pinterest, DoorDash, Spotify, Reddit — has the same shape. You start with hundreds of millions to billions of candidate items and you have ~50–200ms end-to-end to return a sorted list of 10–50 to the user. You cannot run a deep model over 1B items per request, so you build a cascade of progressively more expensive models over progressively smaller candidate sets.

Latency budget — typical

1.2 Retrieval / candidate generation

Retrieval is responsible for recall. It does not need to score perfectly — it just needs to ensure the truly relevant items are among the ~10k passed to ranking. Modern systems run multiple retrieval sources in parallel and union them: collaborative, content, social, geo, freshness, exploration. Each source is allowed to be biased; the union is what matters.

Two-tower (DSSM)

The dominant architecture. A user encoder consumes user features (id embedding, recent history sequence, demographics, context) and produces a vector u ∈ ℝ^d. An item encoder consumes item features (id embedding, content embeddings, taxonomy) and produces v ∈ ℝ^d. Score = ⟨u, v⟩ (or cosine). Trained with sampled-softmax or in-batch negatives so item encoder can be precomputed offline and indexed in an ANN. Only the user tower runs at request time.

Why two-tower wins for retrieval but not for ranking: a dot product cannot model feature crosses across user × item (e.g. "user from SF × restaurant in SF" gets the same score as "user from SF × restaurant in NYC" if those interaction features aren't in the encoders). For retrieval, this is acceptable because ranking will fix it. For ranking, it's a dealbreaker.

Negative sampling — the central question of two-tower training

• In-batch negatives. For each positive (u, v⁺), use other items in the batch as negatives. Cheap, but biased against popular items: popular items appear more often as negatives (they show up in every batch), so the gradient pushes them down — the trained model then under-ranks popular items at serving time. YouTube's seminal paper corrects with logQ subtraction: subtract log(sample frequency) from logits.
• Mixed negatives. Add uniform random negatives (from full corpus) to in-batch — fixes the popularity bias.
• Hard negatives. Mine items the model currently scores high but are not positives. Greatly improves discrimination but can collapse if hard negatives are actually mislabeled positives ("false negatives"). Common recipe: top-100 from a previous-version model, then sample 1–5 per positive.
• Cross-batch negatives. Maintain a queue of recent batch embeddings (MoCo-style) for larger negative pools without OOM.

Item-to-item collaborative filtering

Classical: build a co-occurrence matrix from sessions, normalize (cosine, Jaccard, or PMI), and serve "items frequently watched after X." Old but still extremely strong for fresh sessions and cold users where you have one anchor item. Amazon's "customers who bought this also bought" was item-item CF. Today usually combined with neural retrieval, not replaced.

Matrix factorization (ALS)

Decompose user–item interaction matrix R ≈ U V^T. Implicit-feedback ALS treats observed interactions as confidence-weighted positives, all others as low-confidence zeros. Closed-form per-user/per-item updates make it embarrassingly parallelizable on Spark. Limitation: shallow, no side features (without extensions like LightFM).

Graph-based retrieval — PinSAGE, GraphSAGE, LightGCN

Treat the user–item bipartite graph (and optionally item–item co-engagement graph) as the substrate. Aggregate features from a node's neighborhood with learned weights. PinSAGE scaled this to ~3B node graphs at Pinterest by sampling neighborhoods with random walks. LightGCN strips out non-linearities and feature transforms — just iterative neighborhood averaging — and matches/exceeds heavier GCNs on retrieval benchmarks because in collab-filtering the embedding propagation is what matters, not the MLP.

Sequential models for retrieval — SASRec, BERT4Rec

Treat a user's interaction history as a sequence and use a transformer to predict the next item. SASRec uses causal attention (predict next from prefix); BERT4Rec uses bidirectional masked-item modeling. The "user representation" is the encoder output at the last position. The item embeddings are the input/output embedding table. Retrieval = top-K of W_item^T · h_last.

Generative retrieval — TIGER, RecGPT

The newest paradigm. Items are tokenized into semantic IDs (a short tuple of discrete codes from an RQ-VAE trained on item content embeddings), then a transformer is trained to autoregressively generate the semantic ID of the next item. Retrieval becomes beam search. Strengths: generalizes to cold items (because the semantic ID is content-derived), shares parameters across items, can scale with model size. Weaknesses: beam search is slower than ANN; semantic IDs need re-training as the corpus drifts.

Mixed retrieval — the production pattern

No top-tier system uses one retrieval source. A typical Sr Staff answer:

• Two-tower neural retrieval (~3000 candidates) — captures personalization
• Item-item / "more like this" from current session anchor (~1000) — captures intent
• Trending / freshness pool (~500) — captures discovery
• Social: items engaged by friends in last 24h (~500) — captures network
• Geo / context: nearby restaurants, local creators (~500)
• Exploration source: random sample from underexposed long-tail (~200) — combats popularity feedback loops

Total ~5–10k candidates after dedup, all hydrated and passed to ranking. The diversity of sources is itself a debias mechanism — if your only retrieval is two-tower, popularity bias compounds.

1.3 ANN serving

Once you have item embeddings indexed, retrieval is "given query vector q, return top-K nearest neighbors by inner product or cosine." Exact NN is O(N·d) per query — for N=100M, d=128, that's 12.8B FLOPs per request. Approximate methods get you 100–1000x speedup with <1% recall loss.

HNSW (Hierarchical Navigable Small World)

A multi-layer proximity graph. Top layers are sparse (long-range links); bottom layer contains all points. Search starts at the top entry point, greedily descends by following edges to the closest neighbor of the query, then drops to the next layer. At the bottom layer, performs a beam search of width efSearch.

Key parameters and how they trade off:

• M — max edges per node. Higher M ⇒ better recall but more memory (each node stores ~M neighbor IDs per layer it appears in). Typical 16–48.
• efConstruction — beam width during index build. Higher ⇒ better graph quality, slower build. Typical 100–400.
• efSearch — beam width at query time. Higher ⇒ better recall, slower query. Tunable per query, often the production knob.
• Memory: ~(d × 4 bytes + M × 8 bytes × levels) per item. For N=100M, d=128, M=32: ~75 GB. Doesn't fit on one box → shard or use IVF-PQ.

IVF-PQ (Inverted File + Product Quantization)

Two-stage lossy compression:

1. IVF (coarse). Cluster all vectors into nlist centroids (k-means). At query time, find the nprobe nearest centroids and only search items in those Voronoi cells.
2. PQ (fine). Split each d-dim vector into m sub-vectors, run k-means with K=256 on each sub-space. Each item is now stored as m bytes (one cluster ID per sub-space). Distances are computed via precomputed lookup tables — extremely cache-friendly.

HNSW vs IVF-PQ — when to pick which

Other notable systems

• ScaNN (Google). Anisotropic vector quantization — quantizes to minimize loss in inner product direction specifically (instead of L2). Often best recall/latency tradeoff at moderate scale. Used in Vertex Matching Engine.
• Faiss (Meta). Library, not a service. Provides HNSW, IVF, IVF-PQ, OPQ, GPU implementations. Standard backend.
• Vespa (Yahoo / Verizon). Production search+ranking platform. HNSW + tensor evaluation in the same query — supports retrieval, filtering, and small ranking models in one engine. Used at Spotify, Tumblr, Yahoo.
• Vald, Milvus, Weaviate, Qdrant. Open-source vector DBs. Mostly HNSW under the hood, differentiated on operability, filtering, and hybrid search.
• DiskANN (Microsoft). SSD-resident graph index. Trick: store full vectors on SSD, compressed (PQ) vectors in RAM for routing. Lets you serve 1B+ vectors from a single machine with ~5–10 ms latency.

Filtering / hybrid search — the painful problem

Real recsys queries always have filters: "in stock," "shippable to my zip," "not blocked by user," "language=en." How to combine filters with ANN is non-obvious:

• Pre-filter: filter the corpus to candidates that match, then ANN. Correct but slow if filter selectivity is low (e.g. "in stock" matches 99% of items — you've done nothing) or very high (almost no candidates remain — exact NN is fine).
• Post-filter: ANN over everything, then drop disqualified items from the top-K. Fast but you may return fewer than K, or even 0.
• In-graph filtering (HNSW with attribute predicates): during graph traversal, skip neighbors that don't satisfy the predicate. Most modern systems (Vespa, Milvus, Qdrant) implement this. Caveat: if filter is very selective, the graph becomes disconnected and recall collapses → fall back to exact for those queries.
• Multi-index: partition the corpus by attribute (e.g. one index per country) and query the relevant index. Best when filter cardinality is small.

1.4 Ranking

The ranking stage scores each candidate from retrieval. Unlike retrieval, ranking can use rich user×item interaction features and deep architectures because the candidate set is small (~hundreds to a few thousand). Sr Staff interviews go deep here — be ready to defend choices about loss, architecture, multi-task structure, and calibration.

Loss formulations

• Pointwise (binary cross-entropy). For each (u, i) example with label y ∈ {0,1}, predict p(click). Easy to train, easy to calibrate (probability is meaningful), but doesn't directly optimize ranking — two items both with p=0.3 are "tied" even if one is better.
• Pairwise (BPR, RankNet). Sample pairs (i⁺, i⁻) and train so score(u, i⁺) > score(u, i⁻). Optimizes order, but loses absolute calibration. BPR-MF is the classic implicit-feedback baseline.
• Listwise (LambdaMART, ListNet, NeuralNDCG). Loss is a function of the entire ranked list. LambdaMART (gradient-boosted trees with lambda gradients from NDCG swap deltas) is still the workhorse for search/ads ranking — top-of-leaderboard on classical learning-to-rank tasks.

In practice, a Sr Staff answer is: pointwise BCE for the underlying probability heads (with calibration) plus a pairwise auxiliary loss when the metric you care about is ordering and your data is heavily implicit. For multi-task (CTR, like, share, completion), pointwise BCE per task with shared bottom is the baseline.

Deep ranking architectures — the 10-year lineage

Industrial CTR/CVR rankers all face the same problem: learn high-order feature interactions over hundreds of sparse + dense features at sub-50ms inference. Below is the lineage of architectures that have led the field, with the architectural trick that distinguishes each.

Wide & Deep (Cheng, Google DLRS 2016). Linear/cross-feature "wide" arm + DNN "deep" arm. Wide handles memorization (specific user×item co-occurrence indicators that need exact match); deep handles generalization (embedding lookups + MLP). Trained jointly with one log-loss. The "wide" arm is FTRL-Proximal optimizer (handles billions of sparse weights cheaply); the "deep" arm is AdaGrad. Production at Google Play store. Still a useful mental model for "what features go where" — even today the wide arm is the right place to encode hard business rules ("this category never recommends to under-13 users").

DeepFM (Guo et al., IJCAI 2017). Replaces Wide&Deep's linear arm with an explicit 2nd-order Factorization Machine (FM) interaction over the same embedding table as the deep arm. So each feature has one embedding, used both for FM dot-products (interaction features) and for MLP input (general non-linear features). End-to-end trainable. Beats Wide&Deep on most benchmarks because the FM captures pairwise interactions automatically (no manual cross-feature engineering needed).

DCN — Deep & Cross Network (Wang, ADKDD 2017). Replaces FM's 2nd-order interactions with a cross network that learns polynomial interactions of arbitrary degree. Layer $l$: $x_{l+1} = x_0 \cdot x_l^\top \cdot w_l + b_l + x_l$ — every layer multiplies the original input by the running state plus a residual. $L$ layers give up to degree-$(L+1)$ polynomial interactions. Parameter count is linear in feature count (vs FM's quadratic). Used at Google Search ads.

DCN-V2 (Wang, WWW 2021). Improves on DCN by replacing the vector $w_l$ with a low-rank matrix $W_l = U_l V_l^\top$ which gives much more expressivity per layer without quadratic parameter blow-up. The cross is now $x_{l+1} = x_0 \odot (W_l x_l + b_l) + x_l$. DCN-V2 was the production CTR ranker at Google for years, beating MLPs by ~0.3% AUC on ads data — small numbers, huge dollar value.

AutoInt (Song, CIKM 2019). Stacks multi-head self-attention over the feature embeddings to learn arbitrary feature interactions. Each feature embedding is a "token"; attention weights between tokens capture interaction strength. Heavier than DCN-V2; tends to win on benchmarks with many text-derived features and lose on benchmarks dominated by ID features (where DCN-V2's polynomial form is more parameter-efficient).

xDeepFM / CIN (Lian, KDD 2018). The Compressed Interaction Network (CIN) explicitly enumerates 2nd, 3rd, ..., k-th order interactions via outer-product feature maps compressed by a learned matrix. More expressive than DCN at the same depth but more parameters. Production at Microsoft Bing for some time; less popular post-DCN-V2.

DLRM (Naumov, Meta 2019). The reference Meta architecture for years. Sparse features (id-types) → embedding tables; dense features → bottom MLP; pairwise dot products of all (sparse_emb, dense_proj) pairs → top MLP → logits. The dot-product interaction is the inductive bias that captures "this user × this item × this context" — strictly 2nd-order over learned embedding space. DLRM-V2 (2021) scaled this 10× via TorchRec and tighter ZeRO-style sharding. Replaced by HSTU at Meta ranking in 2024 but still the reference benchmark every recsys paper cites.

MaskNet (Wang, ByteDance 2021). The ByteDance CTR ranker. Insight: feature embeddings are not equally important per instance — a query about "shoes" should down-weight the "country" embedding and up-weight "size." MaskNet adds instance-guided masks: an auxiliary MLP outputs a per-feature mask vector $M(x)$ that element-wise multiplies into the feature embedding before the main DNN. Two variants: serial MaskNet (mask applied once at input) and parallel MaskNet (mask applied at every layer). Beats DCN-V2 by ~0.2% AUC on Criteo benchmark and substantially on ByteDance internal data; powers TikTok-class CTR models.

FinalMLP / FinalNet (Mao, AAAI 2023). The architectural surprise of 2023: a pure two-stream MLP, with one stream applying feature-gating and the other applying feature-aware bilinear interactions, beats DCN-V2 / xDeepFM / AutoInt on every benchmark. The lesson is humbling: feature-aware non-linearity matters more than explicit polynomial-feature-crossing structure. The result has caused a small re-evaluation of whether explicit cross-networks are still worth their complexity.

Wukong — RecSys scaling laws via stacked FM blocks (Zhang, Meta 2024). The most important deep-ranking paper of 2024. Architecture: stacks of FMB (Factorization Machine Block) + LCB (Linear Compressed Block) with residual connections. FMB does a compressed factorization-machine-style interaction; LCB applies linear projection for dimensionality compression. Stacking N of these gives a controllable-depth interaction model. The breakthrough: Wukong exhibits clean scaling laws on industrial recsys data — loss decreases as a power law in compute and parameters. This is the first paper to convincingly show recsys obeys LLM-style scaling laws in the deep-ranker (non-sequence) setting. Production at Meta as the dense-feature-interaction backbone for HSTU. Cite this paper when an interviewer asks "do scaling laws apply to recsys?"

FuxiCTR benchmark (Zhu, FuxiCTR). The standard benchmark for comparing these architectures. Reports AUC and Logloss across Criteo, Avazu, Movielens. When an interviewer says "have you read X paper?" knowing FuxiCTR-published numbers is the next level — you can immediately say "yes, on Criteo it's 0.814 AUC vs DCN-V2's 0.812 — small numbers but consistent."

Comparison table — deep-ranking architectures

Multi-task learning — Shared Bottom, MMoE, CGC/PLE

Real systems optimize multiple objectives jointly: click, like, share, watch-time, complete, follow, hide. Three architectures dominate:

• Shared Bottom. One MLP "trunk," then per-task heads. Simple, parameter-efficient. Fails when tasks conflict — gradients average out and no task wins.
• MMoE (Multi-gate Mixture of Experts). N expert MLPs share the input. Each task has its own gating network producing weights over experts. Tasks can specialize on different experts; shared experts capture common structure. Originated at Google for YouTube ranking.
• PLE / CGC (Tencent). Adds task-specific experts that only their own task can use, plus shared experts. Cleaner separation, better when tasks really do conflict (e.g. "share" and "long-watch" pull in different directions).

Why MMoE outperforms shared-bottom (the interview answer)

In shared-bottom, all tasks must compromise on a single trunk representation. If two tasks have negatively correlated labels (e.g. "fast click" vs "long watch"), their gradients destructively interfere and the trunk learns a weak average. In MMoE, each task routes to a different mixture of experts; the negative correlation can be expressed as the two tasks attending to different experts, and gradients no longer fight at the parameter level. Empirically this shows up most strongly when tasks have different positive rates (e.g. click 5% vs share 0.1%) and different feature dependencies — exactly the regime feed ranking lives in.

Multi-objective combination

You have p(click), p(like), p(share), p(complete), p(follow). What do you optimize? Three approaches:

• Linear scalarization. Score = Σ w_k · f(p_k). Weights are tuned via A/B test guidance from product (e.g. "increase shares 5% without losing CTR"). f might be log or identity. The interview-correct phrasing: "weights are not learned end-to-end — they're a product artifact, set by leadership in service of the ecosystem KPI."
• Pareto frontier. Treat as multi-objective optimization, find Pareto-optimal weight vectors. Useful for offline analysis but practical systems still ship a single weight vector.
• Constrained optimization. Maximize one (e.g. watch time) subject to others ≥ baseline. Dual / Lagrangian methods. More principled but harder to operate.

Calibration

Critical when scores feed into auctions (ads), bid shading, or business rules expecting probabilities. Multi-task models are systematically miscalibrated because:

• Logging policy biases — items shown more often have more training data and shrink toward base rate; rare items inflate.
• Negative downsampling during training breaks the link between predicted probability and true probability (need to back out: p_true = p_pred / (p_pred + (1 − p_pred) / r) where r is sample rate).
• Multi-task gradients pull each head toward an "average" calibration.

Common fixes: Platt scaling (logistic regression on top of logits, fit on holdout), isotonic regression (non-parametric monotonic mapping, more flexible but needs more data), temperature scaling (one scalar T fit to minimize NLL — under-fits but stable). For multi-task, calibrate each head separately, then combine. Track ECE (Expected Calibration Error) and reliability diagrams in dashboards.

1.5 Sequence-aware ranking — the deepest area of 2023–2026 RecSys research

The single biggest accuracy lift in modern recsys, beyond going deep, has been treating user history as a sequence with attention rather than a bag of average-pooled embeddings. The progression has gone: bag-of-items → RNN over history → target-attention (DIN) → transformer (BST) → long-sequence with two-stage retrieval (SIM, TWIN) → lifetime modeling with hierarchical clustering (TWIN-V2) → fully generative formulation (HSTU). Every interviewer at Meta, Kuaishou, TikTok, Pinterest, or Alibaba will probe at least three of these by name.

DIN — Deep Interest Network (Alibaba, KDD 2018)

arxiv 1706.06978. Idea: not all of a user's history is relevant to the current candidate. For target item v with embedding $e_v$ and history items $\{e_{u_1},\ldots,e_{u_T}\}$, compute attention weights $a_t = \mathrm{MLP}([e_{u_t}, e_v, e_{u_t} \odot e_v, e_{u_t} - e_v])$ (the element-wise product and difference are essential — they expose interaction signal to the MLP) then the user-interest representation given target is $u_v = \sum_t a_t \cdot e_{u_t}$. Solves a real production problem: users with broad interests had their representation diluted under sum/mean pooling. Production at Alibaba display advertising; the original paper reports +1.1% offline AUC over GBDT and substantial online RPM gain. Also introduced Dice activation (a data-adaptive variant of PReLU) and mini-batch aware regularization for sparse-feature L2.

DIEN — Deep Interest Evolution Network (Alibaba, AAAI 2019)

arxiv 1809.03672. Two-layer architecture on top of the history: (1) Interest Extractor — GRU over the click sequence to capture temporal hidden states $h_t$; (2) Interest Evolving Layer — AUGRU (GRU with attention-weighted update gate, where the attention weight is computed against the candidate item) so the evolution is target-conditioned. Adds an auxiliary loss at each timestep predicting the next click (essentially next-item prediction) which reportedly improves the hidden-state quality substantially. Handles "user moved from electronics to home goods last week" gracefully. By 2020 DIN/DIEN was the dominant production architecture for sequence-aware ranking in Chinese e-commerce.

BST — Behavior Sequence Transformer (Alibaba, DLP-KDD 2019)

arxiv 1905.06874. Replaces the GRU stack with a transformer encoder over the recent history (typically last 50–100 items). Each item position uses a learned positional embedding (relative time-since-now). The transformer output at the candidate-item position is concatenated with the static features and fed to the deep ranker MLP. By 2019 this was the dominant architecture for sequence-aware ranking; the rest of the field has spent 2020–2026 figuring out how to extend it to much longer sequences.

SIM — Search-based Interest Model (Alibaba/Pi, CIKM 2020)

arxiv 2006.05639. The first paper to convincingly tackle long-sequence (1K–10K item) user histories. The problem: target-attention over 10K items at serving time costs ~10K × d dot products per user per request, which is prohibitive. SIM splits into two stages:

• General Search Unit (GSU). A coarse target-conditioned filter over the entire long history. Two flavors: hard search (keep only history items whose category equals the target's category) — cheap, can be done via inverted index; soft search (use a separate light dot-product model to score every history item against target, keep top-K by score) — slightly more expensive but doesn't need taxonomy.
• Exact Search Unit (ESU). Standard target attention (DIN-style) over the top-K~150 surviving items.

This is the canonical two-stage long-sequence pattern, copied by every Chinese platform afterward. Online at Alibaba: +7.1% CTR over DIN baseline at 1K-length sequence. Critical interview point: the GSU's embedding space is not the same as the ESU's embedding space — they're trained separately — which causes embedding inconsistency that the next generation of models (TWIN) directly fixes.

ETA — End-to-end Target Attention (Alibaba, 2022)

arxiv 2108.04468. Replaces SIM's GSU with SimHash LSH so the coarse search becomes sub-linear in sequence length. Each history-item embedding gets a binary hash code (sign of random projection); target hashes to the same code; Hamming-distance filter yields candidates in $O(T)$ but with very small per-item cost. Co-trains the hash projection with the ranker so the hash bits are accuracy-aware. ETA hits 5K-length sequences in production with similar accuracy to SIM.

SDIM — Sample-based Deep Interest Model (Alibaba, 2022)

arxiv 2205.10249. Another long-sequence approach: hash history items into the same bucket as the target via SimHash, then average-pool within bucket. Sub-linear and collision-aware (uses multiple hash tables to control variance). Competitive with SIM/ETA at half the FLOPs.

CAN — Co-Action Network (Alibaba, KDD 2021)

arxiv 2011.05625. Tackles a different problem: capturing feature interactions with sequential structure. Each user-item pair generates an MLP whose weights come from a co-action unit; this gives orders of magnitude more expressivity per feature pair than a static dot product. Used at Alibaba alongside SIM. The interview-worthy line: "static feature interactions like DLRM can't capture history × target interaction at the depth a recsys needs; CAN gives every pair its own small network."

TWIN — Two-stage Interest Network (Kuaishou, KDD 2023)

arxiv 2302.02352. The Kuaishou breakthrough on long-sequence and arguably the most important industrial sequence paper of 2023. Insight: SIM's GSU and ESU use different embedding spaces, so the GSU's "relevance" is only a proxy for what ESU would actually score — top-K from GSU is not top-K under ESU. TWIN's fix:

• Shared embeddings between GSU and ESU. Both stages consume the same item embeddings; the GSU just uses a compressed/faster scoring formula. So GSU top-K is now a strict subset of what ESU would rank highly.
• GSU's fast scoring: precompute a low-dimensional projection of each history item's embedding (a "compressed key"), cache it, then GSU computes dot-product with target's matching projection at serve time. Sub-linear via caching.
• ESU: standard target attention over the GSU-surviving top-K (typically 100).

Deployed at Kuaishou for short-video FYP ranking over sequences up to 100K items. The paper reports +1.6% Watch Time online over their prior long-sequence baseline. Interview signal: candidates who can explain why embedding consistency between stages matters (because GSU should approximate ESU's relevance, not some separate relevance function) — that's the bar-raising answer.

TWIN-V2 — Lifetime user behavior modeling (Kuaishou, 2024)

arxiv 2407.16357. Extends TWIN to lifetime sequences (think years of clicks per user — millions of items). Key idea: offline hierarchical clustering of each user's history into a tree (k-means at multiple levels), then online retrieval traverses the tree from root toward the leaves most relevant to the target. The GSU becomes "find the few cluster centroids that match target, descend into those subtrees, take their items as candidates"; the ESU runs over the few hundred surviving items. The hierarchy is rebuilt periodically (e.g., daily) offline; online updates only add new items. Production at Kuaishou for short-video; supports 10⁶-length user histories at single-digit ms GSU latency. The interview question this enables: "how would you do million-item user history?" — TWIN-V2 is the answer.

SASRec, BERT4Rec — the academic-canon sequence retrieval models

SASRec (Kang & McAuley, 2018): causal self-attention transformer trained on next-item prediction over user history. BERT4Rec (Sun et al., 2019): same backbone, trained with masked-item prediction (Cloze-style). Both produce strong item-sequence embeddings used heavily in academic benchmarks (MovieLens, Amazon Reviews). At industrial scale they're often frozen feature extractors feeding into the deep ranker — not the ranker itself — because ID-vocabulary issues and sparse-tail items break vanilla transformer training. A practical interview answer: "we use SASRec embeddings as one of many sequence-derived features fed into the DLRM/HSTU; we don't use vanilla SASRec end-to-end in production because of cold-start and freshness."

TransAct — Pinterest's Homefeed sequence ranker (KDD 2023)

arxiv 2306.00248. Pinterest's production sequence model. Combines a short-term transformer (last ~20 actions, captures session intent) with long-term features (aggregated stats over user history). The short-term transformer uses target-attention with action-type embeddings (save vs click vs hide vs close-up) injected at every token, so the model knows the kind of engagement that produced each history pin, not just the pin id. Online at Pinterest: +5.7% engagement on Homefeed at launch. The TransAct paper is one of the few public industrial deep-dives showing how to put a sequence transformer at ranking-tier latency budget (~60ms p99).

HSTU — Hierarchical Sequential Transduction Unit (Meta, ICML 2024)

arxiv 2402.17152 (Zhai et al.). Meta's flagship architecture, replacing DLRM at Reels/IG/Marketplace ranking. The most important industrial recsys paper of 2024. Key claims:

1. Reformulates recommendation as generative sequence modeling. User-action sequence is tokenized as interleaved (item_id, action_type) tokens — so each token represents both "what was shown" and "what the user did." The transformer is trained to predict the next token (next item, or next action conditional on items).
2. Custom HSTU attention. Replaces standard softmax-attention with a point-wise gated formulation: $\mathrm{HSTU}(Q,K,V) = \phi(Q K^\top) \odot \mathrm{Rel}(p) \cdot V$, where $\phi$ is a point-wise SiLU and $\mathrm{Rel}(p)$ is a learned relative-positional bias. No softmax — replaces the normalization with point-wise nonlinearity. This is the architectural surprise: it works, and it's faster (no softmax, no key-norm) at the same accuracy.
3. Million-scale vocabulary handled via sharded embeddings across TorchRec — the same infra that powered DLRM extends naturally.
4. M-FALCON inference: batched serving optimized for the gated-attention pattern. The paper reports 285× FLOP reduction at serving relative to the equivalent softmax-attention.
5. Scaling laws. First industrial recsys paper to show clean scaling-law behavior: loss decreases as a power law in compute, parameters, and data. Meta has scaled HSTU to multi-trillion parameters.
6. Production impact. The paper reports +12.4% E-Task and replaces DLRM at multiple Meta surfaces.

Interview probes: "why replace softmax with point-wise SiLU?" (answer: softmax saturates with very long sequences and the implicit normalization makes value-distribution sensitive to token count; a point-wise gating is more robust and shaves serving cost). "What does HSTU buy you that you couldn't get from stacking BST?" (scaling laws on industrial data, joint retrieval+ranking, generative formulation enables semantic-ID extensions).

MTGR — Meituan's adaptation of HSTU for sponsored search (2025)

arxiv 2505.18654. Meituan reproduced and adapted the HSTU recipe for sponsored search ranking and full-flow deployment. Notable for being the first non-Meta team to publish a working HSTU-class system at production scale. Reports +2.3% CTR over the existing DCN-V2-based ranker. The paper is candid about deployment cost (~3× FLOPs vs the prior ranker) and the engineering needed to fit HSTU in their serving stack. If you're interviewing at any non-Meta platform, this is the paper to mention as evidence that the recipe generalizes.

Comparison table — long-sequence sequence models

LLM-augmented (ReLLA, P5, M6-Rec, TallRec)

Use a pretrained LLM to encode item text/metadata and inject those embeddings into the ranker, or use the LLM directly as a reranker over the top-K. P5 (Geng 2022) reformulated all recommendation tasks as text-to-text. M6-Rec (Cui 2022) scaled this at Alibaba. TallRec (Bao 2023) showed LoRA-tuning a 7B LLM on user history can beat ID-based recsys at small scale. Status in 2026: helpful for cold-start and content-rich domains (news, e-commerce, podcasts) and for reranking the top 20–100 where latency budget is generous, but still too slow for top-of-funnel ranking at feed scale. The dominant production pattern is: traditional cascade does retrieval + first-pass ranking → LLM reranks the top 50.

1.6 Generative recommendation — the frontier any 2026 Sr Staff loop will probe

Three distinct generative families exist; conflating them is a fast way to fail an interview. (1) Semantic-ID generation (TIGER, RQ-VAE flavor): items are quantized to learned discrete codes, a decoder generates code sequences. (2) ID-token generation (HSTU flavor): item IDs are treated as a vocabulary, a decoder generates next ID directly. (3) Text generation (P5, TallRec, OneRec): an LLM-style decoder generates the item slug/title; either as text or constrained to known items via beam search. Know which family each paper sits in.

TIGER — Semantic IDs via RQ-VAE (Rajput et al., NeurIPS 2023)

arxiv 2305.05065. Pipeline:

1. Train a content encoder (text + image, typically Sentence-T5 or CLIP) to produce dense item embeddings.
2. Train an RQ-VAE (Residual Quantized VAE) to compress each item embedding into a tuple of $L$ codes (typically $L=4$, each code from a codebook of $K=256$ entries). The residual quantization is hierarchical: first codebook quantizes the item, the residual is quantized by the second codebook, etc. The result is the item's Semantic ID: semantically similar items share prefix codes.
3. Train a vanilla T5-style transformer to autoregressively predict the next item's Semantic ID given the user's history of Semantic IDs.
4. At serving, beam-search the top-K most likely Semantic IDs and look them up in a Semantic-ID → item table.

Why it matters: (a) generalizes to new items — their Semantic ID is computable from content with no interaction data, (b) the model size scales independently of corpus size (no per-item embedding row), (c) beam search controls retrieval and ranking jointly. The paper reports 14–34% Recall@5 gains on Amazon Reviews benchmark vs SASRec. Caveat for interviews: TIGER on academic benchmarks ≠ TIGER in production. Scaling Semantic IDs to billion-item corpora has known issues: codebook collapse (most items share a small set of codes), Semantic-ID drift when content drifts, and beam-search latency.

HSTU as a generative ranker (Meta, ICML 2024)

Covered architecturally in §1.5. The generative framing here: treat the user's interleaved (item, action) sequence as a sequence of vocabulary tokens, train next-token prediction, sample/score next items in the same forward pass. Retrieval and ranking are unified: the score $P(\text{next item} = v_i \mid \text{history})$ is the ranking score. This collapses the funnel from "retrieve top 500 then rank" to "score all candidate items with one model" — at HSTU's serving FLOPs that's now feasible on Meta-scale hardware. The 2024 paper is the first credible existence proof that the funnel can collapse for one of the top three platforms in the world.

OneRec — Kuaishou's end-to-end generative recsys (2024–2025)

OneRec arxiv 2402.10760. Kuaishou's production generative model for short-video. The most important generative-recsys paper after HSTU; deployed at Kuaishou serving 400M+ DAU. Key components:

• Semantic-ID tokenization of videos via RQ-Kmeans on multi-modal embeddings (visual encoder + text caption encoder). Codebook is hierarchical, ~$O(10^3)$ codes per level × 3 levels gives ~$10^9$ unique Semantic IDs.
• Encoder-decoder transformer. Encoder consumes the user's history (Semantic IDs of recent watched videos plus action types) plus contextual features (location, time, device). Decoder autoregressively generates the Semantic ID of the next recommended video.
• Beam search over Semantic ID tokens at serving — beam width tuned for latency vs diversity. The decoder's output is constrained to valid Semantic IDs (existing items) via a precomputed trie.
• Reward model + DPO-style alignment. The paper's biggest contribution: a separate reward model is trained on user-feedback signals (watch-time, like, share), then the generative model is fine-tuned with a DPO-style preference objective so its sampling distribution prefers high-reward continuations. This is recsys's analogue of RLHF — preference-tune the generator on a learned business-objective reward.
• Production results. +1.7% Watch Time, +3.0% LT7 retention online at Kuaishou main app; the paper reports it replaces the prior cascade (retrieval + DLRM ranker) for >10% of FYP traffic at submission time and was scaling toward 100%.

Interview probes: "why does OneRec need a reward model — isn't next-item prediction enough?" (Answer: next-item prediction optimizes for what the existing system showed, which is already biased; the reward model lets you target a downstream business objective like long-term watch-time, off-policy from the data distribution.) "How does OneRec handle freshness?" (Encoder consumes recent history including very recent items; Semantic IDs of new items computable at upload; reward model retrained nightly.) "Compare OneRec vs HSTU." (Both generative, but HSTU uses ID tokens directly while OneRec uses learned Semantic IDs; OneRec adds explicit reward-model alignment; HSTU pioneered scaling laws while OneRec pioneered RLHF-style alignment for recsys.)

Other generative-rec systems worth knowing

• RecGPT — umbrella framing for autoregressive recsys; many academic instantiations.
• LC-Rec — language-and-code joint generative recsys, useful for cold-start.
• LlamaRec — two-stage: retrieval narrows to ~50 candidates, Llama-2 reranks via prompted next-item-as-text generation.
• A-LLMRec — aligning LLM embeddings with classical recsys ID embeddings via contrastive learning.
• RecRanker — uses LLM as a pairwise/listwise reranker.
• LLaRA — LoRA-tuned LLM for sequential recommendation with hybrid prompt format.
• BiLLP — bi-level LLM planner for long-term recommendation.

The Sr Staff framing: "we are converging recommendation, search, and conversational interfaces into one autoregressive system because (a) Semantic IDs let us share parameters across tasks and across items, (b) scaling laws are starting to apply to recsys for the first time (HSTU, Wukong), and (c) the alignment toolkit from LLM post-training (reward models, DPO) transplants directly to recsys via OneRec. The 2026–2028 question is whether the existing cascade survives or collapses entirely."

Pros and cons vs traditional retrieval+ranking

The Sr Staff "is the cascade dead?" question

Expect this. Calibrated answer: "Not in 2026, but the next 2–3 years will tell. HSTU and OneRec have shown that generative formulations can replace one or two stages of the cascade at FAANG scale. But: (1) the cascade still wins on explainability and filterability — important for ads and policy; (2) cold-start interleaving with millions of new items per day is easier in the cascade because you can just add them to the ANN index; (3) generative models' beam-search latency hits a wall around 50–100ms which is the entire ranking budget on some surfaces; (4) the engineering cost of switching is enormous — Meta and Kuaishou are still running both. My bet: by 2028 the top 5 platforms have generative-first ranking, and the cascade survives for ads, long-tail platforms, and surfaces where filterability is constitutional."

1.7 Real-time learning systems — the freshness floor

For short-video, news, and any high-velocity catalog, model freshness is as critical as model quality. A recommender that learns yesterday's signals can't compete with one that learns the last 5 minutes. This is the engineering layer that powers TikTok, Kuaishou, and increasingly all the others.

Monolith — TikTok/ByteDance's collisionless real-time recsys (RecSys 2022)

arxiv 2209.07663. The most-cited real-time recsys infra paper. Two big innovations:

• Collisionless embedding tables via cuckoo hashing. Standard hash tables sized to fewer slots than IDs force collisions — multiple IDs share a single row and its gradient. For recsys this is catastrophic at the long tail: a brand-new ID lands in the same row as a popular ID, and its embedding gets dominated by the popular one's gradients (because the popular ID's count is 100× higher), so the new ID can never learn its own representation. Monolith uses a cuckoo hash table: each ID has two candidate slots from independent hash functions, and a kick-out chain rebalances on collision. Result: every ID gets its own row, deterministically, even at trillion-row scale. The paper measures +1% engagement just from collision elimination — that's enormous.
• Online training with parameter server. Training and serving share the same embedding table. A streaming pipeline ingests user feedback events, the trainer applies gradients to the parameter server, the serving stack reads updated embeddings within seconds. Asynchronous variants reduce trainer-serving lag at the cost of slight staleness. Per-row update frequencies tuned by item velocity: hot items update every few seconds, cold items every few minutes.

This is what powers TikTok FYP's per-minute freshness. Any TikTok / ByteDance interview will probe Monolith — they want to know if you understand why collisions matter (not just that they do).

Persia — Kuaishou + MSRA's hybrid sync/async PS (NeurIPS 2021)

arxiv 2111.05897. Scales to 100-trillion-parameter recsys models. Hybrid scheme: dense MLP layers train synchronously (low staleness, high accuracy) while sparse embedding tables train asynchronously (high throughput, tolerable staleness). Parameter server with worker-shard topology, all-to-all collective for the dense path, parameter-server pull/push for the sparse path. The paper's table of training-throughput numbers is widely cited; the key claim is that sync training of dense + async of sparse beats both fully-sync (too slow) and fully-async (degrades accuracy at large dense layers).

TorchRec + FBGEMM — Meta's recsys training stack

TorchRec paper (Ivchenko 2024). PyTorch-native; integrates with FSDP for the dense MLP path. Provides table-wise / row-wise / column-wise sharding for embedding tables, fused embedding-lookup kernels via FBGEMM (which is the FLOP-and-memory-optimized C++/CUDA library), and a "ShardingPlan" API that auto-suggests sharding based on table sizes and worker count. Production at Meta for DLRM, HSTU, Wukong. The reason TorchRec is competitive with the open-source TensorFlow ecosystem (which dominated 2018–2021 recsys) is that FBGEMM's quantized embedding ops are 2–4× faster than the cuBLAS equivalents for the small-batch, high-cardinality access patterns of recsys.

DeepRec, HugeCTR, EmbedX

Alibaba's DeepRec is TensorFlow-based with PS and async support; NVIDIA's HugeCTR / Merlin is GPU-first with hierarchical (HBM → host RAM → SSD) embedding storage; Tencent's EmbedX is graph-embedding-focused. Know that these exist as alternatives; in interview a one-line mention is enough unless the role is infra-heavy.

Feature stores and training–serving consistency

The classic recsys bug: feature definition diverges between offline training and online serving. Modern solutions: Feast, Tecton, in-house FAANG equivalents. Core idea: feature definitions are versioned code; online retrieval uses Redis/Cassandra with the same transformation logic; point-in-time joins at training so the feature value at training time matches what was logged at serving time. Without this, models silently train on values that don't match what they see live — debug-hostile, often the root cause when offline AUC doesn't translate to online win.

Streaming feature pipelines

Kafka → Flink → online store (Redis/RocksDB) → serving stack. Latency budget: feature value visible online within ~30 seconds of the underlying event. Flink job per feature; each job emits new feature values to the online store keyed by entity (user_id or item_id). The hard parts: schema evolution (old features in flight when you redeploy), exactly-once semantics for accounting features, and watermark handling for late-arriving events. Interview probe: "what's the difference between a streaming feature and a batch feature, and which would you use for last-7-day click count?" — answer: both are valid; the right choice is streaming for fast-decay features (last 5 minutes) and batch for slow-decay features (last 90 days), with care that the boundary aligns.

1.8 Multi-task learning — the canonical CTR/CVR/dwell-time problem

Every production ranker predicts multiple targets simultaneously: click, save, dwell, convert, like, share, hide, skip<5s, satisfaction-survey. The architectural question is how to share representation across tasks without (a) destructive interference between tasks pulling in opposite directions, or (b) one dominant task drowning out signal for the others.

Shared bottom — the baseline

Stack: input features → shared MLP → split into task-specific heads. Cheap, simple, and the default; works when tasks are highly correlated (click + save). Fails when tasks conflict (CTR ↑ via clickbait vs satisfaction-survey ↑) — the shared bottom forces a compromise representation that's mediocre for both.

Cross-stitch networks (Misra, CVPR 2016)

Each task has its own MLP stack; at each layer, a learnable cross-stitch unit blends the two task stacks' activations $\alpha \cdot h^A_l + \beta \cdot h^B_l$ before the next layer. Generalization of shared-bottom; rarely deployed because the next generation (MMoE/PLE) strictly dominates.

MMoE — Multi-gate Mixture of Experts (Ma, Google KDD 2018)

arxiv 1804.04266. The canonical multi-task architecture for recsys. Architecture: $E$ shared expert MLPs run in parallel on the input features; each task $k$ has its own gating network $g_k(x)$ that outputs a softmax over the $E$ experts; task $k$'s representation is the gated weighted sum $\sum_e g_k(x)_e \cdot \text{expert}_e(x)$, which feeds task $k$'s head. Per-task gates allow each task to attend to different experts, so the experts specialize in feature regimes (popular vs niche items, new vs returning users) without explicit clustering. The MMoE paper is the one that introduced multi-task to YouTube ranking; the 2019 Zhao et al. paper ("Recommending What Video to Watch Next") is the productionized YouTube case study.

PLE / CGC — Progressive Layered Extraction / Customized Gate Control (Tang, Tencent RecSys 2020, Best Paper)

RecSys 2020 paper. Strictly dominates MMoE in nearly every published benchmark. Insight: MMoE's experts are fully shared across all tasks, so even with gates a task can be polluted by another task's gradient signal during training (seesaw phenomenon — improving task A degrades task B). PLE introduces task-specific experts alongside shared experts:

• CGC (Customized Gate Control): one layer of (shared experts + per-task private experts). Task $k$'s gate selects over (shared ∪ task-$k$-private) experts only — it cannot access other tasks' private experts. Task-private gradients flow only through that task's private experts plus the shared ones.
• PLE (multi-level CGC): stack multiple CGC layers; the lower layers still have shared experts feeding the higher layers' shared and private experts. Captures hierarchical task relationships.

Deployed at Tencent for video recommendation (+2.3% VTR, +1.5% VV). The interview-bar answer: "PLE's edge over MMoE is that task-private experts are protected from cross-task gradient interference, so when tasks conflict you don't get the seesaw. We default to PLE for any production multi-task ranker."

AITM — Adaptive Information Transfer Multi-task (Xi, SIGIR 2021)

arxiv 2105.08489. For sequentially-dependent tasks: impression → click → cart → purchase. The funnel is sparse (purchase rate is 1% of click rate); naïvely training a purchase predictor over-fits. AITM stacks the task heads in funnel order; each later head receives the previous head's hidden state through a learned transfer gate that decides how much info to pass. Trained jointly with multi-step losses. Used at Meituan and Alibaba for sequential funnels; the right architecture when CTR / CVR / GMV form a chain.

STAR — Star Topology Adaptive Recommender (Sheng, Alibaba CIKM 2021)

arxiv 2101.11427. For multi-domain recsys: same item catalog, different surfaces (search results page, product detail page recs, post-purchase recs). Each domain shares a "center" MLP (the shared root) and has its own "domain" MLP (the spokes of the star). Domain-specific final score combines center and spoke outputs. Used at Alibaba where they serve thousands of distinct rec surfaces. Beats domain-specific independent models because the shared center transfers learning across data-sparse domains.

PEPNet — Personal Embedding Plug-in Network (Kuaishou, KDD 2023)

arxiv 2302.01115. Kuaishou's multi-domain solution. Each user has a personal embedding that, instead of being a feature, is converted into a gating pattern that modulates every layer of a shared backbone. The PEPNet plug-in for domain $d$ also injects domain-specific gates at every layer. Result: the same backbone serves many domains and many users, but with per-user × per-domain modulation that gives near-domain-specialist accuracy. Deployed across Kuaishou's main feed, Kuaishou Lite, and Kuaishou international apps. Interview-bar answer: "PEPNet shows that gate networks indexed by entity are a general tool for personalization and multi-domain — preferable to giant per-domain models when domains share catalog and audience."

Sparse MoE for recsys (2024–2026)

The transformer-MoE pattern (Switch, GLaM, DeepSeek-V3) is starting to land in industrial recsys. Each ranker has $E$ experts (each a small MLP), a router selects top-$k$ ($k$=1 or 2), and only those experts run per token (per item × user pair). Used at Meta within HSTU and at Kuaishou within OneRec. Benefits: more capacity at the same FLOPs per request; specialization across content types (sports vs news vs entertainment) emerges naturally. Pitfalls: load balancing (some experts dominate, others starve — same problem as LLM MoEs), expert collapse at training start. Solutions inherited from LLM-MoE literature: aux-loss-free balancing (DeepSeek-V3), per-expert bias offsets.

Multi-objective combination — how product weights are tuned

The ranker outputs task probabilities $\{p_{\text{click}}, p_{\text{save}}, p_{\text{watch}}, p_{\text{hide}}\}$. The final score is a weighted combination $\sum_t w_t \cdot p_t$ (or $\sum_t w_t \cdot \log p_t$ for multiplicative). Weights are not learned end-to-end (they encode business priorities that change quarterly). Instead:

1. Define a north-star metric (LT7 retention, GMV, watch-time/DAU).
2. Offline: grid-search weight combinations on a holdout, predict counterfactual outcome via IPS/DR estimator.
3. Online: ship the top 3–5 weight combinations as A/B variants; measure north-star delta over 2–4 weeks; pick winner.
4. Iterate quarterly as product priorities shift.

The bar-raising Sr Staff answer: "weights are a product decision; we keep them out of the model so we can change them without retraining and so different surfaces can have different weights against the same shared backbone."

Loss balancing — when tasks have wildly different loss scales

Watch-time regression (loss ~10) drowns out CTR log-loss (loss ~0.4) if you sum them. Three canonical approaches:

• Manual rescale by per-task moving average — cheap, works fine for most cases.
• Uncertainty weighting (Kendall 2017): learn per-task $\sigma_k$; $L = \sum_k (1/2\sigma_k^2) L_k + \log \sigma_k$. High-noise tasks auto-downweight.
• GradNorm (Chen 2017): balance gradient magnitudes across the shared backbone.
• PCGrad (Yu 2020): project conflicting gradients to prevent destructive interference. Useful when you have a small number of strongly conflicting tasks (CTR + diversity penalty).

Part 1B — Industry deep-dives by company

One paragraph per platform won't cut it for Sr Staff bar. Below, what each major recsys platform actually runs in production as of 2026, the published papers behind it, and what their interviewers will probe.

Part 2 — Eval, calibration, debiasing

2.1 Offline evaluation

The classical metrics

• HitRate@K — fraction of test users for whom the held-out item is in the top-K. Coarse but interpretable.
• Recall@K — for tests with multiple held-out items, the fraction of them that appear in top-K. The retrieval-stage metric.
• Precision@K — fraction of top-K that are positives. Less common in implicit feedback.
• MRR (Mean Reciprocal Rank) — 1 / rank of first relevant item, averaged over users.
• NDCG@K — discounts gain by log of position. The standard ranking metric. Sensitive to relevance labels at top positions.
• AUC — pointwise probability of ranking a positive above a random negative. Useful for binary classifier sanity, but a 0.01 AUC lift can correspond to massive online lift or none at all because AUC is an average over all (pos, neg) pairs while you only show top-K.

AUC vs NDCG — when each matters

AUC is position-insensitive: it asks "given any random pos and any random neg, is pos ranked higher?" NDCG is position-sensitive: lifts at rank 1 matter much more than lifts at rank 50. For a two-tower retrieval model, AUC is fine — you only care that positives are roughly above negatives. For a feed ranker where users see 10 items, NDCG@10 (or even NDCG@3) is what predicts online wins. Sr Staff phrasing: "AUC for retrieval, top-heavy metrics for ranking, and never trust AUC alone for the final ranker."

The offline–online correlation problem

The biggest hard-earned lesson in industrial recsys: offline metrics often do not correlate with online A/B test outcomes. Reasons:

• Selection bias. Logged data came from an old policy. New model is evaluated on items the old policy showed — it never sees the items it would now surface.
• Feedback loops. Offline you predict on historical context; online your predictions change future logs.
• Position bias. Logged labels are confounded with where they were shown.
• Multi-task interactions. A model that improves p(click) AUC may push down p(complete), and the linear combination loses.
• Distribution shift. Model trained on last month, served this month.

Counterfactual / IPS / DR estimators

For more honest offline eval, weight each logged event by 1/p_logging(action), where p_logging is the (logged or estimated) probability that the old policy showed this item to this user. This is the Inverse Propensity Score estimator. It's unbiased but has high variance when propensities are small. Doubly Robust (DR) combines IPS with a reward model: if either the propensity model or the reward model is correct, the estimator is unbiased. Production teams run replay eval (using IPS/DR) alongside traditional offline metrics, and trust the agreement.

2.2 Online evaluation

A/B test fundamentals

Sample size: n ≈ 16 σ² / Δ², for detecting a relative effect Δ at 80% power, 5% significance, two-sided. For typical CTR (5%) and a 1% relative MDE (so absolute Δ = 0.0005), n is in the tens of millions per arm. For low-traffic surfaces, this means month-long tests or no test at all. Sr Staff candidates should know:

• Randomization unit. User-level (default), session-level (for short-lived effects), cluster-level (for network effects).
• Triggered analysis. Only analyze users who actually hit the treatment surface, not all users in the bucket. Reduces variance.
• Multiple comparisons. If you're tracking 50 metrics, expect 2–3 false-positive movements. Pre-register the primary metric.
• Novelty effects. First week of a new model often inflated by curiosity. Run for ≥ 14 days.
• Network effects. If your treatment changes what creators see (e.g. ranking changes affect creator behavior), randomization unit must be a connected component, not a user.

CUPED variance reduction

Subtract a regression-predicted "expected metric value" (from pre-experiment user features) from the observed metric. The residual has lower variance, so you detect smaller effects with the same n. Conceptually: control for "this user was always going to click a lot." Reduces required sample size by 30–60% in most consumer experiments. Microsoft / Booking.com / Netflix all use it.

Interleaving

Instead of A/B (different users see A or B), each user sees a result list interleaved from both rankers (Team Draft Interleaving, Probabilistic Interleaving). Counts wins per ranker. Detects ranking improvements with ~10x less traffic than A/B because variance from "different users" is removed. Microsoft Bing's standard pre-A/B gating tool.

Multi-armed bandits

For exploration / cold-start / many-arm comparison:

• ε-greedy. Pick best arm with prob 1−ε, random arm with prob ε. Trivial; surprisingly hard to beat in practice.
• Thompson sampling. Maintain Bayesian posterior over each arm's reward; sample one parameter per arm per draw; pick the arm with the highest sampled value. Naturally allocates traffic proportional to posterior probability of being best.
• UCB (Upper Confidence Bound). Pick arm with highest mean + √(2 log t / n_arm). Provably good regret bound, but tuning the constant matters.
• Contextual bandits (LinUCB, neural bandits). Reward depends on context (user features). Foundation for personalized exploration.

Bandits in production usually serve either (a) cold-start exposure for new content, or (b) hyperparameter tuning across model variants. Pure bandit-driven feed ranking is rare — bandits don't naturally compose with the cascade.

Long-term metrics

The hardest, most Sr-Staff topic. Short-term CTR may go up while 30-day retention goes down (the "engagement bait" failure mode). Real production teams maintain:

• 30/60/90-day retention
• L7/L28 active days
• Diversity / novelty (e.g. categorical entropy of items shown)
• Creator-side health (creator earnings distribution, cold creator survival)
• Reported satisfaction (star surveys)
• Hide / mute / report rates

The operationalized answer: any launch must move the primary metric without significantly hurting any guardrail. Long-term metrics are tracked in holdback experiments (5% of users on baseline indefinitely) so you can attribute multi-month effects.

2.3 Bias and fairness

Position bias

Items at the top of a list get clicked more regardless of relevance. If you train naively on click logs, your model learns "be like the previous model" rather than relevance. Two correction families:

• Examination hypothesis. Click = examine × relevant. Estimate p(examine | position) from a randomized "swap" experiment, then weight click loss by 1/p(examine).
• PAL (Position-Aware Learning). At training, add position as a feature into a separate "bias" head that is summed with the relevance head. At serving, fix position to a constant (e.g. position 1 or "no position"). Trick: the model learns to put position-dependent variance into the bias head, leaving a clean relevance head for serving. Used at Huawei / Pinterest.

Selection / exposure bias

Items never shown have no labels. Naive training converges to "show items the previous model would've shown." Mitigations: random exploration traffic (e.g. 1% of impressions are random), counterfactual reasoning (IPS), and explicit exploration sources in retrieval.

Popularity bias and item cold start

Popular items appear in training data far more often, both as positives and (via in-batch sampling) as negatives. The negative effect typically dominates and you under-rank popular items unless you correct. YouTube's logQ subtraction (subtract log of sampling probability from logits during training) corrects in-batch negatives. For cold items, hybrid retrieval (content tower) and explicit cold-item exploration boost are standard.

Calibrated logging

The single highest-leverage practice for honest training: log the propensity that the action was taken. If your serving system did Thompson sampling or stochastic rerank, log the probability. This lets you train with IPS or DR estimators, run honest counterfactual eval, and avoid the offline-online correlation collapse. Most teams ship deterministic ranking by default, then quietly add ε-randomization just to enable better training data.

Fairness — not just bias

Beyond position bias, "fairness" in recsys typically means: equal exposure across creator groups, demographic parity, or specific KPIs for protected categories. Common interventions: post-rank reordering to satisfy exposure constraints (Polyak / Lagrangian), in-training fairness regularizers, or pipeline-level changes (e.g. "ensure ≥ 30% of impressions go to creators with < 10k followers"). Be ready to discuss tradeoffs with primary metric.

Part 3 — Embedding tables at scale

The number-one infra problem in industrial recsys. A modern ranker has tens of billions of parameters, ~95% of which are in sparse embedding tables (user IDs, item IDs, creator IDs, all the cross IDs). A single user-id embedding table at Meta or YouTube can be ~100B rows × 128 dim × 4 bytes = ~50 TB. No single GPU has that memory.

Sharding strategies

• Table-wise. Each whole table on one GPU. Simple, but balance is poor (some tables are huge, some small).
• Row-wise. Hash row IDs across GPUs; each GPU holds a slice of rows from each table. Best for one giant table.
• Column-wise. Each GPU holds all rows for a contiguous slice of columns. Useful when row count is moderate but dim is huge.
• Grid (table×row). Combination of the above; most flexible. TorchRec uses a planner (heuristic or ILP) that chooses the best sharding strategy per table given hardware.

Communication: all-to-all

For each minibatch, each GPU has some subset of needed embedding rows; the rest are on other GPUs. The pattern is:

1. Forward all-to-all: each GPU sends "I need rows X, Y, Z from you" and receives back the embedding rows.
2. Compute.
3. Backward all-to-all: gradients flow the other way to update embeddings.

This is the hot loop of training. NVLink/IB bandwidth here is what limits throughput in production.

Compression and capacity tricks

• Hash trick. Map id → hash(id) % B for some bucket count B < cardinality. Loses some uniqueness; reduces memory by orders of magnitude. Used heavily for low-importance categorical features.
• Quantization. Store embeddings in int8 or fp8 instead of fp32. ~4x memory reduction with negligible accuracy loss for ranking. FBGEMM provides fused int8 lookup kernels.
• Mixed precision. Frequent items in fp16/bf16, infrequent in int8. Even fancier: differential dim — popular items get bigger embeddings.
• Composite embeddings. emb(id) = f(coarse_emb(hash1(id)), fine_emb(hash2(id))). Multi-resolution; reduces collision impact.
• UVM (Unified Virtual Memory). Embedding table lives in CPU host memory; GPU pages it in on demand via NVLink/PCIe. The hot rows naturally end up cached on GPU.
• Hot embedding cache. A small explicit GPU cache for top-N most accessed rows (popular items, popular users), backed by a slower CPU-resident store.

Training challenges specific to embeddings

• Skewed access (Zipf). 1% of items get 50% of impressions. The hot rows get massive gradient updates, the cold rows almost none. Need rowwise-adaptive optimizers (RowwiseAdagrad) that maintain per-row state.
• Optimizer state size. Adam doubles or triples table memory (m and v per row). Often forced to use Adagrad / RowwiseAdagrad to keep state small.
• Gradient staleness. In async training, gradients computed against an old version of an embedding row are applied to a newer version. Hot rows are updated by many workers per step.
• Rebalancing. Hot tables become bottlenecks; periodic re-sharding (TorchRec's planner) is run during long training.

Stack landscape

• TorchRec (Meta). PyTorch-native, supports table-wise / row-wise / column-wise sharding, FBGEMM kernels for fast embedding ops, integrates with FSDP for the dense parts. Production at Meta.
• HugeCTR (NVIDIA). CUDA-native, optimized for NVLink/IB. Marketed for ads/recsys at NVIDIA-stack shops.
• Merlin (NVIDIA). The umbrella ecosystem (HugeCTR + NVTabular + Triton-based serving).
• Persia (Bytedance). Heterogeneous training — embeddings on CPU parameter servers, dense on GPU workers. Predecessor pattern.

Part 4 — Six worked ML system designs

Each at the depth a Sr Staff loop expects. The goal isn't to memorize architectures — it's to know which knobs you would turn and why. For every problem, walk the same template: clarifying Qs → scale → data → features → arch → training infra → serving infra → eval → monitoring → gotchas → "if I had more time."

Part 5 — RecSys + LLM convergence (the 2025–2026 frontier)

The most asked-about topic in 2026 Sr Staff loops. Be opinionated. Below: how LLMs are actually used in production recsys, and where they aren't yet.

LLMs as feature extractors

The most boring and most useful pattern. For any item with text (titles, descriptions, reviews, transcripts, captions), pre-compute LLM embeddings (OpenAI text-embedding-3, Cohere, in-house Llama-style encoder). These plug into ranking models as a continuous feature. Wins:

• Dramatic cold-item improvement — content embeddings exist before any interaction.
• Cross-lingual transfer (multilingual encoders).
• Subtle topical similarity beyond what tag systems capture.

This is in production at virtually every modern recsys team. Cost is dominated by one-time embedding compute, refreshed only for changed items.

LLMs as rerankers

Take the top-50 from your traditional ranker, send to an LLM with a prompt like "User has been browsing {history}. Rank these items by relevance: {items}." Used in:

• Search result reranking (Bing, You.com).
• E-commerce product reranking for high-intent queries.
• News reranking with personalized briefs.

Tradeoff: latency. A GPT-4-class call adds 500ms+. Production deployments use distilled smaller models, batch reranking offline for "warm" content, or only invoke LLM rerank for high-value queries (high commercial intent, low-latency tolerance).

Generative recommendation (autoregressive item generation)

TIGER, HSTU, and the broader RecGPT framing — covered in Part 1.6. The future direction. As of 2026, HSTU is in production at Meta scale; most other "generative recsys" are research or small/medium scale.

Conversational recommendation

"Recommend me a sci-fi movie like Arrival but happier." A user expresses preferences in language; an agent translates into retrieval, ranks, presents, and refines based on follow-up. Hot research area, increasingly shipping in product (Amazon Rufus, Klarna, Shopify, Google's product search). Architecture pattern:

• LLM extracts query into structured filters + free-text intent.
• Hybrid retrieval (BM25 + semantic + filtered).
• LLM-style rerank with chain-of-thought rationale.
• Multi-turn refinement: each user reply updates the latent intent state.

Personalized agents (multi-step research / shopping)

"I want to switch from gas to induction stove — what should I buy?" The agent reads product specs, reads reviews, does spec comparisons, answers tradeoffs, narrows to recommendations, possibly checks out. This is recsys + tool use + reasoning. Companies betting here: Anthropic (computer-use agents), Amazon, Klarna, Perplexity. Recsys engineer's role: design how recommendations are surfaced inside an agent loop, how to ground in user history without violating privacy, and how to attribute conversion.

World models for simulation-based eval

The dream: rather than running an A/B test for 14 days, simulate the new ranker against a learned model of user behavior to predict the outcome offline. Active research area (Meta, Google, academia). Risk: the world model itself is just a learned predictor and can suffer from distribution shift. Promising for narrowing the candidate set of A/B tests, not for replacing them.

Part 6 — Common interview questions, with Sr Staff answers

Industry-paper deep-probes (added 2026-05-13)