← finn-hertsch.de
Slides [DE] Deutsch

Kairos: Numerically Robust News Recommendation

Numerically Robust News Recommendation under Item Cold-Start via Cholesky-based LinUCB and Matryoshka Embeddings

In dynamic content spaces like regional news publishing, recommendation systems face a structural paradox: articles display an extremely short Time-to-Live (TTL < 48h) while catalog throughput remains modest. Classical collaborative filtering algorithms collapse because the interaction density required to train neural embeddings rarely materializes before the content decays.

Project Kairos addresses this scarcity by transitioning from passive data-driven learning to active online bandit optimization via Linear Upper Confidence Bound (LinUCB) . However, continuous deployment of LinUCB in production requires unyielding numerical integrity. Here, we examine why conventional implementations relying on recursive Sherman-Morrison matrix inversions break down, and how direct rank-1 updates of Cholesky factors structurally preserve model geometry.

1. The Item Cold-Start Paradox

Empirical evaluations on news interaction datasets confirm that the majority of engagement occurs within hours of publication. Afterwards, article utility decays exponentially . In regional corpora with shallow article pools ($N \approx 350$ items per 48-hour window), recommendation models cannot rely on large collaborative interaction graphs.

LinUCB resolves this by modeling user payoffs as a linear combination of contextual features $x_{t,a} \in \mathbb{R}^d$ and estimated user preference vectors $\hat{\theta}_t \in \mathbb{R}^d$:

$$p_{t,a} = \hat{\theta}_t^\top x_{t,a} + \alpha \sqrt{x_{t,a}^\top A_t^{-1} x_{t,a}}$$

where $\alpha > 0$ balances exploitation with epistemic exploration, and $A_t \in \mathbb{R}^{d \times d}$ denotes the cumulative feature covariance matrix with ridge parameter $I_d$:

$$A_t = I_d + \sum_{\tau=1}^{t-1} x_{\tau, a_\tau} x_{\tau, a_\tau}^\top$$

2. The Geometry of Uncertainty

Geometrically, the quadratic term under the radical, $x_{t,a}^\top A_t^{-1} x_{t,a}$, represents the weighted Mahalanobis distance with respect to the precision matrix $A_t^{-1}$. It defines a confidence ellipsoid in the $d$-dimensional parameter space:

$$\mathcal{E}_t = \left\{ \theta \in \mathbb{R}^d \mid (\theta - \hat{\theta}_t)^\top A_t (\theta - \hat{\theta}_t) \le \alpha^2 \right\}$$

Exploration is non-isotropic. Along feature directions that have received substantial interaction (high data coverage), the ellipsoid contracts quickly, reducing uncertainty. Conversely, along under-explored directions (representing novel or cold-start topics), the ellipsoid remains stretched out, yielding a large exploration bonus that directly counteracts popularity bias.

3. The Showcase: Numerical Divergence vs. Cholesky Stability

To sustain continuous recommendation streams, LinUCB must update $A_t^{-1}$ iteratively as each new article interaction $x_t$ arrives. Conventional implementations employ the Sherman-Morrison formula:

$$A_{t+1}^{-1} = A_t^{-1} - \frac{A_t^{-1} x_t x_t^\top A_t^{-1}}{1 + x_t^\top A_t^{-1} x_t}$$

Under floating-point arithmetic, repeatedly subtracting outer products induces catastrophic cancellation. The condition number $\kappa(A_t^{-1})$ diverges, asymmetry creeps in, and positive definiteness is irreparably destroyed.

Below, we simulate 1,000 streaming interactions on real 128-dimensional Matryoshka embeddings extracted from the Tagesschau API. Both models are initialized identically ($A_0 = 0.1 I_{128} \implies A_0^{-1} = 10 I_{128}$). We project the uncertainty ellipsoid into 3 representative axes: PC1 (dominant category), PC2, and PC120 (cold-start topic):

κ(A⁻¹) = 3.3
Interaction t = 1 / 1000
κ(L) = 1.11 (stable)
Sherman-Morrison (Float32)
Kairos Cholesky (Float64)
Interactive 3D Confidence Ellipsoid Evolution. Drag the slider or press Space to stream 1,000 news interactions. Left: Sherman-Morrison recursive inversion rapidly ill-conditions ($\kappa \to 1400+$), warping axes along unobserved directions. Right: Kairos Cholesky updates operate directly in factor space, ensuring $\kappa(L) \le 13$ and preserving strict SPD geometry.

4. The Solution: Rank-1 Updates in Cholesky Factor Space

Rather than storing and inverting $A_t$, Kairos maintains the lower-triangular Cholesky factor $L_t \in \mathbb{R}^{d \times d}$ such that:

$$A_t = L_t L_t^\top$$

Upon observing action context $x \in \mathbb{R}^d$ and reward $r \in \{0, 1\}$, the factorization is updated directly via rank-1 modification :

Algorithm 1: Incremental Cholesky Ranking Update
1: Normalize context vector: $x \leftarrow x / \|x\|_2$
2: Apply decay discount: $L_t \leftarrow \sqrt{\gamma} \cdot L_t$
3: if $r > 0$ then
4:   $L_{t+1} \leftarrow \text{cholupdate}(L_t, x)$   $\rhd \; L_{t+1} L_{t+1}^\top = L_t L_t^\top + x x^\top$
5: end if
6: Forward substitution: $L_{t+1} y = x \implies y \in \mathbb{R}^d$
7: Uncertainty evaluation: $\sigma^2 = y^\top y = x^\top A_{t+1}^{-1} x$
8: return $L_{t+1}, \sigma^2$

Operating in triangular factor space yields a decisive mathematical property : the condition number satisfies $\kappa(L_t) = \sqrt{\kappa(A_t)}$. Working with square-root factors effectively squares precision and structurally precludes negative eigenvalues.

5. Scaling Up: Matryoshka Representation Learning

Numerical stability ensures system reliability, but real-time news serving requires sub-millisecond inference latency. Kairos combines Cholesky LinUCB with Matryoshka Representation Learning (MRL) via Nomic Embed .

By truncating the 768-dimensional embedding vector to the nested $m = 128$ subspace, Maximum Inner Product Search (MIPS) achieves a 4.85-fold speedup (0.069 ms vs. 0.337 ms per 100 candidates) while retaining over 95% of total semantic variance and 96.4% pairwise cosine similarity precision.

6. Conference Slides (SKILL 2026)

The following slide deck summarizes the mathematical derivation, algorithmic Givens rank-1 factor updates, and empirical latency/stability benchmarks for the SKILL 2026 at the INFORMATIK Festival in Dresden.

SKILL2026_Hertsch_Cholesky_based_LinUCB.pdf
Download PDF
Presentation Slide 1 of 8
1 / 8
Tip: Use ← and → arrow keys to navigate slides. 8 Slides · 16:9 Aspect Ratio

Acknowledgments

This work was conducted at DHBW Ravensburg (School of Business, Data Science & AI). Experimental validation relies on real-time news embeddings sourced from the Tagesschau API. Implementation in Julia 1.10; source code and benchmark scripts are available on GitHub.

Author Contributions

Finn Hertsch: Conceptualization, mathematical formulation of Cholesky LinUCB, MRL integration, empirical evaluation, and software architecture.