King B2B Lead buying intent

UCSD and Together AI Research Introduces Parcae: A Stable Architecture for Looped Language Models That Achieves the Quality of a Transformer Twice the Size The core idea is to recast the looped forward pass as a nonlinear time-variant dynamical system over the residual stream. By analyzing the linearized form of this system, the research team shows that prior injection methods — addition and concatenation-with-projection — produce marginally stable or unconstrained parameterizations of the state transition matrix Ā. Parcae fixes this by constraining Ā via discretization of a negative diagonal parameterization, guaranteeing ρ(Ā) < 1 at all times. Two additional training fixes accompany the architectural change: a normalization layer on the prelude output to prevent late-stage loss spikes, and a per-sequence depth sampling algorithm that corrects a distributional mismatch bug in prior recurrence sampling methods. On results: → Parcae reduces validation perplexity by up to 6.3% over parameter- and data-matched RDMs at 350M scale → A 770M Parcae model matches the Core benchmark quality of a 1.3B standard Transformer → At 1.3B parameters, Parcae outperforms the parameter-matched Transformer by 2.99 points on Core and 1.18 points on Core-Extended On scaling laws: → Compute-optimal training scales mean recurrence µ_rec and tokens D in tandem following power laws (µ_rec ∝ C^0.40, D ∝ C^0.78) → Test-time looping follows a saturating exponential decay — gains plateau near the training recurrence depth µ_rec, setting a hard ceiling on inference-time scaling → A unified law predicts held-out model loss within 0.85–1.31% average error Full analysis: https://lnkd.in/gy6E-gT3 Paper: https://lnkd.in/ge-jG6BK Technical details: https://lnkd.in/gJ_Ytftp Models: https://lnkd.in/giTJ9hVw Together AI UC San Diego Hayden Prairie Zachary Novack