AI21 Labs

I will claim that there is no right chunk size.

Structured Data is finally having its AlexNet/BERT/GPT moment. TabPFN-3 is here, and it is taking things to a whole new level. From release to release, performance and scale improve. It's no longer a gimmick or a promising direction. It's a production-ready solution, significantly outperforming Gradient Boosting Decision Trees on many real-world use-cases. But it's more than that. It's also multimodal, temporal, and relational. It's beyond standard Tabular IID assumptions. It's flexible enough to start influencing not only good-old tabular learning, but so many other, once unrelated traditional ML domains. And yes, I'm biased. I'm fortunate enough to work on this mission at Prior Labs. But this doesn't make it less true ;)

Which model would you use to predict the presence of a disease, given the patient's demographics, clinical notes and X-ray? 🩻 How would you estimate a home's value based on its street-view photos, property description, and real estate metrics? 🏡 And how would you know if a Facebook post will go viral, based on user engagement, the text and the featured image? 👍 It’s 2026, and we still haven't solved Multimodal Tabular Learning. LLMs and VLMs are amazing, but they aren't a good fit for tabular data as they were never built for that. On the other hand, XGBoost and Prior Labs 's TabPFN are perfect for the numerical aspect, but text and image must go through preprocessing to natively fit them. This is suboptimal, as it is usually done using generic embedding models. But sadly, for many cases, these embeddings could be optimized further if they knew what is the target they need to predict. 🎯 We present: "MulTaBench: Benchmarking Multimodal Tabular Learning with Text and Image" 👁️ Arxiv Link: https://lnkd.in/dcjqas_U We show that all models improve on the benchmark when given contextualized embeddings, which were finetuned to explicitly predict the target before they are passed to the tabular learner. This motivates the need for foundation models that natively combine structured and unstructured modalities, and we invite researchers to use MulTaBench to tackle this crucial problem! Thanks to my amazing coauthors: Eilam Shapira , Shoham Grunblat, Mor Ventura , Elad Hoffer , Gioia B. , David Holzmüller , Lennart Purucker , Gael Varoquaux , Frank Hutter , and Roi Reichart ! ❤️