Not a transformer

Almost every large language model since 2017 has been a transformer: stacks of attention layers that let every word look at every other word. Liquid AI's Ramin Hasani, on The Cognitive Revolution, describes what happened when his team stopped assuming that and let an automated search pick the design instead — optimizing not against the usual math proxy but directly against roughly a hundred downstream tests, run on the actual chips inside phones and laptops, under real latency and memory budgets. The design it kept landing on was not a transformer. It was about 70-80% short convolutions with only a minority of attention.

"You still have to compute the models sequentially." — Ramin Hasani, on why the underlying liquid networks resist parallel hardware

The published model bears this out. LFM2 is sixteen blocks: ten short-range convolution blocks and six attention blocks. On a laptop or phone CPU it decodes and loads prompts about twice as fast as a comparable model from Alibaba's Qwen line, and its 230-million-parameter version runs at 213 tokens a second on a recent Galaxy phone. On tasks like pulling structured facts out of text, following instructions, and calling tools, that tiny model beats ones several times its size.

The honest asterisk, which Hasani grants: this is a trade, not a free lunch. The same 230M model loses badly to a bigger Qwen model on broad general-knowledge questions. Convolutions look at a short window of nearby words cheaply; attention's expensive every-word-to-every-word view is what stores wide world knowledge, so a mostly-convolutional model is sharp at local, structured work and thin on trivia.

The claim underneath is about method: when you judge an architecture on the machine it will actually run on rather than on a theoretical score, the field's default choice stops being obvious — a mostly-convolutional model that no one would have picked on paper wins twice the speed at a fraction of the size. For the growing pile of models meant to live on the device in your pocket rather than in a datacenter, the best shape may not be the one everyone copied.