For the modern LLM era, the standard recipe has rested on two pillars: a fixed tokenizer, and next-token prediction. Text becomes tokens, and the model learns by guessing the next one. Two lines of work are now challenging that recipe from opposite sides.

Meta’s Byte Latent Transformer (BLT) asks whether we need tokens at all. Next Concept Prediction (NCP), from researchers at Shanghai AI Lab and Shanghai Jiao Tong University, asks whether predicting the next token is even the right thing to optimize for in the first place. If either one holds up at frontier scale, the implications reach well beyond a single architecture choice.

1. Maybe We Don’t Need Tokens

Modern LLMs first break text into tokens using a fixed vocabulary. This works well, but it creates known weaknesses in areas such as multilingual text, code, numbers, unusual inputs, and character-level understanding.

BLT removes the tokenizer entirely and operates directly on raw bytes. The key idea is dynamic patching: instead of processing every byte independently, BLT groups bytes into patches whose size depends on how predictable that stretch of text is. Predictable regions get larger patches and less compute; complex, high-entropy regions get smaller patches and more compute. Input and output are still bytes, but the expensive transformer compute happens at the patch level. The model spends its budget where the information is actually difficult, rather than treating every token as equally worth thinking about.

That flexibility is not free. Patch boundaries come from an entropy model that estimates how surprising the next byte is, which adds architectural complexity and a bit of overhead. Switching is not an instant win for every workload.

Meta demonstrated this at up to 8B parameters and 4 trillion training bytes. The paper reports training-FLOP parity with tokenization-based models such as Llama 3 in its controlled comparisons, while showing that BLT can trade minor evaluation losses for up to about 50% lower inference FLOPs in the settings they tested. The remaining catch is generation: a naive byte-by-byte decode would take more steps than token-level generation. Meta’s follow-up work — BLT-D and BLT-DV — uses diffusion and verification to generate multiple bytes per step and reports substantial cuts in memory-bandwidth cost.

Alternative byte-level efforts make the same bet from another angle. Ai2’s Bolmo “byteifies” an existing Olmo checkpoint instead of training from scratch. That is a sign this is becoming a real research direction rather than one team’s side project.

2. Maybe Next-Token Prediction Isn’t Enough

The second development goes after something more fundamental than tokenization. Today’s LLMs are fundamentally trained around one question: what token comes next. NCP adds a second question alongside it: what concept comes next.

The researchers build a discrete latent vocabulary of ‘concepts’ from the model’s own hidden states — vector-quantized codes over short spans of tokens, not human-defined ideas — and train the model to predict those concepts in addition to individual tokens. Generation is still token-by-token. The concept head is an extra training target and an internal guide, not a replacement for the decoder. The bet is that this pushes the model toward learning higher-level structure, instead of routing everything it knows through token-by-token predictions.

This isn’t a small-scale proof of concept anymore. The September 2026 NCP-ArchPreview model has 8.9B parameters and was trained on 5.73 trillion tokens. The researchers report reaching the same final training loss as OLMo-3-7B using only 51.3% of its training tokens, along with a 2.45-point improvement on downstream benchmarks and a 5.99-point gain on GSM8K specifically. Those are strong numbers, but they are one team’s results, and the headline loss comparison is against a smaller 7B model. The report separately says NCP-ArchPreview approaches the training loss of a parameter-matched 8.9B baseline using about 85% of the compute. The results still need independent replication before anyone should treat them as settled. They are also still far below the compute used for current frontier training runs.

A Question: What if we combined them?

BLT attacks the representation layer — why should a model think in tokens at all? NCP attacks the prediction objective — why should it optimize mainly for guessing the next one?

Put together, you get something like: raw bytes → dynamic latent patches → concept prediction → concept-guided generation → bytes. That pipeline doesn’t need a fixed vocabulary, doesn’t spend equal compute everywhere, and isn’t forced to route its entire reasoning process through individual token predictions.

This combination is Robotic Magazine’s own question, not something either paper proposes — and stitching them together is not just a diagram. Patches compress local predictability over variable-length byte spans. Concepts compress multi-token meaning. Those boundaries will not line up automatically: a patch can split in the middle of a concept, and a concept can cover many patches. Any combination needs a bridge between local, entropy-driven units and global latent ones. That is an engineering problem, not a free upgrade. The reason to read the papers together is that they are attacking different layers of the same stack.

Questions Worth Asking

Can each one produce better LLMs on its own?
In principle, yes — each one can stand alone. Whether they do at frontier scale is still open. BLT could improve representation and efficiency by eliminating the tokenizer and allocating compute dynamically. NCP could improve abstraction and potentially reasoning, while also improving training efficiency by giving the model a higher-level predictive target. Neither depends on the other working.

Is there real-world use yet?
Not as a proven commercial replacement. Neither approach has publicly displaced the architecture behind any leading frontier model, and no major lab — OpenAI, Anthropic, Google DeepMind, xAI — has said its next flagship model is switching to either one. The research artifacts are public, though: BLT has released code, and both Bolmo and NCP-ArchPreview have open weights, so researchers can already run them. What does not exist yet is a production paradigm. Meta is still developing BLT, Ai2 is pursuing Bolmo, and the Shanghai AI Lab/SJTU team has scaled NCP to multi-trillion-token training runs. Serious research, not a replacement stack.

Could this ultimately be a trillion-dollar difference?
Potentially, yes. If either architecture ends up delivering meaningfully more intelligence per dollar of training and inference, the economics compound: a lab with that advantage can train larger models on the same budget, serve more users more cheaply, or reinvest the savings into the next model. At frontier scale, a structural efficiency advantage becomes a competitive one. But there’s a real gap between “potentially trillion-dollar technology” and “a trillion-dollar investment thesis today,” and right now we’re on the wrong side of that gap.

So is the current LLM era ending?
We don’t know, and that’s the honest answer. Tokenized transformers and next-token prediction have been extraordinarily successful, and neither BLT nor NCP has proven the industry should abandon them. What makes both papers worth paying attention to is that they’re questioning assumptions that had become almost invisible after years of scaling on the same recipe. Maybe tokens aren’t fundamental. Maybe next-token prediction isn’t either. If either turns out to be substantially better at frontier scale — or if someone combines them — the consequences won’t stay contained to architecture diagrams. They’ll show up in the economics of the entire AI industry. That’s the part worth watching.

Sources:

BLT — Meta
The first paper about eliminating tokenization.
Byte Latent Transformer (BLT) — arXiv

NCP — original paper
The paper introducing Next Concept Prediction.
Next Concept Prediction — arXiv

NCP-ArchPreview — newer technical report
This is the large-scale follow-up with the 8.9B model and 5.73T tokens.
NCP-ArchPreview Technical Report — arXiv

---

Post By: A. Tuter

Terms of Use: Unauthorized copying is prohibited; we maintain dated records to document original publication. Content may contain inaccuracies. All trademarks mentioned belong to their respective owners; we are not affiliated with them or any other entities mentioned, and no endorsement is implied. See our Terms Page.