Grok 4.7 costs more than it delivers
- YT :: https://www.youtube.com/watch?v=jLgpzgpsWPc
- Original title :: Elon promised this one would be good...
Theo reviews Grok 4.7, a model Elon Musk spent a month promising would beat every competitor at real-world coding. It does not. It is pleasant to work with and unusually thorough, but it burns roughly twice the tokens of Grok 4.6, costs more per task than GPT-6 Astra, and benches below models a generation older. The broader theme: benchmarks no longer describe what makes a model good or bad, and xAI's own messaging around this release is misleading.
The token-efficiency claim does not survive contact
Musk promised 4.7 would be slower but more token-efficient on a new 2.1T-parameter base. The base is new, but efficiency went the other way: Grok 4.6 xhigh used ~38k tokens per task, 4.7 uses 81k — above Fable 5.1, and nearly 3x GPT-6 Soul's 29k. Cost per task on artificial analysis went from $1.86 to $3.74, above Astra's $3.26.
Cursor CEO Michael Truell pushed back, saying production data shows only 5% more tokens on median requests and 20–30% at P99. Theo's counter: a request is not a prompt. More tool calls mean more requests per prompt, so per-request token growth says nothing about the cost of an agentic task — and the rebuttal was aimed at a benchmark of static prompts anyway.
Theo also traces where the efficiency went. Musk publicly said the model was delayed because RL had penalized response length too much and it gave up on hard tasks early; the fix was training it to work longer and verify more. That is precisely the trade that ate the efficiency — and it produces failure modes like the widely shared loop where the model chants "I'm ready. I'm in. I'm building. I'm the machine." indefinitely.
Benchmarks and cherry-picking
xAI's announcement leans on an odd selection: electrical engineering, legal work, HealthBench clinical reasoning, AA Briefcase (second place, their proudest number). Terminal Bench shows Fable 5.1 roughly 20 points higher. GPT-6 Astra is absent from the internal charts because OpenAI banned xAI from using its models, so third parties cannot fill the gap either.
On artificial analysis's aggregate index, Grok 4.7 scores just above GLM-5.3 Max and below Muse Spark 1.3. On the coding agent index it gets 56 against 62 for Codex and Claude Code. Cognition's Frontier Code bench has it trailing Grok 4.6 outright, which they attribute to 4.7 over-scoping on tasks where restraint was the right call.
Front-end output Theo calls some of the worst he has seen from any model, worse than he remembers GPT-5 producing.
Pricing and subscriptions
Price doubles above 200k tokens and increases again in fast mode, so headline parity with 4.6 is misleading. Ben Davis, Theo's podcast co-host and a committed Grok user, burned 8% of his weekly limit on $40 of usage — a sharp change from the effectively unlimited Grok usage the $300 plan used to offer, and poor against the $8–12k of monthly value in $200 plans elsewhere.
What it is actually good at
Theo does like the model. Its floor is higher than its capability tier suggests: it stays on task, completes real work without hand-holding, and is unusually inquisitive, digging into details other models skip. On his ad-hoc bench asking models to find improvement areas in the T3 Chat codebase, it placed just behind Astra and ahead of Fable 5.1 on suggestion viability — but at up to twice the cost, because it will keep verifying things that do not matter.
His verdict: a stepping stone, not a frontier model. Last-generation performance at current-generation prices, without the subsidized subscription economics that previously made Grok worth switching to.