▶ YouTube Summaries

• YT :: https://www.youtube.com/watch?v=NE006pkiAnk
• Original title :: So I've been testing Gemini 4...

Theo reviews Google's Gemini 4 Argan, the first full-size Gemini release in eight months, after early access from friends at Google. He finds some real strengths: front-end work with Anthropic's design skill, a perfect score on his skateboard-trick naming bench, and unique YouTube and vision capabilities. For real code work he is scathing. The model is token hungry, misses the cache, makes a lot of false claims in codebase audits, and produced a bloated, failing PR that he closed. He calls it one of the worst releases of the year and says he will use it only for fetching YouTube data. He keeps Opus 5.5, with Muse Wonder Full as a pleasant alternative.

Sponsor: WorkOS

• Agents now do most of his work, so products must be signable by agents, not just humans.
• WorkOS is the enterprise sign-in platform, and it co-designed the OMD standard with Firecrawl and Cloudflare so agents can register on a user's behalf. Neon, Monday and Parallel have adopted it.

Release and pricing

• Eight months since the last Pro-size Gemini (February); only Flash models in between. Google pitches Argan for deep work in coding, finance, legal, cyber security and science, and ranks first on the VL index.
• Launch price is $2 in / $10 out per million tokens, which looks cheap next to Opus 5.5. It is an introductory price that doubles to $4 / $20 on December 1st. Theo dislikes this recurring Google habit and says labs should lower prices over time, not raise them.
• Not as cheap as it looks. The model is token hungry, and on Artificial Analysis it costs more than Opus 5.5 on high while scoring lower. It beats Astra and 6.1 Soul on that index but sits well below Opus 5.5.
• Cache hits are poor, only 70 to 80% against 99%+ for other models, so the 95% cache-read discount rarely applies. It might be a bug in Anti-Gravity or in his MCP setup, but he leaves figuring that out to Google.
• A Google employee told him they reach for Opus 5 less. That is hardly a revolution, since Opus 5 is the model he panned.
• Google's claims (35% more agentic code submissions, months of refactoring done in weeks) sound like 2024. Google's launch comparison lists a $4 / $20 price for Muse Wonder Full, which is wrong since Meta published none, so he trusts the page less.
• Google's own software is as broken as ever: a dark-mode-less site, a broken video embed. He jokes that you will know Google has a good model when its software finally works.

Skatebench: a perfect score

• Argan scored 100% on his private skateboard-trick naming bench. Opus 5.5 Max gets 79%, and 3.8 Flash got 96%.
• It was fast and did not overthink, which is rare for Gemini. The API was unstable during testing, with high error rates that forced him to rework the bench.
• A suite's "100% success" claim depends on how the eval relates to your work, and few people make a living naming skateboard tricks.

Front end

• Homepage designs look good, not obviously AI generated, with a functional interactive section. Gemini 3.1 Pro built much of the UI of his video review tool Lawn.
• Tested through whichai.dev, using Anthropic's official design skill. Without it you get cringe nav bars and filler text. The "taste" skill brings the bad layouts back.
• Fable 5.1 is still his best for striking homepages. For iterating on real UX he prefers fast OpenAI models, and Argan generates tokens quickly but does not solve problems quickly.

Unique capabilities

• Every lab has an edge via its own services: OpenAI image generation, Grok's Twitter data. Google's is YouTube, such as transcripts via the Gemini API, which his businesses use heavily.
• Gemini is still not good with Google's own services such as email, and the Gemini button in apps scares him.
• Vision tasks (labeling a motherboard photo, PDFs, video, audio) have historically been a Gemini lead and are worth benchmarking, though others have caught up. Multilingual strength is rumored but unverified.
• Skatebench suggests hidden secret sauce: only Gemini and Muse Wonder break 90%.

Fishlop (3D game)

• Submarine models are mediocre, better than Muse and Grok but far below the 3D demos he saw online. Water surface rendering was badly broken, and the five-tank game loop works.
• Animation is decent, with fish eyes pointing the right way.
• Food that falls to the floor never expires and fish starve staring at it. A second pass fixed feeding and graphics but not the submarine.
• A power-up that boosts attack lasts 30 seconds, expiring before the next monster wave arrives, so it is useless. A smart model would not make that mistake.
• His reading: no "big model smell". 3D is trained separately and does not reflect general intelligence.

Hallucination claims versus his experience

• Artificial Analysis gives a 15% hallucination rate, the lowest among models above 45 on its index. But the omniscience bench rewards refusals as neutral. On the overall score Argan drops to fourth because it answers too few questions correctly.
• This does not match his day-to-day experience, where it is often wrong.

Codebase audit benchmarks

• Method: models audit a codebase, claims are validated, and a judge panel scores them.
• TS Rust (Tyson compiler rewritten in Rust): Soul 8.84 with 38 true claims and no wrong ones; Opus 5.5 8.78 (one wrong claim, 49 min, $24); Argan 4.38, 43 minutes, $3, and 24 of 36 claims false. Astra on max scored worse than on xhigh, so avoid max reasoning levels.
• T3 Code bulk update bug: Opus 5.5 8.22, Soul missed some high-severity items but made no false claims (10 min, $1). Muse Wonder Full scored lower but was readable and fast.
• Lakebed audit: Soul 9, Sonnet 5.5 8.5, Opus 8.25, Muse 5.75, Argan 2.25. Of 78 findings, 56 were wrong, a 70% failure rate.
• T3 Code architecture audit: Argan 3.95. It had 31 true, 14 partially true and 47 false findings, more false findings than Muse had true ones. Only Muse Wonder Standard and Argan fall in the garbage tier.
• The output is walls of text on things that do not matter.

Real task: SQLite connection stability

• Asked to fix SQLite-related connection instability on Windows. It correctly spotted the split read pool with a single writer, but wandered through every file synchronously.
• After 30 minutes it opened a 1,200-line PR (the fix should be about 30 lines), ignored the repo's PR-format skill, wrote a run-on summary, failed CI lint and check, and ignored review bot comments.
• Opus reviewed the thread and the PR and concluded it was slop. The diagnosis lacked evidence, it fixed the wrong half (the reader pool still blocks the main thread), invented a percentage, and missed two existing PRs that already fixed the issue (Muse had pointed such a PR out earlier). It also used a Rx-based route for requests.
• Opus' verdict: Gemini can read and explain code, but does not test its own theories. The real fix was 21 lines and is merged.

Anti-Gravity, ACP and account risk

• A banned Gemini account could cost him his Google account, with email, billing, passwords and the YouTube channel. Google promises to ban only Gemini and Anti-Gravity access, but he has seen full bans.
• Anti-Gravity's ACP integration is poorly done: model slugs are paired with effort levels, and a run failed to connect after over an hour and lost all history. A high share of T3 Code bugs come from Anti-Gravity.
• Next to competent models like Muse, Opus and Sonnet, the first usable Gemini UI hurts more.

Verdict

• He does not trust Gemini 4 Argan or standard Muse Wonder (Muse Wonder Full is fine), and wants to auto-close T3 Code PRs written with them. Vetting its code cost more time and tokens than writing it with a smarter model.
• At best it is useful as a first or second pass on a nice marketing page, plus fetching YouTube data. He has not merged anything it shipped.
• A rumored better snapshot (codename Carbon) may come soon. He sticks with Opus 5.5.