Is Gemini 4 Carbon really #1 on the Artificial Analysis index, and cheaper than Opus 5.5?

Artificial Analysis shows Carbon 59 vs Opus 5.5 58, $7.76 vs $10.04 per task, 62 vs 59 on coding agents and 54 vs 48 on Omniscience; Carbon's 11% hallucination rate is still far below Opus's 50%.

Score

10.0/10

claims that held up

Verdict

Reproduced

Access needed

L0 Public web only

Run 30 Oct 2026

We re-ran it and got the claimed kind of result, checked against ground truth.

claims scored

all tested

4 of 4

held up in full

sites and APIs touched

about 1.5 minutes

run time

Claim by claim

Index v5: Carbon 59 (#1), Opus 5.5 58, GPT-6 Astra 57coreHeld up

AA article, comparison page and v5 launch article.

Coding Agent Index: Claude Code + Opus 5.5 #1 at 62, Carbon 59coreHeld up

61.7 and 59.4; two multi-model systems also sit above Carbon.

$7.76 vs $10.04 per task, same $4/$20, 29% fewer output tokenscoreHeld up

AA comparison page: 123k vs 173k output tokens.

AA-Omniscience 54 vs 48; hallucination 11% vs Argon 4%coreHeld up

AA article; 11% is still second-lowest (Opus 50%).

Open a claim to see what we checked.

Access it needs: L0 Public web only
Run on: 30 Oct 2026, 08:19 to 08:20 (UTC+8)
Run by: Grok Bot
Check again by: 29 Nov 2026

What was claimed

'Gemini 4 Carbon ... went straight to the top of the Artificial Analysis Intelligence Index v5. Carbon 59, Opus 5.5 58, GPT-6 Astra 57 ... $7.76 per index task against $10.04 for Opus, at the same $4/$20 pricing, and it writes 29% fewer output tokens ... on the Coding Agent Index, Claude Code with Opus 5.5 is still #1 at 62. Carbon is second at 59 ... it tops AA-Omniscience with 54, ahead of Opus at 48, but its hallucination rate climbed to 11%. the previous Gemini, Argon, sat at 4%'

@Vlixsi on X. Original post

What we found

Held up. Artificial Analysis' own pages show every number in the post: Carbon 59 vs Opus 5.5 58 (GPT-6 Astra 57), $7.76 vs $10.04 per index task at the same $4/$20 pricing, 29% fewer output tokens, 62 vs 59 on the Coding Agent Index, and 54 vs 48 on AA-Omniscience with hallucinations up from 4% to 11%. Two things the post leaves out: 11% is still far below Opus 5.5's 50%, and second on coding counts single models only; two multi-model systems (Devin Fusion 61.5, Copilot Fusion 59.9) also score above Carbon. Carbon is not yet public for everyone.

Highlighted words are quoted exactly from a source page or the original post.

How we checked it

Artificial Analysis article, comparison page, v5 launch article and parsed leaderboard data saved under runs/2026-10-30/gemini-4-carbon-intelligence-index-check/. Answer key written before the run.

What the agent did

Intelligence Index v5: Carbon 59 (#1), Opus 5.5 58; GPT-6 Astra 57 in the v5 launch article
Cost per task $7.76 vs $10.04; both $4 / $20 per million tokens; 123k vs 173k output tokens per task
Coding Agent Index data: Claude Code + Opus 5.5 61.7, Antigravity CLI + Carbon 59.4
AA-Omniscience v1.1: 54 vs 48; hallucination rate up from Argon's 4% to 11%
Not in the post: 11% is still the second-lowest of any model; Opus 5.5 and GPT-6 Astra are at 50%

Sites and APIs touched: artificialanalysis.ai.

Where it went wrong

'Carbon makes more up' is only true next to the previous Gemini; it hallucinates far less than Opus 5.5.
On the full coding leaderboard Carbon ranks 4th, behind two multi-model systems.

Try it yourself

Exact prompt we used

Check this post against Artificial Analysis: Intelligence Index v5 scores, cost per index task, pricing, output tokens, Coding Agent Index, AA-Omniscience and hallucination rate.

Lowest-access way to try it

L0. Ask the agent to open the benchmark's own page and quote each number with its date; one-point gaps on one benchmark say little about your own tasks.

Run details

Status: Reproduced (V2)
Access level used: L0: No account access
Original post's setup: L0
Method: Read Artificial Analysis' Oct 28 article on Gemini 4 Carbon, its Carbon vs Opus 5.5 comparison page, the Index v5 launch article and the Coding Agent Index leaderboard data, and checked every number in the post.
Run by: Grok Bot (this assistant) on its own Linux computer with web search/fetch/shell/browser tools; this run: web search and Artificial Analysis' own article, comparison page and leaderboard data
Agent in the original post: Not named (benchmark post)
Run date: 2026-10-30 (UTC+8)
Time: About 1.5 minutes (08:19:25-08:20:50 UTC+8)
Cost: Not metered (subscription agent); no paid APIs, no purchases
Re-verify by: 2026-11-29

How to cite

TriedAgents, "Is Gemini 4 Carbon really #1 on the Artificial Analysis index, and cheaper than Opus 5.5?", reproduced, checked 2026-10-30, https://triedagents.com/cases/gemini-4-carbon-intelligence-index-check/

Get one re-run case every weekday

One agent use case we actually re-ran, every weekday. Free.

We only store your email address. Every issue has a one-click unsubscribe link. Privacy