Agent use cases we actually re-ran.

People post what their AI agents did. We re-run the claims, record the exact prompt and every site touched, rate how much account access it needs, and publish the failures too.

19 cases3 reproduced9 partly1 failed6 not yet verified
V2 · ReproducedL0 · No account accessResearchAgent: Grok Bot

Answer a question that needs this week's docs

Reproduced (V2) on 2026-10-19: the official docs say Sign in with ChatGPT usage is for eligible ChatGPT Plus and Pro subscribers, while the search engine's own summary hedged wrongly.

V2 · ReproducedL0 · No account accessLife adminAgent: Grok Bot

Weekly meal plan from what is in your kitchen

Reproduced (V2) on 2026-10-19: a 7-dinner plan built from an 18-item pantry used 0 off-pantry ingredients and planned a next-day lunch for every dinner.

V2 · ReproducedL0 · No account accessTravelAgent: Grok Bot

Which travel sites let an agent see fares?

Reproduced (V2) on 2026-10-19: of 8 travel sites fetched without a browser, only Google Flights returned fares (lowest $1,214 round trip, SFO to Tokyo Haneda, Dec 10-20, 2026).

V1 · Partly reproducedL0 · No account accessLife adminAgent: Grok Bot

90-day menu built from '5-star' recipes

Partly reproduced (V1) on 2026-10-19: recipe sites were readable, but 0 of 8 sampled Allrecipes dinners were rated 5.0 (range 3.7-4.9), so a '5-star recipe' menu needs a stated rule.

V1 · Partly reproducedL0 · No account accessLife adminAgent: Grok Bot

A morning brief (weather + headlines)

Partly reproduced (V1) on 2026-10-19: weather (New York high 16.3 C, 0% rain chance) and 5 BBC World headlines worked from public feeds; the calendar half needs account access and was not tested.

V1 · Partly reproducedL0 · No account accessMoneyAgent: Grok Bot

Am I owed EU261 compensation for a delayed flight?

Partly reproduced (V1) on 2026-10-19: under the official EU rules, a fictional 5-hour Amsterdam to New York delay (5,847 km) qualifies for EUR 600; filing the claim (L3) was not attempted.

V1 · Partly reproducedL0 · No account accessMoneyAgent: Grok Bot

Check if Bitcoin broke out on the 4-hour chart

Partly reproduced (V1) on 2026-10-19: a one-shot 4-hour breakout check worked on Kraken data (close 82,423.4 vs prior-20 high 82,826.0, so no breakout), but 2 of 3 exchange APIs failed and the alert part was not tested.

V1 · Partly reproducedL0 · No account accessShoppingAgent: Grok Bot

Compare one tire's price across retailers

Partly reproduced (V1) on 2026-10-19: all 4 retailer sites blocked an unauthenticated agent; prices of about $205-206 came only from search snippets, and at least one was stale.

V1 · Partly reproducedL0 · No account accessMoneyAgent: Grok Bot

Is a stock below my alert price right now?

Partly reproduced (V1) on 2026-10-19: THANGAMAYL traded at Rs 5,074.00 (two public sources agree), above the Rs 4,770 alert level; the recurring alert itself was not tested.

V1 · Partly reproducedL0 · No account accessTravelAgent: Grok Bot

Pick a trip window that avoids the rainy season

Partly reproduced (V1) on 2026-10-19: 5 seasons of Open-Meteo data show Phu Quoc and Ho Chi Minh City driest for Dec 15-Jan 5 (about 25 mm each) while Ubud stays wet (about 227 mm); flights and budget were not tested.

V1 · Partly reproducedL0 · No account accessTravelAgent: Grok Bot

Price a Japan round trip (and why agents disagree)

Partly reproduced (V1) on 2026-10-19: Google Flights gave a lowest fare of $1,214 round trip for a stand-in SFO to Haneda itinerary; the two-agent price comparison in the claim could not be tested with Grok Bot alone.

V1 · Partly reproducedL0 · No account accessTravelAgent: Grok Bot

Restaurants within walking distance of a Miami Beach hotel

Partly reproduced (V1) on 2026-10-19: OpenStreetMap listed 33 restaurants within 500 m of The Betsy South Beach, with distances and cuisines but no opening hours, ratings or prices.

F · FailedL0 · No account accessResearchAgent: Grok Bot

Build a 'who to follow in AI' list from X

Failed (F) on 2026-10-19: a 25-post X search produced a 'who to follow in AI' list with accounts of 5-21 followers in its top 20.

V0 · Sourced only, not yet verifiedL1 · Reads your private dataMoneyAgent: Not run yet

'Find my missed money' from email and card perks

Sourced only (V0), listed 2026-10-19: “'Find my missed money' from email and card perks” is not yet verified by triedagents because it needs L1 access (reads private data).

V0 · Sourced only, not yet verifiedL1 · Reads your private dataTravelAgent: Not run yet

'Organize me a trip based off today's e-mail'

Sourced only (V0), listed 2026-10-19: “'Organize me a trip based off today's e-mail'” is not yet verified by triedagents because it needs L1 access (reads private data).

V0 · Sourced only, not yet verifiedL4 · Spends, deletes or escalatesTravelAgent: Not run yet

Agent plans a work trip and buys the flight

Sourced only (V0), listed 2026-10-19: “Agent plans a work trip and buys the flight” is not yet verified by triedagents because it needs L4 access (spends money).

V0 · Sourced only, not yet verifiedL1 · Reads your private dataResearchAgent: Not run yet

Amazon replenishment: find high-refund, low-profit ASINs

Sourced only (V0), listed 2026-10-19: “Amazon replenishment: find high-refund, low-profit ASINs” is not yet verified by triedagents because it needs L1 access (reads private data).

V0 · Sourced only, not yet verifiedL1 · Reads your private dataMoneyAgent: Not run yet

Analyse last month's bills and recurring subscriptions

Sourced only (V0), listed 2026-10-19: “Analyse last month's bills and recurring subscriptions” is not yet verified by triedagents because it needs L1 access (reads private data).

V0 · Sourced only, not yet verifiedL3 · Acts as you, visible to othersLife adminAgent: Not run yet

Cancel a pickup order by phone on your behalf

Sourced only (V0), listed 2026-10-19: “Cancel a pickup order by phone on your behalf” is not yet verified by triedagents because it needs L3 access (acts as you).