# Pomerado — pre-registered test plan
Written 2026-11-03 ~08:53 (Asia/Shanghai), BEFORE live keyless MCP/REST calls or OSS install runs.
Commit time of this file is authoritative.

Tool: https://pomerado.ai — hosted keyless MCP `https://mcp.pomerado.ai/mcp/keyless` + REST `https://api.pomerado.ai`; OSS core https://github.com/Pomerado/pomerado (MIT, npm `pomerado`).
Environment: Grok Bot Linux box, curl/Python HTTP client only (no computerUse browser). Free/keyless only; $0; no account unless keyless is blocked by our network and a free signup path is clearly usable without card/SSO we cannot pass.
Budget: finish by ~09:45 if possible; ready file by 10:15. Hard stop on paid paths.

## Fixed test task + ground truth
1. Keyless discover: list MCP tools (or REST GET /v1/tools?query=search+flights) with NO Authorization header against the documented keyless/public endpoints.
2. Find a flight-search tool: call find_website_tool (MCP) or equivalent REST search with query roughly "search flights" (optionally site google.com / kayak / similar public flight search). Record tool_id, effect, input_schema, login.
3. Run one read-only flight search with fixed inputs:
   - origin: SFO
   - destination: JFK
   - departure_date: 2026-11-20
   (match schema field names from the found tool; do not invent extra required fields beyond schema.)
4. Ground truth for the run result (pass marks below): result must be structured JSON (not a raw HTML dump / browser crash), include at least one flight-like entry with identifiable fields (airline and/or price and/or departure time), and origin/destination or route consistent with SFO→JFK (or explicit airports matching those codes). We do NOT assert a specific cheapest price (prices move). Compare sanity against a public Google Flights / similar page fetch only if needed for V1 edge cases.
5. OSS probe (C3): clone or `npm pack`/`npx -y -p pomerado pomerado-mcp --help` (and if cheap, list tools via stdio initialize) WITHOUT OpenAI key / without minting. Pass = package installs and advertised CLI/MCP entrypoint responds; do not rate local mint quality (needs OpenAI key → out of scope / V0 if claimed as free).

## Core claims (verbatim where possible)
C1 (CORE): Keyless access works without account — "No account needed to try it" / "Try Pomerado for free, with no account, sign-in or API key" and can discover/list tools and find a website tool for flight search via find_website_tool or REST GET /v1/tools.
  Pass V2: unauthenticated call returns tools list OR find_website_tool succeeds with ≥1 flight-related tool (effect=read preferred), no sign_in_required for find/list.
  V1: works only on REST or only on MCP, or finds tools but with friction/notice that still allows the try path.
  F: keyless endpoints refuse discovery with auth error when used as documented, or no flight tool exists on keyless catalog.
  V0: our network blocks the host (document + try alternate free path before F).

C2 (CORE): Running a public read-only website tool returns structured usable results (deterministic API-style), not a brittle browser failure. Docs: run_read_only_website_tool / POST /v1/runs/read-only "usually returns the finished job in seconds" with a JSON result (example flights array).
  Task: run the tool found in C1 with SFO→JFK on 2026-11-20; poll get_job if needed up to ~120s.
  Pass V2: job status succeeded; result is parseable structured data with ≥1 flight-like row and route/airports consistent with SFO→JFK.
  V1: succeeded but thin/partial schema, or succeeded after one retry/repair, or structured but route fields ambiguous while still clearly a flight list.
  F: failed with browser/automation error, empty garbage, HTML-only blob, or permanent failure unrelated to try_limit.
  Note: try_limit_reached is itself a documented result — rate as V1 for the free-path claim if discovery worked but run quota exhausted (not F); record the error code.

C3 (CORE): Open-source / installable core exists and is usable as claimed — GitHub MIT "Open source core integration minting harness"; README `npx -y -p pomerado pomerado-mcp --help` / local MCP.
  Pass V2: npm package installs (or npx help works), LICENSE MIT present, CLI/MCP entrypoint responds (help or tools/list over stdio) without requiring a key just to start/list.
  V1: repo+npm exist and help works but listing tools needs extra setup we can still complete without paid keys; or Node version pin friction we work around.
  F: package missing/broken, or help/entrypoint fails on documented Node, or "open source" repo empty of runnable code.
  Non-goal: full mint of a new integration (needs OPENAI_API_KEY) → if vendor implies free local mint without key, that sub-claim is V0/not tested, not F.

## Non-core (do not decide overall; may V0)
N1: "7x speedups" / "2x faster, 10 to 1,000 times cheaper, and 30% more accurate" vs computer-use — V0 unless we can reproduce (we cannot run their 30-task suite today).
N2: Self-healing integrations, 2FA vault, build_website_tool — need account / logins; skip.

## Verdict rules
Overall = worst CORE among C1–C3. Any core V0 → overall max V1. Score = mean of tested claims (V2=10, V1=5, F=0, V0 excluded), one decimal.
Access: expect level L0 (keyless), typical_level L0–L1 (free account for builds/logins).
published:true unless F or disputed → then hold for Lena.
