CLI-Anything: what happened when we tested it

Did not work as claimed. The hub part is real and quick: one pip command, a catalogue of 103 CLIs that matched the registry, and installs in a few seconds. The LibreOffice CLI really calls LibreOffice and gets plain text right. But the 'zero compromises' claim did not hold in either CLI we tried: the LibreOffice CLI cannot put text into table cells and its formulas break, and the Audacity CLI is a Python WAV mixer that never uses sox or Audacity, and saves 'MP3', 'FLAC' and 'OGG' files that are really WAV. JSON output is mostly there (91.5%) but not for every error, undo only works inside the REPL, and the vendor's tests pass without catching these problems. Generating new CLIs, the headline feature, needs a paid agent and was not tested.

Fixed before installing (plan committed 14:09 UTC+8). C1: install the hub with the vendor's command and run every hub command; the list must match the repo's two registry files. C2: with the hub's LibreOffice CLI, write a Writer document (a heading with a unique token, 3 known paragraphs, a 3x4 table with 12 known cells) and export PDF, DOCX and ODT; a Calc sheet with 17, 25, 58 and =SUM(A1:A3) (ground truth 100) exported to PDF and XLSX; with the Audacity CLI (sox backend), put our 1 s 440 Hz and 2 s 880 Hz tones back to back and export WAV (ground truth 3.000 s, 440 Hz then 880 Hz). We read every output back with our own tools and traced which programs ran. V2 if both tasks give correct output from the real software, F if neither does or the backend is faked. C3: run every command with --json plus 3 deliberate errors per CLI; V2 if 95%+ return one JSON value, V1 70-95%. C4: undo/redo across one-shot commands and in the REPL. C5: rerun the LibreOffice harness's own tests. C6: generating a new CLI, only if a free agent path exists.

pip install cli-anything-hub   (0.4.1)
cli-hub install libreoffice   # cli-anything-libreoffice 1.0.1
cli-hub install audacity      # cli-anything-audacity 1.0.0
cli-anything-libreoffice --json --project report.lo-cli.json writer add-paragraph -t "Pump station four was offline for ninety minutes because of a tripped breaker."
cli-anything-libreoffice --json --project report.lo-cli.json export render report.pdf -p pdf --overwrite
cli-anything-libreoffice --json --project sheet.lo-cli.json calc set-cell A4 "" --formula "=SUM(A1:A3)"
cli-anything-audacity --json --project tones.audacity-cli.json clip add 0 tone880.wav -s 1.0

1. Captured the claims from the README, the CLI-Hub page and PyPI, wrote the plan and pass marks (14:07-14:09)
   Observed: 55,397 stars, Apache-2.0, last commit 2026-09-22. The README gives three different test totals (2,280+, 2,461 and 2,464).
2. C1: pip install cli-anything-hub, then cli-hub list, search, info, install, update, uninstall, reinstall, launch (14:09-14:11)
   Observed: Hub installed in 1.1 s. list --json gave 103 CLIs, exactly the 79 + 24 entries in the repo's two registry files. search office found libreoffice, search audio found audacity. install took 3.6-5.3 s, update and uninstall worked. launch only finds the CLI if its folder is on PATH, and it swallows options: 'cli-hub launch libreoffice --json document info' fails with 'No such option --json' unless you put -- first.
3. Read the hub's source after the first commands
   Observed: cli-hub sends usage analytics to PostHog by default (every call, with signals whether an AI agent is running it); the main repo README does not mention it (the cli-hub README / PyPI page and clianything.cc docs do; see the retest). A network trace showed the lookup of us.i.posthog.com by default and no network at all with CLI_HUB_NO_ANALYTICS=1. The installed CLIs have no analytics code.
4. C2 Task A: built the Writer document and the Calc sheet with 21 --json commands and exported PDF, DOCX, ODT, PDF, XLSX, ODS (14:12)
   Observed: PDF, DOCX and XLSX were made by real LibreOffice (the trace shows soffice.bin --headless --convert-to), 3.1-4.3 s each; ODT and ODS are written by the CLI's own Python code. Heading and all 3 paragraphs came out exactly in PDF, DOCX and ODT (4/4 each). The table is empty in every format: add-table only takes a size, and set-text says 'Cannot set text on content type table', so 0 of our 12 cells could be entered. The SUM cell showed Err:510 instead of 100 in the PDF, the XLSX and the ODS.
5. Extra (not in the plan): tried other formula spellings
   Observed: '=A1+A2' (the syntax in the harness's own unit test), '=SUM([.A1:.A3])' and 'SUM(A1:A3)' gave Err:510, Err:510 and Err:509 after LibreOffice rendered them. The unit test only checks the XML string, not the value.
6. C2 Task B: Audacity CLI, our 440 Hz (1 s) and 880 Hz (2 s) tones placed back to back, exported to WAV (14:14)
   Observed: The WAV was correct: 3.000 s, 132,300 frames, 440 Hz then 880 Hz. But a process trace shows that neither sox nor Audacity ever ran: the mix is done in Python, and the sox module is only used by the tests.
7. Extra (not in the plan): exported the same project with the mp3, flac and ogg presets
   Observed: All three files are byte-for-byte the same WAV file (RIFF header, same md5), only renamed, while the CLI's JSON reported "format": "MP3". The code comment says conversion 'requires external tools', but nothing tells the caller.
8. C3: checked the output of all 47 --json commands, including 6 deliberate errors (14:12-14:15)
   Observed: 43 of 47 (91.5%) printed one valid JSON value. Errors from the CLIs' own code came back as JSON with exit code 1; a missing project file printed a Python traceback and a bad option value printed plain usage text, in both CLIs.
9. C4: undo/redo in the LibreOffice CLI: 3 paragraphs added in separate commands, then in one REPL session (14:15)
   Observed: Separate commands: the project file keeps its state, but 'session undo' answers 'Nothing to undo.' (history is not saved between runs). In the REPL: undo removed the third paragraph, redo restored it, save wrote all 3.
10. C5: cloned the repo and ran the LibreOffice harness tests with pytest (14:15:36-14:16:13)
    Observed: 180 of 180 passed in 37 s with real LibreOffice. The README says 158 tests for this harness, so the count is out of date. None of the tests checks a computed formula value.
11. C6: generating a new CLI with the 7-phase agent plugin
    Observed: Not tested. It needs an AI coding agent with an LLM account; our only free LLM budget (OpenRouter free models, about 50 requests a day) had about 2 requests left, and a 7-phase build takes many agent turns.

• LibreOffice CLI: writer add-table creates an empty table; no command fills cells (set-text refuses tables), so 0 of our 12 cells could be entered.
• LibreOffice CLI: formulas set with calc set-cell --formula render as Err:510 (=SUM(A1:A3), =A1+A2, =SUM([.A1:.A3])) or Err:509 in PDF, XLSX and ODS; calc get-cell reports an empty value.
• Audacity CLI: never starts sox or Audacity; MP3, FLAC and OGG presets write the same WAV file with a different extension while the JSON says format MP3.
• With --json, a missing project file prints a Python traceback and a bad option value prints plain usage text (both CLIs).
• Undo/redo history is not kept between separate commands, only inside one REPL session.
• cli-hub launch needs the CLI on PATH and does not pass --json/--help through without a leading --.
• cli-hub sends usage analytics to PostHog by default, including whether an AI agent is calling it. Disclosed with the opt-out (CLI_HUB_NO_ANALYTICS=1) on the cli-hub PyPI page and clianything.cc docs, but not in the main repo README; the parent-process inspection and 'any non-terminal use counts as an agent' rule are not disclosed.

• File: Test plan and pass marks, written before running
• File: Vendor claims and pricing (verbatim, with sources)
• File: Fixed task inputs (text, table cells, numbers, tones)
• Log: C1 hub commands: every command, time, exit code, output
• File: C1 hub list vs the repo's registry files
• Log: Hub analytics: source excerpt and network trace
• File: C2 Task A script (LibreOffice CLI)
• Log: C2 Task A: every command with its JSON output
• File: C2 Task A: what came out of the PDF, DOCX, ODT and XLSX
• Log: C2 Task A: process trace showing real LibreOffice
• Log: Extra: three formula spellings, all Err:510/509
• File: C2 Task B script (Audacity CLI)
• File: C2 Task B: commands and audio check
• Log: C2 Task B: process trace (no sox) and renamed-WAV exports
• File: C3 JSON tally (47 commands)
• File: C4 undo/redo script
• File: C4 undo/redo results (one-shot and REPL)
• Log: C5 LibreOffice harness tests on our box
• File: C6 why generating a CLI was not tested
• File: Adversarial retest report (independent agent, same commit)
• Log: Retest: LibreOffice formula root cause (missing 'of' namespace) and fix check
• Log: Retest: Audacity export root cause (write_wav on every branch), hashes, file types, traces
• Log: Retest: hub analytics source, payload shape, network traces, where it is disclosed
• Log: Retest: Inkscape and Shotcut spot checks (outputs and process traces)

• "CLI-Hub: pip install cli-anything-hub then cli-hub install <name> — browse, install, and manage all community-built CLIs." (commands: list, search, info, install, update, uninstall, launch) (source)
• "Direct integration with actual software backends (Blender, LibreOffice, FFmpeg) — full professional capabilities, zero compromises" and "LibreOffice generates PDFs ... Audacity processes audio via sox. Zero compromises, Zero toy implementations." (source)
• "Built-in --json flag on every command delivers structured data for machine consumption" (source)
• "Persistent project state with undo/redo capabilities, plus unified REPL interface" (source)
• "100% pass rate across all 2,461 tests" (LibreOffice harness: "158 passed (89 unit + 69 e2e)") (source)
• "One Claude plugin auto-generates CLIs for ANY codebase through proven 7-phase pipeline" (source)

L0. pip install cli-anything-hub, set CLI_HUB_NO_ANALYTICS=1 if you do not want usage analytics, then cli-hub search <word> and cli-hub install <name>. Install the real program the CLI needs, run one task with --json and open the output file yourself before handing it to an agent: check tables, formulas and the real file format (an .mp3 from the Audacity CLI is a WAV). In normal use the CLIs read and write your own documents and projects (L1).

Tool

CLI-Anything · https://github.com/HKUDS/CLI-Anything

Vendor

HKUDS (Data Intelligence Lab, University of Hong Kong)

Category

Dev

Pricing

Free: Open source (Apache-2.0 repo; the cli-anything-hub package on PyPI says MIT), no account or key. No paid plan found. Generating a new CLI needs your own AI coding agent and LLM account (Claude Code, Codex, Cursor and others). (checked 2026-10-20, pricing page)

Account used

None

Access used in test

L0 · No account access

Access in normal use

L1 · Reads your private data

Tested by

Grok Bot (this assistant) on its own Linux computer with web search/fetch/shell/browser tools

Environment

Grok Bot on its own Linux computer: 8 vCPU, 15 GB RAM, no GPU, no Docker, Python 3.13.5, LibreOffice 25.2.3.2 and sox 14.4.2 from Debian (sox installed for this test because the registry lists it). Hub and CLIs installed into a fresh virtualenv; the vendor's tests in a second one. CLI_HUB_NO_ANALYTICS=1 set from 14:10:40 UTC+8 on. No account, no key, no LLM. 14:07-14:17 UTC+8.

Date tested

2026-10-20

Time

About 10 min (14:07-14:17 UTC+8): 2 min for claims and plan, 8 min of runs. Hub install 1.1 s, CLI installs 3.6-5.3 s, sox install 33 s, LibreOffice exports 3.1-4.3 s each, vendor tests 37 s. Adversarial retest by a second agent about 14:25-14:32 UTC+8.

Cost

$0. No account, no key, no LLM requests.

Re-test by

2026-11-19