摘要 · Summary
AI Security · Wire-Level Teardowncereblab · The Register · 2026-07
Executive summary · one-page brief
“Do Not Open Any Files” — and the Whole Repo Shipped Anyway
A wire-level teardown of xAI's Grok Build CLI (v0.2.93) showed it ran two independent channels: the model's own file reads, and a separate background upload of the entire workspace as a git bundle. The permission prompt governed only the first. In the decisive control run the agent was told to open nothing — it obeyed, and the repository left the machine anyway, full git history inside it.
By the Numbers one 12 GB repo, one session
- 27,800×more data left the machine than the model's turns needed
- 5.10 GiBuploaded via /v1/storage — against 192 KB of model traffic
- 0settings found that disable the upload — “I did not find one”
- 2unrelated repositories — the same never-read canary recovered
What Was Proven — and What Was Not the teardown states its own limits
Shown on the wire
Proven
- Tracked files plus full git history uploaded as a bundle.
- Upload happened with the agent told to open nothing.
- A .env read by the agent went up unredacted.
- Replicated on a second, unrelated codebase.
Explicitly not claimed
Not proven
- “We did not prove xAI trains on this data. Upload/storage ≠ training.”
- No setting was found that disables it — but enumeration was not exhaustive.
- Some captures — 3 GB direct-to-GCS PUTs — were not retained.
- Whether the code still sits in the binary — not the teardown's claim.
Bottom line
The failure is in the consent model, not the code. The prompt asked “may I read?” while a second channel bundled git-tracked files plus full history and shipped them regardless.
Reframe
Your exposure is whatever the tool uploads — never whatever it reads.
起承 · Evidence · Content 1 / 2
起 — Two Channels, Different Rules the permission prompt sees only one
Channel A · read-driven
/v1/responses
- Carries only what the agent actually opened.
- This is the channel the permission prompt governs.
- 192 KB across five requests in the test session.
- Never-read files are genuinely absent here.
Channel B · repo-driven
/v1/storage
- Bundles git-tracked files plus full history.
- Indifferent to what the model read — or didn't.
- 5.10 GiB from the same session; 73 chunks, all HTTP 200.
- Destination: a Google Cloud Storage bucket.
承 — The Canary That Never Was Read prose left · the decisive run right
The byte ratio alone is suggestive: 192 KB of model turns cannot carry 5.10 GiB, so the storage channel must be a whole-repository snapshot rather than a by-product of reading. But the decisive proof is per-file. A run was prompted “Reply with exactly: OK. Do not read or open any files.” The agent complied and opened nothing — and POST /v1/storage still returned 200, carrying the repository as a git bundle. Cloning the preserved bundle recovered a planted canary file the agent had never touched, verbatim, together with the full four-commit history. It replicated on a second, unrelated codebase. The harness is public, so anyone can re-run it.
- OKthe agent's entire reply — it opened nothing
- 4 commitsof history recovered from the uploaded bundle
On the Wire one session · two destinations
| What went up | Channel | Evidence |
|---|---|---|
| The repository, as a git bundle | /v1/storage | 73 chunks of ~75 MB, every one HTTP 200; cloned back to recover a never-read canary. |
| A .env, unredacted | both | API key and DB password in clear in a 48 KB request body, and again inside a session-state archive. |
| Destination bucket | GCS | Named in the binary's strings and in staged metadata paths — corroborated three ways. |
- 0.2.93the build put behind the proxy — its repro harness is public
- 73chunks of ~75 MB, every one accepted HTTP 200
- 1 GiBper-file cap the server advertised — max_upload_file_bytes
- 5model turns, 196,705 B in total, against 5.10 GiB
轉合 · Pivot & Resolution · Content 2 / 2
轉 — Three Switches, None of Them the Brake assumption vs. behaviour
| Control | What a developer assumes | What it actually does |
|---|---|---|
| “Improve the model” toggle · default ON | opts me out of data collection | governs training/retention policy, not transmission. Switched off, the server still returned trace_upload_enabled: true and the bundle still uploaded. |
| /privacy command | stops the sending | a data-retention setting — “not a block on what's sent”, wire-tested by cereblab after xAI added it. |
| Any user-side off switch | there must be one somewhere | “I did not find a setting that disables the upload in these tests.” The uploads stopped only when xAI disabled them server-side — no switch the user could reach stopped it. |
How It Unfolded chronological
- 07-12cereblab publishes the wire-level teardown of grok 0.2.93, with a public reproduction harness and preserved evidence.
- sincePer the teardown's own update note: xAI disabled the upload server-side (disable_codebase_upload: true) and added a /privacy opt-out — which cereblab wire-tested and found to be a retention setting, “not a block on what's sent”.
- 07-14The Register reports; the thread reaches Hacker News' front page. Musk publicly commits to deleting all previously-uploaded data — “not yet confirmed complete”, in the author's words.
- still openThe deletion remains unconfirmed. Press coverage reports no advisory or changelog accompanied the fix, and that a later build still carries the upload code — both secondary-sourced; neither is the teardown's claim.
合 — Why a Deleted Secret Still Shipped the exposure chain
- 01Commita secret, months ago
- 02Deletefrom the working tree
- 03Persistit lives on in git history
- 04Bundletracked files + history
- 05Uploadregardless of reads
- 06Rotatethe only real remedy
Recommendations highest-return first
- Rotate every credential reachable in tracked git history — deleting it never removed it, and the pledge to erase uploads cannot un-ship it.
- Verify on the wire, not in the prompt — a permission dialog describes one channel; measure what actually leaves the machine.
- Demand default-off — an opt-out you must run every session is not consent. The default is the policy.
- Treat a vendor-side fix as a pause — no switch you could reach stopped this one. Prefer tools whose off switch you hold locally and can verify.
Where it lands
Any tool whose consent surface is scoped to a narrower channel than its transmission has this bug — whatever its logo. Grok Build gated reads while a second channel shipped the repository, and no user-side setting stopped it: the uploads ceased only when the vendor disabled them on its own servers. A remedy you cannot reach is not a control. The right default is off.