# Rosette — Phase 0 testing guide Phase 0 answers one question: **can Rosette actually do what the product promises, on a real Android phone?** Nothing in this phase is product code. It exists to produce evidence. Everything runs locally: no experiment sends data anywhere, and the Rosette source contains no analytics. The current independent VPS evidence host was smoke-tested without Lovable's injected `~flock.js` or any third-party script. The page and service-worker guards remain in place, and E5 still cannot pass unless the handset audit proves zero outbound telemetry bytes (see E5 below). The Lovable deployment remains a read-only rollback checkpoint, not an evidence host. --- ## 1. What you need - A **physical Android phone**. Not an emulator. Emulators lie about GPU memory, thermal behaviour, WebAuthn hardware and the share sheet, and every one of those matters here. If a run happens on an emulator, the report marks the device as likely emulated and the result cannot be recorded as a pass. - **Chrome for Android**, up to date. - A Wi-Fi connection for the model downloads, and the ability to switch it off completely for the offline test. - At least **2 GB of free storage** — the largest model candidate is ~190 MB and the export test allocates large surfaces. Ideally test on two phones: one recent flagship and one mid-range device that is two or three years old. The mid-range result is the one that decides what Rosette can ship. ## 2. Getting the harness onto the phone 1. On a computer, run the dev server (`bun install --frozen-lockfile`, then `bun run dev`) or open the preview build. 2. On the phone, open `/dev/spike`. 3. Chrome menu → **Add to Home screen** / **Install app**. Installing matters: the share sheet test and the offline test only work for an installed app. 4. Open the installed app once while online, so the shell caches. The harness is disabled unless the build is a development build or `VITE_ENABLE_SPIKE=true` is set, so it cannot appear in a normal build. ## 3. Before you start Fill in **Device under test** at the top: the exact model name and the physical RAM. Then tap **Capture environment**. This records the real user agent, the high-entropy client hints, the WebGPU adapter, storage quota, and whether the page is cross-origin isolated. On Android, the reported OS version prefers the high-entropy `platformVersion` hint and records that source explicitly; modern Chrome freezes the legacy user-agent text at `Android 10; K`, so that text is only a fallback. Tap **Request persistent storage** too. If the browser refuses, that is a finding worth having. ## 4. The experiments Run them in order. Each writes into a single report you download at the end. ### E1 — On-device matting quality Pick a candidate and a backend, then run. The harness clears the model cache first so the download is genuinely cold, then runs the model over the seven approved categories, in this order and with no substitutions: 1. people and hair 2. pets and fur 3. hard-edged products 4. everyday objects 5. flat clip-art 6. illustrated flowers 7. real flowers Four supplementary stress cases (translucent glass, low-contrast background, motion blur, multiple subjects) run afterwards. They are extra evidence only and never stand in for one of the seven. Synthetic fixtures are used by default. To judge real quality, pick a category in the dropdown, choose one of your own photos for it, and repeat for as many of the seven as you can. Each subject is tagged in the report with whether it used a synthetic fixture or your photo, and whether it is approved or supplementary. The built-in fixtures are **synthetic**. They stress the pipeline but they are not photographs, and the report labels them as synthetic. For a real quality judgement, attach one of your own photos with the file picker and run again — that run is labelled `tester-supplied`. Look at the matte thumbnails. That visual check is the actual test; the harness deliberately leaves E1 as `unverified` until a human has looked at the mattes and recorded a judgement. Run every candidate at least once: | Candidate | Why it is in the list | | ------------------------ | -------------------------------------------------------- | | `birefnet-lite-512-fp16` | Primary general-purpose candidate | | `birefnet-lite-512-fp32` | Quality reference for the FP16 variant | | `modnet-fp16` | Portrait/hair supplement only | | `modnet-q8` | The only genuinely published Q8 artefact available to us | There is no Q8 build of BiRefNet-lite-512 published anywhere we can pin, so there is no Q8 BiRefNet row. The report records that as unavailable rather than inventing a number. ### E2 — Timing and memory E2 reads the timings E1 just captured and checks them against budgets. Five figures are kept **separate** and never merged: 1. **Download** — cold transfer of the model assets, measured by counting bytes off the response stream. If a request could not be measured, the report says so; it never substitutes the size printed on the model card. 2. **Cold init** — building the session with an empty cache. 3. **First inference** — the first run after a cold init. 4. **Cached init** — building the session with assets already cached. **This is the figure the "under 2 seconds warm start" target applies to.** 5. **Cached inference** — a run after a cached init, judged against its own separate budget. If WebGPU initialisation fails and the harness falls back to WASM, the report records both the requested and the actual backend. A WASM number is never presented as a WebGPU number. After the timings, run **E2 device loss + recovery**. This deliberately destroys the WebGPU device mid-session — the same thing that happens when the phone throttles or the driver resets — and waits for `device.lost` to settle. Recovery must then complete on WASM **in the same page**: no reload, and the unsaved in-memory edit must still be there (the harness checksums it before and after). If the phone has no WebGPU, the experiment records `unavailable` and the recovery half still runs. ### E3 — Full-resolution export Run it once for each tier: 12 MP, 24 MP, 48 MP, and the 64 MP stretch tier. Try at least PNG and JPEG, and try more than one tile size. The scene is deliberately awkward: a curves adjustment, a filter chain, a blur, a text layer crossing a vertical tile boundary, a rotated sticker crossing a horizontal one, and a feathered alpha mask. "The encoder returned a file" is not a pass. Each tier is decoded back and checked for: - decoded dimensions matching what was requested - the container actually being the format asked for (magic bytes, not the MIME string we passed in) - transparency behaving correctly — preserved for PNG/WebP, matted and fully opaque for JPEG - no metadata leaking into the output - **no visible seams**: every internal tile edge is sampled and compared - **source-resolution provenance**: a fine checkerboard is compared against a control that was deliberately enlarged from a 1024 px preview. If the export had been upscaled from a preview, the checkerboard would collapse and the check fails. If a tier fails to allocate at all, that is a real and useful result — record it and move on to the next tier. #### E3 metadata modes Tap **Run E3 metadata modes**. This is separate from the tier exports and tests both directions for JPEG, PNG and WebP. The harness builds a source file that genuinely carries EXIF — camera make and model, DateTime, DateTimeOriginal, GPS, and Orientation 6 — imports it, exports it, and reads the exported bytes back. - **preserve** — those fields must be present in the export, and Orientation must be written as 1 because the rotation is baked into the pixels. An export must never rely on the viewer honouring an orientation tag. - **remove** — no EXIF marker, no GPS, no camera identity anywhere in the file. WebP preserve is expected to report as unsupported: writing EXIF into WebP needs the file rebuilt as an extended VP8X container, which Phase 0 does not implement. That is reported honestly rather than passed silently. ### E4 — Vault unlock This one prompts you: fingerprint or screen lock, several times. WebAuthn PRF is recorded as usable **only if an actual assertion returns key material** on this device. A capability flag is not evidence, and neither is the `prf.enabled` promise returned at registration — the harness stores that separately and then goes and checks. It also repeats the assertion with the same salt to confirm the output is stable, since a key that changes between unlocks is useless for a vault. Only a short fingerprint of the output is ever recorded, never the key. Argon2id is then timed at four cost profiles. On a mid-range phone the target profile may be slow; that number decides what Rosette can actually ask of users. If PRF returns nothing, that is not a failure of the plan. It means the passphrase path stays primary and PRF is an optional accelerator. #### E4 cancellation and migration Two more buttons, neither of which prompts you: - **Argon2id cancellation** — derives once to measure a baseline, then starts a fresh derivation and cancels it at the measured midpoint. The derivation runs in a worker, so the screen must stay responsive (the harness measures the longest main-thread block) and cancelling must actually stop it. No key may be produced afterwards, and a following derivation must still succeed. - **Argon2id parameter migration** — re-wraps the same vault key from 64 MiB to 128 MiB parameters. Your data must not be re-encrypted, the key bytes must be unchanged, existing ciphertext must still decrypt, and the old wrapping key must stop working. ### E5 — Inbound share handoff This one **cannot be run from a browser tab**. You have to use the real share sheet. 1. Open Google Photos (or any gallery) on the phone. 2. Share a photo → choose **Rosette**. 3. The app opens on the spike page with a handoff id in the URL and **claims automatically during boot** — no button press, no race against the 120 s TTL. The boot claim runs the hostile-client check, posts READY from the bound client, takes the files into page memory whatever the lock state, and scrubs the handoff id from the URL and history immediately. 4. Tick "Vault is unlocked right now" **before** sharing if it actually is. Leaving it unticked is the more useful run: it proves the worker hands the bytes over on READY while the vault is still locked. 5. Choose the **scenario** before sharing. "Run E5 manually" only re-runs the diagnostics; the authoritative evidence is the boot claim, recorded in the report as `claimedAutomaticallyOnBoot: true` with `msFromNavigationToReady`. Run all **seven** scenarios in the dropdown, one share each. The evaluator is scenario-aware: the four normal scenarios require a successful bound-client handover, and the three interruption scenarios require the interruption to have actually happened with the payload unrecoverable afterwards. A correct timeout, reload or termination is therefore recorded as a **pass**, not a failure. | Scenario | How to produce it | Terminal condition for a pass | | ------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | Cold launch | Force-stop Rosette first, then share from Google Photos. | Bound client claims automatically on boot and receives the file. | | App already open | Leave Rosette open in the background, then share. | As above. | | Locked vault | Leave "Vault is unlocked right now" unticked, then share. | Files reach client memory on boot **while still locked**; nothing is written at rest. | | Slow unlock | Share while locked and leave the vault locked. The harness holds for a full 60 s by itself. | An immediate locked handover, then a **delayed terminal check**: after ≥ 60 s the held file count and byte sizes are unchanged, the worker can no longer re-grant the handoff, and the telemetry and at-rest audits still pass. The pending record alone is not a pass. | | Timeout, no claim | **Arm "Interruption scenario" first**, share, then leave the page untouched for more than 2 minutes. | The payload was never claimed, the worker's disposal ledger proves `expired-unclaimed` after the full 120 s TTL, and a later claim cannot recover it. A generic "unknown id" answer is _not_ sufficient. | | Reload mid-handoff | **Arm first**, share, and let the harness reload the page while the handoff is still pending (or pull to refresh within a second or two). | A reload genuinely occurred during the pending handoff for **this staged attempt**, the reloaded client was refused specifically with `client-binding-mismatch`, and zero files were recovered. An unknown, expired or already-granted result is a fail. | | Process terminated mid-handoff | **Arm first**, share, then force-stop Rosette from Android app settings while the pending record is showing, and relaunch it. | Every one of: the same staged attempt and scenario; genuine fresh-context relaunch evidence; a worker census (`workerSweep`) is present; `liveHandoffs === 0`; `grantsIssued === 0`; `recoveredFileCount === 0`; no file held in client memory; clean at-rest and telemetry gates. A missing census is a **fail**, not a pass. | Arming is the **only** way automatic claiming is delayed or suppressed; it is bound to the exact staged attempt rather than a global flag. A stale arm value from another attempt cannot suppress READY. Normal share launches always claim on boot. Untick the arm box after the interruption runs. **The staged marker.** It carries the attempt id, scenario, vault state, stage and the exact commit SHA and deployment ID it was armed on — never the token, files, filenames or hashes. The harness waits for the authoritative build identity before resuming a staged attempt. If the marker was armed on a different build it is discarded with an explicit reason, shown in the Completeness panel, and the scenario must be re-run. Otherwise it is cleared only after reading back a terminal journal record matching that marker's **exact attempt id, scenario and build identity** — an older record for the same scenario can never clear a newer attempt. **Terminal predicate for the four normal scenarios.** A pass requires all of: automatic boot claim; exact bound/requesting client match; at least one file granted and retained in client memory; no wait for vault unlock; handoff URL and history scrubbed; forged-id, hostile-client and replay denial; scenario- appropriate locked-state evidence; telemetry gate passed; clean at-rest audit. **Completeness** counts only exact terminal records that pass. A failed terminal run stays in the journal as evidence and is listed separately in the Completeness panel, but it never makes the suite approval-complete. What is being proven: - the shared bytes sit **only in service worker memory** — never Cache Storage, never IndexedDB, never the network - the handoff id is 256 bits of `crypto.getRandomValues` - the handoff is bound to the exact client the share redirect created - the worker hands the file to the bound client **immediately on READY**, without waiting for the vault to unlock; the client then holds it in page memory behind the unlock screen - a forged handoff id is **denied** - the payload is single-use: replaying the same id after handover fails - the binding **fails closed**: if `resultingClientId` is missing on either side, the claim is denied — there is no fallback path - a genuinely hostile same-origin client (a controlled iframe the harness loads) attempts to claim the live handoff id and must be denied, and the legitimate client must still get its file afterwards - if no client ever collects it, it self-destructs after two minutes - the handoff id is scrubbed out of the address bar and session history with `history.replaceState()` as soon as it is read The harness also sweeps Cache Storage for anything resembling the shared file and lists every IndexedDB database, so "no plaintext at rest" is checked rather than asserted. **Telemetry is a mandatory E5 condition.** The Rosette application code contains no analytics, and the independent VPS evidence host does not inject Lovable's `/~flock.js` or any third-party script. A guard installed before the app mounts still blocks anything analytics-shaped at `fetch`, `XMLHttpRequest` and `sendBeacon`, the service worker blocks it again at the network layer, and a `PerformanceObserver` audits what the document actually transferred. E5 **fails** unless the audit shows `zeroOutboundTelemetry: true` — that is, no telemetry entry transferred bytes and none was indeterminate. Blocked injection attempts are recorded separately in `blockedAttemptCount` and do not by themselves fail the run; an actual transfer to `/~api/analytics` or any other telemetry destination does. Confirm independently on the handset via `chrome://inspect` → Network, on **both** the first uncontrolled visit (before the service worker is installed) and the service-worker-controlled share launch. If zero outbound telemetry cannot be demonstrated on both, the Android evidence is invalid even though the host-level smoke test was clean. ### E6 — Offline cold restart Also a physical-device test, and the easiest one to accidentally fake. A backgrounded app resuming from memory proves nothing. 1. Open the installed app once online. 2. Swipe it away from Recents, then **Settings → Apps → Rosette → Force stop**. This is what makes the restart cold. 3. Turn on **Airplane mode** _and_ switch Wi-Fi off. 4. Relaunch from the home screen icon, timing from the tap. 5. Stop timing when buttons respond. 6. Record whether you saw Rosette's own shell or Chrome's offline page. Enter the verdict, the time, and notes. Without notes the harness records the experiment as `unverified`, whatever the preconditions look like. Then, **while still offline**, tap **Run offline capability**. Booting the shell is not enough: the cached model must actually cut out a subject and a full-resolution export must actually be produced. The harness first proves the network really is off with a cache-busting request rather than trusting `navigator.onLine`, and records every network request made during the run — so a silent online fallback cannot be mistaken for an offline pass. If it detects the device is still online, the run is recorded as `not-run`, not as a pass. ## 5. The diagnostic journal (evidence integrity) Every run is written to a durable, append-only journal on the phone, so nothing is lost or overwritten: - results are stored **per run**, not per experiment id: each matting candidate and backend, each megapixel tier / format / tile size, and each E5 scenario is its own labelled record with its own timestamp; - the journal survives refreshes, real share-target navigations and force-stop / cold restarts, so a single complete report is possible; - the E5 scenario and the vault locked/unlocked selection are saved **before** you leave for the share sheet and retained until that exact attempt has a durable terminal record, so a share launch is labelled with the scenario you actually chose rather than the default; - the journal is bound to one commit SHA and deployment ID. If you open a newer build, the older journal is discarded rather than merged, and the discard is recorded in the report. Missing identifiers are never wildcard matches, and unbound evidence is discarded rather than relabelled when an authoritative identity becomes available. Download anything you still need first; - only diagnostic controls and measurements are persisted. Shared `File`/`Blob` data, pixels, matte previews, object URLs, filenames and content hashes are redacted on the way to storage, and E5's plaintext-at-rest audit inspects the journal itself, so a regression fails the experiment instead of passing quietly; - the same redaction is applied before a run enters the in-memory report store, so downloading during the original session cannot expose a filename or content hash that would only disappear after a reload; - **Clear diagnostic data** deletes the whole journal for the device. ## 6. Producing the report Tap **Download report JSON**. The filename includes the device slug and a timestamp. Send it back with the phone model. The report distinguishes four things carefully, and you should preserve that distinction when you talk about the results: - a **measurement** — a real number from a real API - an **estimate** — for example GPU memory, which browsers do not expose; the figure is a sum of the allocations Rosette itself requested - **unavailable** — with the reason the number could not be obtained - **unverified** — ran, but needs a human judgement before it counts A partial report can be downloaded at any time, including after a crash. It carries a **completeness** section naming every experiment, matting candidate, export tier, format, tile size and E5 scenario still missing, so a partial run can never be mistaken for a full one. ## 7. Ground rules - Never mark an Android-only experiment as a pass from a desktop browser. - Never fill a gap with a figure from a model card or a spec sheet. - If something crashes, keep the report — a crash with a stack trace is a finding, not a lost run. - Reload the page between heavy runs. Thermal throttling and residual memory make back-to-back runs untrustworthy; if the phone is warm, let it cool. ## 8. Cleaning up Open `/dev/spike?sw=off`, or tap **Unregister spike worker**. The spike service worker is a Phase 0 artefact; Phase 1 replaces it with a generated Workbox worker that keeps the same handoff contract.