Checks that report, not block

A generated shopping list loads, accepts items, and works on a phone. Its design audit finds that a secondary button is 40 pixels high instead of 44. If the publishing tool refuses to proceed, the assistant may enter another edit, check, and retry cycle. The user still has no link. A useful app is waiting behind a minor finding.

The opposite policy is also bad. Publishing a blank page because every check is advisory breaks trust at once.

The practical answer is a narrow hard-error gate. Stop only when there is direct evidence that the initial app cannot run or would expose a serious secret. Return all other findings as issues beside the result. This classification belongs inside the ship path described in One call to ship.

Policy in one view

hard error -> do not publish, return exact repair evidence
issue      -> publish, return the finding with the live link
clean      -> publish, return the live link

Checks should inform the agent on every path. A gate that throws away its report is worse than no gate.

Terms

Hard error is a deterministic finding that means the initial page is invalid, unsafe to publish, blank, or crashing.

Issue is a flow, layout, accessibility, design, copy, network, or provider finding that may matter but does not prove the app is unusable.

Initial experience means the first page load before optional user actions.

Finding level is the classification attached to one result, not the severity of its wording.

Launch mode is a stricter product state with explicit public promises. It may promote some issues to errors.

Fail open means continue to publication when an optional check is unavailable.

Fail closed means stop because required evidence could not be obtained.

What should be hard

The hard set should be short and easy to explain:

• The HTML cannot be parsed or lacks a usable entry document.
• A JavaScript file has a syntax error.
• A bare import is unmapped, or a required import cannot be resolved.
• Static files contain a recognizable secret or private key.
• The initial request returns a non-success status.
• The first paint has no meaningful visible content.
• The page throws an uncaught exception during initial load.

These findings point to a broken or dangerous artifact. Publishing is unlikely to help the user inspect it, and the agent usually has exact evidence to fix it.

Even here, avoid broad proxies. A console error is not automatically a hard error. Some libraries log recoverable messages. One failed analytics request should not suppress an otherwise working calculator. A runtime classifier should distinguish uncaught execution failure from noisy diagnostics.

What should stay advisory

Advisory findings include:

• An acceptance flow failed after the initial page loaded.
• A multiplayer guest did not observe a change within the allotted wait.
• Elements overlap at one audit width.
• A touch target is undersized.
• Focus styling is weak or missing.
• Text contrast misses a threshold.
• Copy lint finds filler, title case, or placeholder text.
• A network request fails but the main app remains usable.
• A screenshot or design audit is unavailable because the worker pool is busy.
• The hosting provider accepted the upload but its alias has not propagated.

Some of these are serious. Advisory does not mean unimportant. It means the evidence should not force an automated refusal in the normal app-building mode.

Returning issues beside a live URL changes the repair loop. The assistant can tell the user the app is available, fix high-value issues in the same turn when budget permits, and stop before cosmetic work consumes the whole run.

Why broad gates fail in agent workflows

A human release engineer can interpret a failed quality gate, change scope, approve an exception, or defer work. A model often reacts to every red item by editing until the tool turns green. If several checks cover subjective quality, this produces long loops and unrelated regressions.

Rigid prerequisites cause similar trouble. Requiring a prototype audit before a final audit sounds orderly, but it blocks an app whose final UI already exists. Requiring a fresh final report after every byte change makes a typo fix invalidate a full visual process. The workflow begins serving the gate.

The model also pays for each refusal. A tool error triggers another reasoning round. A structured result can carry the same evidence without implying that the whole operation failed.

A classifier should be plain code

Do not ask a model to decide whether its own output may publish. Use deterministic findings and a small policy function:

function classify(findings: Finding[]) {
  const hardErrors = findings.filter((f) =>
    ["secret", "syntax", "blank", "http-status", "uncaught"].includes(f.kind)
  )
  const issues = findings.filter((f) => !hardErrors.includes(f))
  return { hardErrors, issues }
}

Real code should avoid the repeated scan, but the important point is the allowlist. New check kinds default to issues until someone deliberately promotes them. Otherwise adding a linter can accidentally become a release gate.

Store the original check report, classifier version, and resulting levels. Policy changes should be auditable.

Launch mode changes the promise

Casual publishing and public launch are different states. A prototype can ship with a missing description or a layout warning. A launch that claims installability, privacy terms, and search readiness must meet those promises.

Launch mode may promote missing privacy content, malformed manifests, broken canonical URLs, invalid structured data, or undeclared analytics to errors. That stricter policy is justified because the system is generating public claims and durable derived files.

Do not use launch strictness for every app. Users making a weekend scorekeeper should not complete a product-release checklist.

Decisions and alternatives rejected

Allowlist hard errors. A severity threshold sounds flexible, but different checkers use severity inconsistently. A known set of blocking kinds is easier to review.

Return findings instead of throwing. Tool exceptions imply protocol or infrastructure failure. A failed flow is valid check output.

Publish through design warnings. Visual judgment contains false positives, and screenshots may not cover the user's actual path.

Keep initial crashes blocking. A live link to a blank or immediately crashing page is not a useful preview.

Store full evidence, show a compact list. The model needs the top repair facts. Humans may inspect screenshots and full reports elsewhere.

Cap repeated polishing. More than a few ship attempts in one turn usually signals poor classification or unclear stopping guidance.

Common failures

Every warning becomes a blocker. The first live link takes several turns or never arrives.

No finding blocks. Syntax errors and blank pages go public.

Unknown checks default to errors. Adding a new lint rule silently changes release policy.

Unavailable optional checks stop the run. A busy browser pool becomes a product outage.

Issues lack location or evidence. The model guesses and changes unrelated code.

The result returns dozens of near-duplicates. Deduplicate by kind and target, rank them, and cap the model-facing list.

A publish provider failure is called a design issue. Keep check quality and delivery status separate.

Prototype policy leaks into launch mode. Public privacy or indexing promises become false.

Field checklist

• Is the blocking set an explicit allowlist?
• Do unknown finding kinds default to advisory?
• Are syntax, secret, blank-page, status, and uncaught-load failures blocked?
• Are flow, layout, design, copy, and optional-check failures returned as issues?
• Does the ship operation return findings without throwing?
• Does every finding include a kind, location or state, and repair evidence?
• Are findings deduplicated and capped for the model?
• Is the full report stored with a classifier version?
• Can launch mode promote promise-related findings?
• Does guidance tell the agent when to stop polishing?
• Are ship attempts and issue counts measured?
• Can users still request a stricter dry run before publication?

This policy depends on trustworthy runtime evidence. Static checks cannot prove that a button works or that two clients synchronize. Browser tests for generated apps covers bounded browser tests for code that did not exist a minute ago.