The asset & upload pipeline

Stateless edges, stateful core

Benchy's asset pipeline runs on a small set of Cloudflare Workers (the benchy-edge repo) built around one principle: the edge is deliberately dumb. Workers verify signatures and move bytes; they hold no user state and make no authorization decisions. The application database remains the single authority on who owns what, who bought what, and which policy decisions were recorded. Every security review gets shorter when the edge can't make decisions.

The mesh pipeline — grant, PUT, promote, queue, validate, apply — lives in two Workers, and the platform underneath (Cloudflare Queues and R2) carries the operational weight rather than bespoke infrastructure.

The workers

Deployed Worker names carry a benchy- prefix (benchy-upload-gateway and so on); the names above are the directories in the repo. A shared edge-gateway package supplies the common primitives: HMAC-SHA256 signed-token verification, R2 key safety, and HTTP Range parsing.

Where files live

The workers move bytes; Cloudflare R2 buckets hold them. Clear storage boundaries make it easier to understand which files are public, which are entitlement-gated, and which are still being checked.

Data location

Benchy production buckets are configured with Cloudflare R2's location hint Eastern North America (ENAM)— benchy-models-prod, benchy-products-prod, benchy-media-prod, benchy-ingest-prod, and their matching -dev buckets. Application metadata runs on Convex; edge delivery and ingest run on Cloudflare Workers.

Stage A — upload

Clients never write to public storage. An upload starts with an HMAC-signed grant — audience-scoped, short-lived, bound to one pending/v1/… key in the ingest bucket with an explicit byte ceiling. The gateway accepts a single PUT up to 64 MiB, or multipart in 32 MiB parts (create → upload parts → complete, with abort support) up to 512 parts. When the application finalizes the upload, a separate promote credential — a different audience entirely — copies the object from ingest to its destination bucket (models, products, or media), computing the SHA-256 in-stream.

Stage B — validate

model created / new version published
  → fast content preflight (initial feedback only)
  → enqueue { modelId, modelKey, sha256, policyVersion,
              validationProfile, executionMode, unitAssumption, idempotencyKey }
  → Cloudflare Queue "benchy-mesh-validation"
  → worker: fetch bytes from R2
      ≤ 4 MiB    → benchy-mesh compiled to WASM, in-Worker
      > 4 MiB    → FILE_TOO_LARGE
  → report → R2: derived/v2/mesh-reports/<sha256>/<policy>/<profile>/<mode>/<unit>.json
  → POST /mesh/apply → model.meshValidation updated

Details that matter:

• Preflight uses client_advisory mode for fast feedback. It cannot grant eligibility or overwrite a completed server policy decision.
• Jobs are idempotent by key — validation:<modelId>:<sha256>:<policy>:<profile>:<mode>:<unit> — so retries and duplicate enqueues are no-ops without conflating a changed unit context.
• Files up to 4 MiB validate in-Worker via benchy-mesh-wasm. Above that the job returns FILE_TOO_LARGE and the object stays stored and owner-accessible.
• Internal hops (enqueue, apply) are authenticated with shared secrets, and the apply callback is allowlisted to a fixed set of Convex deployment hosts over HTTPS — the worker can't be pointed at arbitrary URLs.

Stage C — apply & gate

The worker writes the full report to R2 under derived/v2/mesh-reports/<sha256>/<policy>/marketplace_publish/server_enforced/<unit>.json, then posts a mapped summary — lifecycle, validation profile, execution mode, decision, check coverage, issue codes, observations, and print-readiness assessment — to the application. The application applies it to the model and enforces the gates: a rejection blocks publishing publicly and blocks commerce, and nothing else. File checks passed appears only when the complete current v2 evidence is bound to the model's exact bytes, format, and unit context. An engine name alone never grants eligibility.

The downloads gateway

Paid downloads on benchy.shop close the loop with the same philosophy. The application checks the buyer's entitlement and signs a 10-minute HMAC claim bound to the exact file; the gateway at downloads.benchy.shop verifies the signature and streams the object from the private bucket with private, no-store caching and Range/206 support. The worker calls no database at all. Authority stays in the core; the edge checks math.

The validation engine these workers embed is fully open — benchy-mesh — and the reasoning behind the architecture is on the blog in Building Benchy.