Ai Pref

AIPREF (RFC draft) signal — operators publish a machine-readable preference about AI training / agent crawling / etc.

Wires three coordinating surfaces into one primitive: the IETF AIPREF Content-Usage HTTP response header (draft-ietf-aipref-attach-04, deadline 2026-08), the matching robots.txt grammar, and Cloudflare's Content Signals Policy + Pay-Per-Crawl (HTTP 402). Operators declare train / infer / snippet preferences once; the middleware emits both the Content-Usage header and Cloudflare's CF-Content-Signals alongside.

Inbound parsing closes the loop when the framework plays the role of crawler — parseHeader decodes a peer's preferences so the caller can refuse training / pay the per-crawl price / respect a snippet=deny.

b.aiPref.serializeHeader(opts) #

{
  train:    "allow" | "deny" | "paid",   // default "deny"
  infer:    "allow" | "deny" | "paid",   // default "allow"
  snippet:  "allow" | "deny",            // default "allow"
  price:    { amountUsd: number, perTokens?: number },
}

Render the AIPREF Content-Usage HTTP response header value from an operator preference object. Output is an RFC 8941 structured- fields list of train=..., infer=..., snippet=... pairs, plus price-usd / per-tokens when any axis is paid. Throws when the preferences are inconsistent (e.g. train=paid with no price).

var v = b.aiPref.serializeHeader({
  train:   "deny",
  infer:   "allow",
  snippet: "allow",
});
// → "train=deny, infer=allow, snippet=allow"

var paid = b.aiPref.serializeHeader({
  train: "paid", infer: "paid", snippet: "allow",
  price: { amountUsd: 0.001, perTokens: 1000 },
});
// → "train=paid, infer=paid, snippet=allow, price-usd=0.001000, per-tokens=1000"

b.aiPref.parseHeader(value) #

Parse an inbound Content-Usage header value into the typed preference shape. Used when the framework acts as a crawler and must respect a publisher's declared preferences. Unknown axes are dropped silently so a forward-compatible publisher can advertise future fields without breaking older clients. Throws when the value is missing or exceeds the 1024-char defensive cap.

var p = b.aiPref.parseHeader(
  "train=deny, infer=allow, snippet=allow"
);
p.train;     // → "deny"
p.infer;     // → "allow"
p.snippet;   // → "allow"

var paid = b.aiPref.parseHeader(
  "train=paid, infer=allow, snippet=allow, price-usd=0.001000, per-tokens=1000"
);
paid.price.amountUsd;   // → 0.001
paid.price.perTokens;   // → 1000

b.aiPref.robotsBlock(opts) #

{
  train:     "allow" | "deny" | "paid",
  infer:     "allow" | "deny" | "paid",
  snippet:   "allow" | "deny",
  price:     { amountUsd: number, perTokens?: number },
  userAgent: string,                     // default "*"
}

Render an AIPREF §3 robots.txt block: a User-agent: line followed by a Content-Usage: line carrying the same grammar as the HTTP header. Authors who serve robots.txt as a static file paste the output verbatim. The userAgent opt defaults to the catch-all *; pass "GPTBot" / "ClaudeBot" / etc. for per-crawler rules. UA strings are capped at 256 chars.

var block = b.aiPref.robotsBlock({
  userAgent: "GPTBot",
  train:     "deny",
  infer:     "allow",
  snippet:   "allow",
});
// → "User-agent: GPTBot\nContent-Usage: train=deny, infer=allow, snippet=allow\n"

b.aiPref.middleware(opts) #

{
  train:             "allow" | "deny" | "paid",
  infer:             "allow" | "deny" | "paid",
  snippet:           "allow" | "deny",
  price:             { amountUsd: number, perTokens?: number },
  cloudflareSignals: boolean,            // default true
}

Build an HTTP middleware that emits Content-Usage (and, by default, the Cloudflare CF-Content-Signals mirror) on every response. Wires the operator's AI-training / inference / snippet preferences into the request lifecycle so every page advertises the same posture without per-route plumbing.

var aiPrefMw = b.aiPref.middleware({
  train:   "deny",
  infer:   "allow",
  snippet: "allow",
});
// mount aiPrefMw on every public route — emits Content-Usage +
// CF-Content-Signals headers on each response.

b.aiPref.refusePaidCrawl(req, res, opts) #

{
  price:    { amountUsd: number, perTokens?: number },
  contact:  string,                      // optional pricing contact
}

Emit HTTP 402 Payment Required with the price manifest in the Cloudflare-compatible JSON body. Operator route handlers detect an unmonetized AI crawler (via UA / signed-token absence / etc.) and call this helper to surface the price + contact channel uniformly. Audits the refusal under aipref.paid_crawl_refused.

function handler(req, res) {
  b.aiPref.refusePaidCrawl(req, res, {
    price:   { amountUsd: 0.005, perTokens: 1000 },
    contact: "https://example.test/ai-licensing",
  });
}
// → res.statusCode === 402; body is JSON { error: "payment_required", ... }

Last updated 2026-08-08T16:39:49.652Z by seeder.