# How to Implement Web Bot Auth

> A step-by-step guide for agents to sign HTTP requests cryptographically
> using Ed25519 + HTTP Message Signatures.

If you operate an AI agent, **Web Bot Auth** lets you prove your identity to
any server using public-key cryptography — no API keys to distribute, no IP
allowlists to maintain. A single key pair authenticates every request your
agent makes, anywhere.

This guide shows you exactly what to build.

## Why do this?

- **Prove your agent is a real actor.** Servers can cryptographically verify
  your requests came from your infrastructure, not a spoofer.
- **Get trusted without direct registration.** Pair this with a public
  allowlist (like [whitelist.dev](https://whitelist.dev)) and your agent is
  instantly recognized as legitimate by any service that checks it.
- **No shared secrets.** Your private key never leaves your servers. Servers
  only need your public key (hosted at a well-known URL on your domain).
- **Standards-based.** Built on published IETF drafts. Cloudflare and others
  are already rolling out verification at the edge.

## The protocol at a glance

1. You generate an Ed25519 key pair.
2. You host a JWKS directory at `/.well-known/http-message-signatures-directory`
   on your agent's operator domain, containing your public key.
3. For every outgoing request your agent makes, it computes a signature over
   a canonical set of HTTP components (using the private key) and attaches
   three headers: `Signature`, `Signature-Input`, `Signature-Agent`.
4. The server reading your agent's request fetches your public key from the
   URL in `Signature-Agent` and verifies the signature.

That's the whole thing. Below are the exact details.

---

## Step 1 — Generate an Ed25519 key pair

Ed25519 is the only algorithm currently required. The private key stays on
your server; the public key goes in your JWKS directory.

```bash
# Private key (keep this secret)
openssl genpkey -algorithm ed25519 -out private-key.pem

# Public key
openssl pkey -in private-key.pem -pubout -out public-key.pem
```

If your OpenSSL is older than 1.1.1 (e.g. LibreSSL on macOS), use Python:

```python
from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PrivateKey
from cryptography.hazmat.primitives import serialization

pk = Ed25519PrivateKey.generate()
priv_pem = pk.private_bytes(
    serialization.Encoding.PEM,
    serialization.PrivateFormat.PKCS8,
    serialization.NoEncryption(),
)
pub_pem = pk.public_key().public_bytes(
    serialization.Encoding.PEM,
    serialization.PublicFormat.SubjectPublicKeyInfo,
)
open("private-key.pem", "wb").write(priv_pem)
open("public-key.pem", "wb").write(pub_pem)
```

## Step 2 — Extract the raw public key (32 bytes)

JWK format uses the raw 32-byte Ed25519 public key, base64url-encoded (no
padding). Extract it from your public key file:

```python
from cryptography.hazmat.primitives import serialization
import base64

with open("public-key.pem", "rb") as f:
    pub = serialization.load_pem_public_key(f.read())

raw = pub.public_bytes(
    serialization.Encoding.Raw,
    serialization.PublicFormat.Raw,
)
x = base64.urlsafe_b64encode(raw).rstrip(b"=").decode()
print(x)  # e.g. "LTzIiCXUxDVBfudGyQsESBPnKyDQ4c8Ecb9fTYrG3Mg"
```

## Step 3 — Compute your JWK thumbprint

This is a stable identifier for your key, per [RFC 8037]. It's the `keyid`
you'll put in every request.

```python
import hashlib, base64

canonical = f'{{"crv":"Ed25519","kty":"OKP","x":"{x}"}}'
thumbprint = base64.urlsafe_b64encode(
    hashlib.sha256(canonical.encode()).digest()
).rstrip(b"=").decode()
print(thumbprint)  # e.g. "oLEigX9jWBZAQlSy3vAr5mT-McHCgEOPVzfC5Ol1oiY"
```

The canonical form **must** have keys sorted as `crv`, `kty`, `x` with no
whitespace — this is the RFC 7638 canonicalization rule for Ed25519 keys.

## Step 4 — Host your JWKS directory

Serve a JSON response at exactly this path on your agent's operator domain:

```
https://your-domain.com/.well-known/http-message-signatures-directory
```

The body is a JWK Set containing your public key(s):

```json
{
  "keys": [{
    "kty": "OKP",
    "crv": "Ed25519",
    "x": "LTzIiCXUxDVBfudGyQsESBPnKyDQ4c8Ecb9fTYrG3Mg"
  }]
}
```

### Sign the directory response itself

To prove you control the domain, the directory **response** must be signed
using the same key listed in it. This prevents anyone from mirroring your
public key and claiming to be you.

Required response headers:
- `Content-Type: application/http-message-signatures-directory+json`
- `Signature: sig1=:<base64 signature>:`
- `Signature-Input: sig1=("@authority";req);alg="ed25519";keyid="<thumbprint>";nonce="<random>";tag="http-message-signatures-directory";created=<unix_ts>;expires=<unix_ts>`

Note: `"@authority";req` uses the `req` parameter because this is a response
signature that references the request's `Host` header.

The signature base for the directory response is:

```
"@authority";req: your-domain.com
"@signature-params": ("@authority";req);alg="ed25519";keyid="...";nonce="...";tag="http-message-signatures-directory";created=...;expires=...
```

Sign those exact bytes with your private key, base64-encode (standard base64,
not base64url) the 64-byte output, and wrap in `:...:`.

## Step 5 — Sign outgoing requests

For each request your agent makes, compute a signature over `@authority` and
`signature-agent`. This binds the signature to both the server the agent is
hitting and the URL of your key directory (preventing replay to a different
server).

### The three headers you send

```
Signature-Agent: "https://your-domain.com"

Signature-Input: sig1=("@authority" "signature-agent");
  created=1735689600;
  keyid="<your-thumbprint>";
  alg="ed25519";
  expires=1735693200;
  nonce="<random base64, ~48 bytes>";
  tag="web-bot-auth"

Signature: sig1=:<base64 signature>:
```

### The signature base

Exactly these bytes, newline-separated, with no trailing newline:

```
"@authority": <target host>
"signature-agent": "<your directory URL, including quotes>"
"@signature-params": <exact contents of Signature-Input after "sig1=">
```

Example for a request to `https://api.example.com/resource`:

```
"@authority": api.example.com
"signature-agent": "https://your-domain.com"
"@signature-params": ("@authority" "signature-agent");created=1735689600;keyid="abc";alg="ed25519";expires=1735693200;nonce="xyz";tag="web-bot-auth"
```

### Important rules

- **Short `expires`** — use 5 minutes or less. This limits replay attack
  windows. A fresh signature per request is cheap.
- **Always include `nonce`** — a random 32–48 byte base64 string. Not yet
  validated by verifiers but forward-compatible.
- **`Signature-Agent` must be in the component list.** Servers that don't see
  it in the signed components will reject the request.
- **Use only ASCII values** in signed components. The spec disallows
  non-ASCII (no `bs`/`sf` serialization).
- **Don't sign `Content-Digest`** unless the request body is encrypted
  end-to-end — any proxy that modifies the body will invalidate the signature.

## Step 6 — Make the request

Full example in Python:

```python
import base64, hashlib, os, time, requests
from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PrivateKey
from cryptography.hazmat.primitives import serialization

# Load your private key
with open("private-key.pem", "rb") as f:
    pk = serialization.load_pem_private_key(f.read(), password=None)

# Compute thumbprint (do this once, cache it)
raw = pk.public_key().public_bytes(
    serialization.Encoding.Raw, serialization.PublicFormat.Raw
)
x = base64.urlsafe_b64encode(raw).rstrip(b"=").decode()
canonical = f'{{"crv":"Ed25519","kty":"OKP","x":"{x}"}}'
keyid = base64.urlsafe_b64encode(
    hashlib.sha256(canonical.encode()).digest()
).rstrip(b"=").decode()

# Build signature for this specific request
target_url = "https://whitelist.dev/verify"
directory_url = "https://your-domain.com"
authority = "whitelist.dev"  # from target URL

now = int(time.time())
expires = now + 300
nonce = base64.b64encode(os.urandom(48)).decode()

params = (
    f'("@authority" "signature-agent");'
    f'created={now};keyid="{keyid}";alg="ed25519";'
    f'expires={expires};nonce="{nonce}";tag="web-bot-auth"'
)

signature_base = (
    f'"@authority": {authority}\n'
    f'"signature-agent": "{directory_url}"\n'
    f'"@signature-params": {params}'
)

sig_b64 = base64.b64encode(pk.sign(signature_base.encode())).decode()

# Send it
resp = requests.get(target_url, headers={
    "Signature-Agent": f'"{directory_url}"',
    "Signature-Input": f"sig1={params}",
    "Signature": f"sig1=:{sig_b64}:",
    "User-Agent": "YourAgent/1.0 (+https://your-domain.com/agent)",
})
print(resp.status_code, resp.json())
```

## Step 7 — Test it

Send a signed request to [whitelist.dev/verify](https://whitelist.dev/verify).
The response tells you exactly what the server saw and whether your signature
verified:

```json
{
  "signature": {
    "status": "VALID",
    "keyid": "oLEigX9jWBZAQlSy3vAr5mT-McHCgEOPVzfC5Ol1oiY"
  },
  "whitelist": {
    "domain": "your-domain.com",
    "trusted": true
  }
}
```

Possible `status` values:

| Status                   | Meaning                                         |
|--------------------------|-------------------------------------------------|
| `VALID`                  | Signature verified successfully                 |
| `INVALID`                | Signature did not verify                        |
| `EXPIRED`                | `expires` timestamp is in the past              |
| `MISSING_HEADERS`        | One of the three required headers wasn't sent   |
| `KEY_NOT_FOUND`          | `keyid` doesn't match any key in your directory |
| `DIRECTORY_FETCH_FAILED` | Couldn't fetch your JWKS endpoint               |
| `ERROR`                  | Other error (check `detail`)                    |

## Common pitfalls

**Signature base has a trailing newline.** It shouldn't. Every line is joined
with `\n`, but no newline after the last line.

**You used `Signature: abc` instead of `Signature: sig1=:abc:`.** The RFC 9421
format requires the label and the colons.

**Your directory response isn't signed.** Servers verifying your bot will
fetch your directory, and they may reject you if the directory isn't self-signed.

**The `x` value has padding.** JWK requires base64url with no padding. Strip `=`.

**The canonical JSON for your thumbprint isn't byte-exact.** `{"crv":"Ed25519","kty":"OKP","x":"..."}` — no spaces, keys in that exact order.

**You sign the wrong authority.** `@authority` must be the `Host` of the
server you're sending *to*, not your own domain.

## Reference implementations

- **Rust:** [web-bot-auth](https://crates.io/crates/web-bot-auth) (Cloudflare)
- **TypeScript / npm:** [web-bot-auth](https://www.npmjs.com/package/web-bot-auth) (Cloudflare)
- **Python (full worked example):** [github.com/mmmglm/web-bot-auth](https://github.com/mmmglm/web-bot-auth)

## Specifications

- [RFC 9421 — HTTP Message Signatures](https://datatracker.ietf.org/doc/html/rfc9421) (signature format, components, canonicalization)
- [RFC 8037 — JWK Thumbprint for OKP Keys](https://www.rfc-editor.org/rfc/rfc8037.html#appendix-A.3) (thumbprint algorithm)
- [RFC 7638 — JSON Web Key (JWK) Thumbprint](https://www.rfc-editor.org/rfc/rfc7638) (canonicalization rules)
- [draft-meunier-web-bot-auth-architecture](https://datatracker.ietf.org/doc/html/draft-meunier-web-bot-auth-architecture) (the Web Bot Auth protocol)
- [draft-meunier-http-message-signatures-directory](https://datatracker.ietf.org/doc/html/draft-meunier-http-message-signatures-directory) (key directory format)
- [Cloudflare blog: Message Signatures and Verified Bots](https://blog.cloudflare.com/verified-bots-with-cryptography/) (context and rationale)

## Questions

File an issue at [github.com/mmmglm/web-bot-auth](https://github.com/mmmglm/web-bot-auth)
or reach out to [@grantm](https://x.com/grantm).
