Skip to content
Zarif Automates

Build a Webhook Receiver: Verify Deliveries and Recover After Failure

ZarifZarif
|Published

An API call starts with your application asking for something. A webhook reverses that direction: another application sends an HTTP request when an event occurs. A new support ticket can trigger a triage workflow without your service repeatedly asking whether anything changed.

The delivery is a notification, not a guarantee that the work ran once. A sender can retry after losing your response. Your receiver can acknowledge an event and then stop before processing it. Build those cases into the example before adding a model.

Download the Python receiver and delivery fixtures. It starts a temporary HTTP server on your machine, sends eight deliveries, and checks a SQLite inbox after an injected worker failure. No model, account, public endpoint or tunnel is needed.

1. Run the complete example

Use Python 3.10 or newer. Save the file in a new folder, then run:

python3 webhook_lesson.py

Use python instead if that is your Python 3 command. There are no packages to install. The script binds only to 127.0.0.1, chooses an available port, and creates a temporary database. It removes that database and stops the server after the checks finish.

The recorded September 17, 2026 run completed eight HTTP cases and left one completed local work record. It also confirmed that an injected exception rolled back an incomplete transaction and that a new database connection could process the pending event. This is an exception-and-reopen test, not a process-kill or power-loss test.

2. Inspect the local delivery protocol

This lesson uses an explicitly local protocol. Its headers are X-Lab-Timestamp and X-Lab-Signature; it is not an implementation of GitHub, Stripe or another provider's webhook format.

The sender posts these exact bytes to /webhook:

{"event_id":"evt-demo-1","type":"ticket.created","ticket_id":"DEMO-42"}

The timestamp is Unix time in seconds. The signature covers its ASCII representation, a period, and the original body bytes:

def signature(secret: bytes, timestamp: str, body: bytes) -> str:
    return "sha256=" + hmac.new(
        secret,
        timestamp.encode("ascii") + b"." + body,
        sha256,
    ).hexdigest()

A new random secret is generated for each test run and shared only by the local sender and receiver. The receiver rejects timestamps more than 300 seconds away from its current time, then compares signatures with hmac.compare_digest. Python's HMAC reference documents that comparison function for signature verification.

Only after those checks does the receiver parse JSON. It accepts exactly the three displayed fields, the event type ticket.created, a nonempty event ID of at most 80 characters, and the synthetic ticket DEMO-42. The HTTP handler also limits the body to 16 KiB. These constraints make the example inspectable; they are not a general support-system schema.

Changing whitespace changes the signed bytes. One fixture appends a space to a valid JSON body while keeping the original signature. The receiver returns 401 invalid_signature, even though a JSON parser would consider the data equivalent.

3. Store the receipt before acknowledging it

The receiver opens a SQLite transaction with BEGIN IMMEDIATE and looks up event_id in an inbox table. A unique primary key prevents a second receipt for that ID. It records a hash of the original body and a pending state, commits, and only then returns:

{
  "event_id": "evt-demo-1",
  "state": "pending",
  "duplicate": false
}

The HTTP status is 202: the event is stored for processing. It does not mean a model has classified the ticket or a record has changed.

An identical delivery returns 200 with the existing state and duplicate: true. Reusing the same event ID with different bytes returns 409 event_id_conflict. That conflict is worth inspecting; silently substituting a different payload under an existing identifier would make recovery ambiguous.

For this protocol, identical means byte-for-byte identical. A provider may resend equivalent data in a different representation, so choose its identity and conflict rules deliberately when adapting the receiver.

4. Follow a failed worker transaction

The worker selects one pending event and inserts a ready_for_triage result into a second table. Both that insert and the inbox transition to completed belong to the same SQLite transaction:

db.execute("INSERT INTO work VALUES (?, 'ready_for_triage')", row)
if fail_before_commit:
    raise RuntimeError("injected failure before commit")
db.execute("UPDATE inbox SET state='completed' WHERE event_id=?", row)

The fixture injects the exception between those statements. The transaction rolls back. A newly opened connection sees zero work rows and a pending receipt. Running the worker again without the injected failure inserts one result and marks the receipt complete. A third worker call finds nothing pending.

This works because both changes are in the same local database transaction. It does not make a remote API call atomic. If an external system accepts a write before your worker loses its connection, use that system's idempotency or operation-status mechanism, or hold the uncertain result for reconciliation. A retry is not proof that the first attempt failed.

5. Read the failure table

These are the eight recorded HTTP outcomes. The worker rollback check occurs between the seventh and eighth deliveries.

DeliveryHTTP resultMeaning
First valid event202Receipt committed as pending
Identical event before processing200Existing pending receipt returned
Changed bytes, old signature401Signature mismatch
Signature made with a different secret401Signature mismatch
Correctly signed timestamp 301 seconds old401Outside the local freshness window
Correctly signed invalid JSON400Authentication does not establish valid data
Same ID, changed bytes, new valid signature409Event identity conflict
Identical event after worker recovery200Existing completed receipt returned

The output ends with:

{
  "http_passed": 8,
  "reopened_after_rollback": true,
  "work_rows": 1
}

There are no AI accuracy measurements here. The single work row is a local marker that the event is ready for triage, not a prediction or customer-facing action.

6. Change a fixture, then adapt one provider

Open the fixtures() function and find the send("stale-delivery", ...) call. Change its timestamp from 301 seconds old to 30 seconds old, and its expected status from 401 to 200. The send() helper recomputes the signature using the supplied timestamp and body; if you edit a saved HTTP request instead, you must re-sign it yourself. It carries the same event ID and body as the accepted event, so the correct result is a duplicate pending receipt, not a second work record. This shows why freshness checks and duplicate handling solve different problems.

Then find send("signed-invalid-json", ...) and replace raw=b"not json" with valid JSON containing an extra field. The helper signs the new body automatically; keep its expected status at 400. A valid signature confirms integrity under the shared secret; the application still decides which events it accepts.

For a real sender, replace the local header and signature logic with that provider's documented verification method. For example, GitHub signs the payload body and sends X-Hub-Signature-256. Do not add this lesson's timestamp format to a GitHub verifier and expect its signatures to match. Check the provider's delivery identity, retry policy and response deadline separately.

Python's development HTTP server is sufficient for this loopback exercise. A deployed receiver needs an appropriate server, HTTPS, request limits, monitored storage and a worker recovery policy. Keep receipt handling quick; run slow classification after durable acceptance.

Your deliverable is the eight-case output, the two changed fixtures, and a short trace showing acceptance, rollback and successful retry. Continue to the prompt-contract lesson to define the model's task after an event has reached that boundary.

Zarif

Zarif

Zarif builds AI agents and automation workflows and writes about what holds up in production: the sources worth following, the roles the AI era is creating, and agent workflows you can inspect end to end.