Designing a reliable email pipeline
Send from a job, not from a request
The single decision that determines whether the integration is reliable. A send inside a web request couples the user’s experience to ours: our latency becomes theirs, our outage becomes their 500, and a retry means they press the button again.
request -> write the intent to your own store -> return
|
background job -> POST /v1/send -> record the outcome
The intent row is the important half. Without it, a crash between “user clicked” and “we sent” leaves nothing to resume from, and no way to tell whether the message went.
What the API already gives you
Worth knowing so you do not build it twice:
| Idempotency | required on both send endpoints, 24h window, byte-exact replay |
| Submission retries | a 4xx or a pre-DATA drop is retried five times at 1m/5m/15m/1h/4h |
| Terminal states | delivered, bounced, deferred with a reason that says it gave up |
| Delivery events | webhooks for each outcome |
So you do not need your own SMTP retry logic, and you should not add a retry
layer on top of the one that exists: a message in queued is being worked on,
and sending it again makes two.
Idempotency keys that work
The key must identify the message, not the attempt. Derive it from something stable in your own domain:
Idempotency-Key: order-1234-shipped-v1
Not a UUID generated at send time, which changes on retry and defeats the point. Not the order id alone, which collides when you later send a second message about the same order.
If the body may legitimately change between attempts, the key must change with it, because a replay returns the first response and silently ignores the new body.
Graceful degradation
Decide in advance what each failure should do, because the default is usually wrong:
| Failure | Reasonable default |
|---|---|
429 rate cap |
pause the worker, resume after Retry-After |
429 allowance |
stop, alert a human, do not drain the queue into failures |
5xx or timeout |
retry the same key with backoff |
400 |
dead-letter it; the same body fails forever |
403 sender |
stop everything; it is configuration, not traffic |
The two worth thinking about hardest are the last two, because both are permanent and a generic retry loop treats them as transient and burns a day discovering it.
Non-critical mail should fail quietly. A marketing send that cannot go out is not a reason to page anyone. A password reset that cannot go out is. Separate those queues, or at least separate their alerting, or the noise from the first will hide the second.
Keep the pipeline observable from both ends
Store our message_id against your row the moment you get it. Without it,
the delivery webhook that arrives twenty minutes later cannot be matched to
anything, and the event log is a list of messages you cannot tie to orders.
See Observability.