envoymail
envoymail: Inbound and Outbound Mail Backbone with a Human Approval Gate
I run more than one system that needs to read and send mail: my own inbox, an AI assistant I built called Sofie, and a separate app called MentorSeed. None of them should hold their own mailbox credentials or their own sending logic. I built envoymail as a single backbone for all three: it receives inbound mail without running an SMTP server, holds outbound mail in a human-approval queue before it leaves, and sends through a pool of providers that fails over when one is capped. It is in production on my own infrastructure, and it is the one place I enforce no mail leaves without a human, once, instead of three times.
The problem
I run several systems that all need mail: a personal inbox, an AI assistant, and a mentoring app. The naive path is for each one to hold its own SMTP credentials and write its own inbox logic. That duplicates secrets across systems, means a provider outage has to be handled three separate times, and gives an AI agent a direct line to send mail with no human in the loop. Off-the-shelf transactional mail SDKs solve sending, not receiving, and they solve neither the multi-provider failover nor the split between an agent drafting and a human approving. Running an actual SMTP server is its own operational burden I did not want on a single Raspberry Pi. What I needed was one backbone: receive inbound mail without owning an SMTP server, hold outbound mail in an approval queue before it leaves, and keep sending when a single provider goes down or hits its cap.
Constraints
- ·One person building and operating it, so every rule had to be enforceable by a test, not by review.
- ·Deployed on a single Raspberry Pi behind a Cloudflare Tunnel, not a cloud fleet, not a managed database.
- ·Three unrelated consumers sharing one service: personal use, an AI assistant, and a separate app, with no consumer allowed logic the others cannot reach.
- ·No mail may leave without an explicit human approval call, because at least one consumer is an autonomous AI agent.
- ·Provider sending quotas are real caps, daily and monthly, enforced by the provider itself.
- ·SQLite is the only datastore, on one disk, so backup and recovery had to be designed in, not assumed.
Architecture
A Node.js and Express service. Inbound mail arrives through Cloudflare Email Routing and a Worker that forwards raw MIME to a POST /inbound webhook, which parses it and threads it on the root of the References header before storing it in SQLite. Outbound mail goes through POST /outbox (queued for human approval) or POST /send (direct), then through the core send engine, Envoy.mail(), which checks a persisted quota store and walks a pool of self-registering provider plugins until one accepts the message. Connected Gmail and IMAP accounts sit behind one MailClient port and are read fetch-on-demand, with no background sync. Three consumers, a web UI, an AI assistant called Sofie, and a separate app called MentorSeed, all call the same HTTP surface with no consumer holding private logic.
- ›API surface (Express): /accounts /inbox /threads /outbox /send /providers, the only place any capability is exposed
- ›Inbound edge: Cloudflare Worker (pure relay, raw MIME) into POST /inbound, its own fail-closed secret, auto-creates an inbound-only account for an unregistered recipient
- ›Threading: groups on the root of the References header, not the immediate parent, so reply chains do not split past message two
- ›Human-approval outbox: POST /outbox drafts, POST /outbox/:id/approve sends, with no approval policy inside this service (that lives in the caller)
- ›Core send engine: Envoy.mail(), the single entry point for anything leaving the system, owns the provider pool walk, quota pre-check, exhaustion marking, per-identity pinning, and a deliberately narrow retry
- ›Transporter registry: a port plus a Map<PROVIDER, factory> kernel, providers self-register, the engine never names a concrete one (Brevo is the primary, others pool alongside it)
- ›Quota: persisted per-provider daily and monthly counters, rollover computed on read by comparing stored date keys, survives a redeploy
- ›Accounts: MailClient port implemented by GmailClient and ImapClient (BYOK), credentials AES-256-GCM encrypted at rest, fetch-on-demand only, no background sync
- ›Storage: one SQLite file in WAL mode, one typed repository per table, no raw SQL in route handlers
How one request flows
- 01A consumer calls the API. The web UI, Sofie, or MentorSeed calls POST /outbox to queue a message, or POST /send to skip the queue for an already-approved case. Every call carries the shared internal API key, a missing key is rejected, never silently allowed through.
- 02A drafted message waits for a human. Anything sent through /outbox is stored with status pending_approval until a human calls POST /outbox/:id/approve. envoymail runs no policy check of its own on that call, the decision belongs to the caller.
- 03Approval hands off to the core engine. Approval, or a direct /send call, reaches Envoy.mail(), the single entry point for anything leaving the system.
- 04Quota is checked before a provider is tried. Quota is read from a persisted store, not an in-memory counter, so a redeploy mid-cycle cannot make an already-capped provider look available again.
- 05A pinned identity is tried first. If the from-address is pinned to a specific provider, that one goes first. An unpinned or unavailable pin falls back to the full pool rather than failing, because delivery matters more than routing precision.
- 06Providers are walked in order. Each is skipped once its cap is hit or once it reports its own quota error. Only failures that plainly never reached the provider are retried. An HTTP 5xx is not, because retrying might send the same mail twice.
- 07Status lands. Success flips the message to sent. Exhausting the whole pool flips it to failed with one serialized error shape, whether the failure surfaced from /send or from an outbox approve.
Engineering decisions
A provider port instead of an SMTP client per vendor
What. Outbound providers implement one Transporter port and self-register into a registry map. The send engine never names a concrete provider.
Why. Adding a fourth provider is a new folder, not an edit to every call site, and it is the pattern the codebase already proved before I formalized it as a rule.
Tradeoff. The provider enum and its registration have to be kept in sync by hand, and an unregistered enum member fails at runtime with a named error rather than at compile time. I judged a full exhaustiveness check not worth the machinery for three providers.
One MailClient contract for every connected mailbox
What. Gmail and IMAP accounts both implement the same MailClient interface, list, read, search, send, reply, flag, archive, trash, and the registry hands back a client, never a type string.
Why. Before this, ten separate account.type === 'gmail' branches were spread across route files, and the two clients' signatures had already drifted apart, archive(ids[]) against archive(id), with nothing to catch it.
Tradeoff. Reconciling the drifted signatures was a real behavior change at every call site, not a pure refactor, so it had to be sequenced deliberately rather than done as a mechanical rename.
Persisted quota instead of an in-memory counter
What. Daily and monthly send counts per provider live in SQLite and roll over by comparing stored date keys on read, not on a scheduled job.
Why. The in-memory version reset on every docker compose up -d, so the caps looked enforced right up until a redeploy-heavy day pushed a provider past its real limit.
Tradeoff. None significant, comparing dates on read is cheap and needs no scheduler, so this beat the in-memory version outright.
Retry narrowed to failures that provably never landed
What. The send path retries only DNS failures, connection refused or reset, and connect timeouts. An HTTP 5xx is never retried.
Why. Sending is not idempotent, a 5xx can mean the provider accepted the message and then failed to answer, so a blind retry risks delivering the same mail twice.
Tradeoff. Some genuinely retryable 5xx failures fall through to the next provider unretried instead. I chose that over the risk of duplicate delivery.
Approval lives outside this service, on purpose
What. POST /outbox/:id/approve sends. It runs no phrase check, no confirmation, no policy of its own.
Why. The actual approval decision belongs to Sofie's own ApprovalAuthority gate. Splitting the decision across two systems would mean neither one owns it.
Tradeoff. envoymail has to trust its caller completely on this endpoint, so the internal API key protecting it is the only thing standing behind that trust, which is why it fails closed rather than open when unset.
Secrets fail closed, not open
What. A gated route with no configured secret returns 503. It never falls through with the check silently skipped.
Why. The earlier behavior treated a missing key as no check needed, so one unset environment variable silently opened every route, including sending mail as any identity.
Tradeoff. A misconfigured deploy is now loudly broken instead of quietly working, which is the correct failure mode for a service holding mailbox credentials.
Raw MIME over the wire, not parsed JSON
What. The Cloudflare Worker forwards the original email bytes to the webhook rather than a parsed, base64-encoded envelope.
Why. Base64 inflates a body by roughly a third, and parsing twice, once in the Worker and once here, burns memory for no benefit, since this service has to parse MIME itself anyway.
Tradeoff. The Worker stays a pure relay with no domain logic, so a bug in inbound parsing can only exist in one place.
Threads key on the References root, not the immediate parent
What. Conversation grouping walks to the root of the References header instead of grouping on In-Reply-To.
Why. The immediate parent looks right for a two-message exchange and then silently splits every later reply into its own thread.
Tradeoff. Mail stored before the fix keeps its fragmented threads. The fix ships a dry-run backfill script rather than rewriting history automatically, because a silent bulk rewrite of real mail is not something I wanted to run without a dry run first.
What changed along the way
- →Started accounts/ with type-string branching, the way transporters/ once did, then rebuilt it behind the same MailClient port once ten duplicated branches and two drifted signatures made the cost obvious.
- →Treated quota as in-memory state at first. It passed every test and then quietly reset on every redeploy in production, so it moved to a persisted store.
- →The first threading rule grouped on the immediate parent, which worked in testing with short threads and split apart in the real world once a conversation ran past two messages.
- →Ran an architecture audit only after roughly 3,200 lines had accumulated two coexisting styles, then named the target pattern in an ADR and added a boundary-gate test so new code cannot drift back into the transaction-script shape.
Security and reliability
- ·An internal API key is required on every gated route, accounts, inbox, threads, outbox, providers, send, configure. A missing key returns 503, never an open route.
- ·The inbound webhook has its own separate secret, also fail-closed, compared in constant time so a match cannot be inferred from response timing.
- ·Stored mailbox credentials, Gmail refresh tokens, IMAP and SMTP passwords, are encrypted at rest with AES-256-GCM. Legacy plaintext rows are migrated on boot rather than left unreadable.
- ·Delete is a soft flag by default. A hard purge is a separate explicit action, since the SQLite file is the only copy of the mail.
- ·Retry is deliberately narrow to avoid duplicate sends, and one corrupt stored row is skipped rather than 500ing the whole endpoint.
- ·API tests run against a throwaway database with stub transporters, never against real credentials or real mail.
Metrics
What shipped
- ✓Runs a working inbound and outbound mail backbone in production: Cloudflare-routed inbound is live, MentorSeed is fully wired end to end on POST /send, and the personal web UI reads and sends through the same surface.
- ✓Named and enforced its own architecture style, a modular monolith with a provider-plugin core, with a boundary-gate test that fails the suite on a new violation rather than a design doc nobody checks again.
- ✓Closed every fail-open security gap found in its own audit: missing secrets now fail closed, credentials are encrypted at rest, quota is persisted, and secret comparison is constant time.
- ✓Fixed a real correctness bug in production data: conversation threads that split apart at message three now group correctly, with a dry-run backfill script for mail stored before the fix.
- ✓Documented to a standard where intent, conventions with reasons, per-module architecture, known gotchas, and decisions are written down and dated, so the service is operable by someone other than its author.
Lessons learned
- ·A pattern the codebase already proves once, the Transporter port for outbound, is the fastest way to fix a second instance of the same problem elsewhere in the same repo. Copy it rather than invent a new shape.
- ·An in-memory counter that passes every test can still be wrong in production the moment the deploy story includes a restart. State that has to survive a redeploy has to be persisted from the start, not added later.
- ·Naming an architecture after the fact, from what the code actually does, produced rules I could enforce with a test. Naming one before writing the code would have been a guess.
- ·The header field that looks obviously correct for threading, In-Reply-To, breaks silently past message two. The one that is actually correct, the References root, is not the intuitive first guess. Worth checking behavior at depth three before trusting a two-message test.