Pilots are easy. Production is the discipline. In the last two years we've shipped AI marketing automations across HubSpot, Klaviyo, Customer.io, and a half-dozen custom stacks — and we've watched many more die before they ever touched a real customer. The pattern that separates the two isn't model choice. It's architecture.
Stop building toys.
An AI feature that depends on a single prompt and a single model is a demo. A system depends on data, evaluation, escalation, and observability. Treat them as first-class concerns from day one or accept that you're shipping a toy.
Three layers, every time.
Foundation: identity, events, and consent — clean. Logic: deterministic rules sit above LLM calls, never below. Surface: the channel-specific renderer that handles delivery, personalization, and fallback. Skip any layer and the system is fragile within a quarter.
Measure incrementality, not output.
If you can't run a holdout, you can't claim the system worked. We instrument incrementality from the first week. It changes the conversation with leadership and it keeps the team honest.