How it's built
A five-stage pipeline: collect account data via Klaviyo MCP, redact PII, classify flows with Haiku, score against a versioned rubric in plain code, then synthesize cited recommendations with Sonnet. Every step is a traced span with latency, tokens and cost.
Architecture
Design decisions
| Decision | Choice | Rationale |
|---|---|---|
| MCP transport | Local Klaviyo MCP server over stdio, inside the API container, with a read-only private key | The remote server requires interactive OAuth and an Owner/Admin/Manager role, so it doesn't suit a public demo. The local server plus a scoped key keeps control server-side. |
| Gaps in the local server | Direct read-only REST calls for metric aggregates, forms and sending domains | The local server exposes 15 read tools; the remote-only tools are replaced by the same REST endpoints with the same key, and labeled REST in the trace. |
| Safety flags | READ_ONLY=true, ALLOW_USER_GENERATED_CONTENT=false | Defense in depth: no writes are possible, and profile, event and review tools are disabled, which removes most PII and prompt-injection surface. |
| Where the LLM is used | Only where judgment is needed | Collection is a fixed tool plan (reproducible, cheap). Scoring is deterministic code (auditable, unit-tested). The LLM does classification and synthesis only. |
| Model routing | Haiku for flow classification, Sonnet for synthesis | Mirrors a production cost-control pattern; the cost meter shows the routed run against an all-Sonnet, no-cache counterfactual. |
| Prompt caching | Cache breakpoint after instructions, rubric and citation catalog | The stable prefix is identical across audits, so repeat runs read it from cache; the cache-hit rate is shown per run. |
| Guardrails | Schema-enforced outputs, one retry, faithfulness check, data wrapped as data | Every number in a recommendation must appear in the scoring output. Names that contain instructions can't change scores, because scoring never reads model output. |
| Replay mode | Recorded or generated tool responses as fixtures | Enables deterministic tests, offline scenario accounts and a zero-cost cached default view. |
Time saved (estimate)
Assumptions are editable and still need validating with a Services practitioner.
Hours saved / consultant / month
50
Agent runtime is under a minute; the estimate assumes 30 minutes of consultant review per audit.
Rubric
* Default threshold from third-party or placeholder values; tune per industry. Checks without data are marked n/a and excluded from the denominator.