Playbook · AI commerce operations · updated 2026-08-18
AI Commerce Operations: What to Automate and What to Keep Human
A practical boundary between the work an agent should own and the decisions that must stay with a person.
Most commerce teams adopt agents in the wrong order. They start with the visible work — writing copy, answering customers, generating product descriptions — because that work is easy to demo. Then they discover the expensive part was never the writing. It was the handoffs, the decision log, and knowing who is accountable when the output is wrong.
The useful framing is narrower than "AI transformation." An agent is good at the repeatable edge of a workflow: gathering context, drafting a first pass, routing work to the right place, and logging what happened. It is bad at the parts of commerce that carry money and trust — pricing, positioning, refunds, and promises made to a customer. Automating the first category buys back hours. Automating the second category buys an incident.
Start with the decision you make three times a week
The best first candidate for automation is not the most painful task. It is the most repeated one. If the same decision gets made three or more times a week, and the inputs look similar every time, that decision has an implicit rule inside it. Automation is the act of making that rule explicit.
Write the rule down before you write the prompt. If you cannot state the rule in two sentences, the workflow is not ready — you would be asking an agent to invent policy on your behalf, and it will, inconsistently, in a way you only notice weeks later when the outputs stop matching each other.
This is also the honest test of whether a task is worth automating at all. Plenty of painful work is painful precisely because it is novel each time. That work wants better tooling, not an agent.
Write acceptance criteria before you write the prompt
An agent without acceptance criteria is a generator of plausible output. You will read the first few results, find them reasonable, and stop checking. The failure mode is not dramatic — it is slow drift, where outputs stay plausible while quietly diverging from what you actually wanted.
Acceptance criteria fix this by making the check mechanical instead of aesthetic. Before the agent runs, define what a correct output contains, what it must never contain, and what a reviewer should look at first. "Draft a product description" is not a spec. "Draft a product description that names the material, states the two most common use cases, avoids any claim about durability we have not tested, and stays under 90 words" is a spec you can grade against.
The side benefit is that acceptance criteria transfer. Once written, they become the QA checklist for human work on the same task, and the regression test when you change models or prompts.
Keep a human review step where money and trust are involved
Some outputs are cheap to get wrong and some are not. A misrouted internal ticket costs minutes. A wrong price, an unauthorized refund, a fabricated product claim, or a commitment made to a customer costs money, and sometimes costs the relationship. The review step is not a lack of confidence in the tooling — it is a recognition that the downside is asymmetric.
Draw the line explicitly and write it into the workflow rather than relying on memory. Anything customer-facing that makes a claim, sets a price, or commits to a delivery gets human sign-off. Everything upstream of that — research, enrichment, drafting, routing, summarizing, logging — can run unattended.
The practical version of this is a queue, not a meeting. The agent does the work and stages it; a person spends ten minutes approving or rejecting a batch. That preserves the speed gain while keeping the accountable decision with someone who can be held to it.
Instrument the handoff, not just the output
When an agent output is wrong, the first question is always the same: where did the bad input come from? Teams that only log final outputs cannot answer that, so every investigation becomes an archaeology project.
Log each step as its own record: what triggered the workflow, what context was retrieved, what the agent produced, who reviewed it, and what they changed. The edits reviewers make are the highest-signal data you have — a pattern in corrections is a defect in your acceptance criteria, and it tells you exactly what to fix.
The checklist
- Name one workflow where the same decision is made at least three times a week.
- Write the rule behind that decision in two sentences before touching a prompt.
- Define acceptance criteria: what a correct output contains, and what it must never contain.
- Draw an explicit line at pricing, claims, refunds, and customer commitments — those keep human sign-off.
- Stage agent output in a review queue rather than publishing it directly.
- Log trigger, context, output, reviewer, and edits as separate records.
- Review reviewer edits monthly and fold recurring corrections back into the criteria.
Signals that it is working
- Response latency drops without a rise in corrections.
- Fewer repeated manual checks on the same class of work.
- When an output is wrong, ownership is obvious within a minute.
Frequently asked
What should a commerce team automate with AI agents first?
Start with a workflow where the same decision recurs at least three times a week and the inputs look similar each time. That repetition means an implicit rule already exists, and automation is just making it explicit. Avoid starting with novel, high-judgment work — that needs better tooling, not an agent.
Which commerce decisions should never be fully automated?
Anything with an asymmetric downside: pricing, refunds, claims about a product, and commitments made to a customer. The cost of a wrong answer in those categories is far higher than the time saved, so they keep a human approval step even when an agent drafts the work.
How do you stop AI output quality from drifting over time?
Write acceptance criteria before deploying the workflow, then grade against them mechanically instead of reading outputs and judging whether they feel reasonable. Track the edits reviewers make — a recurring correction is a defect in the criteria, not a one-off mistake.
Related playbooks
- Build in Public Systems — Build-in-public works when the public record proves movement. Here is the system that produces that record as a byproduct of shipping.
- Solo Founder Stack — A stack is not a trophy shelf. It is a set of defaults that protects attention and makes the next shipped improvement cheaper.
- Agent Revenue Workflows — Revenue agents should remove the dead air between a signal and the follow-up, not replace the judgment in between.
- Commerce Intelligence Layer — Storefronts stall because pages have no memory. The intelligence layer is what turns a view into a useful next action.
- Automated SEO — Useful SEO automation is mostly bookkeeping: one intent per page, canonical URLs that match, and a sitemap that reflects what actually exists.
- Product Validation — A validated experiment is not a brainstorm with a landing page. It is a constrained test with a promise, a signal, and a decision rule written in advance.