Fifty cents a day: the cap on what the writing costs

On a Monday in October the model bill for the four accounts running this product came to four dollars for the day. Not much. Except the price is $49/month, and four dollars a day for four accounts is thirty a month each, which is a business losing money on every customer in the one place a customer cannot see. So I spent the evening finding out where the four dollars went, and the answer is worth writing down, because every tool in this category has the same bill and none of them talk about it.

What one message costs

Two calls on the model per person: one to read them, one to write. Each call carried about four thousand tokens of instructions, the same four thousand every time, at full price, because nothing had told the model it could keep them. Four accounts, twenty-five people a day, two calls each is two hundred calls. That is most of the four dollars, and most of it was paying for the same words two hundred times.

• The instructions are now sent as a cached block. The second call in five minutes pays a tenth for them.
• Readings and drafts go through the model's batch endpoint, which costs half and answers in minutes to an hour. Nobody is waiting on a daily run, so the wait is free. Only the calls a person waits on, a reply suggestion, a reply's intent, stay direct.
• Every call is recorded with its tokens priced, against the workspace it was for, in a ledger I can read by the day.

The cap

Then the rule the ledger enforces. A workspace may spend $0.50 a day on the model. A call that would start past that is refused before it is made, and the day's search sizes itself to what the budget can still draft, so it stops before spending a LinkedIn read on a person it cannot write to. When the cap is hit, the queue says the search resumes tomorrow. Messages already approved still send; sending costs nothing on the model.

That is enough for about twenty-five people a day with the instructions cached and the drafts batched, which is the most the product finds in a day anyway, under its own limits. On a normal day the cap is never touched. It exists for the abnormal day: the deploy that restarted a batch and paid twice, the rewrite that ran across every workspace at once, the week nobody looked.

Why a cap on our spend is a feature of yours

A cap on what we spend per customer is what keeps the price a single number with no usage tier under it. The tools that bill credits for discovery and enrichment are passing their model bill through with a margin. The ones with a flat price and no cap are either losing money or writing less than they say. Fifty cents a day, published, is how this one stays at $49/month and keeps writing every message from the person's own words.

The spend by day, by workspace and by purpose is on an admin page nobody but me sees. The part a customer can see is the result: the accept and reply rates of the accounts running it, every week, on the numbers page.