Unbounded consumption: denial of wallet
Every model call costs money. Without limits, one attacker, one bot or one bug in a loop can burn your budget or take the feature down for everyone.
In 60 seconds
Unbounded consumption: denial of wallet
Every model call costs money. Without limits, one attacker, one bot or one bug in a loop can burn your budget or take the feature down for everyone.
How consumption gets out of control
| Pattern | Example | Control |
|---|---|---|
| Huge inputs | Pasting a whole book or a giant file into the chat | Input size limits; count tokens before calling the model |
| Huge outputs | "Write a 100,000-word story" | A maximum output token setting on every call |
| Loops | An agent calls tools again and again; retries on every error | Maximum steps, maximum time, retry limits with backoff |
| Many requests | A script hammers your free chatbot | Sign-in, per-account rate limits, bot protection |
| Free general-purpose use | People use your shop bot as a free all-purpose AI | Keep the bot on topic; quotas per user |
| Expensive defaults | The largest model and 20 retrieved chunks for every question | Pick the model by task; cap retrieved chunks |
| Exposed keys | Model API key inside the mobile app or web page | Keep keys on the server; call the model from your backend |
Layers of limits
- 1
Per request
Max input size, max output tokens, timeout, max tool steps. - 2
Per user
Requests per minute and tokens per day, per account. - 3
Per feature
A monthly budget for each AI feature, tracked separately. - 4
Organisation-wide
Spend alerts at set levels, and a hard cap if your provider supports it. - 5
Kill switch
A tested way to turn the feature off or fall back quickly.
MAX_INPUT_CHARS = 8_000
MAX_OUTPUT_TOKENS = 800
DAILY_TOKENS_PER_USER = 50_000
REQUEST_TIMEOUT_S = 30
MAX_AGENT_STEPS = 6
class BudgetExceeded(Exception):
pass
def check_budget(user_id: str, text: str, usage) -> None:
if len(text) > MAX_INPUT_CHARS:
raise BudgetExceeded("message too long")
if usage.tokens_today(user_id) >= DAILY_TOKENS_PER_USER:
raise BudgetExceeded("daily limit reached, try tomorrow")
def after_call(user_id: str, tokens_used: int, usage) -> None:
usage.add(user_id, tokens_used) # record real usage from the API responseCaching and cheaper paths
Watch and read more
Lab
A one-page cost budget with limits, alerts and actions for a free AI feature.
The problem
The feature: a free exam-preparation chatbot for a college with about 20,000 students. Use an imaginary price that you write down clearly, for example "₹X per million input tokens and ₹Y per million output tokens". Do not look up or claim real prices.
- Estimate normal use: questions per student per day, tokens per question in and out. Write your assumptions.
- Calculate an expected monthly cost from your assumptions.
- Set limits: per request (input, output, timeout), per user per day, and a monthly feature budget.
- Set alerts at 50%, 80% and 100% of the budget, and write what happens at each (who is told, what changes).
- Describe the kill switch: who can press it and what students see.
You are done when
Hard questions
Try to answer before you reveal. If you can answer these, you understood the lesson.
Q1Exam week arrives and honest usage triples. Your 80% alert fires on day 20 of the month. What do you do, and what should you have planned?Reveal
Look at the data first: is it many students using it normally, or a few accounts using far more? If honest, choose a planned response: a cheaper model for simple questions, shorter answers, caching common questions, or a budget increase approved by the owner. Good planning expects seasonal peaks like exams and sets the budget and limits for them in advance.
Q2Why can an agent loop cost far more than a normal chat, even with the same per-call output limit?Reveal
Because each step is a separate call, and each call re-sends the growing context: instructions, history and tool results. The input tokens grow step after step. Ten or fifty steps can cost much more than one answer. Limits on steps, total time and total cost per task are what stop it.
Questions people ask
What is a token?
A small piece of text, often part of a word. Models read and write text as tokens, and most providers price by tokens in and tokens out. Text in different languages can use different numbers of tokens for the same meaning.
How do we choose limits without annoying real users?
Measure first. Look at how much normal users actually send and receive, then set limits well above that. Give clear messages when a limit is hit, and offer a way to ask for more if there is a real need.
Is caching safe for AI answers?
For public, non-personal answers, usually yes. For anything personal or permissioned, include the user or permission scope in the cache key, or do not cache. Also clear cached answers when their source documents change.
Should agents have a time limit?
Yes. Agents can loop, retry and call tools many times. Set a maximum number of steps, a maximum total time and a maximum cost per task. When a limit is hit, stop and report, instead of trying forever.
Lesson test
5 questions. Get 3 right (60%) to pass and complete this lesson.