Structured outputs and function calling
Get machine-readable JSON from the model instead of free text, check it in code against a schema, and retry cleanly when it is wrong.
In 60 seconds
Structured outputs and function calling
Get machine-readable JSON from the model instead of free text, check it in code against a schema, and retry cleanly when it is wrong.
{"order_id": "ORD-123456"}. There are two common ways to get this:| Way | What it means | Typical use |
|---|---|---|
| Function (tool) calling | The API lets the model return a tool name and JSON arguments that follow your schema. | The agent loop: choosing and calling tools. |
| Structured output (JSON mode) | You ask for the whole answer as JSON matching a schema. Many APIs can enforce or strongly guide this. | Extracting fields, classifying, making a plan as data. |
Why validate if the API already uses a schema?
- Parsing is not the same as correct.
{"people": 4000}for a restaurant booking is valid JSON and wrong. - Schemas cannot express every business rule: "date must be in the future", "only the user's own orders".
- Model output is untrusted input. Treat it like data from a web form (see Treat model output as untrusted input).
- If you switch providers or models later, your checks keep working.
Validate, then retry with the error
- 1
Ask
Send the prompt and the schema. Ask for JSON only. - 2
Parse
Try to read the JSON. If it fails, keep the error message. - 3
Check
Check required fields, types, ranges and your business rules. - 4
Retry with feedback
If something is wrong, send the model its output and the exact error, and ask again. - 5
Give up cleanly
After 2 or 3 tries, stop and return a clear failure. Do not guess.
import json
def call_model(prompt: str) -> str:
# Placeholder. Plug in your provider here. For this demo we fake two replies:
# the first is wrong, the second is right.
call_model.n = getattr(call_model, "n", 0) + 1
if call_model.n == 1:
return '{"name": "Asha", "people": "four"}'
return '{"name": "Asha", "people": 4, "date": "2026-10-02"}'
FIELDS = {"name": str, "people": int, "date": str}
def check(raw: str):
try:
data = json.loads(raw)
except json.JSONDecodeError as err:
return None, "Not valid JSON: " + str(err)
if not isinstance(data, dict):
return None, "Expected one JSON object."
for key, kind in FIELDS.items():
value = data.get(key)
if value is None:
return None, "Missing field: " + key
if not isinstance(value, kind) or isinstance(value, bool):
return None, key + " must be of type " + kind.__name__
if not 1 <= data["people"] <= 20:
return None, "people must be between 1 and 20"
return data, None
def get_booking(text: str, tries: int = 3):
prompt = "Extract name, people (integer), date (YYYY-MM-DD) as JSON only: " + text
for _ in range(tries):
raw = call_model(prompt)
data, error = check(raw)
if data:
return data
prompt += "\nYour last answer: " + raw + "\nProblem: " + error + "\nTry again."
raise ValueError("Model could not produce valid booking data")
print(get_booking("Table for four on 2 October for Asha, please."))jsonschema or pydantic Python packages. Keep your business rules in code either way.Good habits
- Keep schemas small and flat. Deeply nested schemas are harder for the model to fill.
- Use
enumfor fixed choices, like"priority": ["low", "medium", "high"]. - Allow an honest "I don't know": a field like
"found": falseis better than a made-up value. - Set
additionalProperties: falseso extra, unexpected fields are rejected. - Log every failed validation. A spike in failures often means a prompt or model change broke something.
- Use
json.loadsand pass the result straight to the database - Retry by sending the same prompt again
- Retry forever until it works
- Parse, check types and business rules, then use it
- Retry with the exact error, so the model can fix it
- Stop after a few tries and report the failure
{"name": "Asha", "people": 400, "date": "2026-10-02"}. What happens, and is that the behaviour you want?Watch and read more
Lab
A validator for complaint tickets extracted by a model, with a retry loop and a clean failure.
The problem
A municipal helpline gets WhatsApp-style complaints like "Streetlight not working near bus stop, Anna Nagar, since 3 days". You want the model to turn each one into a ticket.
- Write a JSON schema with:
category(enum: water, road, streetlight, garbage, other),area(string),days_open(integer 0 to 365),urgent(boolean). - Write a
check()function in Python that parses and checks these rules by hand, or with thejsonschemapackage. - Add a retry loop (max 3) that sends back the exact error. Use a fake
call_model()that returns one bad and one good answer. - Test with five bad outputs: not JSON, missing field, wrong enum value, days_open of -2, and an extra unknown field.
You are done when
Hard questions
Try to answer before you reveal. If you can answer these, you understood the lesson.
Q1Your schema check passes, but the model sometimes puts the complaint text in <code>area</code>. How would you catch this?Reveal
Add rules the schema cannot express well: a maximum length for area, a check against a list of known areas or wards, or a simple test that area is not almost the same as the whole message. Also collect examples of this failure and add them to your test set so you notice if a prompt change makes it worse.
Q2Why might it be risky to put the raw user complaint into the retry prompt many times?Reveal
Each retry resends the text, which costs tokens, and if the complaint contains instructions aimed at the model (injection), repeating it gives it more chances to work. Keep the user text in one clearly marked data section, keep retries few, and send back only your short error message plus the model's last output.
Questions people ask
If the API guarantees valid JSON, can I skip validation?
You can skip some parsing checks, but not the business checks. A valid object can still have a wrong date, an impossible amount or another user's id. Always check the things that matter before acting.
Why send the error back instead of just asking again?
The model does not know what went wrong unless you tell it. "people must be an integer" gives it a clear fix. Sending the same prompt again often gives the same mistake.
How many retries are sensible?
Usually two or three. If the model fails that often on the same input, the prompt, the schema or the task probably needs to change. More retries add cost and delay without much gain.
Does asking for JSON make the model less accurate?
It can change behaviour in some cases, for example when the model has no room to reason before answering. A common approach is to include a short reason field before the final fields, or to do reasoning and formatting in separate steps. Test both on your own data.
Lesson test
5 questions. Get 3 right (60%) to pass and complete this lesson.