Building AI agentsMiddleLesson 1986 min read

Structured outputs and function calling

Get machine-readable JSON from the model instead of free text, check it in code against a schema, and retry cleanly when it is wrong.

Lesson in motion

In 60 seconds

Structured outputs and function calling

Get machine-readable JSON from the model instead of free text, check it in code against a schema, and retry cleanly when it is wrong.

1/5
In simple words
When you fill a railway reservation form, each box has a job: name here, age there, only numbers in the age box. The clerk can then process it fast. Structured output asks the model to fill a form like that, and your code checks every box before using it.
Agents need to pass data to code, not only to people. Code cannot read "Sure! The order id is probably ORD-123456." It needs {"order_id": "ORD-123456"}. There are two common ways to get this:
WayWhat it meansTypical use
Function (tool) callingThe API lets the model return a tool name and JSON arguments that follow your schema.The agent loop: choosing and calling tools.
Structured output (JSON mode)You ask for the whole answer as JSON matching a schema. Many APIs can enforce or strongly guide this.Extracting fields, classifying, making a plan as data.
Both use JSON Schema: a standard way to describe the shape of JSON data, such as which fields are required, their types and allowed values. Some providers can constrain the model so its output always parses and follows the schema; others only guide it. Features change often, so check your provider's current docs. Either way, validate in your own code.

Why validate if the API already uses a schema?

  • Parsing is not the same as correct. {"people": 4000} for a restaurant booking is valid JSON and wrong.
  • Schemas cannot express every business rule: "date must be in the future", "only the user's own orders".
  • Model output is untrusted input. Treat it like data from a web form (see Treat model output as untrusted input).
  • If you switch providers or models later, your checks keep working.

Validate, then retry with the error

  1. 1

    Ask

    Send the prompt and the schema. Ask for JSON only.
  2. 2

    Parse

    Try to read the JSON. If it fails, keep the error message.
  3. 3

    Check

    Check required fields, types, ranges and your business rules.
  4. 4

    Retry with feedback

    If something is wrong, send the model its output and the exact error, and ask again.
  5. 5

    Give up cleanly

    After 2 or 3 tries, stop and return a clear failure. Do not guess.
structured.py: validate model JSON and retry with the errorpython
import json

def call_model(prompt: str) -> str:
    # Placeholder. Plug in your provider here. For this demo we fake two replies:
    # the first is wrong, the second is right.
    call_model.n = getattr(call_model, "n", 0) + 1
    if call_model.n == 1:
        return '{"name": "Asha", "people": "four"}'
    return '{"name": "Asha", "people": 4, "date": "2026-10-02"}'

FIELDS = {"name": str, "people": int, "date": str}

def check(raw: str):
    try:
        data = json.loads(raw)
    except json.JSONDecodeError as err:
        return None, "Not valid JSON: " + str(err)
    if not isinstance(data, dict):
        return None, "Expected one JSON object."
    for key, kind in FIELDS.items():
        value = data.get(key)
        if value is None:
            return None, "Missing field: " + key
        if not isinstance(value, kind) or isinstance(value, bool):
            return None, key + " must be of type " + kind.__name__
    if not 1 <= data["people"] <= 20:
        return None, "people must be between 1 and 20"
    return data, None

def get_booking(text: str, tries: int = 3):
    prompt = "Extract name, people (integer), date (YYYY-MM-DD) as JSON only: " + text
    for _ in range(tries):
        raw = call_model(prompt)
        data, error = check(raw)
        if data:
            return data
        prompt += "\nYour last answer: " + raw + "\nProblem: " + error + "\nTry again."
    raise ValueError("Model could not produce valid booking data")

print(get_booking("Table for four on 2 October for Asha, please."))
Note
In real projects you can use a library to validate schemas instead of writing checks by hand, such as the jsonschema or pydantic Python packages. Keep your business rules in code either way.

Good habits

  • Keep schemas small and flat. Deeply nested schemas are harder for the model to fill.
  • Use enum for fixed choices, like "priority": ["low", "medium", "high"].
  • Allow an honest "I don't know": a field like "found": false is better than a made-up value.
  • Set additionalProperties: false so extra, unexpected fields are rejected.
  • Log every failed validation. A spike in failures often means a prompt or model change broke something.
Trusting the output
  • Use json.loads and pass the result straight to the database
  • Retry by sending the same prompt again
  • Retry forever until it works
Checking the output
  • Parse, check types and business rules, then use it
  • Retry with the exact error, so the model can fix it
  • Stop after a few tries and report the failure
Try this. Run the code above. Then change the second fake reply to {"name": "Asha", "people": 400, "date": "2026-10-02"}. What happens, and is that the behaviour you want?

Watch and read more

Lab

A validator for complaint tickets extracted by a model, with a retry loop and a clean failure.

~30 min

The problem

A municipal helpline gets WhatsApp-style complaints like "Streetlight not working near bus stop, Anna Nagar, since 3 days". You want the model to turn each one into a ticket.

  1. Write a JSON schema with: category (enum: water, road, streetlight, garbage, other), area (string), days_open (integer 0 to 365), urgent (boolean).
  2. Write a check() function in Python that parses and checks these rules by hand, or with the jsonschema package.
  3. Add a retry loop (max 3) that sends back the exact error. Use a fake call_model() that returns one bad and one good answer.
  4. Test with five bad outputs: not JSON, missing field, wrong enum value, days_open of -2, and an extra unknown field.

You are done when

Hard questions

Try to answer before you reveal. If you can answer these, you understood the lesson.

Q1Your schema check passes, but the model sometimes puts the complaint text in <code>area</code>. How would you catch this?Reveal

Add rules the schema cannot express well: a maximum length for area, a check against a list of known areas or wards, or a simple test that area is not almost the same as the whole message. Also collect examples of this failure and add them to your test set so you notice if a prompt change makes it worse.

Q2Why might it be risky to put the raw user complaint into the retry prompt many times?Reveal

Each retry resends the text, which costs tokens, and if the complaint contains instructions aimed at the model (injection), repeating it gives it more chances to work. Keep the user text in one clearly marked data section, keep retries few, and send back only your short error message plus the model's last output.

Please sign in to continue.

Questions people ask

If the API guarantees valid JSON, can I skip validation?

You can skip some parsing checks, but not the business checks. A valid object can still have a wrong date, an impossible amount or another user's id. Always check the things that matter before acting.

Why send the error back instead of just asking again?

The model does not know what went wrong unless you tell it. "people must be an integer" gives it a clear fix. Sending the same prompt again often gives the same mistake.

How many retries are sensible?

Usually two or three. If the model fails that often on the same input, the prompt, the schema or the task probably needs to change. More retries add cost and delay without much gain.

Does asking for JSON make the model less accurate?

It can change behaviour in some cases, for example when the model has no room to reason before answering. A common approach is to include a short reason field before the final fields, or to do reasoning and formatting in separate steps. Test both on your own data.

Lesson test

5 questions. Get 3 right (60%) to pass and complete this lesson.

Sign in to take the test and save your progress