Tool Use That Doesn't Hallucinate: Designing Function Schemas LLMs Actually Respect
Function calling fails in boring, predictable ways: invented arguments, wrong enums, nested JSON that half-parses. Here's how we design tool schemas so Claude, GPT, and Gemini stop improvising.
Function calling looks like a solved problem until you ship it. Then you start seeing the pattern: an agent invents a user_id that doesn't exist, passes "true" instead of true, or picks an enum value that isn't in your list. The model isn't broken — your schema is leaking ambiguity, and the model is filling the gap with plausible nonsense.
This is a write-up of the heuristics we use when designing tools for agent systems across Claude, OpenAI, and Gemini. It's opinionated, and it assumes you've already read the vendor docs at least once.
Why tool schemas fail in the first place
Every major provider now accepts something close to JSON Schema for function definitions — Anthropic's tools parameter, OpenAI's tools with function type, and Gemini's function_declarations. They all claim strict mode of some flavor. None of them stop a model from generating a semantically wrong call.
The failure modes we see most often:
- Phantom arguments. The model passes a field that looks reasonable but was never in the schema — often a legacy name from its training data (
userIdinstead ofuser_id). - Enum drift. You defined
status: ["open", "closed"]; the model returns"pending"because the user's message mentioned it. - Type coercion. Booleans as strings, numbers as strings, ISO dates as freeform English.
- Over-eager calls. The model calls
delete_accountwhen the user said "I'm thinking about closing my account." - Under-specified objects. Nested objects with
additionalProperties: trueturn into creative writing exercises.
Strict mode helps with the first three. It does nothing for the last two. Those are design problems.
Write schemas the way you'd write a type system
The mental model that works best: treat the function schema as a public API contract, not a hint. If a field can be wrong, it will be. If a field is optional, the model will omit it inconsistently across calls.
Make everything required, then allow null
This is the single highest-leverage change we make to misbehaving tools. Both OpenAI's strict mode and Anthropic's tool use work better when every property is in the required array. If a field is genuinely optional, model it as type: ["string", "null"] and require the model to explicitly pass null.
{
"name": "search_orders",
"description": "Search orders by customer or date range. Returns up to 50 matches.",
"input_schema": {
"type": "object",
"properties": {
"customer_id": {
"type": ["string", "null"],
"description": "UUID of the customer. Pass null if searching by date only."
},
"start_date": {
"type": ["string", "null"],
"format": "date",
"description": "ISO 8601 date, inclusive. Null means no lower bound."
},
"end_date": {
"type": ["string", "null"],
"format": "date",
"description": "ISO 8601 date, inclusive. Null means no upper bound."
}
},
"required": ["customer_id", "start_date", "end_date"],
"additionalProperties": false
}
}
The explicit null forces the model to make a decision for every field, which cuts down on "I forgot that argument existed" errors. Anthropic documents this nullable pattern for Claude; OpenAI requires it under strict mode.
Set additionalProperties: false everywhere
At the top level and at every nested object. Without it, models happily invent fields. With it, the provider's strict validator rejects the call before it reaches your code — which is what you want during the agent loop, because the model gets a clean error and retries correctly.
Descriptions are prompts, not documentation
The description fields in your schema are inside the model's context on every turn. Treat them as prompt engineering.
Things we put in descriptions:
- When to call this tool and when not to. Explicit negative examples help.
- Argument semantics the model can't infer from the type. "UUID v4" vs "may be a UUID or a legacy integer string."
- Side effects. If it writes to a database, say so. If it costs money, say so.
- Idempotency. Can this be retried? Models will retry; tell them whether it's safe.
A bad tool description looks like this:
"Deletes a user."
A tool description that survives contact with a real agent looks like this:
"Permanently delete a user account and all associated data.
This is destructive and cannot be undone. Only call this after
the user has explicitly confirmed deletion in the current turn.
Do not call based on phrases like 'I'm thinking about' or
'maybe I should'. Requires the user_id as a UUID v4."
Yes, it's long. Yes, it burns tokens. It also prevents the kind of incident you only need to have once.
Tool names matter more than you think
Models pattern-match on names. get_user and fetch_user and lookup_user will all get called in situations you didn't intend if they coexist. Pick a verb convention — we use get_* for reads, create_*, update_*, delete_* for writes, and search_* for queries that return lists — and stick to it across your entire tool surface.
Constrain the output surface, not just the input
A lot of teams obsess over input validation and ignore what the tool returns to the model. The return shape matters because the model will use those values in its next call.
If your search_orders tool returns {"id": "ord_123", "ref": "ORD-2026-001"}, the model will sometimes pass ref where you expected id. Either return only one identifier, or make it obvious in the field names which one is the canonical key.
We also strip anything the model doesn't need. A full order object has 40 fields; the agent flow needs 6. Return the 6. Smaller tool outputs mean cheaper context, faster calls, and fewer hallucination surfaces.
Testing tool use without a full agent harness
You do not need LangChain to test this. A tight loop that calls the model once with a crafted user message, inspects the tool call, and asserts on the arguments is enough to catch 80% of regressions.
def assert_tool_call(user_msg, expected_tool, expected_args):
response = client.messages.create(
model="claude-sonnet-4-5",
max_tokens=1024,
tools=TOOLS,
messages=[{"role": "user", "content": user_msg}],
)
tool_uses = [b for b in response.content if b.type == "tool_use"]
assert len(tool_uses) == 1, f"expected 1 tool call, got {len(tool_uses)}"
call = tool_uses[0]
assert call.name == expected_tool
for key, val in expected_args.items():
assert call.input.get(key) == val, f"{key}: {call.input.get(key)} != {val}"
Wrap that in your test runner, feed it 30–50 realistic user messages per tool, and run it on every schema change. In our experience this catches more real bugs than any eval dashboard, because the failures are binary: the right tool got called with the right arguments, or it didn't.
Red-team your own tools
For every destructive tool, write test cases where the user is ambiguous, joking, or asking a hypothetical. "What would happen if I deleted my account?" should not trigger delete_account. If it does, your description isn't strong enough, or you need a confirmation tool in between.
Vendor-specific gotchas worth knowing
The schemas look the same. The behavior isn't.
- Claude (per Anthropic's tool use docs) handles complex nested schemas well and respects descriptions aggressively. It will sometimes call tools in parallel when you didn't ask it to — use
disable_parallel_tool_useif ordering matters. - OpenAI's strict mode is the most rigid: it won't accept schemas with unsupported keywords, and it enforces
requiredon all properties. The payoff is near-zero schema violations. - Gemini historically has been looser with enums and more likely to omit optional fields. Its
function_declarationssupport a subset of OpenAPI — check the current docs before assuming a keyword works.
If you're routing across providers, your lowest common denominator is: all-required fields, nullable types instead of optional, no oneOf/anyOf at the top level, and flat enums.
Where we'd start
If you've got an agent in production that's misbehaving, don't rewrite the orchestration layer first. Open the tool definitions file and do three things: mark every property as required and nullable, set additionalProperties: false everywhere, and rewrite every description as if you were briefing a new engineer on when to call that function. Then build the 30-message assertion suite before you touch anything else.
We've shipped this pattern across customer support agents, internal ops bots, and e-commerce copilots — the full write-up of how we approach agent architecture lives in our AI services overview, and there's more on eval strategy across the blog. Tool schemas are the cheapest, highest-leverage part of an agent to get right. Most teams skip them and go straight to prompt tuning. Do it the other way around.
Want a team like ours?
72Technologies builds production software for the kind of teams who actually read this blog.
Start a projectKeep reading
Structured Outputs in 2026: When to Trust the Schema and When to Validate Twice
Structured output modes from OpenAI, Anthropic, and Google look identical on the slide deck. In production they fail in very different ways. Here's how we handle schemas without getting burned.
Prompt Caching in Practice: What Actually Lands on the Bill
Prompt caching sounds like free money until you read the invoice. Here's how Anthropic, OpenAI, and Gemini caches actually behave, and the patterns that make them pay off.
Agent Loops That Don't Spiral: Budgets, Stop Conditions, and Fallbacks
Most agent failures aren't reasoning failures — they're loop failures. Here's how we design budgets, stop conditions, and fallbacks so our agents don't burn $40 proving they can't book a flight.
