# Tool Use and Function Calling

> Toolformer (Schick et al., 2023) started self-supervised tool annotation. Berkeley Function Calling Leaderboard V4 (Patil et al., 2025) sets the 2026 bar: 40% agentic, 30% multi-turn, 10% live, 10% non-live, 10% hallucination. Single-turn is solved. Memory, dynamic decision-making, and long-horizon tool chains are not.

Canonical: https://www.zambrite.com/ai-training/lessons/tool-use-in-agents

A5Advanced 60 minutes reading and codeUnlocked

Toolformer (Schick et al., 2023) started self-supervised tool annotation. Berkeley Function Calling Leaderboard V4 (Patil et al., 2025) sets the 2026 bar: 40% agentic, 30% multi-turn, 10% live, 10% non-live, 10% hallucination. Single-turn is solved. Memory, dynamic decision-making, and long-horizon tool chains are not.

## Learning Objectives

- Explain Toolformer's self-supervised training signal: keep tool annotations only when execution reduces next-token loss. 
- Name BFCL V4's five evaluation categories and what each measures. 
- Implement a stdlib tool registry with schema validation, argument coercion, and execution sandboxing. 
- Diagnose the three 2026 open problems: long-horizon tool chaining, dynamic decision-making, and memory.

## The Problem

Early tool use asked: can the model predict a correct function call? Modern tool use asks: can the model chain tools across 40 steps, with memory, with partial observability, with recovery from tool failures, without hallucinating tools that do not exist?

Toolformer established the baseline: models can learn when to call tools with self-supervision. BFCL V4 defines the 2026 evaluation target. The gap between them is the space production agents live in.

## The Concept

### Toolformer (Schick et al., NeurIPS 2023)

Idea: let the model annotate its own pretraining corpus with candidate API calls. For each candidate, execute it. Keep the annotation only if including the tool result reduces loss on the next token. Fine-tune on the filtered corpus.

Tools covered: calculator, QA system, search engines, translator, calendar. The self-supervision signal is purely about whether the tool helps predict text — no human labels.

Scale result: tool use emerges at scale. Smaller models hurt from tool annotations; larger models gain. This is why 2026 frontier models have strong tool use baked in while most 7B models need explicit tool-use fine-tuning to be reliable.

### Berkeley Function Calling Leaderboard V4 (Patil et al., ICML 2025)

BFCL is the 2026 de facto evaluation. V4 composition:

- **Agentic (40%)** — full agent trajectories: memory, multi-turn, dynamic decisions. 
- **Multi-Turn (30%)** — interactive conversations with tool chains. 
- **Live (10%)** — user-submitted real prompts (harder distribution). 
- **Non-Live (10%)** — synthetic test cases. 
- **Hallucination (10%)** — detect when no tool should be called.

V3 introduced state-based evaluation: after a tool sequence, check the API's actual state (e.g. "is the file created?") rather than match the AST of the tool calls. V4 added web search, memory, and format sensitivity categories.

Key 2026 finding: single-turn function calling is near-solved. Failures concentrate in memory (carrying context across turns), dynamic decision-making (choosing tools based on prior results), long-horizon chains (drift after 20+ steps), and hallucination detection (refusing to call when no tool fits).

### Tool schema

Every provider has a schema. They differ in details but share the same shape:

```
name: string
description: string (what it does, when to use it)
input_schema: JSON Schema (properties, required, types, enums)
```

Anthropic uses `input_schema` directly. OpenAI uses `function.parameters`. Both accept JSON Schema. Descriptions are load-bearing — the model reads them to pick the right tool. Bad tool descriptions are the #1 root cause of wrong-tool-picked failures.

### Argument validation

Trust no tool call. Validate:

- **Type coercion.** Model may return a string "5" where the schema says int. Coerce if unambiguous; reject if not. 
- **Enum validation.** If the schema says `status in {"open", "closed"}` and model emits `"in_progress"`, reject with a descriptive error. 
- **Required fields.** Missing required field -\> immediate error observation back to the model, not a crash. 
- **Format validation.** Dates, emails, URLs — validate with concrete parsers, not regex.

Every validation failure should return a structured observation so the model can retry with the correct shape.

### Parallel tool calls

Modern providers support parallel tool calls in one assistant turn. The loop:

- Model emits 3 tool calls with distinct `tool_use_id`s. 
- Runtime executes them (in parallel if independent). 
- Each result goes back as a `tool_result` block correlated by `tool_use_id`.

Engineering rule: treat correlation IDs as load-bearing. Swap them and you get wrong-tool-to-wrong-result routing.

### Sandboxing

Tool execution is the sandbox boundary. See Lesson 09 for detail. Short version: every tool should specify read/write surface, network access, timeout, memory cap. Generic `run_shell(cmd)` is a red flag; specific `git_status()` is safer.

## Build it

`code/main.py` implements a production-shape tool registry:

- JSON Schema subset validator (stdlib only). 
- Tool registration with description, input schema, timeout, and executor. 
- Argument coercion and enum validation. 
- Parallel tool dispatch with correlation IDs. 
- Error observations as structured strings.

Run it:

```
python3 code/main.py
```

The trace shows a mini agent calling three tools in one turn, with one deliberately malformed call that is rejected with a descriptive error the model can act on.

## Use it

Every provider has its own tool schema — Anthropic, OpenAI, Gemini, Bedrock. Use a translation layer (OpenAI Agents SDK, Vercel AI SDK, LangChain tool adapter) if you need multi-provider. BFCL is the reference benchmark — run it against your agent before shipping if tool use is central to the product.

## Exercises

- Add a "no-op" tool that lets the model explicitly refuse to use any other tool. Measure on a BFCL-like hallucination test. 
- Implement argument coercion for int-as-string and float-as-string. Where does coercion start to hide real bugs? 
- Add a per-tool timeout and a circuit breaker (refuse the tool for 60s after 3 consecutive failures). What does this change about how the model recovers? 
- Read BFCL V4 description. Pick one category (e.g. "multi-turn") and run 10 example prompts through your agent. Report pass rate. 
- Port the stdlib validator to Pydantic or Zod. What did Pydantic/Zod catch that the toy missed?

## Key Terms

| Term | What people say | What it actually means |
| --- | --- | --- |
| Function calling | "Tool use" | Structured-output tool invocation with validated schema |
| Toolformer | "Self-supervised tool annotation" | Schick 2023 — keep tool calls whose results reduce next-token loss |
| BFCL | "Berkeley Function Calling Leaderboard" | 2026 benchmark: 40% agentic, 30% multi-turn, 10% live, 10% non-live, 10% hallucination |
| Tool schema | "Function signature for the model" | name, description, JSON Schema of arguments |
| tool\_use\_id | "Correlation ID" | Ties a tool call to its result; essential for parallel dispatch |
| Hallucination detection | "Know when not to call" | V4 category: refuse to call when no tool fits |
| Argument coercion | "String-to-int repair" | Narrow fixes for predictable schema-mismatch; reject if ambiguous |
| Sandboxing | "Tool execution boundary" | Per-tool read/write surface, network, timeout, memory cap |

## Further Reading

- [Schick et al., Toolformer (arXiv:2302.04761)](https://arxiv.org/abs/2302.04761) — self-supervised tool annotation 
- [Berkeley Function Calling Leaderboard (V4)](https://gorilla.cs.berkeley.edu/leaderboard.html) — 2026 eval benchmark 
- [Anthropic, Tool use documentation](https://platform.claude.com/docs/en/agent-sdk/overview) — production tool schema in the Claude Agent SDK 
- [OpenAI Agents SDK docs](https://openai.github.io/openai-agents-python/) — function tool type and Guardrails

## The code

### The lesson’s full source

The complete runnable files this lesson builds.

### A working app to start from

A complete starter app built on the same idea, with its setup notes.

Milestone for[An observable back-office agent](https://www.zambrite.com/ai-training/lessons/capstone-back-office-agent)

## Tool errors the agent can recover from

Make every tool return structured errors (not found, ambiguous, rate limited, invalid input) and teach the loop to retry, ask or escalate for each. Test with a statement full of bad data.

### Done when

- Structured error types from every tool: not found, ambiguous, rate limited, invalid
- A recovery rule per error type: retry, ask or escalate
- A messy 50-line statement processed with no crash and every outcome recorded

Stretch: Add backoff with jitter and show it under a simulated rate limit

## Interview practice

Questions on this topic that AI engineering interviews ask, with the companies reported to ask them. Answer the way you would out loud; your answer is scored against production standards with concrete guidance on what to improve.

[A4 The Agent Loop: Observe, Think, Act](https://www.zambrite.com/ai-training/lessons/the-agent-loop)[Next · A6 Agent Memory — Virtual Context and Memory Paging](https://www.zambrite.com/ai-training/lessons/agent-memory)

Adapted from open-source work: AI Engineering from Scratch by Rohit Ghumare (MIT, lesson text, code and quizzes); Awesome LLM Apps by Shubham Saboo (Apache-2.0, starter code); AI Engineering Interview Questions by Outcome School (Apache-2.0, interview questions). Capstones, milestones, data packs and Zambian context by Zambrite.

## Related
- [Home](https://www.zambrite.com/)
- [Contact](https://www.zambrite.com/contact)