Fine grained tool calling

Streaming Basics

• Streaming delivers real-time API responses as a series of events
• Common event type: ContentBlockDelta
• With tools enabled, Claude sends a new event type: input_json_event

input_json_event Structure

• partial_json: a piece of JSON representing part of a tool argument
• snapshot: cumulative sum of all partial JSON pieces received so far

Default Tool Streaming Behavior

• API validates JSON before sending chunks to prevent invalid JSON
• Validation waits for complete top-level key-value pairs (e. g. , closing quote of a string value)
• Once a key-value pair is validated, all its chunks arrive together, creating the appearance of large text blocks
• Result: delays when generating large arguments

Fine-Grained Tool Streaming

• Disables JSON validation on the API side
• Enables traditional streaming: chunks arrive continuously rather than in batches
• Trade-off: your code must handle potentially invalid JSON and implement error handling
• Benefit: faster processing of tool arguments (access values before entire object is generated)

Invalid JSON Handling

• Default mode: API wraps invalid JSON in a string to maintain valid structure
• Fine-grained mode: invalid JSON causes parsing errors on client side (your responsibility)

Key Trade-off

• Choose between validation assurance (default) or faster streaming performance (fine-grained)

Takeaways

• Streaming tool use sends input_json_event with partial_json chunks and a cumulative snapshot of all pieces received
• Default mode validates JSON before sending, causing delays as it waits for complete key-value pairs before chunking
• Fine-grained streaming disables API-side validation, enabling continuous chunks but requiring client-side error handling for potentially invalid JSON
• The core trade-off is between guaranteed valid JSON (default) versus faster argument processing (fine-grained)