Prompt caching in action

Overview

• Prompt caching allows you to cache repeated content (system prompts, tool schemas, messages) to reduce token usage and costs
• Multiple cache breakpoints can be set in a single request

Implementation Basics

• Add cache control field with type: ephemeral to the last element of cached content
• Cache breakpoints are processed in order: tools → system prompt → messages

Caching Tool Schemas

• Best practice: create a copy of the tools list, clone the last tool schema, add cache control field, then overwrite the last element
• This prevents issues if tool order changes later
• Avoids modifying the original tool schema object

Caching System Prompts

• Replace system prompt string with a list containing a text block dictionary
• Include fields: type: text, text: [prompt content], cache_control: {type: ephemeral}

Usage Metrics

• cache_creation_input_tokens: tokens written to cache on first request
• cache_read_input_tokens: tokens read from cache on subsequent requests
• Identical requests reuse cached content; any change to cached content invalidates that cache section

Cache Invalidation

• Changing tool schemas, system prompt, or messages invalidates the corresponding cache breakpoint
• Subsequent requests with changed content trigger new cache writes
• Unchanged cached content continues to be read from cache

Best Practices

• Use prompt caching for repeated, identical content across multiple requests
• Copy and clone objects before adding cache control fields to avoid unintended modifications
• Monitor usage metrics to verify caching is working as expected

Takeaways

• Prompt caching reduces token usage by caching repeated content (system prompts, tool schemas, messages) with cache control fields set to type: ephemeral
• Add cache control to the last element of each cached section; cache breakpoints are processed in order: tools → system prompt → messages
• Clone objects before adding cache control fields to avoid modifying originals and prevent issues if content order changes
• Monitor cache_creation_input_tokens and cache_read_input_tokens metrics to verify caching works; any change to cached content invalidates that cache section