Prompt caching

Problem with Normal Requests

• Claude performs extensive internal processing on input text before generating output (creating data structures and calculations)
• After sending the response, Claude discards all this internal work and calculations
• When a follow-up request includes the same input message, Claude must repeat all the same processing work from scratch
• This wastes computational resources and increases latency and costs

Solution: Prompt Caching

• Prompt caching saves the internal work done on input text instead of discarding it
• When the same input message appears in a follow-up request, Claude can retrieve and reuse the cached work
• Reusing cached work eliminates redundant processing, speeding up response generation and reducing costs

Benefits

• Faster response times for follow-up requests with repeated content
• Decreased text generation costs
• More efficient use of computational resources

How It Works

• Initial request: Claude processes input and caches the internal analysis results
• Follow-up request: If the same input message is detected, Claude retrieves cached work instead of reprocessing
• The cached work is stored in temporary data storage for quick access

Takeaways

• Claude discards internal processing work after each response, forcing redundant reprocessing when the same input appears in follow-up requests
• Prompt caching saves this internal work so it can be retrieved and reused for subsequent requests with identical input
• Reusing cached work reduces latency, computational costs, and improves response speed for follow-up queries