Extended thinking

Extended Thinking in Claude

Definition and Purpose

• Extended Thinking: A feature that allows Claude time to reason about a user's query before generating the final response.
• Goal: Enables Claude to tackle more complex tasks with greater accuracy.

Trade-offs (Costs)

• Cost: Tokens generated during the thinking phase are charged.
• Latency: The process takes a measurable amount of time to complete.

When to Enable Extended Thinking

• Decision Basis: Rely on prompt evaluations (evaluations).
• Trigger: Enable extended thinking when the accuracy of the prompt is not satisfactory, even after significant effort has been put into improving the prompt.

Technical Implementation

• Response Structure Change: When enabled, the response includes a new part type: the reasoning content part.
• Reasoning Content Part: Contains the text generated while Claude was thinking.
• Signature: A cryptographic token within the reasoning content part.
• Function: Ensures the text in the reasoning content block has not been modified.
• Importance: Prevents developers from changing the thinking text, which could steer Claude in an unsafe direction.
• Redacted Content: In some cases, the reasoning content part may only contain a field of redacted content.
• Cause: This occurs when Claude's internal safety systems flag the generated thinking text.
• Function: Provides the full, encrypted thinking text so it can be sent back to Claude later.

Configuration (Code Implementation)

• Thinking Argument: A keyword argument added to the chat helper function.
• Thinking Budget: A parameter (default 1024 tokens) that sets the maximum number of tokens Claude can spend on the thinking portion of the response.
• Configuration Flow:
• Define an empty additional model fields dictionary.
• If thinking is enabled, add a thinking property to this dictionary (specifying type: enabled and budget tokens).
• Pass this dictionary into the parameters sent to the converse function.

Key Takeaway

• The decision to use extended thinking must be driven by prompt evaluations (evaluations) for a specific use case.

Takeaways

• Extended Thinking allows Claude time to reason about a query before responding, improving accuracy for complex tasks.
• It involves trade-offs: it consumes tokens (which are charged) and increases latency.
• Extended Thinking should be enabled based on prompt evaluations when the accuracy of the prompt is unsatisfactory.
• The response structure includes a "reasoning content part" secured by a cryptographic signature to prevent modification of the thinking text.
• Configuration is managed via a thinking budget parameter, which sets the maximum number of tokens Claude can use for reasoning.