Extended thinking

Extended Thinking (Claude)

Definition & Purpose:

• Extended thinking allows Claude time to reason about a query before generating a final response.
• It enables Claude to tackle more complex tasks with greater accuracy.

Trade-offs:

• Cost: Users are charged for tokens generated during the thinking phase.
• Latency: The process takes more time to complete.

When to Enable:

• Enable extended thinking when prompt evaluations (prompt evals) show that the accuracy of the response is not meeting desired standards, even after prompt improvements.

Technical Implementation:

• Response Structure: When enabled, the response contains two main blocks:
• Thinking Block: Contains the text generated while Claude was reasoning.
• Text Block: Contains the final, generated response.
• Signature: A cryptographic token within the thinking block.
• Function: Ensures the thinking text has not been modified.
• Importance: Prevents developers from altering the reasoning, which is crucial for maintaining safety and response integrity.
• Redacted Content:
• Occurs when Claude's internal safety systems flag the generated thinking text.
• The actual thinking text is provided in a fully encrypted form (redacted content).
• This allows the full message to be passed back to Claude without losing context.

Configuration (Code Parameters):

• thinking: Boolean flag to enable or disable the feature.
• thinking_budget: The number of tokens allocated for the thinking portion of the response.
• Minimum value is 1024.
• max_tokens: The total maximum tokens allowed in the response.
• Crucial Rule: max_tokens must be greater than thinking_budget.
• Example: If thinking_budget is 1024, max_tokens should be significantly larger (e. g. , 4000) to allow tokens for the final text block.
• params Dictionary: When enabled, a nested dictionary is added to params containing the thinking type (enabled) and budget_tokens.

Testing/Debugging:

• A specific, specially formatted string (e. g. , "entropic magic string triggered redacted thinking") can be sent to force Claude to return a redacted thinking block for testing purposes.

Takeaways

• Extended Thinking ermöglicht Claude, komplexe Aufgaben durch eine dedizierte Denkphase zu lösen, um die Antwortgenauigkeit zu erhöhen.
• Die Nutzung hat einen Trade-off: Es erhöht die Kosten (Tokens) und die Latenz (Verarbeitungszeit).
• Die Antwortstruktur besteht aus einem Denkblock (Reasoning) und einem Textblock (final response).
• Bei der Konfiguration muss der Parameter max_tokens immer größer sein als der thinking_budget.