Accessing the API

Accessing and Using Claude: Request Lifecycle

**I. Architectural Requirements (Security)**

• Server-Side Access: Never access Vertex directly from a client-side application (web or mobile app).
• Security: All requests to Vertex must be routed through a developer-implemented and controlled server to keep secret credentials secure.

**II. The Request Process**

• Step 1 (Client to Server): The user's input message is sent from the client (e. g. , web browser) to the developer's server.
• Step 2 (Server to Vertex): The server makes a request to Vertex, typically using an SDK (e. g. , Anthropic SDK for Python, TypeScript, Go, Ruby, or Google Vertex SDK).
• Required Request Data:
• Model name.
• List of messages (including the user's submitted text).
• Max tokens (a limit on the length of the generated text).

**III. Internal Text Generation Process (The Model)** The text generation process within the language model involves four main stages:

• Tokenization: The user's input is broken down into smaller strings called tokens (which can be whole words, parts of words, spaces, or symbols).
• Embedding: Each token is converted into an embedding—a long list of numbers that acts as a numerical definition of the word.
• Contextualization: Embeddings are adjusted based on the surrounding tokens. This process refines the embedding to highlight the single, precise meaning of a word given its context.
• Generation: The final processed embeddings are passed to an output layer, which produces probabilities for every possible next word.
• Selection: The model uses a mix of probability and randomness to select the next word, ensuring natural and varied responses.
• Iteration: The selected word is added to the list of embeddings, and the entire process repeats.

**IV. Completion and Response**

• Stopping Criteria: The model stops generating text when:
• It reaches the max tokens limit set in the input request.
• It generates a special "end of sequence token" (a signal indicating a natural end to the response).
• API Response: Once complete, the API sends a response to the server containing:
• The generated text.
• Usage (count of input tokens + count of generated tokens).
• Stop Reason (explaining why generation stopped).
• Final Step: The server sends the generated text back to the client/app for display.

Takeaways

• Security mandates that all requests to Vertex must be routed through a developer-controlled server, never directly from the client application.
• The request process flows from the client to the server, which then uses an SDK to send the request to Vertex, requiring the model name, message list, and max tokens.
• The internal text generation process involves four stages: Tokenization (breaking input), Embedding (numerical definition), Contextualization (contextual refinement), and Generation (probabilistic selection).
• Text generation concludes when the model hits the max tokens limit or produces an "end of sequence token," at which point the API returns the generated text and usage statistics.