What is the primary goal of a grader in a prompt evaluation workflow?	To take model output and provide an objective signal, such as a number or a true/false value.
What are the three types of graders discussed in the transcript?	Code-based, model-based, and human-based.
How does a code-based grader function?	It feeds the model's output into a custom snippet of code to perform programmatic checks (e.g., syntax validation, length checks).
What is the mechanism of a model-based grader?	It takes the original model output and feeds it into an additional model API request for evaluation.
What is the main disadvantage of human-based grading?	It is generally time-consuming and tedious.
Why is it beneficial to ask a model grader for strengths, weaknesses, and reasoning instead of just a score?	It forces the model to hone in and provide a more concrete, less middling score.
Name three examples of evaluation criteria mentioned for a specific use case.	Output format (e.g., Python, JSON, regex), syntax validity, and general task following/accuracy.
What is a key requirement when designing a prompt for a model grader?	The prompt must set a role, clearly state the task, list the solution, and provide specific directions on how to respond.