What is the primary function of a grader in a prompt evaluation workflow?	To provide an objective signal (like a number or true/false) about the quality of the model's output.
Name the three types of graders discussed in the transcript.	Code-based, Model-based, and Human-based.
How does a code-based grader function?	It feeds the model's output into custom code snippets to perform programmatic checks.
Provide examples of checks that can be performed using a code-based grader.	Checking output length, verifying the presence of specific words, and validating JSON or code syntax.
What is the defining characteristic of a model-based grader?	It uses an additional model API call to evaluate the original model's response.
What is the main requirement for a model-based grader?	The model must provide a hard, objective signal, typically a number between 1 and 10.
What is the primary disadvantage of human-based grading?	It is generally time-consuming and tedious work.
What are the three evaluation criteria used in the example use case?	Format (Python, JSON, or Regex), Valid Syntax, and General Task Following/Accuracy.
Which grader type is most appropriate for validating the format and syntax of generated code?	A code-based grader.
Why is it recommended to ask a model grader for strengths, weaknesses, and reasoning instead of just a score?	It helps the model hone in and provide a more concrete, less middling score.