Prompt engineering

📝 Prompt Engineering Study Notes

**I. Core Concepts**

• Prompt Engineering: The process of improving an initial prompt to achieve more reliable and higher quality outputs from a language model.
• Module Goal: To iteratively improve a prompt by applying various prompt engineering techniques, using evaluation to measure performance gains.
• Target Prompt Goal: To generate a one-day meal plan for an athlete based on specific inputs:
• Height
• Weight
• Physical Goal
• Dietary Restrictions

**II. Evaluation Pipeline Components**

• Prompt Evaluator Class: A wrapper that manages the entire evaluation process, including dataset generation and model grading.
• Concurrency: The evaluator supports running multiple API calls simultaneously to dramatically speed up the evaluation process.
• Note: If rate limit errors occur, reduce the concurrency setting (e. g. , from 50 to 1).
• Dataset Generation: Requires defining the prompt's overall purpose and an input specification (a dictionary listing all required properties, such as height in CM, weight in kilograms, etc. ).
• Initial Prompt: The starting point is a simple, often poor, initial prompt designed to test the baseline performance.

**III. Evaluation Process**

• Function Call: The evaluation function takes the test case prompt inputs as its primary argument.
• Extra Criteria: A crucial string argument used during model grading that specifies additional requirements the output must meet (e. g. , "output must include daily caloric total," "macro nutrient breakdown," "exact food portions and timing").
• Initial Performance: The first, unoptimized prompt typically yields a very poor evaluation score.
• Output: Evaluation results, including scores, reasoning, and solutions, are compiled into an output. html report.

Takeaways

• Prompt Engineering is the iterative process of refining initial prompts to achieve more reliable and higher quality outputs from a language model.
• The evaluation pipeline manages this process, utilizing dataset generation and concurrency to speed up performance testing.
• The process begins with an initial prompt to establish a baseline, which is then optimized through iterative improvements.
• Model grading relies on "Extra Criteria," a crucial string argument that specifies additional requirements for the output (e. g. , caloric totals or macro breakdowns).