@rohanpaul_ai: This 76-page paper on Promptin...
@rohanpaul_ai
59 views
Jul 14, 2024
1
This 76-page paper on Prompting Techniques has become quite popular. A nice read for your weekend.
- "The Prompt Report: A Systematic Survey of Prompting Techniques": ✨
Explores structured understanding and taxonomy of 58 text-only prompting techniques, and 40 techniques for other modalities.
📌 The paper focuses on discrete prefix prompts rather than cloze prompts, because prefix prompts are widely used with modern LLM architectures like decoder-only models. It excludes soft prompts and techniques using gradient-based updates.
📌 The paper identifies 58 text-based prompting techniques broken into 6 major categories:
1) In-Context Learning (ICL) - learning from exemplars/instructions in the prompt
2) Zero-Shot - prompting without exemplars
3) Thought Generation - prompting the LLM to articulate reasoning
4) Decomposition - breaking down complex problems
5) Ensembling - using multiple prompts and aggregating outputs
6) Self-Criticism - having the LLM critique its own outputs
📌 For ICL, it discusses key design decisions like exemplar quantity, ordering, label quality, format, and similarity that critically influence output quality. It also covers ICL techniques like K-Nearest Neighbor exemplar selection.
📌 Extends the taxonomy to multilingual prompts, discussing techniques like translate-first prompting and cross-lingual ICL. It also covers multimodal prompts spanning image, audio, video, segmentation, and 3D modalities.
📌 More complex techniques like agents that access external tools, code generation, and retrieval augmented generation are also taxonomized. Evaluation techniques using LLMs are discussed.
📌 Prompting issues like security (prompt hacking), overconfidence, biases, and ambiguity are highlighted. Two case studies - benchmarking techniques on MMLU and an entrapment detection prompt engineering exercise - are presented.
- "The Prompt Report: A Systematic Survey of Prompting Techniques": ✨
Explores structured understanding and taxonomy of 58 text-only prompting techniques, and 40 techniques for other modalities.
📌 The paper focuses on discrete prefix prompts rather than cloze prompts, because prefix prompts are widely used with modern LLM architectures like decoder-only models. It excludes soft prompts and techniques using gradient-based updates.
📌 The paper identifies 58 text-based prompting techniques broken into 6 major categories:
1) In-Context Learning (ICL) - learning from exemplars/instructions in the prompt
2) Zero-Shot - prompting without exemplars
3) Thought Generation - prompting the LLM to articulate reasoning
4) Decomposition - breaking down complex problems
5) Ensembling - using multiple prompts and aggregating outputs
6) Self-Criticism - having the LLM critique its own outputs
📌 For ICL, it discusses key design decisions like exemplar quantity, ordering, label quality, format, and similarity that critically influence output quality. It also covers ICL techniques like K-Nearest Neighbor exemplar selection.
📌 Extends the taxonomy to multilingual prompts, discussing techniques like translate-first prompting and cross-lingual ICL. It also covers multimodal prompts spanning image, audio, video, segmentation, and 3D modalities.
📌 More complex techniques like agents that access external tools, code generation, and retrieval augmented generation are also taxonomized. Evaluation techniques using LLMs are discussed.
📌 Prompting issues like security (prompt hacking), overconfidence, biases, and ambiguity are highlighted. Two case studies - benchmarking techniques on MMLU and an entrapment detection prompt engineering exercise - are presented.
2
