Midterm Study Guide

Online Canvas Exam

Format

• 20 multiple choice questions based on assigned readings
• Identical format to quizzes
• Questions may differ between students to prevent sharing
• Students found sharing information will receive zero credit and may face disciplinary action according to the Student Honor Code
• Required use of LockDown Browser

Resources

Permitted Materials

• Handwritten notes
• Printed materials (e.g., printed word files)
• LLM-generated study notes (must be printed, not shared)

Prohibited Materials

• ChatGPT, Claude, LLama, or any other LLMs during exam
• Collaboration with other students
• Sharing of exam questions or content

Time Limit

• 20 minutes maximum

Exam Material

Week 2: AI Product

• AI Product Development Chapter 1: Creating Value with AI Driven Products
• 16 Changes to the Way Enterprises Are Building and Buying Generative AI
• Your AI Product Needs Evals
• Why Host Your Own LLMs?

Week 3: Prompting

• AI Product Development Chapter 6: Steering Models with Prompt Engineering
• Prompting Fundamentals and How to Apply them Effectively
• F*** you, Show me the Prompt
• Explainprompt.com

Week 5: LLM Technicals

• AI Product Development: Making Products Work
• Understanding LLM Tokenization
• Weird Properties of Large Language Models
• Building a Generative AI Platform

Week 6: RAG I

• Prompting Guide: RAG (skip RAG Research Insights section)
• Verba: Building an Open Source, Modular RAG Application
• Improving Retrieval with LLM-as-a-Judge
• Levels of Complexity: RAG Applications

Week 7: RAG II

• Emerging UX Patterns for Generative AI Apps & Copilots
• LLM From the Trenches: 10 Lessons Learned Operationalizing Models at GoDaddy
• Open Challenges in LLM Research

Required Videos

1. Video Reflection 1: Simon Willison's PyCon 2024 Keynote
2. Video Reflection 2: Lessons From A Year Building With LLMs

Important Notes

• LockDown Browser is required - test it in advance
• Using LLMs during the exam will likely hurt your grade
• You may use LLMs to help prepare study notes before the exam
• All notes must be printed or handwritten - no digital resources during exam