LLMs in Production

Objective: Learn best practices for deploying LLMs in production environments. Students will understand the challenges and solutions for running LLMs at scale.

Slides

Case Study

LLMs in Production by the course's LLM Ops Developers

Required Readings

• The Shift from Models to Compound AI Systems
• Optimizing Latency
• A Visual Guide to Quantization

Optional Reading

• Every Way To Get Structured Output From LLMs
• Berkeley Function-Calling Leaderboard