Skip to main content

Project Roles

Project Overview​

The class will work together to conceptualize, design, and prototype an LLM-based application for group-based course projects.

In groups of four students, you will work and iterate on delivering a Proof-of-Concept (MVP, Minimum Viable Product) LLM application across four different project milestones.

To choose groups/roles, each student will complete the role/project questionnaire after the first class, ranking their preference of the four roles. You may request preferred students who you know who you want to work with, which will be taken into consideration.

Roles​

Product Manager (Lead)​

Acts as project lead and is responsible for defining and maintaining project scope, objectives, and success criteria in a Product Requirements Document (PRD). Manages documentation, coordinates team efforts, and ensures alignment with business requirements. Develops and communicates project strategy and roadmap. Serves as an advocate for customer (UX) experience and journey, ensuring the product aligns with a customer need (resource).

Data/Prompt Engineer​

Oversees data quality and management throughout the project lifecycle. Responsible for data collection, preprocessing, and augmentation strategies. Manages data annotation processes and explores synthetic data generation techniques. Collaborates closely with the AI Application Developer on prompt engineering and iterative refinement based on model performance.

AI Application Developer​

Leads the technical implementation of AI components, including LLM selection and integration. Develops prompt engineering pipelines, implements advanced LLM features (e.g., function calling, retrieval-augmented generation), and explores fine-tuning opportunities. Collaborates with Data/Prompt Engineer for prompt/data pipelines and LLM Ops for implementation and deployment.

LLM Ops​

Manages the operational aspects of LLM deployment and maintenance. Responsible for setting up and maintaining the development environment, implementing CI/CD pipelines, and version control systems. Focuses on optimizing model inference, monitoring system performance (latency, memory usage), and ensuring scalability. Collaborates with the AI Application Developer on model deployment strategies.

Presentation Schedule by Roles​

AI Product Manager​

Data/Prompt Engineer​

AI Application Developer​

LLMOps Developer​

Project Timeline​

This project will follow a spiral development pattern, where students will have four deliverables:

  1. Project Milestone 1 (Sept 12): Initial prototype of an LLM inference engine (e.g., Modal) with initial prompt engineering

  2. Project Milestone 2 (Oct 24): 2nd iteration of the prototype, improving on identified gaps in the product and integrating at least (initial) Retrieval Augmented Generation

  3. Project Milestone 3 (Nov 21): 3rd iteration which could include adding Fine Tuning, Evals, and/or improved Production latency

  4. Final Submission and Final Presentation (Dec 12, final exam date): Operational prototype, GitHub Repo, write up and in-class presentation

Example Implementation​

Simple example: Llama 3 8B Modal Service Repo + Streamlit Chat App

Possible Project Ideas​

These are just a few initial ideas for your project:

  • Copilot generator: Develop a custom "Copilot" code generator with RAG and as a VSCode Extension

  • Edge LLM on a raspberry pi: Serving an open LLM onto a Raspberry Pi with the goal of "how low can you go?" or on an iPhone using MLC Chat

  • Weather forecaster app: Use a weather API (ex: api.weather.gov) to auto-generate daily weather reports, augmenting it with a knowledge base. Alternatively, you could do something similar with other APIs like stocks.

  • Retrieval Augmented Generation with custom "in-the-wild" dataset:

For example: