Evaluating Patterns in Gen AI Outputs (alternative: "What can we expect of LLMs as software engs?")

How can we expect LLM products to impact software engineering? Join Chelsea Troy for an examination of how LLM products are built. She'll walk through the domain: what data LLMs are trained on and how they're trained: and spell out how those factors impact what LLMs tend to do well with and where they tend to struggle. Then, we'll talk about how that affects the design of the products built around LLMs, like ChatGPT, Gemini, Claude, Copilot, or Devon. Finally, take a tour with Chelsea through current research elucidating the impact of LLM product use on developer workflows, and dig into what that means for us as practitioners in an LLM-enabled world.