The ROI of AI: Why you need Eval Framework - Beyang Liu

Video Description

There has been a ton of hype about AI's applications in software development, but leaders must look beyond the hype to assess the ROI for their organizations. We'll cover some quantitative and qualitative frameworks that have proven useful for our customers, highlight the benefits and drawbacks of popular metrics in use, and share the themes that have emerged as important pillars of the value prop for AI dev tools. I'll also share a vision for where the next 18 months of AI will lead in software development. Recorded live in San Francisco at the AI Engineer World's Fair. See the full schedule of talks at https://www.ai.engineer/worldsfair/2024/schedule & join us at the AI Engineer World's Fair in 2025! Get your tickets today at https://ai.engineer/2025 Beyang Liu is CTO and co-founder of Sourcegraph. Sourcegraph is the world's best code search and understanding engine, which enables developers to find and grok code across very large codebases and dependency graphs. Beyang is also the creator of Cody, the context-aware AI coding assistant, which uses Sourcegraph's search and understanding engine to generate code, answer questions, and enable a wide variety of AI automation in the context of your private codebase. Prior to Sourcegraph, Beyang built large-scale data analysis engines for Fortune 500 companies with complex codebases as an engineer at Palantir Technologies. Beyang's interest in AI sprang from his experience studying computer science at Stanford University, where he first encountered the Chomsky and Norvig models of intelligence and discovered a love for compilers while publishing research as a member of the Stanford AI Lab.