Evals 101 — Doug Guthrie, Braintrust
AI Engineer
•
June 27, 2025
AI Engineer
@aiDotEngineerAbout
Talks, workshops, events, and training for AI Engineers.
Video Description
This hands-on workshop guides participants through the full AI evaluation lifecycle with Braintrust, from initial prompt testing to production monitoring. Attendees will build evaluation frameworks, practice offline and online strategies, and implement logging systems. About Doug Guthrie Doug Guthrie is a solutions engineer at Braintrust. Previously, he helped customers deploy data infrastructure at dbt Labs. He is also a proud girl dad. Recorded at the AI Engineer World's Fair in San Francisco. Stay up to date on our upcoming events and content by joining our newsletter here: https://www.ai.engineer/newsletter
AI Engineer's Top Picks for Eval Enthusiasts
AI-recommended products based on this video
Loading...
Pishengfa DIY Stitch Book, Embroidery Stitch Book, DIY Sewing Books, EmbroideryS Books of Stitches, with 86/128 Sewing Tips for Beginners, Unique Embroidery Handmade Gifts (Grey)
(0)
$29.99
$6.99 delivery Dec 23, 2025 - Jan 2, 2026
700+ bought in past month
Loading...
Google Pixel Buds Pro 2 - Noise Canceling Earbuds - Up to 31 Hour Battery Life with Charging Case - Bluetooth Headphones - Compatible with Android - Hazel
(183)
199.99
PrimeFREE delivery Saturday, June 14
400+ bought in past month
Loading...
Best Seller
Deeyaple USB C to Aux, 4FT/1.2M, Type C to 3.5mm Audio Cable Headphone Jack Cable for Car Mobile Phone, iPhone 16 15, iPad Pro, Samsung Galaxy S24 S23 S2010, Google Pixel,Oneplus Grey (1)
(78)
$6.99
FREE delivery Fri, Sep 5 on your first order
1K+ bought in past month
Loading...
Car Carplay Woven Cable for iPhone 16 15 3.3FT USB A to USB C 3.2 Gen 2 Carplay Adapter Wire for iPhone 16 15 Pro Max, iPad Pro/Air, Samsung Galaxy S25/S24/S23/S22/S21 Google Pixel, Car Charger Cable
(213)
$12.99
FREE delivery Sat, Jun 14 on your first order
100+ bought in past month
Loading...
Seasonic Focus V4 GX-1000 (ATX3) - 1000W - 80+ Gold - ATX 3.0 & PCIe 5.1 Ready -Full-Modular -ATX Form Factor -Premium Japanese Capacitor -10 Year Warranty -Nvidia RTX 30/40 Super & AMD GPU Compatible
(22)
$423.35
FREE delivery Oct 8 - 10
Loading...
NEXPOW Car Jump Starter,Car Battery Jump Starter Pack 1500A Peak Q10S for Up to 7.0L Gas and 5.5L Diesel Engine12V Auto Battery Booster,Jumper Cables,Portable Lithium Jump Box with LED Light/USB QC3.0
(4,578)
$71.99
FREE delivery Sat, Jan 3
2K+ bought in past month
Loading...
Firefly Variety 8 Pack - Fire Starter Accessory for Swiss Army Victorinox Knives (Neon Green-Yellow Glow)
(181)
$70.15$61.13
$5.05 delivery Thu, Jun 26Only 3 left in stock.
Loading...
9-in-1 5000A 150PSI Car Battery Booster Jump Starter with Air Compressor (All Gas/9L Diesel), Portable Car Battery Booster Pack, Safe Durable Car Jump Starter with Extended Jumper Cables, Glove, Light
(261)
$99.99
FREE delivery Sat, Sep 20
1K+ bought in past month
Loading...
UCANUUP 130W Tip 4.5mm Compatible with Dell XPS 15 7590 9530 9550 9560 9570 9575 Inspiron 7348 Precision M3800 5510 5520 5530 06TTY6 AC Adapter Power Replacement Supply Cord
(61)
$35.99
FREE delivery Sun, Aug 24
Loading...
tomtoc 360° Protective Laptop Sleeve for 15-inch MacBook Air M4/A3241 2025, M3/A3114 2024, M2/A2941 2023, 15-inch MacBook Pro A1990 A1707, Dell XPS 15 Plus Laptop, Water-Resistant Computer Case Bag Global Recycled Standard
(12,968)
$39.55
FREE delivery Tue, Jun 17
Loading...
MOSISO 360 Protective Laptop Bag 15 inch, 15 inch Computer Shoulder Bag Compatible with MacBook Air 15 M4 M3 M2 2025-2023, Dell XPS 15, Side Open Messenger Case &4 Zipper Pockets&Handle, Black Global Recycled Standard
(3,362)
$39.32
FREE delivery Tue, Jun 17
Loading...
UGREEN 130W Car Charger USB C Fast Charger PD3.0 QC4.0 PPS Fast Charging Car Adapter with 100W USB C Cable Compatible with Dell XPS, MacBook, iPhone 16 15 14 Pro Max, Galaxy S25 S24 Ultra, iPad Pro
(4,725)
$31.99
500+ bought in past month
Loading...
10.1 Inch Touch Portable Monitor IPS Screen 1366x768P 60Hz 400 Brightness 99% sRGB HDMI USB-C Monitors Switch for Xbox PS3/4/5 Laptop Compatible with Raspberry Pi, Mini Touch Screen
(1)
$99.99
FREE delivery Mon, Aug 11
Loading...
ELECROW 8 Inch Portable Monitor, 1280x800 Mini HD Display with Built-in Speakers, USB Powered, Non-Touch LCD Screen for Raspberry Pi, PC, Laptop, Jetson Nano, Game Consoles
(49)
$117.29$102.21
$6.86 delivery Thu, Aug 14
Loading...
7 Inch Portable Monitor Touchscreen HD 1024x600 LED Display Dual HDMI Port Small Monitor for PC Raspberry Pi Laptop Computer Xbox PS4/5 Switch Built-in Speakers
(61)
$68.99
FREE delivery Tue, Aug 12
Loading...
BrosTrend 1800Mbps WiFi 6 Linux WiFi Adapter for PC and Raspberry Pi 2+, Long Range USB WiFi Dongle Linux for Ubuntu, Mint, Debian, Kubuntu, Lubuntu, Zorin, Windows 11/10, Dual Band Wireless Antenna
(288)
$54.14
FREE delivery Sun, Jun 22