Marketplace lessons from Uber, Airbnb, Bumble, and more | Ramesh Johari (Stanford professor)

Lenny's Podcast November 9, 2023
Video Thumbnail
Lenny's Podcast Logo

Lenny's Podcast

@lennyspodcast

About

Interviews with world-class product leaders and growth experts to uncover concrete, actionable, and tactical advice to help you build, launch, and grow your own product.

Video Description

Ramesh Johari is a professor at Stanford University focusing on data science methods and practice, as well as the design and operation of online markets and platforms. Beyond academia, Ramesh has advised some incredible startups, including Airbnb, Uber, Bumble, and Stitch Fix. Today we discuss: • What exactly a marketplace is, if you boil it down • What you need to get right to build a successful marketplace • How to optimize any marketplace • An easy litmus test to see if there’s an opportunity to build a marketplace in the space • The role of data science in successful marketplaces • Ramesh’s philosophy on experimentation and AI • Advice on implementing rating systems • Why learning isn’t free — Brought to you by Sanity—The most customizable content layer to power your growth engine: https://www.sanity.io/lenny | Hex—Helping teams ask and answer data questions by working together: https://www.hex.tech/lenny | Eppo—Run reliable, impactful experiments: https://www.geteppo.com/ Find the full transcript at: https://www.lennyspodcast.com/marketplace-lessons-from-uber-airbnb-bumble-and-more-ramesh-johari-stanford-professor-startup/ Where to find Ramesh Johari: • LinkedIn: https://www.linkedin.com/in/rameshjohari/ • Website: https://web.stanford.edu/~rjohari/ • X: https://twitter.com/rameshjohari Where to find Lenny: • Newsletter: https://www.lennysnewsletter.com • X: https://twitter.com/lennysan • LinkedIn: https://www.linkedin.com/in/lennyrachitsky/ In this episode, we cover: (00:00) Ramesh’s background (04:31) A brief overview of what a marketplace is (08:10) The role of data science in marketplaces (11:21) Common flaws of marketplaces (16:43) Why every founder is a marketplace founder (20:26) How Substack increased value to creators by driving demand (20:58) An example of overcommitting at eBay (22:24) An easy litmus test for marketplaces  (25:52) Thoughts on employees vs. contractors (28:02) How to leverage data scientists to improve your marketplace (34:10) Correlation vs. causation (35:27) Decisions that should be made using data (39:29) Ramesh’s philosophy on experimentation (41:06) How to find a balance between running experiments and finding new opportunities (44:11) Badging in marketplaces (46:04) The “superhost” badge at Airbnb (49:59) How marketplaces are like a game of Whac-A-Mole (52:41) How to shift an organization’s focus from impact to learning (55:43) Frequentist vs. Bayesian A/B testing  (57:50) The idea that learning is costly (1:01:55) The basics of rating systems (1:04:41) The problem with averaging (1:07:14) Double-blind reviews at Airbnb (1:08:55) How large language models are affecting data science (1:11:27) Lightning round Referenced: • Riley Newman on LinkedIn: https://www.linkedin.com/in/rileynewman/ • Upwork (formerly Odesk): https://www.upwork.com/ • Ancient Agora: https://en.wikipedia.org/wiki/Ancient_Agora_of_Athens • Trajan’s Market: https://en.wikipedia.org/wiki/Trajan%27s_Market • Kayak: https://www.kayak.com/ • UrbanSitter: https://www.urbansitter.com/ • Thumbtack: https://www.thumbtack.com/ • Substack: https://substack.com/ • Ebay: https://www.ebay.com/ • Coase: “The Nature of the Firm”: https://en.wikipedia.org/wiki/The_Nature_of_the_Firm • Stitch Fix: https://www.stitchfix.com/ • A/B Testing with Fat Tails: https://www.journals.uchicago.edu/doi/abs/10.1086/710607 • The ultimate guide to A/B testing | Ronny Kohavi (Airbnb, Microsoft, Amazon): https://www.lennyspodcast.com/the-ultimate-guide-to-ab-testing-ronny-kohavi-airbnb-microsoft-amazon/ • Servaes Tholen on LinkedIn: https://www.linkedin.com/in/servaestholen/ • Bayesian A/B Testing: A More Calculated Approach to an A/B Test: https://blog.hubspot.com/marketing/bayesian-ab-testing • Designing Informative Rating Systems: Evidence from an Online Labor Market: https://arxiv.org/abs/1810.13028 • Reputation and Feedback Systems in Online Platform Markets: https://faculty.haas.berkeley.edu/stadelis/Annual_Review_Tadelis.pdf • How to Lie with Statistics: https://www.amazon.com/How-Lie-Statistics-Darrell-Huff/dp/0393310728 • David Freedman’s books on Amazon: https://www.amazon.com/stores/David-Freedman/author/B001IGLSGA • Four Thousand Weeks: Time Management for Mortals: https://www.amazon.com/Four-Thousand-Weeks-Management-Mortals/dp/0374159122 • The Alpinist on Prime Video: https://www.amazon.com/Alpinist-Peter-Mortimer/dp/B09KYDWVVC • Only Murders in the Building on Hulu: https://www.hulu.com/series/only-murders-in-the-building-ef31c7e1-cd0f-4e07-848d-1cbfedb50ddf Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email [email protected]. Lenny may be an investor in the companies discussed.

You May Also Like

Uberize Your Business: Must-Have Startup Essentials

AI-recommended products based on this video

Loading...
tomtoc 360° Protective Laptop Sleeve for 15-inch MacBook Air M4/A3241 2025, M3/A3114 2024, M2/A2941 2023, 15-inch MacBook Pro A1990 A1707, Dell XPS 15 Plus Laptop, Water-Resistant Computer Case Bag Global Recycled Standard

tomtoc 360° Protective Laptop Sleeve for 15-inch MacBook Air M4/A3241 2025, M3/A3114 2024, M2/A2941 2023, 15-inch MacBook Pro A1990 A1707, Dell XPS 15 Plus Laptop, Water-Resistant Computer Case Bag Global Recycled Standard

(12,968)
$39.55
FREE delivery Tue, Jun 17
Loading...
tomtoc 360° Protective Laptop Sleeve for 13-inch New MacBook Air M4/A3240 2025, M3/A3113, M2/A2681 M1/A2337 A2179 A1932, 13-inch MacBook Pro M2/A2686 M1/A2338, Water-Resistant Computer Case Bag Global Recycled Standard

tomtoc 360° Protective Laptop Sleeve for 13-inch New MacBook Air M4/A3240 2025, M3/A3113, M2/A2681 M1/A2337 A2179 A1932, 13-inch MacBook Pro M2/A2686 M1/A2338, Water-Resistant Computer Case Bag Global Recycled Standard

(12,952)
$36.07
PrimeFREE delivery Sat, Jun 14
100+ bought in past month
Loading...
Phone Case for iPhone 12 Pro Max Case Compatible with MagSafe, YUANDMI Case for iPhone 12 Pro Max Magnetic Slim Thin Translucent Matte Back Shockproof Anti-Yellow Protective Cover Case Black

Phone Case for iPhone 12 Pro Max Case Compatible with MagSafe, YUANDMI Case for iPhone 12 Pro Max Magnetic Slim Thin Translucent Matte Back Shockproof Anti-Yellow Protective Cover Case Black

(130)
$12.99
FREE delivery Sat, Jun 14 on your first order
Loading...
iPhone Charger Fast Charging [Apple MFi Certified] USB C to Lightning Cable 6FT USBC to Lightning Cable Nylon Braided Type C Cord for iPhone 14 13 12 11 Pro Max XR XS X 8 7 6 Plus SE, iPad

iPhone Charger Fast Charging [Apple MFi Certified] USB C to Lightning Cable 6FT USBC to Lightning Cable Nylon Braided Type C Cord for iPhone 14 13 12 11 Pro Max XR XS X 8 7 6 Plus SE, iPad

(4,036)
$8.99
FREE delivery Sun, Jun 29 on your first order
2K+ bought in past month
Loading...
iPhone Charger [Apple MFi Certified] 3FT, 3Pack USB A to Lightning Cable Braided Fast Charging Apple Cord Compatible with iPhone 14 13 12 11 Pro Max XS XR X 8 7 6 5 Plus iPad/iPod/AirPods

iPhone Charger [Apple MFi Certified] 3FT, 3Pack USB A to Lightning Cable Braided Fast Charging Apple Cord Compatible with iPhone 14 13 12 11 Pro Max XS XR X 8 7 6 5 Plus iPad/iPod/AirPods

(5,626)
$17.99
FREE delivery Sat, Jun 14 on your first order
50+ bought in past month
Loading...
2Pack 10ft iPhone Charger 90 Degree Lightning Cable Right Angle Long USB Charging Cord Compatible with Apple iPhone 14 13 12 11 Pro Max XS XR X 8 7 6 5 Plus iPad iPod

2Pack 10ft iPhone Charger 90 Degree Lightning Cable Right Angle Long USB Charging Cord Compatible with Apple iPhone 14 13 12 11 Pro Max XS XR X 8 7 6 5 Plus iPad iPod

(2,193)
$14.24
FREE delivery Sat, Jun 14 on your first order
400+ bought in past month
Loading...
Wireless Charging Case for AirPods Pro 1st / 2nd Gen with Type-C Charging Port, Compatible with AirPod Pro 2nd Generation Replacement Case, with Blue-Tooth Sync Button, No Earbuds

Wireless Charging Case for AirPods Pro 1st / 2nd Gen with Type-C Charging Port, Compatible with AirPod Pro 2nd Generation Replacement Case, with Blue-Tooth Sync Button, No Earbuds

(15)
$42.98
PrimeFREE delivery Sat, Jun 14
Loading...
Cleaner Kit for Airpods Pro 1 2 3 Multi-Function Cleaning Pen with Soft Brush Flocking Sponge for Bluetooth Earphones Case Cleaning Tools White
Best Seller

Cleaner Kit for Airpods Pro 1 2 3 Multi-Function Cleaning Pen with Soft Brush Flocking Sponge for Bluetooth Earphones Case Cleaning Tools White

(9,766)
$9.89
FREE delivery Sat, Jun 14 on your first order
500+ bought in past month
Loading...
Apple AirPods Pro 2 Wireless Earbuds, Bluetooth Headphones, Active Noise Cancellation, Transparency, Personalized Spatial Audio, High-Fidelity Sound, H2 Chip, USB-C Charging

Apple AirPods Pro 2 Wireless Earbuds, Bluetooth Headphones, Active Noise Cancellation, Transparency, Personalized Spatial Audio, High-Fidelity Sound, H2 Chip, USB-C Charging

(26,678)
$259.99$194.99
FREE delivery Wed, Aug 20
5K+ bought in past month
Loading...
Wireless Charger, 3 in 1 Charging Station, Fast Charging Station Foldable Charging Stand, Wireless Charging Station for Multiple Devices, Compatible with lPhone 17/16/15/14/13 Pro Max Airpods Watch

Wireless Charger, 3 in 1 Charging Station, Fast Charging Station Foldable Charging Stand, Wireless Charging Station for Multiple Devices, Compatible with lPhone 17/16/15/14/13 Pro Max Airpods Watch

(156)
$25.64
FREE delivery Wed, Dec 24 on your first order
3K+ bought in past month
Loading...
SmallRig ZV-E1 Camera Baseplate for Sony ZV-E1, Built-in Cold Shoe Mount and QR Plate (for Arca) Quick Switch Between Tripod and Stabilizer for DJI RS 2 / RSC 2 / RS 3 / RS 3 Pro - 4314

SmallRig ZV-E1 Camera Baseplate for Sony ZV-E1, Built-in Cold Shoe Mount and QR Plate (for Arca) Quick Switch Between Tripod and Stabilizer for DJI RS 2 / RSC 2 / RS 3 / RS 3 Pro - 4314

(3)
$44.99
FREE delivery Thu, Jun 19
Loading...
SmallRig FX3 / FX30 HawkLock Quick Release Camera Cage Kit for Sony with Cable Clamp, Bulit-in QR Cold Shoe NATO Rail/QR Plate for Arca, Modular Design Compatible with Original XLR Handle - 4770

SmallRig FX3 / FX30 HawkLock Quick Release Camera Cage Kit for Sony with Cable Clamp, Bulit-in QR Cold Shoe NATO Rail/QR Plate for Arca, Modular Design Compatible with Original XLR Handle - 4770

(35)
$124.94
FREE delivery Thu, Jun 19
Loading...
SmallRig FX3 / FX30 HawkLock Quick Release Camera Cage Kit for Sony with XLR Handle Extension, Wrist Strap and Cable Clamp for HDMI, QR Cold Shoe, NATO Rail/QR Plate for Arca,Titanium Gray - 4771

SmallRig FX3 / FX30 HawkLock Quick Release Camera Cage Kit for Sony with XLR Handle Extension, Wrist Strap and Cable Clamp for HDMI, QR Cold Shoe, NATO Rail/QR Plate for Arca,Titanium Gray - 4771

(72)
$174.24$162.04
FREE delivery Thu, Jun 19
Loading...
AC Power Cord Cable Fit for Sony PS5 PS4 PS3 Slim PlayStation 5 4 3, Xbox Series X/S, Xbox One X/S, TV, Printer Universal 2 Prong Supply Replacement - (ETL Listed Cable)

AC Power Cord Cable Fit for Sony PS5 PS4 PS3 Slim PlayStation 5 4 3, Xbox Series X/S, Xbox One X/S, TV, Printer Universal 2 Prong Supply Replacement - (ETL Listed Cable)

(5,351)
$8.49
Prime
500+ bought in past month
Loading...
INIU USB C Charger Block, 100W 3-Port PD Fast Charging, Advanced GaN Power Adapter, Foldable Charger with USB C Cable for MacBook Air 13 Pro, iPad, iPhone 17 16 Pro Max, Samsung S25 S24, Switch 2 etc ClimatePartner certified

INIU USB C Charger Block, 100W 3-Port PD Fast Charging, Advanced GaN Power Adapter, Foldable Charger with USB C Cable for MacBook Air 13 Pro, iPad, iPhone 17 16 Pro Max, Samsung S25 S24, Switch 2 etc ClimatePartner certified

(804)
$36.99
FREE delivery Fri, Feb 13
1K+ bought in past month
Loading...
UGREEN USB to USB C Adapter & USB C to USB Adapter Combo 4-Pack, 10Gbps Type-C Converter, Car Charger Compatible with MacBook Pro, iPad Mac mini, iPhone 17/16, Galaxy, PC/Laptop, Hard Drive Enclosure

UGREEN USB to USB C Adapter & USB C to USB Adapter Combo 4-Pack, 10Gbps Type-C Converter, Car Charger Compatible with MacBook Pro, iPad Mac mini, iPhone 17/16, Galaxy, PC/Laptop, Hard Drive Enclosure

(86)
$9.74
Prime
Loading...
Metapen Air 8 Apple Pencil for iPad 2018-2025, 20H Battry Life, 2Min Fast Charging, Palm Rejection, Stylus Pen for iPad 11/10/9/8/7/6th, iPad Pro 12.9/11/13“M4, iPad Air 3/4/5/6/M2/M3, iPad mini 5/6

Metapen Air 8 Apple Pencil for iPad 2018-2025, 20H Battry Life, 2Min Fast Charging, Palm Rejection, Stylus Pen for iPad 11/10/9/8/7/6th, iPad Pro 12.9/11/13“M4, iPad Air 3/4/5/6/M2/M3, iPad mini 5/6

(1,985)
$14.22
FREE delivery Tue, Sep 9 on your first order
1K+ bought in past month
Loading...
Deeyaple USB C to Aux, 4FT/1.2M, Type C to 3.5mm Audio Cable Headphone Jack Cable for Car Mobile Phone, iPhone 16 15, iPad Pro, Samsung Galaxy S24 S23 S2010, Google Pixel,Oneplus Grey (1)
Best Seller

Deeyaple USB C to Aux, 4FT/1.2M, Type C to 3.5mm Audio Cable Headphone Jack Cable for Car Mobile Phone, iPhone 16 15, iPad Pro, Samsung Galaxy S24 S23 S2010, Google Pixel,Oneplus Grey (1)

(78)
$6.99
FREE delivery Fri, Sep 5 on your first order
1K+ bought in past month
Loading...
Metapen Air 8 Apple Pencil for iPad 2018-2025, 20H Battry Life, 2Min Fast Charging, Palm Rejection, Stylus Pen for iPad 11/10/9/8/7/6th, iPad Pro 12.9/11/13“M4, iPad Air 3/4/5/6/M2/M3, iPad mini 5/6

Metapen Air 8 Apple Pencil for iPad 2018-2025, 20H Battry Life, 2Min Fast Charging, Palm Rejection, Stylus Pen for iPad 11/10/9/8/7/6th, iPad Pro 12.9/11/13“M4, iPad Air 3/4/5/6/M2/M3, iPad mini 5/6

(1,985)
$14.22
FREE delivery Tue, Sep 9 on your first order
1K+ bought in past month
Loading...
Bose QuietComfort Wireless Noise Cancelling Headphones, Bluetooth Over Ear Headphones with Up to 24 Hours of Battery Life, Moonlight Grey - Limited Edition

Bose QuietComfort Wireless Noise Cancelling Headphones, Bluetooth Over Ear Headphones with Up to 24 Hours of Battery Life, Moonlight Grey - Limited Edition

(12,251)
$269.00
FREE delivery Tue, Oct 14
Loading...
Graduation Gifts, Bus Driver Appreciation Gifts Ideas for Women/Men, End of Year Retirement Inspirational Thank You Gift, Natural Stone Bracelets

Graduation Gifts, Bus Driver Appreciation Gifts Ideas for Women/Men, End of Year Retirement Inspirational Thank You Gift, Natural Stone Bracelets

(7)
$18.99
FREE delivery Sat, Feb 14 on your first order
Loading...
Buruis Extra Large Capacity Toiletry Bag for Women and Men, Hanging Toiletry Organizer Cosmetics Makeup Bag, Water-Resistant Dopp kit Travel Bag for Full Sized Toiletries, Travel Essentials (Black)

Buruis Extra Large Capacity Toiletry Bag for Women and Men, Hanging Toiletry Organizer Cosmetics Makeup Bag, Water-Resistant Dopp kit Travel Bag for Full Sized Toiletries, Travel Essentials (Black)

(9,847)
$37.99
Prime$6.49 delivery Fri, Jun 13