Ray at Scale: Apple's Approach to Elastic GPU Management | Ray Summit 2024

Video Description

As Ray's ecosystem expands, efficient GPU resource management becomes crucial for scaling AI/ML workloads. In this session, Apple's Weiwei Yang and Abin Shahab unveil their innovative approach to building a multi-tenancy ready platform based on Ray, tackling common challenges like GPU fragmentation, low utilization, and compromised SLAs. Yang and Shahab delve into the intricacies of their queuing and GPU quota management system, powered by Apache YuniKorn. They explore advanced techniques for achieving resource fairness, GPU preemption, and gang scheduling across diverse Ray workloads. This talk offers valuable insights for organizations looking to optimize their GPU resource management and enhance the scalability and efficiency of their AI/ML operations. -- Interested in more? - Watch the full Day 1 Keynote: https://youtu.be/jwZHJthQvXo - Watch the full Day 2 Keynote https://youtu.be/Lury2ad6KG8 -- 🔗 Connect with us: - Subscribe to our YouTube channel: https://www.youtube.com/@anyscale - Twitter: https://x.com/anyscalecompute - LinkedIn: https://linkedin.com/company/joinanyscale/ - Website: https://www.anyscale.com