GPUs in Kubernetes for AI Workloads
DevOps & AI Toolkit
@devopstoolkitAbout
We want to help you learn the tools and the processes that you should be using and applying in your day-to-day job. We want to help you make decisions. What works well, what doesn't work, why you should choose one tool over the other, and how to get up-to-speed quickly. Which tool works the best for a given task? What should we explore in more depth, and what is a waste of time? This channel has AI & DevOps in the name because we believe that the only way forward is to combine different types of expertise, be it development, or operations, or AI, or anything else. Ultimately, we need to be able to develop, test, deploy, and operate our systems without friction caused by silos formed around distinct types of expertise. Hence, our focus is on bridging the gap by focusing on the topics that allow developers, operators, and everyone else to work together by adopting tools and processes that are relevant today and foster collaboration. Viktor Farcic & Darin Pope
Video Description
Today we dive into running AI models on Kubernetes with GPU support. Learn how to manage GPUs in Kubernetes clusters, create GPU nodes, and optimize resource usage without breaking the bank. We'll walk you through setting up a Google Cloud Kubernetes cluster (the same logic should apply to other Cloud providers), deploying AI models like Ollama's Llama2, and handling GPU partitioning. Watch now to master GPU-based AI workloads in Kubernetes! #Kubernetes #GPU #AI ▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬ Sponsor: CAST AI 🔗 https://cast.ai/devopstoolkit ▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬ Consider joining the channel: https://www.youtube.com/c/devopstoolkit/join ▬▬▬▬▬▬ 🔗 Additional Info 🔗 ▬▬▬▬▬▬ ➡ Transcript and commands: https://devopstoolkit.live/ai/unlock-the-power-of-gpus-in-kubernetes-for-ai-workloads ▬▬▬▬▬▬ 💰 Sponsorships 💰 ▬▬▬▬▬▬ If you are interested in sponsoring this channel, please visit https://devopstoolkit.live/sponsor for more information. Alternatively, feel free to contact me over Twitter or LinkedIn (see below). ▬▬▬▬▬▬ 👋 Contact me 👋 ▬▬▬▬▬▬ ➡ Twitter: https://twitter.com/vfarcic ➡ LinkedIn: https://www.linkedin.com/in/viktorfarcic/ ▬▬▬▬▬▬ 🚀 Other Channels 🚀 ▬▬▬▬▬▬ 🎤 Podcast: https://www.devopsparadox.com/ 💬 Live streams: https://www.youtube.com/c/DevOpsParadox ▬▬▬▬▬▬ ⏱ Timecodes ⏱ ▬▬▬▬▬▬ 00:00 AI Inference with GPUs 01:30 CAST AI (sponsor) 02:29 Using GPUs for AI Inference in Kubernetes
Essential Camera Gear for Travel
AI-recommended products based on this video
COTUBLR 31 Inch Computer Desk, Home Office Desk, Simple Modern Small Desk for Bedroom, Writing Desk with Storage Bag, Study Table for Students, Grey Oak
Hand Warmers Rechargeable, 10000mAh Electric Heated Gloves Power Bank Portable Graphene Handwarmers Pouch with 3 Levels & Double-Sided Heating for Hunting Camping Golf Xmas Gifts for Women Men Kids
MAXGROOM Bikini Trimmer Women, 3-in-1 Waterproof Electric Razors Shaver for Pubic Hair, Rechargeable Body Hair Trimmer for Underarm Leg Arm with LED Light and Travel Pouch
Waxed Canvas Multi-Purpose Waterpoof Tool Organizer Bags, Portable Carry On Pouch with 1 Small Detachable Tool pouch and 1 Zipper Pockets, 23.5" X 10.5"|ArmyGreen
ANXRE 71" Phone Tripod, NT79 Extendable Selfie Stick for Cell Phone＆Camera Photo Video Kits, Travel Tripod Stand with Remote for Video Recording, Phone Holder Tripod Compatible with Phone Camera Gopro
Kaiess 62" Tripod for iPhone, Selfie Stick Tripod & Phone Tripod Stand with Remote, Cell Phone Tripod for iPhone, Extendable Travel Tripod Compatible with iPhone 14/13/12 Pro Max/Android
EUCOS 62" Phone Tripod, Tripod for iPhone & Selfie Stick Tripod with Remote, Extendable Phone Tripod Stand & Travel Tripod, Solidest Cell Phone Tripod Compatible with iPhone/Android
Jhcztrk 17.3 Inch Portable Monitor HDMI Type-C Laptop Gaming Monitor, 1600X900 with Built-in Speakers and Tripod External Slim Travel Monitors Second Monitors for Mac Smartphone PS4/PS5/ Gaming
NEEWER a6700 L Plate Compatible with Sony Alpha 6700, Quick Switch Aluminum Extendable Side Plate & Anti Twist Baseplate Compatible with Arca Swiss QR System DJI RS 2 RSC 2 RS 3 Pro Gimbals, CA048
Anker USB C Charger Cable (6ft 100W, 2Pack), USB 2.0 Type C Fast Charging Cable for iPhone 15 / 15Pro / 15Plus / 15ProMax MacBook Pro 2020, iPad Pro 2020, iPad Air 4, Samsung Galaxy S23+/S23 Ultra ClimatePartner certified
![Anker USB C Cable, [2-Pack, 6 ft] Type C Charger Premium Nylon USB Cable, USB A to Type C Charging Cable Fast Charge for Samsung Galaxy S10 S10+ / Note 8, LG V20 and Other USB C Charger (Black)](https://m.media-amazon.com/images/I/71N7xhCJnoL._AC_UL960_FMwebp_QL65_.jpg)
Anker USB C Cable, [2-Pack, 6 ft] Type C Charger Premium Nylon USB Cable, USB A to Type C Charging Cable Fast Charge for Samsung Galaxy S10 S10+ / Note 8, LG V20 and Other USB C Charger (Black)
Anker Nano USB C Wall Charger,45W Fast Charging Smart Display Charger,with 180°Foldable Plug,Smart Recognition,Built-in Care Mode,for iPhone17/16/15 (Non-Battery,One USB-C Port,No Cable Included) ClimatePartner certified
eazpower for Magsafe Power Bank 10000mAh, Magnetic Battery Pack Wireless, 20W PD Fast Charging, Compact Portable Charger with Built-in Cables and Stand for iPhone 16/15/14/13/12/ Apple Watch/Airpods
