Local Ai Proxmox GPU Passthrough LXC with Docker Ultimate Guide

Video Description

UPDATED GUIDES These guides are meant to be followed in this order: ▶️ Ollama Openwebui Video https://youtu.be/Met9pEfxsF8 📝 Ollama Openwebui Article https://digitalspaceport.com/how-to-setup-an-ai-server-homelab-beginners-guides-ollama-and-openwebui-on-proxmox-lxc/ ▶️ Llamacpp Unsloth Video https://youtu.be/CBNbM-RZgIQ 📝 Llamacpp Unsloth Article https://digitalspaceport.com/how-to-setup-an-ai-server-homelab-beginners-guides-llama-cpp-on-proxmox-9-lxc/ ▶️ vLLM Video https://youtu.be/CBNbM-RZgIQ 📝 vLLM Article https://digitalspaceport.com/how-to-setup-vllm-local-ai-homelab-ai-server-beginners-guides/ Getting a stable functional Proxmox GPU virtualization setup for GPU Passthrough in for LXC's to a Proxmox host setup is a bit challenging, but very worth the effort. I show you every step for a complete proxmox installation, setting up LXC container and installing Docker into that. We use Nvidia Container Toolkit to enable seemless access to quad 3090 GPUs. Getting docker support is easy with this video and companion written guide so you can run multiple processes off a single or multiple GPUs all at the same time. QUAD 3090 AI HOME SERVER BUILD https://youtu.be/JN4EhaM7vyw GPU Rack Frame https://geni.us/GPU_Rack_Frame Supermicro H12ssl-i MOBO (better option vs mz32-ar0) https://geni.us/MBD_H12SSL-I-O Gigabyte MZ32-AR0 MOBO https://geni.us/mz32-ar0_motherboard AMD 7V13 (newer, faster vs 7702) https://geni.us/EPYC_7V13_CPU RTX 3090 24GB GPU (x4) https://geni.us/GPU3090 256GB (8x32GB) DDR4 2400 RAM https://geni.us/256GB_DDR4_RAM PCIe4 Risers (x4) https://geni.us/PCIe4_Riser_Cable AMD SP3 Air Cooler (easier vs water cooler) https://geni.us/EPYC_SP3_COOLER iCUE H170i water cooler https://geni.us/iCUE_H170i_Capellix (sTRX4 fits SP3 and retention kit comes with the CAPELLIX) CORSAIR HX1500i PSU https://geni.us/Corsair_HX1500iPSU 4i SFF-8654 to 4i SFF-8654 (x4, not needed for H12SSL-i) https://geni.us/SFF8654_to_SFF8654 ARCTIC MX4 Thermal Paste https://geni.us/Arctic_ThermalPaste Kritical Thermal GPU Pads https://geni.us/Kritical-Thermal-Pads HDD Rack Screws for Fans https://geni.us/HDD_RackScrews Be sure to 👍✅Subscribe✅👍 for more content like this! Join this channel https://www.youtube.com/@DigitalSpaceport/join Digital Spaceport Website https://digitalspaceport.com Patreon https://www.patreon.com/digitalspaceport Buy Me a Coffee https://buymeacoffee.com/digitalspaceport Please share this video to help spread the word and drop a comment below with your thoughts or questions. Thanks for watching! Chapters 0:00 LXC GPU Passthrough 0:50 Install Proxmox 5:57 Proxmox Configuration 7:08 Install NVIDIA Drivers Proxmox 12:42 Install Debian LXC with GPU Passthrough 18:12 Install Nvidia Container Toolkit 20:09 Install Docker and Dockge 21:39 Install Docker Containers On LXC 23:23 Testing GPU Passthrough ***** As an Amazon Associate I earn from qualifying purchases. When you click on links to various merchants on this site and make a purchase, this can result in this site earning a commission. Affiliate programs and affiliations include, but are not limited to, the eBay Partner Network. *****