How to Turn Your AMD GPU into a Local LLM Beast: A Beginner's Guide with ROCm

Video Description

RX 7600 XT on Amazon (affiliate): https://locally.link/kEJG LM Studio: https://lmstudio.ai/rocm Products provided by Gigabyte Those of us with NVIDIA GPUs, particularly ones with enough VRAM, have been able to run large language models locally for quite a while. I did a guide last year showing you how to run Vicuna locally, but that really only worked on NVIDIA GPUs. Support has improved a little, including running it all on your CPU instead, but with the launch of AMD’s ROCm software, it’s now not only possible to run large language models locally, but insanely easy. Now if you are just here for the guide, skip here. If you want to know how this works so damn well, stick around! BUY AN OSRTT (Open Source Response Time Tool): https://osrtt.com Become a Youtube Member and get sponsor free videos and access to our private Discord chat: https://www.youtube.com/techteamgb/join Locally Links - Global Short Linking for Creators: https://locally.link/techteamgb Use referral code "techteamgb20" when signing up! Patreon: https://www.patreon.com/techteamgb Donations: https://streamlabs.com/techteamgb OverclockersUK Affiliate link: http://techteamgb.co.uk/ocuk Discord! https://discord.gg/NxFzaWy As an Amazon Associate I earn from qualifying purchases, using the links below or other Amazon affiliate links here. Want a cool T-Shirt or hoodie? https://teespring.com/en-GB/stores/techteamgb Private Internet Access (VPN): http://techteamgb.co.uk/PIA NordVPN: https://nordvpn.org/techteamgb HUMBLE BUNDLE: https://www.humblebundle.com/monthly?partner=techteamgb Zen Internet - My (UK) ISP: https://locally.link/b3DC WEB HOSTING: http://techteamgb.co.uk/domaincom Check out our awesome website! http://techteamgb.co.uk My Monitor - Philips EVNIA 8600 QD-OLED: https://locally.link/H33q - My PC - AMAZON AFFILIATE LINKS - 5900X on Amazon: https://techteamgb.co.uk/5900x 32GB Corsair 3000MHz on Amazon: https://techteamgb.co.uk/cors32gbrgb GBT X570 Master on Amazon: https://techteamgb.co.uk/gbtx570master RTX 2080 on Amazon: https://techteamgb.co.uk/RTX2080 H100i Pro on Amazon: https://techteamgb.co.uk/h100ipro WD 1TB NVME on Amazon: https://techteamgb.co.uk/wdbl1tb Patriot VP4100 on Amazon: https://techteamgb.co.uk/vp4100 Sabrent Rocket Q 4TB on Amazon: https://techteamgb.co.uk/rocketq4 If you are interested in contacting us, then please email: [email protected] and we will respond as soon as possible.