Run State-of-the-art LLMs on RTX | NVIDIA NIM x AnythingLLM

Tim Carambat March 25, 2025
Video Thumbnail
Tim Carambat Logo

Tim Carambat

@timcarambat

About

Software Engineer, Founder and CEO of Mintplex Labs. Creator of AnythingLLM and Senate/House Stock Watcher.

Video Description

Hey everyone! In this video, I'm thrilled to announce our exciting partnership with NVIDIA, showcasing their groundbreaking NIM technology paired with AnythingLLM. NVIDIA NIM provide a supercharged method to run state-of-the-art models on your RTX AI PCs, effortlessly enhancing your AI capabilities. With AnythingLLM, you can easily harness this power without any complicated setup – just bring your GPU and go! Join us as we explore NVIDIA NIM and the numerous advantages of integrating this cutting-edge technology with AnythingLLM. Discover how you can run powerful LLMs entirely on your local hardware, optimizing your AI inference and workflows like never before. AnythingLLM Docs https://docs.anythingllm.com/nvidia-nims/introduction Available NIM models: https://build.nvidia.com/search?q=chat-run-on-rtx NVIDIA NIM Developer Docs: TBD CHAPTERS 0:00 Introduction 1:00 What is NVIDIA NIM? 2:11 CES 2025 Announcement 3:14 What NIMs are available today? 4:18 Lets run a NIM on a 4090 5:00 NVIDIA NIM set up in AnythingLLM 6:40 Choosing your preferred LLM 8:00 Pulling in a model 8:30 Starting a NIM for inference 9:50 Lets use the model for chats, agents, and more! 13:00 Running Deepseek-R1 15:55 Summary of NVIDIA NIM w/AnythingLLM 16:30 Thanking everyone for supporting AnythingLLM

You May Also Like

RTX GPUs for AI Work

AI-recommended products based on this video

Loading...
EZDIY-FAB RTX 3000 Series 12 Pin to Dual 8 Pin PCIe Sleeved Extension Cable 300 MM- Connector for NVIDIA Ampere GEFORCE RTX 3060ti 3070 3080 FE Funder Edition- White

EZDIY-FAB RTX 3000 Series 12 Pin to Dual 8 Pin PCIe Sleeved Extension Cable 300 MM- Connector for NVIDIA Ampere GEFORCE RTX 3060ti 3070 3080 FE Funder Edition- White

(122)
$15.99
FREE delivery Mon, Feb 16 on your first order
Loading...
Gigabyte GeForce RTX 3090 Gaming OC 24G Graphics Card, 3X WINDFORCE Fans, 24GB 384-Bit GDDR6X, GV-N3090GAMING OC-24GD Video Card

Gigabyte GeForce RTX 3090 Gaming OC 24G Graphics Card, 3X WINDFORCE Fans, 24GB 384-Bit GDDR6X, GV-N3090GAMING OC-24GD Video Card

(57)
$5,149.06
Loading...
for 90MM Video Card Cooler Fan for GPU for MSI GeForce for RTX 3060 3070 3080 3090 3060Ti 3070TI for Gaming X Trio Graphics(1pcs)

for 90MM Video Card Cooler Fan for GPU for MSI GeForce for RTX 3060 3070 3080 3090 3060Ti 3070TI for Gaming X Trio Graphics(1pcs)

(0)
$38.13
FREE delivery Mar 19 - Apr 10
Loading...
95MM T129215SU Cooling Fan for ASUS ROG Strix for GeForce RTX 3060 3070 3080 3090 Ti Graphics Card CF1010U12S/D T129215BU(Black A-Fan T12)

95MM T129215SU Cooling Fan for ASUS ROG Strix for GeForce RTX 3060 3070 3080 3090 Ti Graphics Card CF1010U12S/D T129215BU(Black A-Fan T12)

(0)
$43.56
FREE delivery Mar 19 - Apr 10
Loading...
85MM for Cooling Fan for KFA2 Galax GTX 1660 Super TI RTX 2060 EX Graphics Card Cooler T129215SU GFY09215M12SPA

85MM for Cooling Fan for KFA2 Galax GTX 1660 Super TI RTX 2060 EX Graphics Card Cooler T129215SU GFY09215M12SPA

(0)
$29.13
FREE delivery Mar 19 - Apr 10
Loading...
for 85MM FDC10H12S9-C Cooling Fan for Lenovo Dell HP RTX 3060 3060TI OEM Graphics Card for Cooler FDC10H12D9-C(Black B-Fan)

for 85MM FDC10H12S9-C Cooling Fan for Lenovo Dell HP RTX 3060 3060TI OEM Graphics Card for Cooler FDC10H12D9-C(Black B-Fan)

(0)
$24.17
FREE delivery Mar 19 - Apr 10
Loading...
88MM PLD09210S12H Cooling Fan for ASUS for Dual RTX 2080 2070 2060 Super EVO Graphics Card Cooler T129215SU FDC10H12S9-C(Black A-Fan)

88MM PLD09210S12H Cooling Fan for ASUS for Dual RTX 2080 2070 2060 Super EVO Graphics Card Cooler T129215SU FDC10H12S9-C(Black A-Fan)

(0)
$23.22
FREE delivery Mar 19 - Apr 10
Loading...
95MM T129215SH for Cooling Fan for Gigabyte GTX 1650 for Gaming 1660 Ti RTX 2060 for Super 2070 WINDFORCE Graphics Card Cooler

95MM T129215SH for Cooling Fan for Gigabyte GTX 1650 for Gaming 1660 Ti RTX 2060 for Super 2070 WINDFORCE Graphics Card Cooler

(0)
$31.43
FREE delivery Mar 19 - Apr 10
Loading...
for 88MM T129215SU PLD09210S12HH for Cooling Fan for Gigabyte GTX 1650 1660 Ti for Super RTX 2060 Graphics Video Card Fans(Black A-Fan PLD)

for 88MM T129215SU PLD09210S12HH for Cooling Fan for Gigabyte GTX 1650 1660 Ti for Super RTX 2060 Graphics Video Card Fans(Black A-Fan PLD)

(0)
$21.53
FREE delivery Mar 19 - Apr 10
Loading...
EVA Storage Bag for GTX 1660 Super, RTX 2060 Super, 3060 Ti, and 3070 Graphics Cards, 40 x 36 x 5 cm, Compact Protective Case with Dust-Resistant Design, Black

EVA Storage Bag for GTX 1660 Super, RTX 2060 Super, 3060 Ti, and 3070 Graphics Cards, 40 x 36 x 5 cm, Compact Protective Case with Dust-Resistant Design, Black

(0)
$30.69
$0.88 delivery Mar 9 - 27
Loading...
MSI Gaming GeForce RTX 2060 Super 8GB GDRR6 256-bit HDMI/DP G-SYNC Turing Architecture Graphics Card (RTX 2060 Super Gaming)

MSI Gaming GeForce RTX 2060 Super 8GB GDRR6 256-bit HDMI/DP G-SYNC Turing Architecture Graphics Card (RTX 2060 Super Gaming)

(44)
$3,014.86
Loading...
RTX 2060 Super 8GB Placa De Video Gaming Graphics Card GDDR6 256Bit 8Pin PCI Express 3.0x16 RTX 2060 Super 8gb Game PC

RTX 2060 Super 8GB Placa De Video Gaming Graphics Card GDDR6 256Bit 8Pin PCI Express 3.0x16 RTX 2060 Super 8gb Game PC

(0)
$570.27
FREE delivery Apr 2 - 24
Loading...
GIGABYTE GeForce RTX 3070 Ti Gaming OC 8G Graphics Card, WINDFORCE 3X Cooling System, 8GB 256-bit GDDR6X, GV-N307TGAMING OC-8GD Video Card

GIGABYTE GeForce RTX 3070 Ti Gaming OC 8G Graphics Card, WINDFORCE 3X Cooling System, 8GB 256-bit GDDR6X, GV-N307TGAMING OC-8GD Video Card

(594)
$2,761.68
Loading...
Gigabyte AERO 16 Gaming & Entertainment Laptop (Intel i7-12700H 14-Core, 16.0" 60Hz 4K (3840x2400), GeForce RTX 3070 Ti, 32GB RAM, 4TB PCIe SSD, Backlit KB, WiFi, USB 3.2, Win 11 Pro)

Gigabyte AERO 16 Gaming & Entertainment Laptop (Intel i7-12700H 14-Core, 16.0" 60Hz 4K (3840x2400), GeForce RTX 3070 Ti, 32GB RAM, 4TB PCIe SSD, Backlit KB, WiFi, USB 3.2, Win 11 Pro)

(0)
$11,403.11
FREE delivery Jun 23 - Jul 3
Loading...
95MM for PLD10010S12HH Ventus Cooling Fan for MSI for RTX 3060 3070 Ti 2X OC Graphics Card Cooler(Black B-Fan)

95MM for PLD10010S12HH Ventus Cooling Fan for MSI for RTX 3060 3070 Ti 2X OC Graphics Card Cooler(Black B-Fan)

(0)
$23.88
FREE delivery Mar 19 - Apr 10
Loading...
EVA Graphics Card Storage Bag 1660 Super, for RTX 2060 Super, for RTX 3060 Ti 3070 40 x 36 x 5 cm, Black

EVA Graphics Card Storage Bag 1660 Super, for RTX 2060 Super, for RTX 3060 Ti 3070 40 x 36 x 5 cm, Black

(0)
$34.09
FREE delivery Mar 9 - 27
Loading...
ASUS TUF F16 16" WUXGA 165Hz Gaming Laptop, Intel i7-14650HX, NVIDIA GeForce RTX 5070 8GB, 16GB DDR5, 1TB SSD, Backlit Keyboard, Number Pad, IR Camera, Wi-Fi 6E, Win 11, Gray, 1TB Docking Station Set

ASUS TUF F16 16" WUXGA 165Hz Gaming Laptop, Intel i7-14650HX, NVIDIA GeForce RTX 5070 8GB, 16GB DDR5, 1TB SSD, Backlit Keyboard, Number Pad, IR Camera, Wi-Fi 6E, Win 11, Gray, 1TB Docking Station Set

(10)
$2,429.00
FREE delivery Feb 24 - Mar 5Only 9 left in stock.
Loading...
Lenovo LOQ 15.6" FHD 144Hz Gaming Laptop, Intel Core i5-12450HX, NVIDIA GeForce RTX 2050, 16GB DDR5, 2TB Storage (1TB SSD+1TB Docking Station Set), Number Pad, 720p Camera, Wi-Fi 6, Win 11, Gray

Lenovo LOQ 15.6" FHD 144Hz Gaming Laptop, Intel Core i5-12450HX, NVIDIA GeForce RTX 2050, 16GB DDR5, 2TB Storage (1TB SSD+1TB Docking Station Set), Number Pad, 720p Camera, Wi-Fi 6, Win 11, Gray

(2)
$1,369.00
FREE delivery Feb 24 - Mar 5
Loading...
MSI Stealth 18 HX AI 18” 240Hz QHD+ Gaming Laptop: Intel Ultra 9-275HX, NVIDIA Geforce RTX 5080, 32GB DDR5, 2TB NVMe SSD, Wi-Fi 7, Win 11 Home :Midnight Black A2XWIG-045US

MSI Stealth 18 HX AI 18” 240Hz QHD+ Gaming Laptop: Intel Ultra 9-275HX, NVIDIA Geforce RTX 5080, 32GB DDR5, 2TB NVMe SSD, Wi-Fi 7, Win 11 Home :Midnight Black A2XWIG-045US

(12)
$3,999.00$3,587.85
FREE delivery Sat, Jan 3
Loading...
GIGABYTE - AORUS Elite 16 Gaming Laptop - 165Hz 2560x1600 WQXGA - NVIDIA GeForce RTX 5070 - Intel Core Ultra 9 275HX - 1TB SSD with 32GB DDR5 RAM - Windows 11 Home AD (AORUS Elite 16 BWHC3USC64SH)

GIGABYTE - AORUS Elite 16 Gaming Laptop - 165Hz 2560x1600 WQXGA - NVIDIA GeForce RTX 5070 - Intel Core Ultra 9 275HX - 1TB SSD with 32GB DDR5 RAM - Windows 11 Home AD (AORUS Elite 16 BWHC3USC64SH)

(0)
$9,050.69
FREE delivery Aug 28 - Sep 8Only 3 left in stock.
Loading...
GIGABYTE GeForce RTX 3080 Ti Eagle OC 12G Graphics Card, 3X Windforce Fans, 12GB 384-bit GDDR6X, GV-N308TEAGLE OC-12GD Video Card

GIGABYTE GeForce RTX 3080 Ti Eagle OC 12G Graphics Card, 3X Windforce Fans, 12GB 384-bit GDDR6X, GV-N308TEAGLE OC-12GD Video Card

(24)
$1,069.00
Loading...
GIGABYTE AORUS GeForce RTX 3080 Ti Master 12G Graphics Card, Max Covered Cooling, 12GB 384-bit GDDR6X, GV-N308TAORUS M-12GD Video Card

GIGABYTE AORUS GeForce RTX 3080 Ti Master 12G Graphics Card, Max Covered Cooling, 12GB 384-bit GDDR6X, GV-N308TAORUS M-12GD Video Card

(44)
$5,301.22$1,169.99
FREE delivery Feb 26 - Mar 5
Loading...
95MM T129215SU Cooling Fan for ASUS ROG Strix for GeForce RTX 3060 3070 3080 3090 Ti Graphics Card CF1010U12S/D T129215BU(Black A-Fan T12)

95MM T129215SU Cooling Fan for ASUS ROG Strix for GeForce RTX 3060 3070 3080 3090 Ti Graphics Card CF1010U12S/D T129215BU(Black A-Fan T12)

(0)
$43.56
FREE delivery Mar 19 - Apr 10
Loading...
Skytech Archangel Gaming PC Desktop – AMD Ryzen 5 3600 3.6 GHz, NVIDIA RTX 3060, 1TB NVME SSD, 16GB DDR4 RAM 3200, 600W Gold PSU, 11AC Wi-Fi, Windows 11 Home 64-bit

Skytech Archangel Gaming PC Desktop – AMD Ryzen 5 3600 3.6 GHz, NVIDIA RTX 3060, 1TB NVME SSD, 16GB DDR4 RAM 3200, 600W Gold PSU, 11AC Wi-Fi, Windows 11 Home 64-bit

(31)
$2,657.01$2,656.90
FREE delivery Feb 24 - Mar 3
Loading...
Skytech Blaze 3.0 Gaming PC Desktop – Intel Core i5 12400F 2.5 GHz, NVIDIA RTX 3060, 500GB NVME SSD, 16GB DDR4 RAM 3200, 600W Gold PSU, 11AC Wi-Fi, Windows 11 Home 64-bit

Skytech Blaze 3.0 Gaming PC Desktop – Intel Core i5 12400F 2.5 GHz, NVIDIA RTX 3060, 500GB NVME SSD, 16GB DDR4 RAM 3200, 600W Gold PSU, 11AC Wi-Fi, Windows 11 Home 64-bit

(786)
$1,414.11
FREE delivery Thu, Feb 19Only 4 left in stock.
Loading...
Yeston RTX 3050 6GB GDDR6 Graphics Cards Nvidia pci Express 4.0 x8 Video Cards Desktop Computer PC Video Gaming Graphics Card

Yeston RTX 3050 6GB GDDR6 Graphics Cards Nvidia pci Express 4.0 x8 Video Cards Desktop Computer PC Video Gaming Graphics Card

(94)
$448.71$429.58
FREE delivery Feb 18 - 20
Loading...
Skytech Chronos 3 Gaming PC Desktop, Ryzen 7 9700X 3.8 GHz (5.5GHz Turbo), NVIDIA RTX 5060 8GB, 1TB NVMe SSD, 32GB DDR5 RAM 5200 RGB, 850W Gold ATX 3 PSU, 360mm ARGB AIO, Wi-Fi, Win 11

Skytech Chronos 3 Gaming PC Desktop, Ryzen 7 9700X 3.8 GHz (5.5GHz Turbo), NVIDIA RTX 5060 8GB, 1TB NVMe SSD, 32GB DDR5 RAM 5200 RGB, 850W Gold ATX 3 PSU, 360mm ARGB AIO, Wi-Fi, Win 11

(50)
$1,995.40
FREE delivery Jul 6 - 30
Loading...
Magnetic Nasal Strips Starter Kit: Comfortable Nasal Breathing Support for Sleep, Helps Reduce Snoring Noise, Includes 60 Tabs (30 Uses) with 4 Sizes

Magnetic Nasal Strips Starter Kit: Comfortable Nasal Breathing Support for Sleep, Helps Reduce Snoring Noise, Includes 60 Tabs (30 Uses) with 4 Sizes

(40)
$17.99
FREE delivery Wed, Mar 18 on your first orderOnly 7 left in stock.
1K+ bought in past month
Loading...
Environet Hydroponic Growing Kit, Self-Watering Mason Jar Herb Garden Starter Kit Indoor, Windowsill Herb Garden, Grow Your Own Herbs from Organic Seeds (Basil)

Environet Hydroponic Growing Kit, Self-Watering Mason Jar Herb Garden Starter Kit Indoor, Windowsill Herb Garden, Grow Your Own Herbs from Organic Seeds (Basil)

(786)
$69.15$33.51
FREE delivery Jun 13 - 17
Loading...
Herb Garden Planter Indoor Kit 21Pcs Kitchen Herb Garden Starter Kit Growing Kit Including Wooden Box Burlap Pots Soil Discs Gardening Tools Unique Easter Birthday Christmas Gift Ideas for Women Mom

Herb Garden Planter Indoor Kit 21Pcs Kitchen Herb Garden Starter Kit Growing Kit Including Wooden Box Burlap Pots Soil Discs Gardening Tools Unique Easter Birthday Christmas Gift Ideas for Women Mom

(0)
$30.99
FREE delivery Jun 19 - Jul 3
Loading...
Bonsai Starter Kit – 1x Bonsai Tree | Complete Indoor Starter Kit for Growing Plants with Bonsai Seeds, Tools & Planters – Gardening Gifts for Women & Men
Best Seller

Bonsai Starter Kit – 1x Bonsai Tree | Complete Indoor Starter Kit for Growing Plants with Bonsai Seeds, Tools & Planters – Gardening Gifts for Women & Men

(243)
$19.99
FREE delivery Sat, Jun 14 on your first order
300+ bought in past month