I built a private AI mini-cluster with Framework Desktop

Video Description

Can we build a private AI cluster in a mini rack with Framework's Desktop motherboard? Mentioned in this video: - Framework Desktop: https://frame.work/desktop - Level1Techs review of Framework Desktop: https://www.youtube.com/watch?v=L-xgMQ-7lW0 - Beowulf AI Cluster project: https://github.com/geerlingguy/beowulf-ai-cluster - Blog post with all test results: https://www.jeffgeerling.com/blog/2025/i-clustered-four-framework-mainboards-test-huge-llms - Other test results and benchmarks for the Framework Mainboard: https://github.com/geerlingguy/sbc-reviews/issues/80 - DeskPi RackMate T1 (Black - US): https://amzn.to/456iuXv - DeskPi RackMate T1 (Black - EU): https://www.amazon.de/dp/B0FBFDZD4C?th=1 - DeskPi RackMate 2U Mini ITX Tray: https://frame.work/products/deskpi-rackmate-10-inch-2u-mini-itx-shelf Framework sent the units used in this review, and DeskPi sent a mini rack and four rack trays to mount the Framework hardware. Neither company paid me, nor had any input into my testing or the production of the video. See my review video policies: https://github.com/geerlingguy/youtube?tab=readme-ov-file#review-videos Support me on Patreon: https://www.patreon.com/geerlingguy Sponsor me on GitHub: https://github.com/sponsors/geerlingguy Merch: https://www.redshirtjeff.com 2nd Channel: https://www.youtube.com/@GeerlingEngineering 3rd Channel: https://www.youtube.com/@Level2Jeff Contents: 00:00 - Desktop schmeshtop 00:32 - Nirav Patel explains the mini cluster 02:06 - Mainboard overview 04:16 - Single node performance 06:44 - AI and AMD mini-rant 07:47 - Single node AI benchmarks 09:03 - It's clusterin' time! 11:07 - A mini rack for the AI stack 14:27 - Large LLMs, clustered 16:14 - DeepSeek 671B and Llama 405B testing 18:02 - What about value?