Zero to Hero LLMs with M3 Max BEAST

Alex Ziskind โ€ข December 1, 2023
Video Thumbnail
Alex Ziskind Logo

Alex Ziskind

View Channel

About

Cracking code and curiosities, I turn caffeine-fueled late nights into bytes of wisdom. Welcome aboard! Iโ€™m Alex, with 20+ years of mastering the art of coding and now, the craft of content creation. Subscribe to my channel and join me in unraveling software enigmas with in-depth tech reviews tailored for developers, with a dash of humor. Emojis suck ๐Ÿš€ https://www.youtube.com/@azisk?sub_confirmation=1

Video Description

M3 Max is a Machine Learning BEAST. So I took it for a spin with some LLM's running locally. I also show how to gguf quantizations with llama.cpp Temperature/fan on your Mac: https://www.tunabellysoftware.com/tgpro/index.php?fpr=alex (affiliate link) Run Windows on a Mac: https://prf.hn/click/camref:1100libNI (affiliate) Use COUPON: ZISKIND10 ๐Ÿ›’ Gear Links ๐Ÿ›’ * ๐Ÿ๐Ÿ’ฅ New MacBook Air M1 Deal: https://amzn.to/3S59ID8 * ๐Ÿ’ป๐Ÿ”„ Renewed MacBook Air M1 Deal: https://amzn.to/45K1Gmk * ๐ŸŽงโšก Great 40Gbps T4 enclosure: https://amzn.to/3JNwBGW * ๐Ÿ› ๏ธ๐Ÿš€ My nvme ssd: https://amzn.to/3YLEySo * ๐Ÿ“ฆ๐ŸŽฎ My gear: https://www.amazon.com/shop/alexziskind ๐ŸŽฅ Related Videos ๐ŸŽฅ * ๐ŸŒ— RAM torture test on Mac - https://youtu.be/l3zIwPgan7M * ๐Ÿ› ๏ธ Set up Conda on Mac - https://youtu.be/2Acht_5_HTo * ๐Ÿ‘จโ€๐Ÿ’ป 15" MacBook Air | developer's dream - https://youtu.be/A1IOZUCTOkM * ๐Ÿค– INSANE Machine Learning on Neural Engine - https://youtu.be/Y2FOUg_jo7k * ๐Ÿ’ป M2 MacBook Air and temps - https://youtu.be/R7F-TxEukdY * ๐Ÿ’ฐ This is what spending more on a MacBook Pro gets you - https://youtu.be/iLHrYuQjKPU * ๐Ÿ› ๏ธ Developer productivity Playlist - https://www.youtube.com/playlist?list=PLPwbI_iIX3aQCRdFGM7j4TY_7STfv2aXX ๐Ÿ”— AI for Coding Playlist: ๐Ÿ“š - https://www.youtube.com/playlist?list=PLPwbI_iIX3aSlUmRtYPfbQHt4n0YaX0qw Timestamps 00:00 Intro 00:40 Build from scratch - manual 09:44 Bonus script - automated 11:21 LM Studio - one handed Repo https://github.com/ggerganov/llama.cpp/ Commands //assuming you already have a conda environment set up, and dev tools installed (see videos above for instructions) *Part 1 - manual* brew install git-lfs git lfs install git clone https://github.com/ggerganov/llama.cpp cd llama.cpp pip install -r requirements.txt make git clone https://huggingface.co/teknium/OpenHermes-2.5-Mistral-7B openhermes-7b-v2.5 mv openhermes-7b-v2.5 models/ python3 convert.py ./models/openhermes-7b-v2.5 --outfile ./models/openhermes-7b-v2.5/ggml-model-f16.gguf --outtype f16 ./quantize ./models/openhermes-7b-v2.5/ggml-model-f16.gguf ./models/openhermes-7b-v2.5/ggml-model-q8_0.gguf q8_0 ./quantize ./models/openhermes-7b-v2.5/ggml-model-f16.gguf ./models/openhermes-7b-v2.5/ggml-model-q4_k.gguf q4_k ./batched-bench ./models/openhermes-7b-v2.5/ggml-model-f16.gguf 4096 0 99 0 2048 128,512 1,2,3,4 ./server -m models/openhermes-7b-v2.5/ggml-model-q4_k.gguf --port 8888 --host 0.0.0.0 --ctx-size 10240 --parallel 4 -ngl 99 -n 512 *Part 2 - auto* bash -c "$(curl -s https://ggml.ai/server-llm.sh)" ๐Ÿ’ป MacBooks in this video M2 Max 16" MacBook Pro 64GB/2TB โ€” โ€” โ€” โ€” โ€” โ€” โ€” โ€” โ€” โค๏ธ SUBSCRIBE TO MY YOUTUBE CHANNEL ๐Ÿ“บ Click here to subscribe: https://www.youtube.com/@AZisk?sub_confirmation=1 โ€” โ€” โ€” โ€” โ€” โ€” โ€” โ€” โ€” Join this channel to get access to perks: https://www.youtube.com/channel/UCajiMK_CY9icRhLepS8_3ug/join #m3max #macbook #macbookpro โ€” โ€” โ€” โ€” โ€” โ€” โ€” โ€” โ€” ๐Ÿ“ฑ ALEX ON X: https://twitter.com/digitalix

You May Also Like

Upgrade Your Gaming Today

AI-recommended products based on this video

Loading...
Asus ROG Strix 1000W Gold Aura Edition (Fully Modular Power Supply, 80+ Gold Certified, ATX 3.0, Cybenetics Lambda A+ Certification, PCIe Gen 5.0 Ready, Axial-tech Fan, Aura Sync, 10-Year Warranty)

Asus ROG Strix 1000W Gold Aura Edition (Fully Modular Power Supply, 80+ Gold Certified, ATX 3.0, Cybenetics Lambda A+ Certification, PCIe Gen 5.0 Ready, Axial-tech Fan, Aura Sync, 10-Year Warranty)

(38)
$267.83$262.70
FREE delivery Oct 9 - 10
Loading...
100W USB C Universal Laptop Charger Fit for Lenovo MacBook HP Dell Acer Asus Thinkpad Google MSI LG Samsung Huawei, ROG Ally, Steam Deck, Chromebook Computer and All USB-C Devices

100W USB C Universal Laptop Charger Fit for Lenovo MacBook HP Dell Acer Asus Thinkpad Google MSI LG Samsung Huawei, ROG Ally, Steam Deck, Chromebook Computer and All USB-C Devices

(19)
$25.99$24.69
FREE delivery Tue, Jun 17 on your first order
100+ bought in past month
Loading...
Logitech M185 Wireless Mouse, 2.4GHz with USB Mini Receiver, 12-Month Battery Life, 1000 DPI Optical Tracking, Ambidextrous, Compatible with PC, Mac, Laptop - Black

Logitech M185 Wireless Mouse, 2.4GHz with USB Mini Receiver, 12-Month Battery Life, 1000 DPI Optical Tracking, Ambidextrous, Compatible with PC, Mac, Laptop - Black

(43,809)
$14.99$11.99
FREE delivery Mon, Sep 22 on your first order
1K+ bought in past month
Loading...
Logitech G305 LIGHTSPEED Wireless Gaming Mouse, Hero 12K Sensor, 12,000 DPI, Lightweight, 6 Programmable Buttons, 250h Battery Life, On-Board Memory, PC/Mac - Black

Logitech G305 LIGHTSPEED Wireless Gaming Mouse, Hero 12K Sensor, 12,000 DPI, Lightweight, 6 Programmable Buttons, 250h Battery Life, On-Board Memory, PC/Mac - Black

(21,223)
$44.98
FREE delivery Tue, Oct 14
1K+ bought in past month
Loading...
Logitech K400 Plus Wireless Touch TV Keyboard With Easy Media Control and Built-in Touchpad, HTPC Keyboard for PC-connected TV, Windows, Android, Chrome OS, Laptop, Tablet - Black
Best Seller

Logitech K400 Plus Wireless Touch TV Keyboard With Easy Media Control and Built-in Touchpad, HTPC Keyboard for PC-connected TV, Windows, Android, Chrome OS, Laptop, Tablet - Black

(38,516)
$39.98$35.98
FREE delivery Thu, Sep 4
500+ bought in past month
Loading...
Logitech G203 Wired Gaming Mouse, 8,000 DPI, Rainbow Optical Effect LIGHTSYNC RGB, 6 Programmable Buttons, On-Board Memory, Screen Mapping, PC/Mac Computer and Laptop Compatible - Black
Best Seller

Logitech G203 Wired Gaming Mouse, 8,000 DPI, Rainbow Optical Effect LIGHTSYNC RGB, 6 Programmable Buttons, On-Board Memory, Screen Mapping, PC/Mac Computer and Laptop Compatible - Black

(17,871)
$27.98$22.38
FREE delivery Tue, Oct 14 on your first order
500+ bought in past month
Loading...
New SteelSeries Arctis Nova Pro for Xbox Multi-System Gaming Headset - Premium Hi-Fi Drivers - Hi-Res Audio - 360ยฐ Spatial - GameDAC Gen 2 - Quad-DAC - ClearCast Gen 2 Mic - Xbox, PC, PS5/PS4, Switch

New SteelSeries Arctis Nova Pro for Xbox Multi-System Gaming Headset - Premium Hi-Fi Drivers - Hi-Res Audio - 360ยฐ Spatial - GameDAC Gen 2 - Quad-DAC - ClearCast Gen 2 Mic - Xbox, PC, PS5/PS4, Switch

(517)
$209.41$122.65
PrimeFREE delivery Sat, Jun 14
Loading...
CORSAIR iCUE Link XD5 RGB Elite LCD Pump-Reservoir Unit - D5 PWM Pump - 480x480 IPS LCD Screen - 22 Addressable RGB LEDs - 440ml Nylon Reservoir - White

CORSAIR iCUE Link XD5 RGB Elite LCD Pump-Reservoir Unit - D5 PWM Pump - 480x480 IPS LCD Screen - 22 Addressable RGB LEDs - 440ml Nylon Reservoir - White

(17)
$379.99
FREE delivery Sun, Aug 24
Loading...
CORSAIR iCUE Link XC7 RGB Elite CPU Water Block - Transparent Flow Chamber - 24 RGB LEDs - Fits Intelยฎ LGA 1700, AMDยฎ AM5 and Older - White

CORSAIR iCUE Link XC7 RGB Elite CPU Water Block - Transparent Flow Chamber - 24 RGB LEDs - Fits Intelยฎ LGA 1700, AMDยฎ AM5 and Older - White

(56)
$149.99$125.33
FREE delivery Sun, Aug 24
Loading...
CORSAIR Hydro X Series iCUE Link XH405i Custom Cooling Kit โ€“ Hardline Water Cooling Loop โ€“ XC7 Elite CPU Water Block โ€“ XD5 Elite D5 Pump Res โ€“ XR5 360mm Radiator โ€“ 3X QX120 RGB Fans

CORSAIR Hydro X Series iCUE Link XH405i Custom Cooling Kit โ€“ Hardline Water Cooling Loop โ€“ XC7 Elite CPU Water Block โ€“ XD5 Elite D5 Pump Res โ€“ XR5 360mm Radiator โ€“ 3X QX120 RGB Fans

(235)
$879.99
FREE delivery Sun, Aug 24
Loading...
Western Digital 2TB WD Blue SA510 SATA Internal Solid State Drive SSD - SATA III 6 Gb/s, 2.5"/7mm, Up to 560 MB/s - WDS200T3B0A

Western Digital 2TB WD Blue SA510 SATA Internal Solid State Drive SSD - SATA III 6 Gb/s, 2.5"/7mm, Up to 560 MB/s - WDS200T3B0A

(74,910)
$169.97
FREE delivery Tue, Jun 17
50+ bought in past month
Loading...
Razer BlackShark V2 X Gaming Headset: 50mm Drivers - Cardioid Mic - Memory Foam Cushion - Lightweight- Comfortable, Noise Isolating Earcups - for PC, PS4, PS5, Switch - 3.5mm Audio Jack - Black ECOLOGO

Razer BlackShark V2 X Gaming Headset: 50mm Drivers - Cardioid Mic - Memory Foam Cushion - Lightweight- Comfortable, Noise Isolating Earcups - for PC, PS4, PS5, Switch - 3.5mm Audio Jack - Black ECOLOGO

(17,168)
$47.97$43.17
FREE delivery Sat, Dec 27
3K+ bought in past month
Loading...
Asus ROG Strix 1000W Gold Aura Edition (Fully Modular Power Supply, 80+ Gold Certified, ATX 3.0, Cybenetics Lambda A+ Certification, PCIe Gen 5.0 Ready, Axial-tech Fan, Aura Sync, 10-Year Warranty)

Asus ROG Strix 1000W Gold Aura Edition (Fully Modular Power Supply, 80+ Gold Certified, ATX 3.0, Cybenetics Lambda A+ Certification, PCIe Gen 5.0 Ready, Axial-tech Fan, Aura Sync, 10-Year Warranty)

(38)
$267.83$262.70
FREE delivery Oct 9 - 10
Loading...
230W Charger for Lenovo Legion Slim Pro 5 5P 7 S7 C7 Y720 Y730 Y740 Y920 Y530 Y540 Y545 R720 ThinkPad P1 P16 P16v P15 P17 T15g P50 P50s P51 P52 P53 P70 P71 P72 P73 Laptop ideacentre/ThinkStation AIO

230W Charger for Lenovo Legion Slim Pro 5 5P 7 S7 C7 Y720 Y730 Y740 Y920 Y530 Y540 Y545 R720 ThinkPad P1 P16 P16v P15 P17 T15g P50 P50s P51 P52 P53 P70 P71 P72 P73 Laptop ideacentre/ThinkStation AIO

(10)
$47.99
FREE delivery Sat, Jun 21
Loading...
New Slim 170W AC Charger Fit for Lenovo Legion Slim 5 5i Yoga Pro 9i ThinkPad P1 P16 P16v P53 P70 P71 P72 P73 IdeaPad Pro 5i 5 3 LOQ 16 15 ADL170SCC3A 20V 8.5A Laptop Power Supply Adapter Cord

New Slim 170W AC Charger Fit for Lenovo Legion Slim 5 5i Yoga Pro 9i ThinkPad P1 P16 P16v P53 P70 P71 P72 P73 IdeaPad Pro 5i 5 3 LOQ 16 15 ADL170SCC3A 20V 8.5A Laptop Power Supply Adapter Cord

(3)
$36.89
FREE delivery Sat, Jun 21
Loading...
170W AC Charger for Lenovo ThinkPad P1 P50 P51 P52 P53 P70 P71 P72 P73 P16 P16v P17 P15 T15g P15v T15p X1 Extreme Legion Y520 IdeaPad Pro 5 Yoga Slim 7 Laptop ThinkStation ThinkCentre IdeaCentre AIO

170W AC Charger for Lenovo ThinkPad P1 P50 P51 P52 P53 P70 P71 P72 P73 P16 P16v P17 P15 T15g P15v T15p X1 Extreme Legion Y520 IdeaPad Pro 5 Yoga Slim 7 Laptop ThinkStation ThinkCentre IdeaCentre AIO

(8)
$35.99
FREE delivery Sun, Jun 22
Loading...
135W USB C AC Adapter Charger for Lenovo ThinkPad Z16 P16s P15s P14s E16 E15 E14 T16 Yoga/Slim Pro 9 7 6 7i 9i Book 9 ThinkBook 16 14p 14 15 14s IdeaPad Slim/Pro 9 7 5 Legion 9 Laptop ADL135YSDC3A

135W USB C AC Adapter Charger for Lenovo ThinkPad Z16 P16s P15s P14s E16 E15 E14 T16 Yoga/Slim Pro 9 7 6 7i 9i Book 9 ThinkBook 16 14p 14 15 14s IdeaPad Slim/Pro 9 7 5 Legion 9 Laptop ADL135YSDC3A

(1)
$59.99
FREE delivery Sun, Jun 22