Fine-tune Mixtral 8x7B (MoE) on Custom Data - Step by Step Guide

Video Description

In this tutorial, we will walk through a step by step tutorial on how to fine tune Mixtral MoE from Mistral AI on your own dataset. LINKS: Colab (free T4 will not work): http://tinyurl.com/2hfk2fru Mistral 7B fine-tune video: https://youtu.be/lCZRwrRvrWg @AI-Makerspace Want to Follow: 🦾 Discord: https://discord.com/invite/t4eYQRUcXB ▶️️ Subscribe: https://www.youtube.com/@engineerprompt?sub_confirmation=1 Want to Support: ☕ Buy me a Coffee: https://ko-fi.com/promptengineering |🔴 Support my work on Patreon: https://www.patreon.com/PromptEngineering Need Help? 📧 Business Contact: [email protected] 💼Consulting: https://calendly.com/engineerprompt/consulting-call Join this channel to get access to perks: https://www.youtube.com/channel/UCDq7SjbgRKty5TgGafW8Clg/join Timestamps: [00:00] Introduction [00:57] Prerequisites and Tools [01:52] Understanding the Dataset [03:35] Data Formatting and Preparation [06:16] Loading the Base Model [09:55] Setting Up the Training Configuration [13:22] Fine-Tuning the Model [16:28] Evaluating the Model Performance All Interesting Videos: Everything LangChain: https://www.youtube.com/playlist?list=PLVEEucA9MYhOu89CX8H3MBZqayTbcCTMr Everything LLM: https://youtube.com/playlist?list=PLVEEucA9MYhNF5-zeb4Iw2Nl1OKTH-Txw Everything Midjourney: https://youtube.com/playlist?list=PLVEEucA9MYhMdrdHZtFeEebl20LPkaSmw AI Image Generation: https://youtube.com/playlist?list=PLVEEucA9MYhPVgYazU5hx6emMXtargd4z