Fine-tuning Multimodal Embeddings on Custom Text-Image Pairs
About
No channel description available.
Latest Posts
Video Description
🤝 Work with me: https://aibuilder.academy/yt/W4s6b2ZM6kI 🚀 Ship AI apps in weeks, not months: https://aibuilder.academy/courses/yt/W4s6b2ZM6kI In this video, I walk through how to fine-tune CLIP on my YouTube titles and thumbnails using the Sentence Transformers Python library. Resources: 📰 Blog: https://medium.com/towards-data-science/fine-tuning-multimodal-embedding-models-bf007b1c5da5?source=friends_link&sk=df620c0d7aa4959d566771fe09766c41 💻 GitHub Repo: https://github.com/ShawhinT/YouTube-Blog/tree/main/multimodal-ai/4-ft-mm-embeddings 🤗 Model: https://huggingface.co/shawhin/clip-title-thumbnail-embeddings 💿 Dataset: https://huggingface.co/datasets/shawhin/yt-title-thumbnail-pairs References: [1] https://youtu.be/YOvxh_ma5qE [2] arXiv:2103.00020 [cs.CV] [3] arXiv:1705.00652 [cs.CL] [4] https://youtu.be/hOLBrIjRAj4 Intro - 0:00 Multimodal Embeddings - 0:44 0-shot Use Cases - 2:30 Limitations of CLIP - 3:50 Fine-tuning CLIP - 5:14 Step 1: Gather training data - 6:46 Step 2: Preprocess data - 15:20 Step 3: Define evals - 17:20 Step 4: Fine-tune model - 19:22 Step 5: Evaluate model - 26:04
AI & ML Must-Have Essentials
AI-recommended products based on this video
AocBook 15.6'' FHD Laptop, Intel N95, Nvidia GTX 1060 4GB, 32GB DDR4 RAM, M.2 SSD, Sleek Notebook with Type-C, HDMI, RJ45 Ethernet, Backlit Keyboard, Fingerprint (32GB DDR4 | 1TB SSD)
acer Nitro 50 N50-620-UA91 Gaming Desktop | 11th Gen Intel Core i5-11400F 6-Core Processor | NVIDIA GeForce GTX 1650 | 8GB DDR4 | 512GB NVMe M.2 SSD | Intel Wi-Fi 6 AX201 | Keyboard and Mouse
Dell Inspiron 7640 Laptop - 16.0-inch 16:10 2.5K (2560x1600) Display, Intel Core Ultra 7-155H Processor, 16GB DDR5 RAM, 1TB SSD, Windows 11 Home, Onsite Service & Migrate - Ice Blue EPEAT
Dell Inspiron 3535 Laptop - 15.6 inch FHD (1920 x 1080) 120Hz Display, AMD Ryzen 5-7520U Processor, 8GB LPDDR5 RAM, 512GB SSD, AMD Radeon Graphics, Windows 11 Home, Migrate Service - Carbon Black EPEAT
45W 65W AC Adapter Laptop Charger for Dell Inspiron 15 14 13 17 3000 5000 7000 Series 5558 5559 Charger Latitude E6440 E6430 3520 3420 XPS 13 14 9333 9350 Notebook Dell Computer Power Supply Cord
