How the VLLM inference engine works?

Video Description

In this video, we understand how VLLM works. We look at a prompt and understand what exactly happens to the prompt as it travels through the VLLM inference engine. We cover the following concepts: - Prefill stage - Decoding stage - Paged Attention - Continuous batching - GPU VRAM Allocation Code file: https://colab.research.google.com/drive/1rb2rgy5ve6eNje2iKop_k1JBVSBflhPy?usp=sharing