Python Machine Learning From Scratch Batching

Video Description

Okay, here is your edited paragraph: Alright, here is what we are going to do. We are going to add batching real quick. It is super late, but I want to see if we can get batching into the code today. In earlier streams, we built a deep learning framework from scratch using only NumPy, which is helpful for making loops and math a lot easier to write and read. Without NumPy, you would need several nested for loops for some of the stuff we do in just one line with NumPy. Anyway, thanks to July for suggesting the MNIST dataset, which actually worked, even though we got some divide by zero warnings that do not break our code but are worth fixing later. Right now, our model just loops over all our samples in one big batch, so we need to change our train method to do batch training. This means breaking the data into smaller groups, say, batches of 32, and training on those one at a time. We can use random sampling with NumPy to pick batch indices, and then get the feature and label batches out of those indices. It is better to pass the batch size into the train function instead of using global variables. To find the number of batches per epoch, we just divide the total number of samples by the batch size, making sure it is an integer so the loop works. Note that with random batches, we might miss some samples every epoch, since the batch picks are not guaranteed to cover all the data. This is not perfect, but it is good enough for now. We solved float division turning our count into a non-integer by using double slashes in Python. Finally, we saw an error with the input shape, which we fixed for now, but we will have to revisit how we separate training and test data later. The main point here is we are learning how batching works under the hood by writing it ourselves, even though in real projects we would use something like PyTorch for hardware acceleration and more features. We now have batching, even though it is not perfect and not all data is used in each epoch with this setup. Our next step is to improve the batching so every sample gets trained on equally, and I will add a to-do at the top of the code for that tomorrow.