Learn PyTorch in a day YouTube video corrections

[mrdbourke]

Hello hello!

The first~25 hours of the course are published as a single YouTube video here: https://youtu.be/Z_ikDlimN6A

Such a long video means there will likely be a few errors.

This discussion collects notes on them.

If you'd like to add one, feel free to leave a comment with a timestamp (such as "3:48:19") and what you think the correction should be (text explanations and code examples are welcome).

Corrections

• torch.mm != torch.matmul at [02:40:00] - I say these two functions are aliases of one another but this is not the case, torch.mm does not perform broadcasting. See more here: torch.mm != torch.matmul #391 thank you @AntonOfTheWoods

• Tensors and NumPy - If you make a tensor out of an array and then you change the array, the tensor will also change. I think that in your case (video time 3:38:50) the value of the tensor did not change because you changed the data type (from float 64 to float 32), that was the operation that created a new tensor in memory. From Rogelio Garcia.

• requires_grad=False by default - 5:09:14 requires_grad is False by default not True in the documentations. From Ameera Ali.

• Accumulating gradients - 6:29:12 I think it is accumulating the gradients because SDG is actually Mini-Batch Gradient Descent in Pytorch and the weights are going to be updated after the entire batch instead of after each training sample. From Ameera Ali.

• RuntimeError: DataLoader worker - 22:25:56 If you are running this video on your local machine and getting issues with multiple workers, try removing num_workers from train_dataloader_custom and test_dataloader_custom. See more here Video 142. Turning custom datasets into DataLoaders: "RuntimeError: DataLoader worker" #107.

[whydoubt]

Reproducibility - Some of the statements in the time range [03:49:25] - [03:50:45] seem to imply that the seed only affects the first torch.rand(...) call following torch.manual_seed(...). That is inaccurate, as may be illustrated by the following code

import torch
torch.manual_seed(42)
tensor_run1_A = torch.rand(2, 3)
tensor_run1_B = torch.rand(2, 3)
torch.manual_seed(42)
tensor_run2_A = torch.rand(2, 3)
tensor_run2_B = torch.rand(2, 3)
print(tensor_run1_A, tensor_run2_A, tensor_run1_A == tensor_run2_A, sep='\n')
print(tensor_run1_B, tensor_run2_B, tensor_run1_B == tensor_run2_B, sep='\n')

For this example, it is necessary to call torch.manual_seed() multiple times. However, I would suggest that in general use, torch.manual_seed() need only be called once (before any randomness is called for within a cell) for reproducibility.

[MoesiAndrew]

Hello Daniel,

It is Andrew from South Africa, on 03_pytorch_computer_vision chapter 3.3.Creating a training loop and training a model on batches of data, on line 17 of the code "y_pred = model_0(X)" is giving me an error here is the description of an error: "mat1 and mat2 shapes cannot be multiplied (32x784 and 748x10)"

[MoesiAndrew]

I managed to figure out what was the problem, I actually made a mistake instead of putting 784 I typed 748 when instantiating the model. But thank you anyway because you just gifted us with a wealth of free knowledge.

Kind regards
Andrew, all the way from South Africa

[MubasharaRehman]

I followed this video, code is exactly same but when i am trying this chunk of code,
........................................................................................................................

torch.manual_seed(42)

An Epoch is one loop through the data...

epochs = 1

TRAINING LOOP

Step 0 >> Loop through the data

for epoch in range(epochs):
model_0.train()

Step 1 >> fORWARD PASS

y_pred = model_0(X_train)

Step 2 >> CALCULATE THE LOSS

loss = loss_fn( y_pred, y_train )

Step 3 >> OPTIMIZER ZERO GRAD

optimizer.zero_grad()

Step 4 >> PERFORM BACKPROPAGATION ON THE LOSS WITH RESPECT TO THE PARAMETERS OF THE MODEL

loss.backward()

Step 5 >> STEP THE OPTIMIZER (PERFORMS GRADIENT DESCENT)

optimizer.step()
model_0.eval() # turns off gradient tracking

/////////////////////////////////////////////////////////////////

i get error on this line,
---> 12 loss = loss_fn( y_pred, y_train )

RuntimeError: Boolean value of Tensor with more than one value is ambiguous

I am new learner, kindly help me where is the mistake and how to fix it

[pritesh2000]

Check out what is y_pred, and y_train and its type and length if type and length isn't same it will cause error

[LyleZiegelmiller]

At 06:33:32 - you state that if we get to a gradient value of zero, the loss function will also be zero. Are you sure it will be zero, rather than just being a minimum? That f'(c) == 0 does not generally imply that f(c) == 0.

[LyleZiegelmiller]

In 03_pytorch_computer_vision_video.ipynb, "# Create a sinlge conv2d layer" has the word "single" misspelled.

[Samadarshi-Maity]

The 'Accuracy' module from torchmetrics requires: ...(task='multiclass', num_classes=num_classes) as compulsory arguments which was discussed at c.a. "13:54:12"