Video 45: In the training loop, I am getting a RuntimeError.

[Hari-Shankar-Karthik]

import numpy as np
import matplotlib.pyplot as plt
import torch
from torch import nn

# Linear regression parameters
weight = 4.39
bias = -0.04
noise_std = 0.49
collection_std = 0.10
N = 50

# Synthesize the data
np.random.seed(42)
X = np.linspace(0, 1, num=N) + np.random.normal(loc=0, scale=collection_std, size=N)
y = weight * X + bias + np.random.normal(loc=0, scale=noise_std, size=N)

# Split the data into training and testing sets
# Here, we will not be splitting randomly
train_frac = 0.80
split_index = int(train_frac * X.shape[0])
X_train, X_test, y_train, y_test = X[:split_index], X[split_index:], y[:split_index], y[split_index:]

# Create the class for the linear regression model
class LinearRegressionModel(nn.Module):
    def __init__(self):
        super().__init__()
        self.weight = nn.Parameter(torch.randn(1, requires_grad=True))
        self.bias = nn.Parameter(torch.randn(1, requires_grad=True))
    
    def forward(self, x: torch.Tensor) -> torch.Tensor:
        return self.weight * x + self.bias
    
# Instantiate the model
torch.manual_seed(42)
model = LinearRegressionModel()

# Choose a loss function and an optimizer
loss_function = nn.L1Loss()
optimizer = torch.optim.SGD(model.parameters(), lr=0.001)

# Create the training loop
num_epochs = 1
for _ in range(num_epochs):
    # Set the model to training mode
    model.train()
    # Feed the data forward
    training_predictions = model(X_train)
    # Calculate the loss on the training data
    training_loss = loss_function(training_predictions, y_train)
    # Reset the optimizer's gradient to zero
    optimizer.zero_grad()
    # Perform backpropagation to calculate the gradient of the loss function w.r.t. the model's parameters
    training_loss.backward()
    # Move the parameters in the direction of steepest descent
    optimizer.step()

I am getting an error in the line training_predictions = model(X_train) in the feed-forward step of the training loop.
The error is RuntimeError: Can't call numpy() on Tensor that requires grad. Use tensor.detach().numpy() instead.

[Dhruvil-Joshi]

Ok, I found the error here!! It says that it requires a Tensor to process the training of model but you are passing a numpy ndarray, so its mismatching the conditions. You can do one thing, just convert the numpy ndarray to Tensor by adding this following line :

X = torch.from_numpy(X).unsqueeze(dim=1)

This will convert your numpy ndarray into a Tensor and will solve the problem.

But you output variable has size of 50, so just check it once whether its correct or not , As I think it must be of size 1

Thus, I have also changed in the line of tensor y as :

y = weight * X + bias + np.random.normal(loc=0, scale=noise_std, size=1)
instead of size = N.

If still any problem occurs, feel free to message me.

Hope this helps!!