PyTorch fundamentals - What is squarebrackets changes?

[Anti4ris98]

Hello, My name is Ilya.
I am on 20th video of "Learn PyTorch. Become a Deep Learning Engineer. Get Hired. course."
And I understand that square brackets change amount of dimensions, but I could just add a new pair of square brackets, and I don`t know what changes in "Background".

[aldragan0]

As it's currently written it's not that clear to me what do you mean with 'what changes in "Background"'.
Are you talking about how the internal representation of the data changes?

If so, here's a StackOverflow link I've found useful: https://stackoverflow.com/questions/49724954/how-are-pytorchs-tensors-implemented

[pritesh2000]

explain further with example...

[tal7aouy]

Hi Ilya! Understanding how dimensions and shapes of tensors work in PyTorch is crucial for building and debugging neural networks.

Tensor Dimensions and Shapes

In PyTorch, tensors are multi-dimensional arrays, and their dimensions (also called ranks) are determined by the number of square brackets you use. Here are a few examples to illustrate:

1. 0-D Tensor (Scalar):

   scalar = torch.tensor(5)
   print(scalar.shape)  # Output: torch.Size([])

2. 1-D Tensor (Vector):

   vector = torch.tensor([5, 6, 7])
   print(vector.shape)  # Output: torch.Size([3])

3. 2-D Tensor (Matrix):

   matrix = torch.tensor([[5, 6, 7], [8, 9, 10]])
   print(matrix.shape)  # Output: torch.Size([2, 3])

4. 3-D Tensor:

   tensor3d = torch.tensor([[[5, 6, 7], [8, 9, 10]], [[1, 2, 3], [4, 5, 6]]])
   print(tensor3d.shape)  # Output: torch.Size([2, 2, 3])

Adding Dimensions with Square Brackets

When you add square brackets around a tensor, you are essentially creating a new dimension:

1. Adding a Dimension to a Scalar:

   scalar = torch.tensor(5)
   tensor1d = torch.tensor([scalar])
   print(tensor1d.shape)  # Output: torch.Size([1])

2. Adding a Dimension to a Vector:

   vector = torch.tensor([5, 6, 7])
   tensor2d = torch.tensor([vector])
   print(tensor2d.shape)  # Output: torch.Size([1, 3])

3. Adding a Dimension to a Matrix:

   matrix = torch.tensor([[5, 6, 7], [8, 9, 10]])
   tensor3d = torch.tensor([matrix])
   print(tensor3d.shape)  # Output: torch.Size([1, 2, 3])

Using unsqueeze and view for Dimension Manipulation

In addition to directly adding square brackets, PyTorch provides functions like unsqueeze and view to manipulate tensor dimensions more flexibly.

1. unsqueeze Method:
   The unsqueeze method adds a new dimension of size 1 at the specified position.

   vector = torch.tensor([5, 6, 7])
   tensor2d = vector.unsqueeze(0)
   print(tensor2d.shape)  # Output: torch.Size([1, 3])
   
   tensor3d = vector.unsqueeze(1)
   print(tensor3d.shape)  # Output: torch.Size([3, 1])

2. view Method:
   The view method reshapes the tensor to the specified shape.

   matrix = torch.tensor([[5, 6, 7], [8, 9, 10]])
   tensor1d = matrix.view(-1)  # Flatten the matrix
   print(tensor1d.shape)  # Output: torch.Size([6])
   
   tensor3d = matrix.view(2, 3, 1)
   print(tensor3d.shape)  # Output: torch.Size([2, 3, 1])

Background Operations

In the background, adding dimensions or reshaping tensors involves modifying the tensor's metadata (like its shape and stride) rather than copying the actual data. This means that operations like unsqueeze and view are generally very efficient because they do not require additional memory allocation for the data itself.

Understanding these concepts will help you manipulate tensors effectively and ensure that they are in the correct shape for your models. If you have more specific examples or questions, feel free to ask!