Matrices.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": [],
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/SrivinaySridhar/MLT_workshop/blob/main/Matrices.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Matrices\n"
      ],
      "metadata": {
        "id": "H729Vxqsdh5_"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Import"
      ],
      "metadata": {
        "id": "iQLGiDT32DHY"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import numpy as np"
      ],
      "metadata": {
        "id": "NTY1DDOO6aQS"
      },
      "execution_count": 1,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Matrix as a `NumPy` array\n",
        "\n",
        "Everything in `NumPy` is an array. A matrix is also an array. Let us create a simple matrix:\n",
        "\n",
        "$$\n",
        "\\textbf{M} = \\begin{bmatrix}\n",
        "1 & 2 & 3\\\\\n",
        "4 & 5 & 6\\\\\n",
        "7 & 8 & 9\n",
        "\\end{bmatrix}\n",
        "$$\n",
        "\n",
        "In `NumPy`:"
      ],
      "metadata": {
        "id": "QchFdrAp6c0E"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "M = np.array([[1, 2, 3],\n",
        "              [4, 5, 6],\n",
        "              [7, 8, 9]])\n",
        "M"
      ],
      "metadata": {
        "id": "DURrZ-wb6426",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "f4c69b14-7611-4f72-eb63-ac1bda75a369"
      },
      "execution_count": 2,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[1, 2, 3],\n",
              "       [4, 5, 6],\n",
              "       [7, 8, 9]])"
            ]
          },
          "metadata": {},
          "execution_count": 2
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Adding two matrices\n",
        "\n",
        "Let us now add the following matrices:\n",
        "\n",
        "$$\n",
        "\\textbf{A} = \\begin{bmatrix}\n",
        "1 & 2\\\\\n",
        "3 & 4\n",
        "\\end{bmatrix}, \\textbf{B} = \\begin{bmatrix}\n",
        "5 & 6\\\\\n",
        "7 & 8\n",
        "\\end{bmatrix}\n",
        "$$\n",
        "\n",
        "then,\n",
        "\n",
        "$$\n",
        "\\textbf{C} = \\textbf{A} + \\textbf{B}  = \\begin{bmatrix}\n",
        "6 & 8\\\\\n",
        "10 & 12\n",
        "\\end{bmatrix}\n",
        "$$\n",
        "\n",
        "In `NumPy`:"
      ],
      "metadata": {
        "id": "L7h4kYjw7XKe"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "A = np.array([[1, 2], [3, 4]])\n",
        "B = np.array([[5, 6], [7, 8]])\n",
        "C = A + B\n",
        "C"
      ],
      "metadata": {
        "id": "rOV92c-Y7SJT",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "79a61000-a5e1-4ad4-8cd7-e3d202a0d539"
      },
      "execution_count": 3,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[ 6,  8],\n",
              "       [10, 12]])"
            ]
          },
          "metadata": {},
          "execution_count": 3
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Scaling a matrix\n",
        "\n",
        "Scaling a matrix is nothing but element-wise multiplication:\n",
        "\n",
        "$$\n",
        "\\textbf{M} = \\begin{bmatrix}\n",
        "1 & 2 & 3\\\\\n",
        "4 & 5 & 6\\\\\n",
        "7 & 8 & 9\n",
        "\\end{bmatrix}\n",
        "$$\n",
        "\n",
        "then,\n",
        "\n",
        "$$\n",
        "3 \\textbf{M} = \\begin{bmatrix}\n",
        "3 & 6 & 9\\\\\n",
        "12 & 15 & 18\\\\\n",
        "21 & 24 & 27\n",
        "\\end{bmatrix}\n",
        "$$\n",
        "\n",
        "In `NumPy`:"
      ],
      "metadata": {
        "id": "m_pCQww198OD"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "M = np.array([[1, 2, 3], [4, 5, 6], [7, 8, 9]])\n",
        "3 * M"
      ],
      "metadata": {
        "id": "jM_o92DpBoYe",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "4199a4f3-d9c9-4134-e398-0d9a3d22d8ef"
      },
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[ 3,  6,  9],\n",
              "       [12, 15, 18],\n",
              "       [21, 24, 27]])"
            ]
          },
          "metadata": {},
          "execution_count": 4
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Element-wise multiplication of matrices\n",
        "\n",
        "Consider two matrices:\n",
        "\n",
        "$$\n",
        "\\textbf{A} = \\begin{bmatrix}\n",
        "1 & 2\\\\\n",
        "3 & 4\n",
        "\\end{bmatrix}, \\textbf{B} = \\begin{bmatrix}\n",
        "5 & 6\\\\\n",
        "7 & 8\n",
        "\\end{bmatrix}\n",
        "$$\n",
        "\n",
        "The element-wise product is given by $\\textbf{A} \\odot \\textbf{B}$:\n",
        "\n",
        "$$\n",
        "\\textbf{C} = \\textbf{A} \\odot \\textbf{B} = \\begin{bmatrix}\n",
        "5 & 12\\\\\n",
        "21 & 32\n",
        "\\end{bmatrix}\n",
        "$$\n",
        "\n",
        "In `NumPy`:"
      ],
      "metadata": {
        "id": "hmE4xEve7y0n"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "A = np.array([[1, 2], [3, 4]])\n",
        "B = np.array([[5, 6], [7, 8]])\n",
        "C = A * B\n",
        "C"
      ],
      "metadata": {
        "id": "USCv3DX18gbD",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "c80f9b62-5b9f-45b9-da9b-032094d025e9"
      },
      "execution_count": 5,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[ 5, 12],\n",
              "       [21, 32]])"
            ]
          },
          "metadata": {},
          "execution_count": 5
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Element-wise functions of matrices\n",
        "\n",
        "Given a matrix, we sometimes would want to apply a function to every element of the matrix. We will consider two examples."
      ],
      "metadata": {
        "id": "V3iWGy8-8xho"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Example-1\n",
        "\n",
        "For example, we may want to take the absolute value of all the elements. Let us say $f(x) = |x|$, then:\n",
        "\n",
        "$$\n",
        "\\mathbf{A} = \\begin{bmatrix}\n",
        "-1 & 2\\\\\n",
        "-3 & -4\n",
        "\\end{bmatrix}\n",
        "$$\n",
        "\n",
        "then:\n",
        "\n",
        "$$\n",
        "\\begin{bmatrix}\n",
        "f(-1) & f(2)\\\\\n",
        "f(-3) & f(-4)\n",
        "\\end{bmatrix} =\n",
        "\\begin{bmatrix}\n",
        "1 & 2\\\\\n",
        "3 & 4\n",
        "\\end{bmatrix}\n",
        "$$\n",
        "\n",
        "In `NumPy`, this becomes:"
      ],
      "metadata": {
        "id": "gVkkmUzNvppN"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "A = np.array([[-1, 2], [-3, -4]])\n",
        "np.abs(A)"
      ],
      "metadata": {
        "id": "8svqjjFy-QG9",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "5c4ad415-36f6-431a-c7b2-94d89110a879"
      },
      "execution_count": 6,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[1, 2],\n",
              "       [3, 4]])"
            ]
          },
          "metadata": {},
          "execution_count": 6
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Example-2\n",
        "\n",
        "We might want to square each element of the matrix. If $\\textbf{A}$ is a matrix, then $\\textbf{B}$ could be defined element-wise as follows:\n",
        "\n",
        "$$\n",
        "B_{ij} = A_{ij}^2\n",
        "$$\n",
        "\n",
        "Let us compute $\\mathbf{B}$ for the following matrix:\n",
        "\n",
        "$$\n",
        "\\mathbf{A} = \\begin{bmatrix}\n",
        "1 & \\sqrt{2}\\\\\n",
        "\\sqrt{3} & 2\n",
        "\\end{bmatrix}\n",
        "$$\n",
        "\n",
        "In `NumPy`:"
      ],
      "metadata": {
        "id": "FYiMrhwx-hXT"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "A = np.array([[1,np.sqrt(2)], [np.sqrt(3), 2]])\n",
        "B = A ** 2\n",
        "B"
      ],
      "metadata": {
        "id": "uqym9th4-ywH",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "262f10d9-5a9c-428f-a61c-1eaac5401f8d"
      },
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[1., 2.],\n",
              "       [3., 4.]])"
            ]
          },
          "metadata": {},
          "execution_count": 7
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Transpose of a matrix\n",
        "\n",
        "Given a matrix $\\textbf{M}$:\n",
        "\n",
        "$$\n",
        "\\textbf{M} = \\begin{bmatrix}\n",
        "1 & 2 & 3\\\\\n",
        "4 & 5 & 6\n",
        "\\end{bmatrix}\n",
        "$$\n",
        "\n",
        "then, its transpose $\\textbf{M}^{T}$ is:\n",
        "\n",
        "$$\n",
        "\\textbf{M}^{T} = \\begin{bmatrix}\n",
        "1 & 4\\\\\n",
        "2 & 5\\\\\n",
        "3 & 6\n",
        "\\end{bmatrix}\n",
        "$$\n",
        "\n",
        "In `NumPy`:"
      ],
      "metadata": {
        "id": "-Pfh-LgH_AHi"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "M = np.array([[1, 2, 3], [4, 5, 6]])\n",
        "# np.transpose(M)\n",
        "M.T"
      ],
      "metadata": {
        "id": "9uBXpMH3_5pw",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "a8a8ebb5-2a21-4f5c-dde2-f7478c6b2bbb"
      },
      "execution_count": 8,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[1, 4],\n",
              "       [2, 5],\n",
              "       [3, 6]])"
            ]
          },
          "metadata": {},
          "execution_count": 8
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Shape and dimension of a matrix\n",
        "\n",
        "Matrices are \"two dimensional\" arrays. So all matrices in `NumPy` have array-dimension equal to two. The shape of the `NumPy` array gives what we usually call the dimension of the matrix in the linear algebra sense.\n",
        "\n",
        "Explore these two ideas for:\n",
        "\n",
        "$$\n",
        "\\mathbf{M} = \\begin{bmatrix}\n",
        "1 & 2 & 3\\\\\n",
        "4 & 5 & 6\\\\\n",
        "\\end{bmatrix}\n",
        "$$\n",
        "\n",
        "In `NumPy`:"
      ],
      "metadata": {
        "id": "wDsjZjevz1eg"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "M = np.array([[1, 2, 3], [4, 5, 6]])\n",
        "\n",
        "# rows X cols\n",
        "print(f\"Rows: {M.shape[0]}, Columns: {M.shape[1]}\")\n",
        "\n",
        "# number of axes\n",
        "print(f\"Axes: {M.ndim}\")"
      ],
      "metadata": {
        "id": "8WPQSzfzgSO1",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "d248965f-262e-461f-cb4b-f0ac889a4b7f"
      },
      "execution_count": 10,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Rows: 2, Columns: 3\n",
            "Axes: 2\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Vectors as matrices\n",
        "\n",
        "Each vector can be viewed as a matrix. Column vectors are matrices of shape $(d, 1)$. Row vectors are matrices of shape $(1, d)$. Let us look at how NumPy treats both these cases:\n"
      ],
      "metadata": {
        "id": "4wKzNg0afnIo"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "x = np.array([1, 2, 3])\n",
        "print('x is a vector of shape', x.shape)\n",
        "\n",
        "x_col_mat = np.expand_dims(x, axis = 1)\n",
        "print('x_col_mat is a matrix of shape', x_col_mat.shape)\n",
        "\n",
        "x_row_mat = np.expand_dims(x, axis = 0)\n",
        "print('x_row_mat is a matrix of shape', x_row_mat.shape)"
      ],
      "metadata": {
        "id": "v2Tqn0AGfzKA",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "8b4368d9-94a0-4022-ba02-019594595f26"
      },
      "execution_count": 13,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "x is a vector of shape (3,)\n",
            "x_col_mat is a matrix of shape (3, 1)\n",
            "x_row_mat is a matrix of shape (1, 3)\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Products involving matrices and vectors\n",
        "\n",
        "We will look at the following products:\n",
        "- matrix - matrix\n",
        "- matrix - vector\n",
        "- vector - matrix\n",
        "- vector - vector"
      ],
      "metadata": {
        "id": "GpQXZMCve8Wf"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Product of two matrices\n",
        "\n",
        "Given two matrices:\n",
        "\n",
        "$$\n",
        "\\textbf{A} = \\begin{bmatrix}\n",
        "1 & 2 & 3\\\\\n",
        "4 & 5 & 6\n",
        "\\end{bmatrix}, \\textbf{B} = \\begin{bmatrix}\n",
        "6 & 7\\\\\n",
        "8 & 9\\\\\n",
        "10 & 11\n",
        "\\end{bmatrix}\n",
        "$$\n",
        "\n",
        "then,\n",
        "\n",
        "$$\n",
        "\\textbf{C} = \\textbf{A} \\times \\textbf{B} = \\begin{bmatrix}\n",
        "52 & 58\\\\\n",
        "124 & 139\n",
        "\\end{bmatrix}\n",
        "$$\n",
        "\n",
        "In `NumPy`:"
      ],
      "metadata": {
        "id": "ugLU7PhgA548"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "A = np.array([[1, 2, 3], [4, 5, 6]])\n",
        "B = np.array([[6, 7], [8, 9], [10, 11]])\n",
        "print(A.shape, B.shape)\n",
        "C = A @ B\n",
        "C"
      ],
      "metadata": {
        "id": "sbiOJrIzBkFg",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "2caabe7f-63b7-4954-bc12-435f85aeb84b"
      },
      "execution_count": 14,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "(2, 3) (3, 2)\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[ 52,  58],\n",
              "       [124, 139]])"
            ]
          },
          "metadata": {},
          "execution_count": 14
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Product of a matrix and a (column) vector\n",
        "\n",
        "Given the matrix $\\mathbf{A}$ and the vector $\\mathbf{x}$:\n",
        "\n",
        "$$\n",
        "\\mathbf{A} = \\begin{bmatrix}\n",
        "1 & 2 & 3\\\\\n",
        "4 & 5 & 6\\\\\n",
        "7 & 8 & 9\n",
        "\\end{bmatrix}, \\mathbf{x} = \\begin{bmatrix}\n",
        "6\\\\\n",
        "7\\\\\n",
        "8\n",
        "\\end{bmatrix}\n",
        "$$\n",
        "\n",
        "The product $\\mathbf{Ax}$ is given by:\n",
        "\n",
        "$$\n",
        "\\mathbf{C} = \\mathbf{A x} = \\begin{bmatrix}\n",
        "44\\\\\n",
        "107\\\\\n",
        "170\n",
        "\\end{bmatrix}\n",
        "$$\n",
        "\n",
        "In `NumPy`:"
      ],
      "metadata": {
        "id": "zdFPst7KCtgl"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "A = np.array([[1, 2, 3], [4, 5, 6], [7, 8, 9]])\n",
        "x = np.array([6, 7, 8])\n",
        "A @ x"
      ],
      "metadata": {
        "id": "F9tQwGumDfgv",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "56f872c3-2539-4a2c-abcb-e14652932aad"
      },
      "execution_count": 15,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([ 44, 107, 170])"
            ]
          },
          "metadata": {},
          "execution_count": 15
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Product of a (row) vector and a matrix\n",
        "\n",
        "Given the matrix $\\mathbf{A}$ and the vector $\\mathbf{x}$:\n",
        "\n",
        "$$\n",
        "\\mathbf{A} = \\begin{bmatrix}\n",
        "1 & 2 & 3\\\\\n",
        "4 & 5 & 6\\\\\n",
        "7 & 8 & 9\n",
        "\\end{bmatrix}, \\mathbf{x} = \\begin{bmatrix}\n",
        "6\\\\\n",
        "7\\\\\n",
        "8\n",
        "\\end{bmatrix}\n",
        "$$\n",
        "\n",
        "The product $\\mathbf{x}^T \\mathbf{A}$ is given by:\n",
        "\n",
        "$$\n",
        "\\mathbf{C} = \\mathbf{x}^T \\mathbf{A} = \\begin{bmatrix}\n",
        "90 & 111 & 132\n",
        "\\end{bmatrix}\n",
        "$$\n",
        "\n",
        "In `NumPy`:"
      ],
      "metadata": {
        "id": "czO_GwynKHWQ"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "A = np.array([[1, 2, 3], [4, 5, 6], [7, 8, 9]])\n",
        "x = np.array([6, 7, 8])\n",
        "x @ A"
      ],
      "metadata": {
        "id": "C_n2CZRhKTio",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "730f6dcc-4598-4f50-d98d-3c4833fd56d4"
      },
      "execution_count": 16,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([ 90, 111, 132])"
            ]
          },
          "metadata": {},
          "execution_count": 16
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### (Inner) Product of a (row) vector and a (column) vector\n",
        "\n",
        "The product of a row vector and a column vector is nothing but the usual dot product:\n",
        "\n",
        "$$\n",
        "\\mathbf{x}^T = \\begin{bmatrix}\n",
        "1 & 2 & 3\n",
        "\\end{bmatrix}, \\quad\n",
        "\\mathbf{y} = \\begin{bmatrix}\n",
        "4\\\\\n",
        "5\\\\\n",
        "6\n",
        "\\end{bmatrix}\n",
        "$$\n",
        "\n",
        "The product $\\mathbf{x}^T \\mathbf{y}$ is then:\n",
        "\n",
        "$$\n",
        "\\mathbf{x}^T \\mathbf{y} = 32\n",
        "$$\n",
        "\n",
        "In `NumPy`:"
      ],
      "metadata": {
        "id": "d0impiXDL6_s"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "x = np.array([1, 2, 3])\n",
        "y = np.array([4, 5, 6])\n",
        "x @ y"
      ],
      "metadata": {
        "id": "Wch5wXY3MZol",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "693c5589-4b8d-4395-c946-f8cf8e6df917"
      },
      "execution_count": 17,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "32"
            ]
          },
          "metadata": {},
          "execution_count": 17
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### (Outer) Product of a (column) vector and a (row) vector\n",
        "\n",
        "The product of a column vector and a row vector is an outer product:\n",
        "\n",
        "$$\n",
        "\\mathbf{x} = \\begin{bmatrix}\n",
        "1\\\\\n",
        "2\\\\\n",
        "3\n",
        "\\end{bmatrix}, \\quad\n",
        "\\mathbf{y} = \\begin{bmatrix}\n",
        "4 & 5 & 6\n",
        "\\end{bmatrix}\n",
        "$$\n",
        "\n",
        "The product $\\mathbf{x} \\mathbf{y}^T$ is then:\n",
        "\n",
        "$$\n",
        "\\mathbf{x} \\mathbf{y}^T = \\begin{bmatrix}\n",
        "4 & 5 & 6\\\\\n",
        "8 & 10 & 12\\\\\n",
        "12 & 15 & 18\n",
        "\\end{bmatrix}\n",
        "$$\n",
        "\n",
        "In `NumPy`:"
      ],
      "metadata": {
        "id": "xtZ4gUhdMrvM"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "x = np.array([1, 2, 3])\n",
        "y = np.array([4, 5, 6])\n",
        "x_new = np.expand_dims(x, axis = 1)\n",
        "y_new = np.expand_dims(y, axis = 0)\n",
        "x_new @ y_new"
      ],
      "metadata": {
        "id": "p5a1jU1cNDVm",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "f55ae585-0ea0-47e0-f274-e24278ef9c77"
      },
      "execution_count": 18,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[ 4,  5,  6],\n",
              "       [ 8, 10, 12],\n",
              "       [12, 15, 18]])"
            ]
          },
          "metadata": {},
          "execution_count": 18
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Matrix of zeros\n",
        "\n",
        "In many algorithms, we might have to initialize a matrix with zeros. For example, consider a $2 \\times 4$ matrix:\n",
        "\n",
        "$$\n",
        "\\mathbf{M} = \\begin{bmatrix}\n",
        "0 & 0 & 0 & 0\\\\\n",
        "0 & 0 & 0 & 0\n",
        "\\end{bmatrix}\n",
        "$$\n",
        "\n",
        "In `NumPy`:"
      ],
      "metadata": {
        "id": "KShBdBZcQpTN"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "M = np.zeros((2, 4))\n",
        "M"
      ],
      "metadata": {
        "id": "x38VjAKRRAFk",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "20c4f951-fe97-4617-b9dc-cd69d2367ea5"
      },
      "execution_count": 19,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[0., 0., 0., 0.],\n",
              "       [0., 0., 0., 0.]])"
            ]
          },
          "metadata": {},
          "execution_count": 19
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Matrix of ones\n",
        "\n",
        "Similar to a matrix of zeros, we can come up with a matrix of ones.\n",
        "\n",
        "$$\n",
        "\\mathbf{M} = \\begin{bmatrix}\n",
        "1 & 1\\\\\n",
        "1 & 1\\\\\n",
        "1 & 1\n",
        "\\end{bmatrix}\n",
        "$$\n",
        "\n",
        "In `NumPy`:"
      ],
      "metadata": {
        "id": "hEg3EbveRWyc"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "M = np.ones((3, 2))\n",
        "M"
      ],
      "metadata": {
        "id": "kdApYPy7RlUd",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "5ebc08e7-e817-4432-e7b9-295bae2a6117"
      },
      "execution_count": 20,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[1., 1.],\n",
              "       [1., 1.],\n",
              "       [1., 1.]])"
            ]
          },
          "metadata": {},
          "execution_count": 20
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Identity matrix\n",
        "\n",
        "Often, we might have to deal with identity matrices. A $3 \\times 3$ identity matrix is as follows:\n",
        "\n",
        "$$\n",
        "\\mathbf{I} = \\begin{bmatrix}\n",
        "1 & 0 & 0\\\\\n",
        "0 & 1 & 0\\\\\n",
        "0 & 0 & 1\n",
        "\\end{bmatrix}\n",
        "$$\n",
        "\n",
        "In `NumPy`:"
      ],
      "metadata": {
        "id": "hhHfmOEiSknK"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "np.eye(3)"
      ],
      "metadata": {
        "id": "sTLClyKSS8s-",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "5bf594b0-8d12-40de-f8bc-bf3966b15504"
      },
      "execution_count": 21,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[1., 0., 0.],\n",
              "       [0., 1., 0.],\n",
              "       [0., 0., 1.]])"
            ]
          },
          "metadata": {},
          "execution_count": 21
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Diagonal matrices\n",
        "\n",
        "Another special kind of matrix. Let us create the following matrix:\n",
        "\n",
        "$$\n",
        "\\mathbf{D} = \\begin{bmatrix}\n",
        "1 & 0 & 0 & 0\\\\\n",
        "0 & 2 & 0 & 0\\\\\n",
        "0 & 0 & 3 & 0\\\\\n",
        "0 & 0 & 0 & 4\n",
        "\\end{bmatrix}\n",
        "$$\n",
        "\n",
        "In `NumPy`:"
      ],
      "metadata": {
        "id": "gsSwBuSExQX1"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "np.diag([1, 2, 3, 4])"
      ],
      "metadata": {
        "id": "fLZ8y1ODxgq6",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "681fe5c1-db2d-4d03-94d9-8f8c1f979dcf"
      },
      "execution_count": 22,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[1, 0, 0, 0],\n",
              "       [0, 2, 0, 0],\n",
              "       [0, 0, 3, 0],\n",
              "       [0, 0, 0, 4]])"
            ]
          },
          "metadata": {},
          "execution_count": 22
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Indexing and Slicing\n",
        "\n",
        "Just like lists in Python, `NumPy` arrays can be indexed and sliced. Slicing is useful if we want to work with a portion of an array. We will look at some examples."
      ],
      "metadata": {
        "id": "LYEO0xG8Tmdo"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Example-1: Row-slice\n",
        "\n",
        "We will extract the third row of the matrix $\\mathbf{M}$:\n",
        "\n",
        "$$\n",
        "\\mathbf{M} = \\begin{bmatrix}\n",
        "1 & 2\\\\\n",
        "3 & 4\\\\\n",
        "5 & 6\\\\\n",
        "7 & 8\\\\\n",
        "9 & 10\n",
        "\\end{bmatrix}\n",
        "$$\n",
        "\n",
        "\n",
        "\n",
        "In `NumPy`:"
      ],
      "metadata": {
        "id": "jjVMaWjETs1R"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "M = np.array([[1, 2], [3, 4], [5, 6], [7, 8], [9, 10]])\n",
        "M[2, :]"
      ],
      "metadata": {
        "id": "J1rnADwuTtXz",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "1a33607a-c4e6-4b44-d771-679c0a4475e8"
      },
      "execution_count": 23,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([5, 6])"
            ]
          },
          "metadata": {},
          "execution_count": 23
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Example-2: Column slice\n",
        "\n",
        "Let us now extract the second column of the following matrix:\n",
        "\n",
        "$$\n",
        "\\mathbf{M} = \\begin{bmatrix}\n",
        "1 & 2 & 3\\\\\n",
        "4 & 5 & 6\\\\\n",
        "7 & 8 & 9\n",
        "\\end{bmatrix}\n",
        "$$\n",
        "\n",
        "In `NumPy`:"
      ],
      "metadata": {
        "id": "Sg0PsBYSTtsy"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "M = np.array([[1, 2, 3], [4, 5, 6], [7, 8, 9]])\n",
        "M[:, 1]"
      ],
      "metadata": {
        "id": "6VC5H1v4Tvp9",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "26769c54-232c-45a9-ebd9-c40c7118a4d0"
      },
      "execution_count": 24,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([2, 5, 8])"
            ]
          },
          "metadata": {},
          "execution_count": 24
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Example-3: Submatrix slice\n",
        "\n",
        "Now, we want to extract the $2 \\times 2$ submatrix colored in blue from $M$:\n",
        "\n",
        "$$\n",
        "\\mathbf{M} = \\begin{bmatrix}\n",
        "1 & 2 & 3 & 4\\\\\n",
        "5 & 6 & \\color{blue}7 & \\color{blue}8\\\\\n",
        "9 & 10 & \\color{blue}{11} & \\color{blue}{12}\\\\\n",
        "13 & 14 & 15 & 16\n",
        "\\end{bmatrix}\n",
        "$$\n",
        "\n",
        "In `NumPy`:"
      ],
      "metadata": {
        "id": "STtPO2rjTvzq"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "M = np.arange(1, 17).reshape(4, 4)\n",
        "M[1:3, 2:4]"
      ],
      "metadata": {
        "id": "18Ktzj0nTyOx",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "f1538e6b-bb4c-4930-c230-c2556cc85d17"
      },
      "execution_count": 26,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[ 7,  8],\n",
              "       [11, 12]])"
            ]
          },
          "metadata": {},
          "execution_count": 26
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Matrix algebra\n",
        "\n",
        "There are several important matrix operations that we will list down here. The `np.linalg` module helps us perform these operations effortlessly."
      ],
      "metadata": {
        "id": "USo33Hw9nSPP"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Rank, Inverse\n",
        "\n",
        "We can get the rank of a matrix and its inverse as follows:"
      ],
      "metadata": {
        "id": "F0OVX6CtiF70"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "M = np.array([[1, 2, 3], [4, 5, 6], [7, 8, 8]])\n",
        "np.linalg.matrix_rank(M)"
      ],
      "metadata": {
        "id": "DTskqJVZiMuP",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "84034ae2-e027-48c2-e6fa-acf26fab4a1f"
      },
      "execution_count": 32,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "3"
            ]
          },
          "metadata": {},
          "execution_count": 32
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "np.linalg.inv(M)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "s9BhgNzgcT-T",
        "outputId": "e58852e5-a73a-4ef9-fd6d-12cc27919d07"
      },
      "execution_count": 33,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[-2.66666667,  2.66666667, -1.        ],\n",
              "       [ 3.33333333, -4.33333333,  2.        ],\n",
              "       [-1.        ,  2.        , -1.        ]])"
            ]
          },
          "metadata": {},
          "execution_count": 33
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Pseuodoinverse\n",
        "\n",
        "The pseudoinverse $\\mathbf{M}^{\\dagger}$ of a matrix $\\mathbf{M}$ with real entries satisfies the following properties:\n",
        "\n",
        "- $\\mathbf{M} \\mathbf{M}^{\\dagger} \\mathbf{M} = \\mathbf{M}$\n",
        "- $\\mathbf{M}^{\\dagger} \\mathbf{M} \\mathbf{M} = \\mathbf{M}^{\\dagger}$\n",
        "- $\\mathbf{M} \\mathbf{M}^{\\dagger}$ is symmetric\n",
        "- $\\mathbf{M}^{\\dagger} \\mathbf{M}$ is symmetric\n",
        "\n",
        "Let us go ahead and verify this for the following matrix:\n",
        "\n",
        "$$\n",
        "\\mathbf{M} = \\begin{bmatrix}\n",
        "1 & 2 & 3\\\\\n",
        "3 & 6 & 9\n",
        "\\end{bmatrix}\n",
        "$$\n",
        "\n",
        "Note: For a complete list of properties of the pseudoinverse, check out this [link](https://en.wikipedia.org/wiki/Moore%E2%80%93Penrose_inverse#Definition)."
      ],
      "metadata": {
        "id": "P77YDapdn1eW"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "M = np.arange(1, 17).reshape(4, 4)\n",
        "np.linalg.pinv(M)"
      ],
      "metadata": {
        "id": "vHcx2d9HnWf6",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "3e8d08f4-afb5-430a-801f-a019cd847016"
      },
      "execution_count": 35,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[-0.285 , -0.145 , -0.005 ,  0.135 ],\n",
              "       [-0.1075, -0.0525,  0.0025,  0.0575],\n",
              "       [ 0.07  ,  0.04  ,  0.01  , -0.02  ],\n",
              "       [ 0.2475,  0.1325,  0.0175, -0.0975]])"
            ]
          },
          "metadata": {},
          "execution_count": 35
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Eigenvalues and Eigenvectors\n",
        "\n",
        "Given a matrix $\\mathbf{M}$ let us find its eigenvalues and eigenvectors:\n",
        "\n",
        "$$\n",
        "\\mathbf{M} = \\begin{bmatrix}\n",
        "1 & 0 & -3\\\\\n",
        "0 & 5 & 2\\\\\n",
        "-3 & 2 & 8\n",
        "\\end{bmatrix}\n",
        "$$"
      ],
      "metadata": {
        "id": "6v1KRPi3sqqk"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "M = np.array([[1, 0, -3], [0, 5, 2], [-3, 2, 8]])\n",
        "eigval, eigvec = np.linalg.eigh(M)\n",
        "print(f\"Eigenvalues: {eigval}\")\n",
        "print(f\"Eigenvectors: {eigvec}\") # Each column corresponds to an eigenvector"
      ],
      "metadata": {
        "id": "LkH6ayZ7s6dZ",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "052f1ded-40ea-4148-8b0b-fc0e4168d17c"
      },
      "execution_count": 39,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Eigenvalues: [-0.20942046  4.36588492  9.84353554]\n",
            "Eigenvectors: [[ 0.91805853  0.26010485 -0.2991889 ]\n",
            " [-0.14209114  0.92042494  0.36418131]\n",
            " [ 0.37010626 -0.29182767  0.88196257]]\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## SVD\n",
        "\n",
        "We can compute the singular value decomposition of a matrix $\\mathbf{M}$ as:\n",
        "\n",
        "$$\n",
        "\\mathbf{M} = \\mathbf{U} \\boldsymbol{\\Sigma} \\mathbf{V}^T\n",
        "$$\n",
        "\n",
        "Here, the symbols have their usual meanings."
      ],
      "metadata": {
        "id": "Rq2jpFjIhIsI"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "SVD_M = np.linalg.svd(M)\n",
        "U = SVD_M[0]\n",
        "Sigma = SVD_M[1]\n",
        "V_T = SVD_M[2]\n",
        "\n",
        "print(f\"U: {U}\", \"\\n\")\n",
        "print(f\"Sigma: {Sigma}\", \"\\n\")\n",
        "print(f\"V_T: {V_T}\", \"\\n\")"
      ],
      "metadata": {
        "id": "ieo70cvZhZhc",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "f851cb32-7ea1-45d6-c40e-bdf3720819ff"
      },
      "execution_count": 45,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "U: [[-0.2991889  -0.26010485  0.91805853]\n",
            " [ 0.36418131 -0.92042494 -0.14209114]\n",
            " [ 0.88196257  0.29182767  0.37010626]] \n",
            "\n",
            "Sigma: [9.84353554 4.36588492 0.20942046] \n",
            "\n",
            "V_T: [[-0.2991889   0.36418131  0.88196257]\n",
            " [-0.26010485 -0.92042494  0.29182767]\n",
            " [-0.91805853  0.14209114 -0.37010626]] \n",
            "\n"
          ]
        }
      ]
    }
  ]
}