Problem 1a, HW9.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "c15020af",
   "metadata": {},
   "source": [
    "# Problem 1a, HW 9\n",
    "\n",
    "Let $X, Y$ be independent, positive random variables with finite expected values.\n",
    "\n",
    "Give an example where $E(\\frac{X}{X+Y}) \\neq \\frac{E(X)}{E(X+Y)}$, computing both sides exactly. (Hint: start by thinking about the simplest examples you can think of!)\n",
    "\n",
    "For ease of notation, let's define $W = \\frac{X}{X+Y}$. Now let's compare the above result to $E(W)$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "f4add891",
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "%config InlineBackend.figure_format = \"retina\"\n",
    "%load_ext autoreload\n",
    "%autoreload 2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "6c0e3400",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Import dependencies\n",
    "import numpy as np\n",
    "\n",
    "# For displaying plots in dark mode\n",
    "from jupyterthemes import jtplot\n",
    "jtplot.style(theme='monokai', context='notebook', ticks=True, grid=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d82d691b",
   "metadata": {},
   "source": [
    "### Independent _and_ identically distributed\n",
    "\n",
    "Let me first model the roll of two dice whose outcomes are the random variables $X, Y \\stackrel{\\text{i.i.d}}{\\sim} \\text{DUnif}(1,6)$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "c8de2c9b",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Marginals fo X, Y\n",
    "X_support = np.arange(1, 6+1).reshape(1, 6)\n",
    "Y_support = np.arange(1, 6+1).reshape(6, 1)\n",
    "X_marg_PMF = np.ones_like(X_support) / X_support.shape[1]\n",
    "Y_marg_PMF = np.ones_like(Y_support) / Y_support.shape[0]\n",
    "\n",
    "# Joint distribution X, Y\n",
    "joint_dist = np.repeat(X_support, 6, axis=0) + np.repeat(Y_support, 6, axis=1)\n",
    "joint_PMF = Y_marg_PMF.dot(X_marg_PMF)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "47c14ca4",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.5"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Calculate the RHS term\n",
    "exp_X_plus_Y = (joint_dist * joint_PMF).sum()\n",
    "exp_X = X_marg_PMF.dot(X_support.T).item(0)\n",
    "exp_X_div_X_plus_Y = RHS = exp_X / exp_X_plus_Y\n",
    "\n",
    "RHS"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "f5dac433",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.49999999999999994"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Calculate the LHS term\n",
    "W_support = X_support / joint_dist\n",
    "exp_W = LHS = (W_support * joint_PMF).sum()\n",
    "\n",
    "LHS"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b38105a0",
   "metadata": {},
   "source": [
    "In this set up, $E(\\frac{X}{X+Y}) = \\frac{E(X)}{E(X+Y)}$. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a728f711-7fd6-4284-8c1d-1d4aada3c662",
   "metadata": {},
   "source": [
    "### Independent, but _not_ identically distributed\n",
    "Let's try a different experiment: let $X \\sim \\text{Bern}(0.5)$ be the outcome of a fair coin flip (where $1$ signifies heads and $0$ corresponds to tails) and $Y$ continue being the outcome of a fair die roll."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "fbd99a1d",
   "metadata": {},
   "outputs": [],
   "source": [
    "# New definition of X (coin flip)\n",
    "X_support = np.arange(0, 1+1).reshape(1, 2)\n",
    "X_marg_PMF = np.ones_like(X_support) / X_support.shape[1]\n",
    "\n",
    "# Joint distribution X, Y\n",
    "joint_dist = np.repeat(X_support, 6, axis=0) + np.repeat(Y_support, 2, axis=1)\n",
    "joint_PMF = Y_marg_PMF.dot(X_marg_PMF)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "ee72663b",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.125"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Calculate the RHS term\n",
    "exp_X_plus_Y = (joint_dist * joint_PMF).sum()\n",
    "exp_X = X_marg_PMF.dot(X_support.T).item(0)\n",
    "exp_X_div_X_plus_Y = RHS = exp_X / exp_X_plus_Y\n",
    "\n",
    "RHS"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "3061f2db",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.13273809523809527"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Calculate the LHS term\n",
    "W_support = X_support / joint_dist\n",
    "exp_W = LHS = (W_support * joint_PMF).sum()\n",
    "\n",
    "LHS"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1b41ba8f",
   "metadata": {},
   "source": [
    "Ah ha! So, even with two independent random variables, we can quite easily find an example of $E(W) \\neq \\frac{E(X)}{E(X+Y)}$."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}