INVADERS_script.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<span style=\"font-weight:normal;font-size:32pt\">\n",
    "    INVADERS</span>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Introduction \n",
    " \n",
    "By the end of 2019 the SARS-CoV-2 virus strain emerged and developed into a global pandemic. Consequently (virtually) all aspects of our lives were affected. However, despite the feeling that the virus paralyzed the world, the pandemic itself was far from being still. The spread of the virus has taken many forms and raised many questions concerning our safety and well-being: the virus mortality rate, the imposition of nation-wide lockdowns or the effectivity of masks and vaccines... just to name a few. The urgency of such topics pushed them into the front page of all our discussions, as we showed in the <a href=\"https://jpalm3r.github.io/INVADER/\">first issue</a>  of this project. This continuous presence raised a very simple question: <u>do political beliefs affect COVID discussion?</u>\n",
    "\n",
    "Our intuition (and prejudices) might want to answer these questions right away. For this reason, this work intends to quantify and visualize if such political bias exists. To do it, we have used data from <a href=\"www.reddit.com\">Reddit</a>, and analyzed post submissions and comments of key communities. Specifically, we identified key words that represent the topics and questions that the pandemic has raised. Afterwards, these topics have been analyzed in mainstream political communities to evaluate if there are differences in their usage.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# A grim trip down memory lane\n",
    "\n",
    "\n",
    "The year 2020 will be forever linked to the pandemic. The virus started to globally spread during January, and by December the first people started to get vaccinated.\n",
    "Thus, 2020 encapsulates most of the major breakthroughs and controversies that defined COVID-19 and this work has focused only on studying this strange year.\n",
    "To represent the major COVID-related controversies that flooded 2020, 19 words have been selected from the corpus of all submission titles in the\n",
    "<a href=\"www.reddit.com/r/coronavirus\">r/Coronavirus</a> community. This community is dedicated to sharing global news about the evolution of\n",
    "the pandemic so it should work as a proxy of public discussion.\n",
    "\n",
    "The following group of infamous words<sup>(1)</sup> are the most used during 2020:\n",
    "\n",
    "<span class = 'corona'>\n",
    "China, confirm, death, die, health, hospit, infect, lockdown, mask, outbreak, pandem, patient, posit, record, report, spread, studi, test, vaccin.\n",
    "</span>\n",
    "\n",
    "\n",
    "<span class = 'footnote'> (1) Notice that the words are <span style=\"font-style:italic\"><a href=\"https://en.wikipedia.org/wiki/Stemming\">stemmed</a></span>.\n",
    "\n",
    "\n",
    "Looking at all of them together, they appear as a foggy cloud of concepts that range from little surprising to not surprising at all. Checking them individually helps to untangle this cloud and sets a timeline that all of us remember well.\n",
    "\n",
    "__CALENDAR PLOT__\n",
    "\n",
    "Throughout the year, the relative importance of each topic changed as we were collectively adapting and overcoming new challenges. This can be easily seen by comparing the usage of each word to the other 18 words, the higher up a word appears, the more important it was during a given period.\n",
    "\n",
    "\n",
    "__RANKING PLOT__\n",
    "\n",
    "It is interesting to see the correlation between some of the words; as a starter, these are clashes worth checking:\n",
    "\n",
    "  <li>The take over of <span class = 'corona'>pandemic</span> over\n",
    "  <span class = 'corona'>outbreak</span></li>\n",
    "  <li>The inverse correlation between <span class = 'corona'>hospital</span> and\n",
    "   <span class = 'corona'>lockdown</span></li>\n",
    "  <li>The ominous importance of the word <span class = 'corona'>death</span> ,\n",
    "  despite the hopeful race upwards of <span class = 'corona'>vaccine</span></li>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Lose-lose\n",
    "\n",
    "\n",
    "The presented words describe a landscape of topics that took over public discussion.\n",
    "However, the reactions that they caused might have been different depending on the communities they were discussed on.\n",
    "To sort this out, this section compares the usage of the 19 words in the <u>comment</u> sections of two major political subreddits.\n",
    "<br>\n",
    "Given that <a href=\"www.reddit.com\">Reddit</a> is a US-based website, this study has focused on US politics.\n",
    "So, in order to check differences in political discussion we have focused on communities dedicated to the two major US political parties:\n",
    " <a href=\"http://www.reddit.com/r/republican\"><span class = 'republican'> r/republican </span></a> and\n",
    " <a href=\"http://www.reddit.com/r/republican\"><span class = 'democrat'> r/democrats </span></a>.\n",
    "<br>\n",
    "<br>\n",
    "First of all, an easy check: the total use of the 19 corona words.\n",
    "\n",
    "__CORONA FOOTPRINT__\n",
    "\n",
    "It already looks like the words were used more in the republican category; still, the size is comparable in both.\n",
    "We can go a bit deeper by comparing the total volume<sup>(2)</sup> of each word in each political community.\n",
    "This gives us a better picture of what is causing the difference that we see in the previous figure.\n",
    "\n",
    "__WORDS BY COMMUNITY__\n",
    "\n",
    "(2) The words are normalized by the total number of comments in each community.\n",
    "\n",
    "Notice that, neutral words, such as <span class=\"corona\">test</span>\n",
    "and <span class=\"corona\">confirm</span> have comparable sizes. On the other hand, other words with more controversial undertones show significant differences in their volumes.\n",
    "It is fair to assume that controversy generates arguments. Typically, in such arguments, people will post longer comments to make sure that their point is carried accross.\n",
    "Then, with the next figure, we wanted to check whether comments containing corona-related words are longer than others.\n",
    "\n",
    "\n",
    "__COMMENT LENGTH__\n",
    "\n",
    "It looks like corona words tend to appear in longer comments, and the longest comments contain words that relate to health (<span class = 'corona'> hospit</span> and <span class = 'corona'>patient </span>).\n",
    "Still, focusing on each particular word, there is no clear differences between republican and democrat communities.\n",
    "\n",
    "As we have seen above studying the <a href=\"www.reddit.com/r/coronavirus\">r/Coronavirus</a> community, the volume of these 19 words changed with time, as the relevance of\n",
    "the topics they described shifted. Then, it might be interesting to study the frequency use of these words in the two political communities.\n",
    "However, <a href=\"http://www.reddit.com/r/republican\"><span class = 'republican'> r/republican </span></a> and <a href=\"http://www.reddit.com/r/democrats\"><span class = 'democrat'> r/democrats </span></a>\n",
    "are as broad as communities as politics can be, so covid-related discussion will share the space with many other different topics.\n",
    "To clear the search, we focused on days when the use of COVID-words was abnormally high and we marked them with a &#11035;; the larger the &#11035;, the higher the surge.\n",
    "\n",
    "\n",
    "\n",
    "__SURGES TIMELINE__\n",
    "\n",
    "\n",
    "Notice that the surges propagate over time, given that they seldomly disappear after one day. Also, in general, they do not appear alone as one can see that\n",
    "when the volume of a word peaks so do many of the others, creating vertical patterns in the previous figure. A priori, the triggers behind the surges are not known,\n",
    "but we included different 2020 milestones to give temporal context, and we invite the reader to explore and come up with a reasonable explanation.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Mapping political bias\n",
    "\n",
    "The previous analysis suggests that there is a difference in how the words are treated in each political community and we would like to visualize\n",
    "it in a compact and clear way. Comparing the volumes of each word might not be enough, since we have seen that the time when a word is\n",
    "used can be as important. To combine the volume and temporal use we have computed the monthly volume of each word, and correlated the trend in both\n",
    "political communities.\n",
    "\n",
    "__METRIC__\n",
    "\n",
    "If the usage of a word in <a href=\"http://www.reddit.com/r/republican\"><span class='republican'> r/republican </span></a> is similar to\n",
    "its use in <a href=\"http://www.reddit.com/r/democrats\"><span class='democrat'> r/democrats </span></a>,\n",
    "then the cloud of dots will fall over the reference line. If the dots are \"tilted\" towards one community's axis,\n",
    "then that word was systematically more used in that community. Then, the slope synthetizes the likeness of the use of a word during 2020 in\n",
    "both political communities.\n",
    "<br>\n",
    "<br>\n",
    "But these communities already existed before COVID hit. So, was the usage of these words \"normal\" or was indeed triggered by the pandemic?\n",
    "We measured this by measuring the volumes of each word during 2019 and computing the ratio between the volumes in 2020 and the previous year.\n",
    "Finally, the slope of the previous figure and the volume ratio represent two metrics that are easily comparable, which allowed us to map\n",
    "map the political bias in one single final plot.\n",
    "\n",
    "\n",
    "__MAP__\n",
    "\n",
    "\n",
    "The map shows a clear bias towards the <a href=\"http://www.reddit.com/r/republican\"><span class = 'republican'> r/republican </span></a> community.\n",
    "The severity of this bias is striking and raises the question of whether this community is an appropiate representative of the republican mindset, or <a href=\"http://www.reddit.com/r/republican\"><span class = 'republican'> r/republican </span></a>\n",
    "caters to a more polarized audience than <a href=\"http://www.reddit.com/r/democrats\"><span class='democrat'> r/democrats </span></a>. However, this question is out of\n",
    "the scope of this work.\n",
    "<br>\n",
    "Notice, that the lockdown was a far more important topic in the <a href=\"http://www.reddit.com/r/republican\"><span class = 'republican'> r/republican </span></a> community,\n",
    "while the discussion of masks and vaccines were closer in both political communities. As already hinted in previous figures, there is a big difference in the usage\n",
    "of the word China, even though, since it appears closer to the horizontal axis, it looks like it was a recurrent republican topic in 2019 already. In general, the lower a word appears, the\n",
    "the more broad and politically neutral, which again highlights that the reaction to COVID-discussion was different in both communities. Yet, this map, and the rest of the work for that matter, is open for discussion (it deals with politics after all).\n",
    "<br>\n",
    "<br>\n",
    "Now, to conclude, we believe there is one thing left to highlight about the last figure. Despite the noticeable differences, after all this effort and discussion,\n",
    "the <span class = 'corona'>pandemic</span> is still up there unbiased."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}