Notebooks/1-MLPs/2-Classification.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Multilayer Perceptron Classifiers\n",
    "\n",
    "**This notebook shows how to create a MLP classification model with TensorFlow 2 and Keras**\n",
    "\n",
    "## The Data\n",
    "\n",
    "We will be using the Wine Quality dataset from [UCI Machine Learning Repository](https://archive.ics.uci.edu/):\n",
    "\n",
    "https://archive.ics.uci.edu/ml/datasets/wine+quality\n",
    "\n",
    "**Information**\n",
    "\n",
    "The two datasets are related to red and white variants of the Portuguese \"Vinho Verde\" wine. Due to privacy and logistic issues, only physicochemical (inputs) and sensory (the output) variables are available (e.g. there is no data about grape types, wine brand, wine selling price, etc.).\n",
    "\n",
    "\n",
    "**Attributes**\n",
    "\n",
    "1. fixed acidity\n",
    "2. volatile acidity\n",
    "3. citric acid\n",
    "4. residual sugar\n",
    "5. chlorides\n",
    "6. free sulfur dioxide\n",
    "7. total sulfur dioxide\n",
    "8. density\n",
    "9. pH\n",
    "10. sulphates\n",
    "11. alcohol\n",
    "12. quality (score between 0 and 10)\n",
    "\n",
    "## Table of Contents\n",
    "- ### [Python libraries](#libraries)\n",
    "- ### [Data Exploration and Feature Engineering](#exploration)\n",
    "- ### [Split the Data](#split)\n",
    "- ### [Normalize the Data](#normalize)\n",
    "- ### [Create the Model](#model)\n",
    "- ### [Train the Model](#training)\n",
    "    ### [1. Choosing too many epochs and overfit](#overfit)\n",
    "    ### [2. Add Dropout layers](#dropout)\n",
    "    ### [3. Early Stopping](#early_stop)\n",
    "- ### [Evaluate the Model](#evaluation)\n",
    "- ### [Predictions](#predictions)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id='libraries'></a>\n",
    "## Python libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/ikersanchez/anaconda3/lib/python3.7/site-packages/statsmodels/tools/_testing.py:19: FutureWarning: pandas.util.testing is deprecated. Use the functions in the public API at pandas.testing instead.\n",
      "  import pandas.util.testing as tm\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "import seaborn as sns\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id='exploration'></a>\n",
    "## Data Exploration and Feature Engineering"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Read the data**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>fixed.acidity</th>\n",
       "      <th>volatile.acidity</th>\n",
       "      <th>citric.acid</th>\n",
       "      <th>residual.sugar</th>\n",
       "      <th>chlorides</th>\n",
       "      <th>free.sulfur.dioxide</th>\n",
       "      <th>total.sulfur.dioxide</th>\n",
       "      <th>density</th>\n",
       "      <th>pH</th>\n",
       "      <th>sulphates</th>\n",
       "      <th>alcohol</th>\n",
       "      <th>quality</th>\n",
       "      <th>type</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>7.4</td>\n",
       "      <td>0.70</td>\n",
       "      <td>0.00</td>\n",
       "      <td>1.9</td>\n",
       "      <td>0.076</td>\n",
       "      <td>11.0</td>\n",
       "      <td>34.0</td>\n",
       "      <td>0.9978</td>\n",
       "      <td>3.51</td>\n",
       "      <td>0.56</td>\n",
       "      <td>9.4</td>\n",
       "      <td>5</td>\n",
       "      <td>red</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>7.8</td>\n",
       "      <td>0.88</td>\n",
       "      <td>0.00</td>\n",
       "      <td>2.6</td>\n",
       "      <td>0.098</td>\n",
       "      <td>25.0</td>\n",
       "      <td>67.0</td>\n",
       "      <td>0.9968</td>\n",
       "      <td>3.20</td>\n",
       "      <td>0.68</td>\n",
       "      <td>9.8</td>\n",
       "      <td>5</td>\n",
       "      <td>red</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>7.8</td>\n",
       "      <td>0.76</td>\n",
       "      <td>0.04</td>\n",
       "      <td>2.3</td>\n",
       "      <td>0.092</td>\n",
       "      <td>15.0</td>\n",
       "      <td>54.0</td>\n",
       "      <td>0.9970</td>\n",
       "      <td>3.26</td>\n",
       "      <td>0.65</td>\n",
       "      <td>9.8</td>\n",
       "      <td>5</td>\n",
       "      <td>red</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>11.2</td>\n",
       "      <td>0.28</td>\n",
       "      <td>0.56</td>\n",
       "      <td>1.9</td>\n",
       "      <td>0.075</td>\n",
       "      <td>17.0</td>\n",
       "      <td>60.0</td>\n",
       "      <td>0.9980</td>\n",
       "      <td>3.16</td>\n",
       "      <td>0.58</td>\n",
       "      <td>9.8</td>\n",
       "      <td>6</td>\n",
       "      <td>red</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>7.4</td>\n",
       "      <td>0.70</td>\n",
       "      <td>0.00</td>\n",
       "      <td>1.9</td>\n",
       "      <td>0.076</td>\n",
       "      <td>11.0</td>\n",
       "      <td>34.0</td>\n",
       "      <td>0.9978</td>\n",
       "      <td>3.51</td>\n",
       "      <td>0.56</td>\n",
       "      <td>9.4</td>\n",
       "      <td>5</td>\n",
       "      <td>red</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   fixed.acidity  volatile.acidity  citric.acid  residual.sugar  chlorides  \\\n",
       "0            7.4              0.70         0.00             1.9      0.076   \n",
       "1            7.8              0.88         0.00             2.6      0.098   \n",
       "2            7.8              0.76         0.04             2.3      0.092   \n",
       "3           11.2              0.28         0.56             1.9      0.075   \n",
       "4            7.4              0.70         0.00             1.9      0.076   \n",
       "\n",
       "   free.sulfur.dioxide  total.sulfur.dioxide  density    pH  sulphates  \\\n",
       "0                 11.0                  34.0   0.9978  3.51       0.56   \n",
       "1                 25.0                  67.0   0.9968  3.20       0.68   \n",
       "2                 15.0                  54.0   0.9970  3.26       0.65   \n",
       "3                 17.0                  60.0   0.9980  3.16       0.58   \n",
       "4                 11.0                  34.0   0.9978  3.51       0.56   \n",
       "\n",
       "   alcohol  quality type  \n",
       "0      9.4        5  red  \n",
       "1      9.8        5  red  \n",
       "2      9.8        5  red  \n",
       "3      9.8        6  red  \n",
       "4      9.4        5  red  "
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_red = pd.read_csv('../../Data/winequality-red.csv').drop(columns='Unnamed: 0')\n",
    "df_red['type'] = 'red'\n",
    "df_white = pd.read_csv('../../Data/winequality-white.csv').drop(columns='Unnamed: 0')\n",
    "df_white['type'] = 'white'\n",
    "\n",
    "df = pd.concat([df_red, df_white])\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>count</th>\n",
       "      <th>mean</th>\n",
       "      <th>std</th>\n",
       "      <th>min</th>\n",
       "      <th>25%</th>\n",
       "      <th>50%</th>\n",
       "      <th>75%</th>\n",
       "      <th>max</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>fixed.acidity</th>\n",
       "      <td>6497.0</td>\n",
       "      <td>7.215307</td>\n",
       "      <td>1.296434</td>\n",
       "      <td>3.80000</td>\n",
       "      <td>6.40000</td>\n",
       "      <td>7.00000</td>\n",
       "      <td>7.70000</td>\n",
       "      <td>15.90000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>volatile.acidity</th>\n",
       "      <td>6497.0</td>\n",
       "      <td>0.339666</td>\n",
       "      <td>0.164636</td>\n",
       "      <td>0.08000</td>\n",
       "      <td>0.23000</td>\n",
       "      <td>0.29000</td>\n",
       "      <td>0.40000</td>\n",
       "      <td>1.58000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>citric.acid</th>\n",
       "      <td>6497.0</td>\n",
       "      <td>0.318633</td>\n",
       "      <td>0.145318</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.25000</td>\n",
       "      <td>0.31000</td>\n",
       "      <td>0.39000</td>\n",
       "      <td>1.66000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>residual.sugar</th>\n",
       "      <td>6497.0</td>\n",
       "      <td>5.443235</td>\n",
       "      <td>4.757804</td>\n",
       "      <td>0.60000</td>\n",
       "      <td>1.80000</td>\n",
       "      <td>3.00000</td>\n",
       "      <td>8.10000</td>\n",
       "      <td>65.80000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>chlorides</th>\n",
       "      <td>6497.0</td>\n",
       "      <td>0.056034</td>\n",
       "      <td>0.035034</td>\n",
       "      <td>0.00900</td>\n",
       "      <td>0.03800</td>\n",
       "      <td>0.04700</td>\n",
       "      <td>0.06500</td>\n",
       "      <td>0.61100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>free.sulfur.dioxide</th>\n",
       "      <td>6497.0</td>\n",
       "      <td>30.525319</td>\n",
       "      <td>17.749400</td>\n",
       "      <td>1.00000</td>\n",
       "      <td>17.00000</td>\n",
       "      <td>29.00000</td>\n",
       "      <td>41.00000</td>\n",
       "      <td>289.00000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>total.sulfur.dioxide</th>\n",
       "      <td>6497.0</td>\n",
       "      <td>115.744574</td>\n",
       "      <td>56.521855</td>\n",
       "      <td>6.00000</td>\n",
       "      <td>77.00000</td>\n",
       "      <td>118.00000</td>\n",
       "      <td>156.00000</td>\n",
       "      <td>440.00000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>density</th>\n",
       "      <td>6497.0</td>\n",
       "      <td>0.994697</td>\n",
       "      <td>0.002999</td>\n",
       "      <td>0.98711</td>\n",
       "      <td>0.99234</td>\n",
       "      <td>0.99489</td>\n",
       "      <td>0.99699</td>\n",
       "      <td>1.03898</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>pH</th>\n",
       "      <td>6497.0</td>\n",
       "      <td>3.218501</td>\n",
       "      <td>0.160787</td>\n",
       "      <td>2.72000</td>\n",
       "      <td>3.11000</td>\n",
       "      <td>3.21000</td>\n",
       "      <td>3.32000</td>\n",
       "      <td>4.01000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>sulphates</th>\n",
       "      <td>6497.0</td>\n",
       "      <td>0.531268</td>\n",
       "      <td>0.148806</td>\n",
       "      <td>0.22000</td>\n",
       "      <td>0.43000</td>\n",
       "      <td>0.51000</td>\n",
       "      <td>0.60000</td>\n",
       "      <td>2.00000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>alcohol</th>\n",
       "      <td>6497.0</td>\n",
       "      <td>10.491801</td>\n",
       "      <td>1.192712</td>\n",
       "      <td>8.00000</td>\n",
       "      <td>9.50000</td>\n",
       "      <td>10.30000</td>\n",
       "      <td>11.30000</td>\n",
       "      <td>14.90000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>quality</th>\n",
       "      <td>6497.0</td>\n",
       "      <td>5.818378</td>\n",
       "      <td>0.873255</td>\n",
       "      <td>3.00000</td>\n",
       "      <td>5.00000</td>\n",
       "      <td>6.00000</td>\n",
       "      <td>6.00000</td>\n",
       "      <td>9.00000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                       count        mean        std      min       25%  \\\n",
       "fixed.acidity         6497.0    7.215307   1.296434  3.80000   6.40000   \n",
       "volatile.acidity      6497.0    0.339666   0.164636  0.08000   0.23000   \n",
       "citric.acid           6497.0    0.318633   0.145318  0.00000   0.25000   \n",
       "residual.sugar        6497.0    5.443235   4.757804  0.60000   1.80000   \n",
       "chlorides             6497.0    0.056034   0.035034  0.00900   0.03800   \n",
       "free.sulfur.dioxide   6497.0   30.525319  17.749400  1.00000  17.00000   \n",
       "total.sulfur.dioxide  6497.0  115.744574  56.521855  6.00000  77.00000   \n",
       "density               6497.0    0.994697   0.002999  0.98711   0.99234   \n",
       "pH                    6497.0    3.218501   0.160787  2.72000   3.11000   \n",
       "sulphates             6497.0    0.531268   0.148806  0.22000   0.43000   \n",
       "alcohol               6497.0   10.491801   1.192712  8.00000   9.50000   \n",
       "quality               6497.0    5.818378   0.873255  3.00000   5.00000   \n",
       "\n",
       "                            50%        75%        max  \n",
       "fixed.acidity           7.00000    7.70000   15.90000  \n",
       "volatile.acidity        0.29000    0.40000    1.58000  \n",
       "citric.acid             0.31000    0.39000    1.66000  \n",
       "residual.sugar          3.00000    8.10000   65.80000  \n",
       "chlorides               0.04700    0.06500    0.61100  \n",
       "free.sulfur.dioxide    29.00000   41.00000  289.00000  \n",
       "total.sulfur.dioxide  118.00000  156.00000  440.00000  \n",
       "density                 0.99489    0.99699    1.03898  \n",
       "pH                      3.21000    3.32000    4.01000  \n",
       "sulphates               0.51000    0.60000    2.00000  \n",
       "alcohol                10.30000   11.30000   14.90000  \n",
       "quality                 6.00000    6.00000    9.00000  "
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.describe().transpose()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Class balancing**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x1a1e009610>"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAEGCAYAAACUzrmNAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAR6ElEQVR4nO3de5BedX3H8feHIN4VkICYUMNoqmJbEXYQpZ1RUUCrhnHA4qhEZZo6xarTqsVOp1iVGZ1eqFIvpQUNVkW8Qq0VM3i/IGwUkYtM4pUMlESDeBsvwW//eH6Rh2Q3v4Xu2d1k36+Zneec7/mds9+H2cmHc09VIUnSruw13w1IkhY+w0KS1GVYSJK6DAtJUpdhIUnq2nu+GxjCAQccUCtWrJjvNiRpt7J+/fofVNXSqZbtkWGxYsUKJicn57sNSdqtJPnedMs8DCVJ6ho0LJJ8N8k3klyVZLLV9k+yLsmG9rlfqyfJW5JsTHJ1kiPGtrO6jd+QZPWQPUuSdjYXexZPqqrDq2qizZ8BXFZVK4HL2jzA04CV7WcN8HYYhQtwJvA44CjgzO0BI0maG/NxGGoVsLZNrwVOHKtfUCOXA/smORg4HlhXVVur6lZgHXDCXDctSYvZ0GFRwCeTrE+yptUOqqqbAdrnga2+DLhxbN1NrTZd/U6SrEkymWRyy5Yts/w1JGlxG/pqqGOq6qYkBwLrknxzF2MzRa12Ub9zoepc4FyAiYkJn44oSbNo0D2LqrqpfW4GPsLonMMt7fAS7XNzG74JOGRs9eXATbuoS5LmyGBhkeS+Se6/fRo4DrgGuATYfkXTauDiNn0JcGq7Kupo4LZ2mOpS4Lgk+7UT28e1miRpjgx5GOog4CNJtv+e91bVJ5JcCVyU5DTg+8DJbfzHgacDG4GfAy8CqKqtSV4PXNnGva6qtg7YtyRpB9kTX340MTFR3sGtPdX3X/f7892CFqDf+btv/L+3kWT92G0Od+Id3JKkLsNCktRlWEiSugwLSVKXYSFJ6jIsJEldhoUkqcuwkCR1GRaSpC7DQpLUZVhIkroMC0lSl2EhSeoyLCRJXYaFJKnLsJAkdRkWkqQuw0KS1GVYSJK6DAtJUpdhIUnqMiwkSV2GhSSpy7CQJHUZFpKkLsNCktRlWEiSugwLSVKXYSFJ6jIsJEldhoUkqcuwkCR1GRaSpC7DQpLUNXhYJFmS5GtJPtbmD03ylSQbkrw/yT6tfs82v7EtXzG2jde0+g1Jjh+6Z0nSnc3FnsXLgevH5t8EnF1VK4FbgdNa/TTg1qp6OHB2G0eSw4BTgEcDJwBvS7JkDvqWJDWDhkWS5cAfA//R5gM8GfhgG7IWOLFNr2rztOXHtvGrgAur6pdV9R1gI3DUkH1Lku5s6D2LfwFeDfymzT8I+FFVbWvzm4BlbXoZcCNAW35bG//b+hTr/FaSNUkmk0xu2bJltr+HJC1qg4VFkmcAm6tq/Xh5iqHVWbarde4oVJ1bVRNVNbF06dK73K8kaXp7D7jtY4BnJXk6cC/gAYz2NPZNsnfbe1gO3NTGbwIOATYl2Rt4ILB1rL7d+DqSpDkw2J5FVb2mqpZX1QpGJ6g/VVXPAz4NnNSGrQYubtOXtHna8k9VVbX6Ke1qqUOBlcAVQ/UtSdrZkHsW0/lr4MIkbwC+BpzX6ucB706ykdEexSkAVXVtkouA64BtwOlVdfvcty1Ji9echEVVfQb4TJv+NlNczVRVvwBOnmb9s4CzhutQkrQr3sEtSeoyLCRJXYaFJKnLsJAkdRkWkqQuw0KS1GVYSJK6DAtJUpdhIUnqMiwkSV2GhSSpy7CQJHUZFpKkLsNCktRlWEiSugwLSVKXYSFJ6jIsJEldhoUkqcuwkCR1GRaSpC7DQpLUZVhIkroMC0lSl2EhSeoyLCRJXYaFJKnLsJAkdRkWkqQuw0KS1GVYSJK6DAtJUpdhIUnqGiwsktwryRVJvp7k2iR/3+qHJvlKkg1J3p9kn1a/Z5vf2JavGNvWa1r9hiTHD9WzJGlqQ+5Z/BJ4clU9BjgcOCHJ0cCbgLOraiVwK3BaG38acGtVPRw4u40jyWHAKcCjgROAtyVZMmDfkqQdDBYWNfLTNnuP9lPAk4EPtvpa4MQ2varN05YfmyStfmFV/bKqvgNsBI4aqm9J0s4GPWeRZEmSq4DNwDrgW8CPqmpbG7IJWNamlwE3ArTltwEPGq9Psc7471qTZDLJ5JYtW4b4OpK0aA0aFlV1e1UdDixntDfwqKmGtc9Ms2y6+o6/69yqmqiqiaVLl97dliVJU5iTq6Gq6kfAZ4CjgX2T7N0WLQduatObgEMA2vIHAlvH61OsI0maA0NeDbU0yb5t+t7AU4DrgU8DJ7Vhq4GL2/QlbZ62/FNVVa1+Srta6lBgJXDFUH1Lkna2d3/I3XYwsLZdubQXcFFVfSzJdcCFSd4AfA04r40/D3h3ko2M9ihOAaiqa5NcBFwHbANOr6rbB+xbkrSDwcKiqq4GHjtF/dtMcTVTVf0COHmabZ0FnDXbPUqSZsY7uCVJXTMKiySXzaQmSdoz7fIwVJJ7AfcBDkiyH3dcxvoA4CED9yZJWiB65yz+DHgFo2BYzx1h8WPgrQP2JUlaQHYZFlX1ZuDNSf6iqs6Zo54kSQvMjK6GqqpzkjwBWDG+TlVdMFBfkqQFZEZhkeTdwMOAq4Dt9zgUYFhI0iIw0/ssJoDD2h3VkqRFZqb3WVwDPHjIRiRJC9dM9ywOAK5LcgWjlxoBUFXPGqQrSdKCMtOweO2QTUiSFraZXg312aEbkSQtXDO9Guon3PHCoX0YvSL1Z1X1gKEakyQtHDPds7j/+HySE/E92JK0aNytp85W1UeBJ89yL5KkBWqmh6GePTa7F6P7LrznQpIWiZleDfXMseltwHeBVbPejSRpQZrpOYsXDd2IJGnhmunLj5Yn+UiSzUluSfKhJMuHbk6StDDM9AT3O4FLGL3XYhnwX60mSVoEZhoWS6vqnVW1rf28C1g6YF+SpAVkpmHxgyTPT7Kk/Twf+OGQjUmSFo6ZhsWLgecA/wvcDJwEeNJbkhaJmV46+3pgdVXdCpBkf+AfGYWIJGkPN9M9iz/YHhQAVbUVeOwwLUmSFpqZhsVeSfbbPtP2LGa6VyJJ2s3N9B/8fwK+lOSDjB7z8RzgrMG6kiQtKDO9g/uCJJOMHh4Y4NlVdd2gnUmSFowZH0pq4WBASNIidLceUS5JWlwMC0lSl2EhSeoyLCRJXYOFRZJDknw6yfVJrk3y8lbfP8m6JBva536tniRvSbIxydVJjhjb1uo2fkOS1UP1LEma2pB7FtuAv6qqRwFHA6cnOQw4A7isqlYCl7V5gKcBK9vPGuDt8NsbAM8EHgccBZw5foOgJGl4g4VFVd1cVV9t0z8Brmf0LoxVwNo2bC1wYpteBVxQI5cD+yY5GDgeWFdVW9sjR9YBJwzVtyRpZ3NyziLJCkbPkvoKcFBV3QyjQAEObMOWATeOrbap1aar7/g71iSZTDK5ZcuW2f4KkrSoDR4WSe4HfAh4RVX9eFdDp6jVLup3LlSdW1UTVTWxdKnvZZKk2TRoWCS5B6OgeE9VfbiVb2mHl2ifm1t9E3DI2OrLgZt2UZckzZEhr4YKcB5wfVX989iiS4DtVzStBi4eq5/aroo6GritHaa6FDguyX7txPZxrSZJmiNDPmb8GOAFwDeSXNVqfwO8EbgoyWnA94GT27KPA08HNgI/p72Jr6q2Jnk9cGUb97r2Pg1J0hwZLCyq6gtMfb4B4Ngpxhdw+jTbOh84f/a6kyTdFd7BLUnqMiwkSV2GhSSpy7CQJHUZFpKkLsNCktRlWEiSugwLSVKXYSFJ6jIsJEldhoUkqWvIBwnu1o581QXz3YIWoPX/cOp8tyDNC/csJEldhoUkqcuwkCR1GRaSpC7DQpLUZVhIkroMC0lSl2EhSeoyLCRJXYaFJKnLsJAkdRkWkqQuw0KS1GVYSJK6DAtJUpdhIUnqMiwkSV2GhSSpy7CQJHUZFpKkLsNCktQ1WFgkOT/J5iTXjNX2T7IuyYb2uV+rJ8lbkmxMcnWSI8bWWd3Gb0iyeqh+JUnTG3LP4l3ACTvUzgAuq6qVwGVtHuBpwMr2swZ4O4zCBTgTeBxwFHDm9oCRJM2dwcKiqj4HbN2hvApY26bXAieO1S+okcuBfZMcDBwPrKuqrVV1K7COnQNIkjSwuT5ncVBV3QzQPg9s9WXAjWPjNrXadHVJ0hxaKCe4M0WtdlHfeQPJmiSTSSa3bNkyq81J0mI312FxSzu8RPvc3OqbgEPGxi0HbtpFfSdVdW5VTVTVxNKlS2e9cUlazOY6LC4Btl/RtBq4eKx+arsq6mjgtnaY6lLguCT7tRPbx7WaJGkO7T3UhpO8D3gicECSTYyuanojcFGS04DvAye34R8Hng5sBH4OvAigqrYmeT1wZRv3uqra8aS5JGlgg4VFVT13mkXHTjG2gNOn2c75wPmz2Jok6S5aKCe4JUkLmGEhSeoyLCRJXYaFJKnLsJAkdRkWkqQuw0KS1GVYSJK6DAtJUpdhIUnqMiwkSV2GhSSpy7CQJHUZFpKkLsNCktRlWEiSugwLSVKXYSFJ6jIsJEldhoUkqcuwkCR1GRaSpC7DQpLUZVhIkroMC0lSl2EhSeoyLCRJXYaFJKnLsJAkdRkWkqQuw0KS1GVYSJK6DAtJUpdhIUnq2m3CIskJSW5IsjHJGfPdjyQtJrtFWCRZArwVeBpwGPDcJIfNb1eStHjsFmEBHAVsrKpvV9WvgAuBVfPckyQtGnvPdwMztAy4cWx+E/C48QFJ1gBr2uxPk9wwR70tBgcAP5jvJhaC/OPq+W5Bd+bf5nZnZja28tDpFuwuYTHVf4W600zVucC5c9PO4pJksqom5rsPaUf+bc6d3eUw1CbgkLH55cBN89SLJC06u0tYXAmsTHJokn2AU4BL5rknSVo0dovDUFW1LclLgUuBJcD5VXXtPLe1mHh4TwuVf5tzJFXVHyVJWtR2l8NQkqR5ZFhIkroMC91tSZ6Y5GPz3YcWhyQ/nab+kiSntukXJnnI3Ha2OOwWJ7g1t5KE0fms38x3L1JPVb1jbPaFwDV4af2sc89CACRZkeT6JG8Dvgq8IMmXk3w1yQeS3K+NOyHJN5N8AXj2vDatPUqSVyd5WZs+O8mn2vSxSf6zTZ+V5OtJLk9yUKu9Nskrk5wETADvSXJVknsnOTLJZ5OsT3JpkoPn6/vt7gwLjXsEcAHwVOA04ClVdQQwCfxlknsB/w48E/gj4MHz1aj2SJ9j9HcFo3/075fkHsAfAp8H7gtcXlWPaWP/dHzlqvogo7/V51XV4cA24BzgpKo6EjgfOGsuvsieyMNQGve9qro8yTMYPd33i6MjUuwDfBl4JPCdqtoA0P5vb810G5PuovXAkUnuD/yS0R7uBKMAeRnwK+BjY2Of2tneI4DfA9a1v+MlwM2z3/biYFho3M/aZ4B1VfXc8YVJDmeHZ3JJs6Wqfp3ku8CLgC8BVwNPAh4GXA/8uu64Mex2+v9+Bbi2qh4/TMeLi4ehNJXLgWOSPBwgyX2S/C7wTeDQJA9r45473Qaku+lzwCvb5+eBlwBX1czvHv4JcP82fQOwNMnjAZLcI8mjZ7nfRcOw0E6qagujq0rel+RqRuHxyKr6BaPDTv/dTnB/b/661B7q88DBwJer6hbgF602U+8C3pHkKkaHnU4C3pTk68BVwBNmt93Fw8d9SJK63LOQJHUZFpKkLsNCktRlWEiSugwLSVKXYSHNgiT7Jvnz+e5DGophIc2OfQHDQnssw0KaHW8EHtaedvqBJKu2L0jyniTPau9auDjJJ5LckOTMsTHPT3JFW//fkiyZl28hTcOwkGbHGcC32tNO/5XR841I8kBGdw1/vI07CngecDhwcpKJJI8C/gQ4pq1/exsjLRg+SFCaZVX12SRvTXIgo3d+fKiqtrUnn66rqh8CJPkwo8dvbwOOBK5sY+4NbJ6X5qVpGBbSMN7NaO/gFODFY/Udn69TjJ6OuraqXjNHvUl3mYehpNkx/rRTGD3Q7hUAVXXtWP2pSfZPcm/gROCLwGXASW1PhLb8oXPStTRD7llIs6Cqfpjki0muAf6nql6V5HrgozsM/QKjvY6HA++tqkmAJH8LfDLJXsCvgdPxqb5aQHzqrDSAJPcBvgEcUVW3tdoLgYmqeul89ibdHR6GkmZZkqcwelHUOduDQtrduWchSepyz0KS1GVYSJK6DAtJUpdhIUnqMiwkSV3/B0W5gl9xH4KGAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "sns.countplot(x='type',data=df)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x1a1f3a4d90>"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAEGCAYAAACUzrmNAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAU3ElEQVR4nO3de5CldX3n8fdHRlC8AU6jODNkCI4azBqFDqJutoyIgDEMZcEGSsOI1M66QY3LGoXNVkhpUaWJu6wXgjsJI5CwEIJGZl0SnEUj3rgMBLnK0ouR6UCYxkFiNIKj3/3j/CYcZ7rnado+5/TQ71dV13me7/M753yH6uoPv+eaqkKSpF15yqgbkCQtfIaFJKmTYSFJ6mRYSJI6GRaSpE5LRt3AICxdurRWrlw56jYkabdy0003PVRVY9Nte1KGxcqVK9m0adOo25Ck3UqSb8+0zd1QkqROhoUkqZNhIUnqZFhIkjoZFpKkTgMLiyTrk2xJcvsO9XcluTvJHUn+oK9+VpKJtu3ovvoxrTaR5MxB9StJmtkgT529EPgEcPH2QpJfBVYDL6uqR5Ps3+qHACcBLwVeAPyfJC9qbzsPOAqYBG5MsqGq7hxg35KkHQwsLKrq2iQrdyj/B+BDVfVoG7Ol1VcDl7X6t5JMAIe3bRNVdS9AksvaWMNCkoZo2McsXgT8SpLrk3wpyS+3+jJgc9+4yVabqS5JGqJhX8G9BNgXOAL4ZeDyJD8PZJqxxfRhNu3TmpKsBdYCHHjggT9zo4f9zsXdg7To3PSHp4y6BQDu+8C/GnULWoAO/L3bBvbZw55ZTAKfqZ4bgJ8AS1t9Rd+45cD9u6jvpKrWVdV4VY2PjU17axNJ0hwNOyw+C7wOoB3A3hN4CNgAnJRkryQHAauAG4AbgVVJDkqyJ72D4BuG3LMkLXoD2w2V5FLgtcDSJJPA2cB6YH07nfYxYE31HgJ+R5LL6R243gacXlU/bp/zTuBqYA9gfVXdMaieJUnTG+TZUCfPsOmtM4w/BzhnmvpVwFXz2Jok6QnyCm5JUifDQpLUybCQJHUyLCRJnQwLSVInw0KS1MmwkCR1MiwkSZ0MC0lSJ8NCktTJsJAkdTIsJEmdDAtJUifDQpLUybCQJHUyLCRJnQwLSVKngYVFkvVJtrRHqO647b1JKsnStp4kH0sykeTWJIf2jV2T5J72s2ZQ/UqSZjbImcWFwDE7FpOsAI4C7usrHwusaj9rgfPb2P3oPbv7lcDhwNlJ9h1gz5KkaQwsLKrqWmDrNJvOBd4HVF9tNXBx9VwH7JPkAOBoYGNVba2qh4GNTBNAkqTBGuoxiyTHAX9fVd/YYdMyYHPf+mSrzVSf7rPXJtmUZNPU1NQ8di1JGlpYJNkb+F3g96bbPE2tdlHfuVi1rqrGq2p8bGxs7o1KknYyzJnFwcBBwDeS/B2wHLg5yfPpzRhW9I1dDty/i7okaYiGFhZVdVtV7V9VK6tqJb0gOLSq/gHYAJzSzoo6Anikqh4ArgbekGTfdmD7Da0mSRqiQZ46eynwdeDFSSaTnLaL4VcB9wITwB8DvwVQVVuBDwI3tp8PtJokaYiWDOqDq+rkju0r+5YLOH2GceuB9fPanCTpCfEKbklSJ8NCktTJsJAkdTIsJEmdDAtJUifDQpLUybCQJHUyLCRJnQwLSVInw0KS1MmwkCR1MiwkSZ0MC0lSJ8NCktTJsJAkdTIsJEmdDAtJUqdBPlZ1fZItSW7vq/1hkm8muTXJXybZp2/bWUkmktyd5Oi++jGtNpHkzEH1K0ma2SBnFhcCx+xQ2wj8YlW9DPi/wFkASQ4BTgJe2t7zR0n2SLIHcB5wLHAIcHIbK0kaooGFRVVdC2zdofb5qtrWVq8Dlrfl1cBlVfVoVX0LmAAObz8TVXVvVT0GXNbGSpKGaJTHLN4O/FVbXgZs7ts22Woz1XeSZG2STUk2TU1NDaBdSVq8RhIWSX4X2AZcsr00zbDaRX3nYtW6qhqvqvGxsbH5aVSSBMCSYX9hkjXAm4Ajq2r7H/5JYEXfsOXA/W15prokaUiGOrNIcgzwfuC4qvpB36YNwElJ9kpyELAKuAG4EViV5KAke9I7CL5hmD1LkgY4s0hyKfBaYGmSSeBsemc/7QVsTAJwXVW9o6ruSHI5cCe93VOnV9WP2+e8E7ga2ANYX1V3DKpnSdL0BhYWVXXyNOULdjH+HOCcaepXAVfNY2uSpCfIK7glSZ0MC0lSJ8NCktTJsJAkdTIsJEmdDAtJUifDQpLUybCQJHUyLCRJnQwLSVInw0KS1MmwkCR1MiwkSZ0MC0lSJ8NCktTJsJAkdTIsJEmdBhYWSdYn2ZLk9r7afkk2Jrmnve7b6knysSQTSW5Ncmjfe9a08fckWTOofiVJMxvkzOJC4JgdamcC11TVKuCatg5wLLCq/awFzodeuNB7dvcrgcOBs7cHjCRpeAYWFlV1LbB1h/Jq4KK2fBFwfF/94uq5DtgnyQHA0cDGqtpaVQ8DG9k5gCRJAzbsYxbPq6oHANrr/q2+DNjcN26y1Waq7yTJ2iSbkmyampqa98YlaTFbKAe4M02tdlHfuVi1rqrGq2p8bGxsXpuTpMVu2GHxYNu9RHvd0uqTwIq+ccuB+3dRlyQN0bDDYgOw/YymNcCVffVT2llRRwCPtN1UVwNvSLJvO7D9hlaTJA3RkkF9cJJLgdcCS5NM0jur6UPA5UlOA+4DTmzDrwLeCEwAPwBOBaiqrUk+CNzYxn2gqnY8aC5JGrCBhUVVnTzDpiOnGVvA6TN8znpg/Ty2Jkl6ghbKAW5J0gJmWEiSOhkWkqROhoUkqdOswiLJNbOpSZKenHZ5NlSSpwF70zv9dV8ev6L62cALBtybJGmB6Dp19t8D76EXDDfxeFj8I3DeAPuSJC0guwyLqvoo8NEk76qqjw+pJ0nSAjOri/Kq6uNJXg2s7H9PVV08oL4kSQvIrMIiyZ8CBwO3AD9u5QIMC0laBGZ7u49x4JB2Ww5J0iIz2+ssbgeeP8hGJEkL12xnFkuBO5PcADy6vVhVxw2kK0nSgjLbsPj9QTYhSVrYZns21JcG3YgkaeGa7dlQ3+PxZ1/vCTwV+H5VPXtQjUmSFo7Zziye1b+e5Hjg8IF0JElacOZ019mq+izwurl+aZL/mOSOJLcnuTTJ05IclOT6JPck+fMke7axe7X1ibZ95Vy/V5I0N7PdDfXmvtWn0LvuYk7XXCRZBryb3nUb/5zkcuAkes/gPreqLkvySeA04Pz2+nBVvTDJScCHgd+Yy3dLkuZmtjOLX+/7ORr4HrD6Z/jeJcDTkyyhd1fbB+jNVK5o2y8Cjm/Lq9s6bfuRSYIkaWhme8zi1Pn6wqr6+yQfAe4D/hn4PL072n63qra1YZPAsra8DNjc3rstySPAc4GH+j83yVpgLcCBBx44X+1Kkpj9w4+WJ/nLJFuSPJjk00mWz+UL23MxVgMH0bv1+TOAY6cZun0313SziJ12gVXVuqoar6rxsbGxubQmSZrBbHdDfQrYQO+P+zLgf7XaXLwe+FZVTVXVj4DPAK8G9mm7pQCWA/e35UlgBUDb/hxg6xy/W5I0B7MNi7Gq+lRVbWs/FwJz/d/3+4Ajkuzdjj0cCdwJfBE4oY1ZA1zZlje0ddr2L3hDQ0kartmGxUNJ3ppkj/bzVuA7c/nCqrqe3oHqm4HbWg/rgPcDZySZoHdM4oL2lguA57b6GcCZc/leSdLczfbeUG8HPgGcS+94wdeAOR/0rqqzgbN3KN/LNBf6VdUPgRPn+l2SpJ/dbMPig8CaqnoYIMl+wEfohYgk6UlutruhXrY9KACqaivwisG0JElaaGYbFk9pp7wC/zKzmO2sRJK0m5vtH/z/CnwtyRX0jln8W+CcgXUlSVpQZnsF98VJNtG7JUeAN1fVnQPtTJK0YMx6V1ILBwNCkhahOd2iXJK0uBgWkqROhoUkqZNhIUnqZFhIkjoZFpKkToaFJKmTYSFJ6mRYSJI6GRaSpE6GhSSp00jCIsk+Sa5I8s0kdyV5VZL9kmxMck973beNTZKPJZlIcmuSQ0fRsyQtZqOaWXwU+OuqegnwS8Bd9J6tfU1VrQKu4fFnbR8LrGo/a4Hzh9+uJC1uQw+LJM8G/g1wAUBVPVZV3wVWAxe1YRcBx7fl1cDF1XMdsE+SA4bctiQtaqOYWfw8MAV8KsnfJvmTJM8AnldVDwC01/3b+GXA5r73T7baT0myNsmmJJumpqYG+y+QpEVmFGGxBDgUOL+qXgF8n8d3OU0n09Rqp0LVuqoar6rxsbGx+elUkgSMJiwmgcmqur6tX0EvPB7cvnupvW7pG7+i7/3LgfuH1KskiRGERVX9A7A5yYtb6Uh6T+DbAKxptTXAlW15A3BKOyvqCOCR7burJEnDMevHqs6zdwGXJNkTuBc4lV5wXZ7kNOA+4MQ29irgjcAE8IM2VpI0RCMJi6q6BRifZtOR04wt4PSBNyVJmpFXcEuSOhkWkqROhoUkqZNhIUnqZFhIkjoZFpKkToaFJKmTYSFJ6mRYSJI6GRaSpE6GhSSpk2EhSepkWEiSOhkWkqROhoUkqZNhIUnqZFhIkjqNLCyS7JHkb5N8rq0flOT6JPck+fP2yFWS7NXWJ9r2laPqWZIWq1HOLH4buKtv/cPAuVW1CngYOK3VTwMerqoXAue2cZKkIRpJWCRZDvwa8CdtPcDrgCvakIuA49vy6rZO235kGy9JGpJRzSz+O/A+4Cdt/bnAd6tqW1ufBJa15WXAZoC2/ZE2/qckWZtkU5JNU1NTg+xdkhadoYdFkjcBW6rqpv7yNENrFtseL1Stq6rxqhofGxubh04lSdstGcF3vgY4LskbgacBz6Y309gnyZI2e1gO3N/GTwIrgMkkS4DnAFuH37YkLV5Dn1lU1VlVtbyqVgInAV+oqrcAXwROaMPWAFe25Q1tnbb9C1W108xCkjQ4C+k6i/cDZySZoHdM4oJWvwB4bqufAZw5ov4kadEaxW6of1FVfwP8TVu+Fzh8mjE/BE4camOSpJ+ykGYWkqQFyrCQJHUyLCRJnQwLSVInw0KS1MmwkCR1MiwkSZ0MC0lSJ8NCktTJsJAkdTIsJEmdDAtJUifDQpLUybCQJHUyLCRJnQwLSVInw0KS1GnoYZFkRZIvJrkryR1JfrvV90uyMck97XXfVk+SjyWZSHJrkkOH3bMkLXajmFlsA/5TVf0CcARwepJD6D1b+5qqWgVcw+PP2j4WWNV+1gLnD79lSVrchh4WVfVAVd3clr8H3AUsA1YDF7VhFwHHt+XVwMXVcx2wT5IDhty2JC1qIz1mkWQl8ArgeuB5VfUA9AIF2L8NWwZs7nvbZKvt+Flrk2xKsmlqamqQbUvSojOysEjyTODTwHuq6h93NXSaWu1UqFpXVeNVNT42NjZfbUqSGFFYJHkqvaC4pKo+08oPbt+91F63tPoksKLv7cuB+4fVqyRpNGdDBbgAuKuq/lvfpg3Amra8Briyr35KOyvqCOCR7burJEnDsWQE3/ka4DeB25Lc0mr/GfgQcHmS04D7gBPbtquANwITwA+AU4fbriRp6GFRVV9h+uMQAEdOM76A0wfalCRpl7yCW5LUybCQJHUyLCRJnQwLSVInw0KS1MmwkCR1MiwkSZ0MC0lSJ8NCktTJsJAkdTIsJEmdDAtJUifDQpLUybCQJHUyLCRJnQwLSVInw0KS1Gm3CYskxyS5O8lEkjNH3Y8kLSa7RVgk2QM4DzgWOAQ4Ockho+1KkhaP3SIsgMOBiaq6t6oeAy4DVo+4J0laNJaMuoFZWgZs7lufBF7ZPyDJWmBtW/2nJHcPqbfFYCnw0KibWAjykTWjbkE78/dzu7Pzs37Cz820YXcJi+n+C9RPrVStA9YNp53FJcmmqhofdR/SdPz9HI7dZTfUJLCib305cP+IepGkRWd3CYsbgVVJDkqyJ3ASsGHEPUnSorFb7Iaqqm1J3glcDewBrK+qO0bc1mLi7j0tZP5+DkGqqnuUJGlR2112Q0mSRsiwkCR1Miw0Z0lem+Rzo+5Di0OSf5qh/o4kp7TltyV5wXA7Wxx2iwPcGq4koXc86yej7kXqUlWf7Ft9G3A7nlo/75xZCIAkK5PcleSPgJuB30zy9SQ3J/mLJM9s445J8s0kXwHePNKm9aSS5H1J3t2Wz03yhbZ8ZJI/a8vnJPlGkuuSPK/Vfj/Je5OcAIwDlyS5JcnTkxyW5EtJbkpydZIDRvXv290ZFur3YuBi4CjgNOD1VXUosAk4I8nTgD8Gfh34FeD5o2pUT0rX0vu9gt4f/WcmeSrwr4EvA88ArquqX2pj/13/m6vqCnq/q2+pqpcD24CPAydU1WHAeuCcYfxDnozcDaV+366q65K8id7dfb/a2yPFnsDXgZcA36qqewDa/+2tnenDpCfoJuCwJM8CHqU3wx2nFyDvBh4DPtc39qiOz3sx8IvAxvZ7vAfwwPy3vTgYFur3/fYaYGNVndy/McnL2eGeXNJ8qaofJfk74FTga8CtwK8CBwN3AT+qxy8M+zHdf78C3FFVrxpMx4uLu6E0neuA1yR5IUCSvZO8CPgmcFCSg9u4k2f6AGmOrgXe216/DLwDuKVmf/Xw94BnteW7gbEkrwJI8tQkL53nfhcNw0I7qaopemeVXJrkVnrh8ZKq+iG93U7/ux3g/vboutST1JeBA4CvV9WDwA9bbbYuBD6Z5BZ6u51OAD6c5BvALcCr57fdxcPbfUiSOjmzkCR1MiwkSZ0MC0lSJ8NCktTJsJAkdTIspHmQZJ8kvzXqPqRBMSyk+bEPYFjoScuwkObHh4CD291O/yLJ6u0bklyS5Lj2rIUrk/x1kruTnN035q1Jbmjv/x9J9hjJv0KagWEhzY8zgf/X7nb6CXr3NyLJc+hdNXxVG3c48Bbg5cCJScaT/ALwG8Br2vt/3MZIC4Y3EpTmWVV9Kcl5Sfan98yPT1fVtnbn041V9R2AJJ+hd/vtbcBhwI1tzNOBLSNpXpqBYSENxp/Smx2cBLy9r77j/XWK3t1RL6qqs4bUm/SEuRtKmh/9dzuF3g3t3gNQVXf01Y9Ksl+SpwPHA18FrgFOaDMR2vafG0rX0iw5s5DmQVV9J8lXk9wO/FVV/U6Su4DP7jD0K/RmHS8E/mdVbQJI8l+Azyd5CvAj4HS8q68WEO86Kw1Akr2B24BDq+qRVnsbMF5V7xxlb9JcuBtKmmdJXk/vQVEf3x4U0u7OmYUkqZMzC0lSJ8NCktTJsJAkdTIsJEmdDAtJUqf/DyxAfgEtA+lKAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "df = df.iloc[:2*len(df_red)]\n",
    "sns.countplot(x='type',data=df)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Converting categorical data into numbers"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'red': 0, 'white': 1}"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dict_cat = dict(zip(df['type'].unique(), np.arange(df['type'].nunique())))\n",
    "dict_cat"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "df['type'] = df['type'].apply(lambda x: dict_cat[x])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Correlations**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x1a1fb75550>"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAoMAAAIqCAYAAABffUCoAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOzdeZwlZXn3/8+XAQUBQUUJuI1RkCDCKAOKLKKiwSUSEBQlymjMhBhEMRjJgzHqLzyB6BN/7mQkChKiBkwUDUEQZQlujDCsgvoARgKCrLJvfT1/nJqmbE4vM9Pd1TP1efM6r65z133fdVX3MHP1dVfVSVUhSZKkflqr6wAkSZLUHZNBSZKkHjMZlCRJ6jGTQUmSpB4zGZQkSeoxk0FJkqQeWy2SwSSHJPlJkluTHD5Nc945DXMsTPKJcfZdk2STZvt7zdf5Sd60qseVJEmaLlkdnjOY5ArglVV19TTOeWdVbTBd8w2Z/xpgYVXd1GrbHTisql4zU8eVJElaEXO+MpjkGOB3gVOSHJrkU03715O8pdn+0yQnNtvPTHJakh8nOTfJVk37M5J8P8n5Sf6/CY732SRLk1yW5EOt9h2SfC/JRUl+lGTDJLsn+Waz/wlJTk9yYZJ/BNIau7wKeRSwa5Jlzbmcm2RBq995Sbadpm+dJEnSpOZ8MlhVBwHXAS8Bbm3tWgx8IMmuwF8A72zalwDvrKrtgcOAzzTtHwc+W1U7AL+a4JBHVNVCYFvgxUm2TfIo4CvAu6pqO2AP4J4x4/4G+K+qeh5wCvC0IXMfDpxbVQuq6mPAscAigCRbAo+uqosn/IZIkiRNo7W7DmBlVdUNST4AfBfYu6puSbIB8CLgpGS0MPfo5uvOwOua7ROAo8eZ+vVJFjP43mwGbA0UcH1Vnd8c+zcArWMA7Abs0+z/jyTtxHU8JwF/neS9wNuA48br2MS0GOAz/+dvt3/7W944helnznnPeV+nx2/b9kU3dh0CAB+58MldhwDAHz/m5q5DGPXuO+fGXzF3jdzfdQgAfOOADbsOYdQfnHhH1yEAcMKTH+o6BAC+9T+bdx3CqJGuA2j840O/6DqEUUuvPzeT95o+D9x01YxcQ7fOJr87q+cxVXPjb+qV91zgZmD5/8VrAbdV1YJx+k/4w03yDAbVxB2q6tYkxwHrMljyncofjBX6w1NVdyc5A9gLeD2wcIK+SxhUPWfsD6kkSeqfOb9MPJ4kOwKvBJ4HHJbkGU3F7uok+zV9kmS7Zsh5wP7N9gHjTPtY4C7g9iSbNvMDXAFsnmSHZt4Nk4xNpM9ZPm+SVwKPGzL/HcDY0sCxwCeA86vqlklOW5IkzbSRh2bmNUetlslgkkcDnwPeVlXXMbhm8PMZrNseAPxxkouAyxhU3QDeBfx5kvOBjcbMtwygqi4CLmzGfZ5BAklV3Q+8AfhkM+8ZDCqGbR8CdktyAfAK4L+HhH4x8GBzE8qhzdw/Bn4DfGElvx2SJGk61cjMvOao1WKZuKrmN5vH8fB1ddu19p/C4KYNgKuBPYfMcTWwU6vpqNa+Ba3tRePEcD7wwjHNZzUvqupmBkngcoe2xm7QfH0AeFl7giSbM0jKTx92XEmSpJm0WiSDa6rm0ThHAu+pmsO/MkiS1Ccj/fon2WSwQ1X1ReCLXcchSZL6y2RQkiSppW+LdavlDSSSJEmaHlYGJUmS2rxmUJIkqcdcJpYkSVJfWBmUJElqm8OfFjITrAxKkiT1mJVBSZKktp5dM2gyKEmS1Nazu4ldJpYkSeoxK4OrofOe876uQ2Dny47uOoRRW221b9chALDLBvd1HQIAj93s3q5DGHXzFXPj983bHrir6xAAuP2cuXNR+hc2mdd1CADc9KsNug4BgN/MjT+qAJzJrV2HAMCyG6/qOoTO+AkkkiRJ6g0rg5IkSW09u2bQZFCSJKnNZWJJkiT1hZVBSZKkNj+BRJIkSbMtyZ5Jrkzy8ySHD9n/9CRnJrk4yVlJnjIdxzUZlCRJaquRmXlNIMk84NPAK4GtgTcm2XpMt48CX6yqbYEPA383HadrMihJktS9HYGfV9VVVXU/8GVgrzF9tgbObLa/O2T/SjEZlCRJahsZmZFXksVJlrZei1tHfTLwy9b7a5u2touA1zXbewMbJnnCqp6uN5BIkiS1zdCjZapqCbBknN0ZNmTM+8OATyVZBJwD/A/w4KrGZTIoSZLUvWuBp7bePwW4rt2hqq4D9gFIsgHwuqq6fVUPbDIoSZLU1s0nkJwPbJHkGQwqfvsDb2p3SLIJcEsNPjz5r4DPT8eBvWZQkiSpY1X1IHAw8C3gJ8C/VtVlST6c5LVNt92BK5P8FNgUOHI6jm1lUJIkqaWqm4dOV9WpwKlj2j7Q2j4ZOHm6jzunK4NJ5ie5dAp93tR6vzDJJ5rtRUk+NYPxHZTkLePEdOmQeHZP8qKZikeSJE2DDp4z2KU1oTI4n8Ga+r8AVNVSYOlsHLiqjplCn3Y8uwN3At+bwbAkSZKmbNYrg0mOTvKO1vsPJvmLJB9JcmmSS5K8Yci4+UnOTXJB81peYTsK2DXJsiSHNtW3bw4Z/8QkX01yfvPaeUifDZqPebmgiWOv1r63NB//clGSE1qxH9Zsb9/s+z7w561xuyf5ZpL5wEHAoU2suya5Osk6Tb/HJrlm+XtJktSRGXrO4FzVxTLxl4F2svd64CZgAbAdsAfwkSSbjRl3I/Dyqnp+M/4TTfvhwLlVtaCqPjbBcT8OfKyqdmDwwMZjh/S5F9i7OcZLgP+TgecARwAvrartgHcNGfsF4JCq2mnYwavqGuCYJoYFVXUucBbw6qbL/sBXq+qBCc5BkiRpWs16MlhVFwJPSrJ5ku2AWxkkgl+qqoeq6gbgbGCHMUPXAT6X5BLgJAYfybIi9mDwoMZlwCnAY5NsOKZPgP+d5GLg2wye/L0p8FLg5Kq6qTmHW35rULIRsHFVnd00nTDFmI4F3tpsv5VBQjlU+6nl37jnqilOL0mSVpjXDM6Kk4F9gd9hUCl85hTGHArcwKB6uBaDKt6KWAvYqarumaDPAcATge2r6oEk1wDrMkgSxz4FvG2y/UNV1XnN8veLgXlVNe7NMu2nlp+16X4rfCxJkjRFI93cTdyVru4m/jKDZdF9GSSG5wBvSDIvyROB3YAfjRmzEXB986DFNwPzmvY7gLEVvmFOZ/D8HgCSLBjSZyPgxiYRfAnw9Kb9TOD1yz//L8nj24Oq6jbg9iS7NE0HjBPDsFi/CHyJCaqCkiRJM6WTZLCqLmOQFP1PVV0P/DtwMYMPYP4O8JdV9asxwz4DHJjkB8CWwF1N+8XAg83NG4dOcNhDgIXNTSCXM7iZY/mjX5ZfP3hi02cpg4Tuila8RwJnJ7kI+Ich878V+HRzA8l41cdvAHsvv4GkdczHMUgIJUlS13q2TJwqVxy7lGRfYK+qevNUx8yFZeKdLzu66xBGbbXVvl2HAMAuGzyj6xAA+Mgzb+o6hFF7XTE3HmV62wN3Td5pFnz7mRt0HcKoB+6ZN3mnWXDbret1HQIA363Hdh3CqDO5tesQADjtV8u6DmHUg/f/T2bzePf+6KQZ+Xd23R33m9XzmKo14TmDq60knwReCbyq61gkSVJjDj8GZiaYDHaoqt7ZdQySJKnfTAYlSZLa5vD1fTPBZFCSJKmtZ8vEc+PqbkmSJHXCyqAkSVKblUFJkiT1hZVBSZKklqp+fRydyaAkSVKby8SSJEnqCyuDkiRJbT17zqCVQUmSpB6zMrga2vZFN3YdAltttW/XIYy64oqTuw4BgCt3PKTrEAA47Yqndh3CqE/mzq5DAGDDx6/bdQgAXHLVRl2HMOovuLrrEAD44FpP6joEAJ5C8WC6jmLgr0bW6zoEAD6+5TZdh9Cdnl0zaDIoSeq9uZIIao5wmViSJEl9YWVQkiSprWfLxFYGJUmSeszKoCRJUlvPrhk0GZQkSWpzmViSJEl9YWVQkiSpzcqgJEmS+sLKoCRJUlvPbiCxMihJktRjVgYlSZLaenbNoMmgJElSm8vEkiRJ6gsrg5IkSW09WybuTWUwyUFJ3tJsL0qy+QR9P5xkjxmK47VJDh9n350zcUxJkqTx9KYyWFXHtN4uAi4FrhvbL8m8qvrADMZxCnDKTM0vSZJWUc+uGVxjk8GmCngYUMDFwP8F7gSuARYCJya5B9gJ+AnweeAVwKeS7Al8s6pOTrID8HFgfeA+4GVVdceYY30NeCqwLvDxqlrStO8J/G9gHnBTVb0sySJgYVUdnOQZwL8w+DmcNlPfC0mStAJcJl79JXkOcATw0qraDnjX8n1VdTKwFDigqhZU1T3Nrnurapeq+nJrnkcBXwHe1cyzB7C8f9vbqmp7BknmIUmekOSJwOeA1zVj9xsy7uPAZ6tqB+BXk5zT4iRLkyw9/prrp/R9kCRJmsyaWhl8KXByVd0EUFW3JJlszFeGtD0buL6qzm/m+c04Yw9Jsnez/VRgC+CJwDlVdfXyGIaM2xl4XbN9AnD0eME11cYlALfs/eKa+FQkSdJK61llcE1NBsNgeXhF3LUy8yTZnUHFcKequjvJWQyWi6cag4mdJEnqzBq5TAycCbw+yRMAkjx+zP47gA2nMM8VwObNdYMk2TDJ2AR6I+DWJhHcCnhh0/594MXNdYHDYgA4D9i/2T5gCvFIkqSZVjUzrzlqjawMVtVlSY4Ezk7yEHAhgxtHljsOOKZ1A8l489yf5A3AJ5Osx+B6wT2SPBY4tqpexeDGj4OSXAxcCfygGfvrJIuBf0uyFnAj8PIxh3gX8C9J3gV8dVXPW5IkTQOXidcMVXU8cPw4+77Kbydf88fsX9TaPp+Hq33L3Qm8qtl/H/DKcY7zn8B/jmk7jkEySnM9YTsZPWroyUiSJM2QNTYZlCRJWik9qwyuqdcMSpIkaQqsDEqSJLX17BNIrAxKkiT1mJVBSZKktp5dM2gyKEmS1DaHnwk4E1wmliRJ6jErg5IkSW09Wya2MihJktRjVgYlSZLaelYZNBmUJElq8zmDkiRJ6gsrg6uhj1z45K5DYJcN7us6hFFX7nhI1yEA8OwffaLrEAD4p4X/q+sQRu2zX7oOAYCsv3HXIQDwxIuu6zqEUdtesFnXIQDwn5kbf5fcOHJP1yGM2rMe13UIAPzRdrd0HUJnasRHy0iSJKknrAxKkiS1eQOJJElSj3kDiSRJkvrCyqAkSVKbN5BIkiSpL6wMSpIktfXsBhIrg5IkST1mZVCSJKmtZ5VBk0FJkqS28gYSSZIk9YTJoCRJUtvIyMy8JpFkzyRXJvl5ksPH6fP6JJcnuSzJv0zH6bpMLEmS1LEk84BPAy8HrgXOT3JKVV3e6rMF8FfAzlV1a5InTcexTQYlSZLaunno9I7Az6vqKoAkXwb2Ai5v9fkT4NNVdStAVd04HQdebZeJkxyU5C1D2ucnuXQV5j0rycJVi06SJK22amRGXkkWJ1naei1uHfXJwC9b769t2tq2BLZMcl6SHyTZczpOd85UBpMESNXUPh26qo6Z4ZA6k2Ttqnqw6zgkSdL0qaolwJJxdmfYkDHv1wa2AHYHngKcm2SbqrptVeLqtDLYVPF+kuQzwAXAm5N8P8kFSU5KskHT76jmYsmLk3y0aftgksOa7e2TXJTk+8Cft+ZflORTrfffTLJ7s/3ZJiu/LMmHphDrIa0Yvjw2hub9pUnmN9t/neSKJGck+VIr1j9Jcn4T71eTPKZpPy7JPyT5LnD0KnxbJUnSqhipmXlN7Frgqa33TwGuG9Ln61X1QFVdDVzJIDlcJXNhmfjZwBcZXDD5x8AeVfV8YCnwniSPB/YGnlNV2wJ/O2SOLwCHVNVOK3DcI6pqIbAt8OIk207S/3DgeU0MB03UsVlmfh3wPGAfoL3s/G9VtUNVbQf8hME5L7clg/P/iyFzjpaWl93x88nOTZIkrV7OB7ZI8owkjwL2B04Z0+drwEsAkmzCIG+4alUPPBeSwV9U1Q+AFwJbA+clWQYcCDwd+A1wL3Bskn2Au9uDk2wEbFxVZzdNJ0zxuK9PcgFwIfCc5tgTuRg4MckfAZMt4e7CIHO/p6ruAL7R2rdNknOTXAIc0Bx7uZOq6qFhE1bVkqpaWFULF2z4rEkOL0mSVlaNjMzIa8JjDi4POxj4FoNi0b9W1WVJPpzktU23bwE3J7kc+C7w3qq6eVXPdy5cM3hX8zXAGVX1xrEdkuwIvIxBlnww8NL2bh65pr7cg/x2wrtuM98zgMOAHZpbs49bvm8CrwZ2A14L/HWS54w3P8PX/Zc7DvjDqrooySIG6/7L3TVsgCRJmkXd3E1MVZ0KnDqm7QOt7QLe07ymzVyoDC73A2DnJM8CSPKYJFs21w1u1HyD3g0saA9qLpq8PckuTdMBrd3XAAuSrJXkqQxu2wZ4LIPE6/YkmwKvnCiwJGsBT62q7wJ/CWwMbNDM//ymz/OBZzRD/gv4gyTrNvG/ujXdhsD1SdYZE6skSdKsmwuVQQCq6tdNpexLSR7dNL8fuAP4epJ1GVTcDh0y/K3A55PczaCEutx5wNXAJcClDG5SoanKXQhcxmCt/bxhMSU5FjgGuAj452ZJOsDHquq2JF8F3tIsa58P/LSZ//wkpzTjfsHg+sfbm2n/Gvhh034Jg+RQkiTNFVN7sMkao9NksKquAbZpvf8OsMOQrjuObaiqD7a2fwxs19r9waa9GKf6VlWLxmnfvbX99tauXYb0vQd4xbB5gI9W1Qebu4XPAf5PM+azwGenGo8kSdJMmjOVwTXQkiRbM7iO8PiquqDrgCRJ0hR0dM1gV0wGZ0hVvanrGCRJkiZjMihJktQ2yWNg1jQmg5IkSW09WyaeS4+WkSRJ0iyzMihJktTWs0fLWBmUJEnqMSuDkiRJbT27ZtBkUJIkqaV6djexy8SSJEk9ZmVQkiSprWfLxFYGJUmSeszK4Grojx9zc9ch8NjN7u06hFGnXfHUrkMA4J8W/q+uQwDg75f+765DGPWnC/+y6xAA+NXI7V2HAMDxz+46goft8dAGXYcAwH6/f0PXIQDw12c+oesQRv2UB7oOAYAdT/9N1yGM+ulsH7BnlUGTQUmSpDafMyhJkqS+sDIoSZLU1rNlYiuDkiRJPWZlUJIkqaWsDEqSJKkvrAxKkiS19awyaDIoSZLU5mcTS5IkqS+sDEqSJLX1bJnYyqAkSVKPWRmUJElq61ll0GRQkiSppapfyaDLxJIkST3W22QwyXFJ9h3SPj/JpSs41+ZJTh5n31lJFq5snJIkaZaN1My85iiXiVdRkrWr6jrgEYmlJEnSXNebymCStyS5OMlFSU5omndL8r0kV41TJVw3yReSXJLkwiQvadoXJTkpyTeA09vVxCTrJflyc6yvAOu15ntFku8nuaAZv0HTflSSy5sxH53xb4YkSRqflcE1T5LnAEcAO1fVTUkeD/wDsBmwC7AVcAowdqn3zwGq6rlJtmKQ+G3Z7NsJ2LaqbkkyvzXmz4C7q2rbJNsCFzQxbAK8H9ijqu5K8j7gPUk+BewNbFVVlWTj6T5/SZI0dTWHE7eZ0JfK4EuBk6vqJoCquqVp/1pVjVTV5cCmQ8btApzQjLkC+AWwPBk8ozVP227APzdjLgYubtpfCGwNnJdkGXAg8HTgN8C9wLFJ9gHuHnYCSRYnWZpk6ZdvvXbqZy5JkjSBXlQGgQDD0vz7xvQZNm48d02wb9ixwiCBfOMjdiQ7Ai8D9gcOZpC8/vaEVUuAJQA/3/r3+/UriyRJs8nK4BrpTOD1SZ4A0CwTT8U5wAHNmC2BpwFXrsCYbYBtm/YfADsneVaz7zFJtmyuG9yoqk4F3g0smPJZSZIkraJeVAar6rIkRwJnJ3kIuHCKQz8DHJPkEuBBYFFV3ZdMVDDks8AXklwMLAN+1MTw6ySLgC8leXTT9/3AHcDXk6zLoHp46IqdnSRJmlYjXQcwu3qRDAJU1fHA8RPs36D5eg2wTbN9L7BoSN/jgONa79tj7mGw3DvsGN8Bdhiya8cpnIIkSdK0600yKEmSNBV9u5vYZFCSJKmtZ8lgX24gkSRJ0hBWBiVJktp6dgOJlUFJkqQeszIoSZLU4g0kkiRJfeYysSRJkvrCyqAkSVJL35aJrQxKkiT1mJVBSZKktp5dM2gyKEmS1FImg5rr3n1n9z+2m6+YO1cYfDJ3dh0CAPvsl65DAOBPF/5l1yGM+self991CAA8eMFpXYcAwGfe/r2uQxh17doPdh0CAGd+e4OuQwBgwzzUdQijXnNv93/HA/zdO5/ddQiaJXPjT5wkSdJc0bPK4Nwp70iSJGnWWRmUJElq6ds1g1YGJUmSeszKoCRJUlvPKoMmg5IkSS0uE0uSJKk3rAxKkiS1WBmUJElSb1gZlCRJaulbZdBkUJIkqa3mxseLzhaXiSVJknrMyqAkSVJL35aJrQxKkiT12Aolg0kOSfKTJCfOVEBTiGF+kktb77+U5OIkh07T/IuSfKrZPijJW1ZynmOTbD3R/JIkae6pkczIa65a0WXidwCvrKqrlzckWbuqHpzesKYmye8AL6qqp6/AmCnHW1XHrGxsVfX2lR0rSZK64zLxOJIcA/wucEqS25MsSXI68MUk85J8JMn5TZXuT1vj3ttq/9A4cx+V5PKmz0ebtuOS7Nvqc+eQoacDT0qyLMmuSc5KsrDpv0mSa5rtRUlOSvKNZszY4781yU+TnA3s3Gr/YJLDmu0FSX7QxPjvSR6XZO3m3HZv+vxdkiOb7XYs483/xCRfbeY4P8nOSJIkzaIpVwar6qAkewIvAQ4G/gDYparuSbIYuL2qdkjyaOC8JlHconntCIRBIrlbVZ2zfN4kjwf2Braqqkqy8QrE/1rgm1W1oJlror47AdtW1S3txiSbAR8CtgduB74LXDhk/BeBd1bV2Uk+DPxNVb07ySLg5CSHAHsCL1iB+T8OfKyq/ivJ04BvAb83xXOXJEkzoHy0zJSdUlX3NNuvAN6SZBnwQ+AJDJLAVzSvC4ELgK2a9rbfAPcCxybZB7h7FWKayBljE8HGC4CzqurXVXU/8JWxHZJsBGxcVWc3TccDuwFU1WXACcA3gLc1c0x1/j2ATzXft1OAxybZcFjwSRYnWZpk6X/f+d9TPWdJkrSaSLJnkiuT/DzJ4UP2H5TkkmZF9L+G3ZuwMlbl0TJ3tbbDoGr2rXaHJL8P/F1V/eN4k1TVg0l2BF4G7M+g6vhS4EGaZDWDkt+jphDT6Bhg3QnifUQYU5h7Is8FbgM2XcH51wJ2aiXV46qqJcASgNc87dWrGq8kSRpHF9cMJpkHfBp4OXAtcH6SU6rq8la3f1l+P0OS1wL/wGBVcpVM16NlvgX8WZJ1AJJsmWT9pv1tSTZo2p+c5Entgc2+jarqVODdwIJm1zUMllYB9gLWmUIc7TH7TtCv7YfA7kme0MS/39gOVXU7cGuSXZumNwNnN/Hvw6ASuhvwiSHL3BPNfzqD5JdmrgVIkqQ+2hH4eVVd1awkfplB/jOqqn7Ters+q17MAqbvodPHAvOBC5oq3q+BP6yq05P8HvD95nq+O4E/Am5McirwdgYn8vUk6zKoMC5/RMznmvYfAWcycWVvuY8C/5rkzcB3JuqYZFlVLaiq65N8EPg+cD2D5ex5Q4YcCByT5DHAVcBbk2wCHAW8rKp+2Twy5uNNXwAmmf8Q4NNJLmbwszgHOGgK5ylJkmbITD0GprnHYnGraUmz8gfwZOCXrX3XMuY+hGaOPwfew2DF9KXTEleVK46rm7mwTHzzQzN1aeeK+2Q26joEALZ649y44PiQk4b9LtONf1z6912HAMCDF5zWdQgAfObt3+s6hFHXrtXJE8Ee4Ya6r+sQANgwc+cDuV5z79yI5aXvmDufS7H+B06c1b9g/3vhy2bk39mnLT1z3PNIsh/w+8sfTdcUtnasqneO0/9NTf8Dh+1fEXPnJy1JktRf1wJPbb1/CnDdBP2/DPzhdBx4bvz6IUmSNEd09Gkh5wNbJHkG8D8Mbqp9U7tDki2q6mfN21cDP2MamAxKkiR1rHm6ysEMbr6dB3y+qi5rnm28tKpOAQ5OsgfwAHArrXsUVoXJoCRJUktXnyPcPFnl1DFtH2htv2smjmsyKEmS1NK3e2u9gUSSJKnHrAxKkiS1dLVM3BUrg5IkST1mZVCSJKmlql+VQZNBSZKklhrpOoLZ5TKxJElSj1kZlCRJahlxmVhz3V0j93cdArc9cFfXIYza8PHrdh0CAFl/465DAOBXI7d3HcKoBy84resQAFj7+Xt2HQIAN651btchjPpF3d11CADcOQf+PgNYZ976XYcw6va15nUdAgBrveCFXYegWWIyKEmS1OINJJIkST3mcwYlSZLUG1YGJUmSWvxsYkmSJPWGlUFJkqQWrxmUJElSb1gZlCRJavGh05IkST3Wt+cMukwsSZLUY1YGJUmSWny0jCRJknrDyqAkSVKLN5BIkiT1mDeQSJIkqTcmTAaTbJzkHZP0mZ/kTZMdqOl36YoGOGSeRUk+1Ww/MckPk1yYZNdVnbuZ87gk+zbbxybZeiXn+d5k80uSpLmnamZec9VklcGNgQmTQWA+MGkyOENeBlxRVc+rqnOnMiDJlJfGq+rtVXX5ygRWVS9amXGSJEmzabJk8CjgmUmWJflI87o0ySVJ3tDqs2vT59CmAnhukgua14RJUZL1k/xHkouaud/QtF+TZJNme2GSs8aMWwD8PfCq5tjrJbmztX/fJMc128cl+Yck3wWOHjNPknwqyeVJ/gN4UmvfWUkWNttvbM770iRHN21PT/KzJJskWas571c0++6cwvzbJzk7yY+TfCvJZpP8PCRJ0gwbqczIa66arEp2OLBNVS1I8jrgIGA7YBPg/CTnNH0Oq6rXACR5DPDyqro3yRbAl4CFExxjT+C6qnp1M36jqQReVcuSfABYWFUHN2MnGrIlsEdVPTSmfW/g2cBzgU2By4HPtzsk2ZxBErk9cCtwepI/rKqvNYnhMcAPgcur6vSpzJ9kHeCTwF5V9esmCT4SeNuw4JMsBhYDbLnxVmy+/lMmOldJkrSSvIFkfLsAX6qqh6rqBuBsYIch/dYBPpfkErEvNLUAACAASURBVOAkYLJr7i4B9khydJJdq+r2FYhpRZw0JBEE2I2Hz+s64DtD+uwAnFVVv66qB4ETm3FU1bHAhgwS5cNWYP5nA9sAZyRZBrwfGDfDq6olVbWwqhaaCEqSpOmyIo+WmWqafChwA4MK4lrAvRN1rqqfJtkeeBXwd0lOr6oPAw/ycLK67hSP3b48c+yYu6Y4bphxz72phC7PzjYA7pji/AEuq6qdJjm2JEmaRXN5SXcmTFYZvINB1QvgHOANSeYleSKDitePxvQB2Ai4vqpGgDcD8yY6QLMEe3dV/TPwUeD5za5rGCzLArxuSmcDNyT5vSRrMVienYpzgP2b89oMeMmQPj8EXtxcGzgPeCODyigMlo9PBD4AfG4F5r8SeGKSnQCSrJPkOVOMWZIkaVpMmAxW1c3Aec0jYXYCLgYuYrDU+ZdV9aum7cHmBpBDgc8AByb5AYPr9B5RkUuyeZJTm7fPBX7ULJUeAfxt0/4h4ONJzgWGLe8OczjwzSa+68fr1NyQcmzz9t+BnzFYrv4sDyd57e/D9cBfAd9tzv+Cqvp6khczWEI+uqpOBO5P8tYxw4fOX1X3A/sCRye5CFgGeAeyJEkdqxl6zVWpufzgGw31kqe8vPMf2q/un6lLO1fc1x7/hK5DAOAp+2/cdQgA7Pf5ufOz+eqxr+k6BADWfv6eXYcAwPsXHtF1CKN+VhNdOTN77hy5v+sQANhs3vpdhzDq5fev13UIALzuCy/sOoRR6/3+wbO6bvuDzfeZkX9nX3jdv83J9Wc/jk6SJKmlb9cMmgxKkiS1+GgZSZIk9YaVQUmSpJaRrgOYZVYGJUmSeszKoCRJUktN+XM21gwmg5IkSS0jnT/AbXa5TCxJktRjVgYlSZJaRnq2TGxlUJIkqcesDEqSJLV4A4kkSVKP9e05g6nq2S0za4A73zczH6C9Im4/57auQxh1yVVP6joEAHZ80fVdhwDAQ3d2HcHDvnjlU7sOAYAb13qo6xAA+NulR3YdwqhjnveBrkMAYO/Nr+s6BAAWXjk3/v8F2GWjLboOAYCf3vfrrkMYdfGvvj+rpbozNn3DjPw7+/IbvjInS45WBiVJklr6tkzsDSSSJEk9ZmVQkiSppW/XDFoZlCRJ6jErg5IkSS19qwyaDEqSJLV4A4kkSZJ6w8qgJElSy0i/CoNWBiVJkvrMyqAkSVLLSM+uGTQZlCRJaun8M19nmcvEkiRJPWZlUJIkqaVvzxm0MihJktRjJoNDJPlgksOmcb5Tk2zcvN4xXfNKkqTpN5LMyGuuMhmcBVX1qqq6DdgYMBmUJGkOqxl6zVUmg40kRyS5Msm3gWc3bc9MclqSHyc5N8lWTftxST6R5HtJrkqyb9O+WZJzkixLcmmSXZv2a5JsAhwFPLPZ/5EkJyTZqxXDiUleO+snL0mSestkEEiyPbA/8DxgH2CHZtcS4J1VtT1wGPCZ1rDNgF2A1zBI8gDeBHyrqhYA2wHLxhzqcOD/VtWCqnovcCzw1iaGjYAXAaeOE+PiJEuTLP38sqtX5XQlSdIERmboNVd5N/HArsC/V9XdAElOAdZlkJydlIfX+R/dGvO1qhoBLk+yadN2PvD5JOs0+8cmg7+lqs5O8ukkT2KQhH61qh4cp+8SBskpd75vn7lcbZYkSasRk8GHjU2w1gJua6p8w9zX2g5AVZ2TZDfg1cAJST5SVV+c5LgnAAcwqEy+bcXDliRJ08nPJu6nc4C9k6yXZEPgD4C7gauT7AeQge0mmiTJ04Ebq+pzwD8Bzx/T5Q5gwzFtxwHvBqiqy1b1RCRJklaElUGgqi5I8hUG1/j9Aji32XUA8Nkk7wfWAb4MXDTBVLsD703yAHAn8JYxx7k5yXlJLgX+s6reW1U3JPkJ8LVpPSlJkrRS/GzinqqqI4Ejh+zac0jfRWPeb9B8PR44fkj/+a3tN7X3JXkMsAXwpZUIW5IkTbO+XZjvMnGHkuwBXAF8sqpu7zoeSZLUPyaDHaqqb1fV06rq/+86FkmSNDCSmXlNJsmezTOPf57k8CH7H53kK83+HyaZPx3nazIoSZLUsSTzgE8DrwS2Bt6YZOsx3f4YuLWqngV8DDh6Oo5tMihJktTS0UOndwR+XlVXVdX9DG5a3WtMn714+N6Ek4GXJav+occmg5IkSS0dfTbxk4Fftt5f27QN7dN8SMXtwBNW9PzGMhmUJEmaBe2Plm1ei9u7hwwZm0NOpc8K89EykiRJLTP1CSTtj5Yd4lrgqa33TwGuG6fPtUnWBjYCblnVuKwMSpIkde98YIskz0jyKAYfU3vKmD6nAAc22/sC36kqK4OSJEnTaQo3e0y7qnowycHAt4B5wOer6rIkHwaWVtUpDD7q9oQkP2dQEdx/Oo5tMihJktTSRTIIUFWnAqeOaftAa/teYL/pPq7LxJIkST1mZVCSJKmlZugGkrnKZHA19Acn3tF1CHxhk3ldhzDqL7i66xAA2PaCzboOAYA9Htqg6xBGXbv2g12HAMAv6u6uQwDgmOd9YPJOs+SgCz/cdQgAnPec93UdAgCnbTR3/r+58v4Nuw4BgBMe9VDXIWiWmAxKkiS1dHXNYFe8ZlCSJKnHrAxKkiS19K0yaDIoSZLUsspPcV7NuEwsSZLUY1YGJUmSWmbqs4nnKiuDkiRJPWZlUJIkqcUbSCRJknqsb8mgy8SSJEk9ZmVQkiSpxUfLSJIkqTesDEqSJLX07dEyJoOSJEkt3kCiWZPkrCQLW+/nJ7m0y5gkSVK/WBmUJElq8QYSTbum4ndFkuOTXJzk5CSP6TouSZIkK4Oz59nAH1fVeUk+D7yjaT8xyT3N9qMY51KFJIuBxQBbbrwVm6//lJmOV5KkXhrpWW3QyuDs+WVVndds/zOwS7N9QFUtqKoFwKvGG1xVS6pqYVUtNBGUJEnTxcrg7Bn7a0a/fu2QJGk14d3EmilPS7JTs/1G4L+6DEaSJA1XM/Saq0wGZ89PgAOTXAw8Hvhsx/FIkiS5TDyLRqrqoDFtu7ffVNU1wDazFZAkSXokl4klSZLUG1YGZ4EVP0mSVh9+NrEkSVKP+ZxBSZIk9YaVQUmSpJZ+1QWtDEqSJPWalUFJkqSWvj1axmRQkiSpxRtIJEmS1BtWBiVJklr6VRe0MihJktRrVgYlSZJavIFEc94JT36o6xC46VcbdB3CqA+u9aSuQwDgP3Nf1yEAsN/v39B1CKPO/Pbc+HNy58j9XYcAwN6bX9d1CKPOe877ug4BgJ0vO7rrEAA4bZsjug5h1LpzZJHysruu7ToEzRKTQUmSpJa+3U1sMihJktTSr1TQG0gkSZJ6zcqgJElSS99uILEyKEmS1GNWBiVJklqqZ1cNmgxKkiS1uEwsSZKk3rAyKEmS1NK35wxaGZQkSeoxK4OSJEkt/aoLmgxKkiT9FpeJJUmS1BtWBiVJklp8tEwPJVmU5FOr2mfImHcnecyqRSdJkjRzTAZn1rsBk0FJklYjNUP/zVVrbDKYZP0k/5HkoiSXJnlDkmuSbNLsX5jkrCHjjktyTJJzk/w0yWtauzdPclqSnyX5+9aYzyZZmuSyJB9q2g4BNge+m+S7Tdsrknw/yQVJTkqyQdN+VJLLk1yc5KMz912RJEn6bWvyNYN7AtdV1asBkmwEHD3FsfOBFwPPZJDMPatpXwA8D7gPuDLJJ6vql8ARVXVLknnAmUm2rapPJHkP8JKquqlJQt8P7FFVdyV5H/CeZul5b2CrqqokG0/L2UuSpJXiNYNrjkuAPZIcnWTXqrp9Bcb+a1WNVNXPgKuArZr2M6vq9qq6F7gceHrT/vokFwAXAs8Bth4y5wub9vOSLAMObMb/BrgXODbJPsDdwwJKsripPi498cbrVuBUJEnSiujbMvEaWxmsqp8m2R54FfB3SU4HHuThBHjdiYaP8/6+VttDwNpJngEcBuxQVbcmOW6cuQOcUVVvfMSOZEfgZcD+wMHAS4eczxJgCcC1L3jp3P0TJUmSVitrbGUwyebA3VX1z8BHgecD1wDbN11eN8Hw/ZKsleSZwO8CV07Q97HAXcDtSTYFXtnadwewYbP9A2Dn5UvOSR6TZMvmusGNqupUBjecLFiB05QkSdNsZIZec9UaWxkEngt8JMkI8ADwZ8B6wD8l+V/ADycYeyVwNrApcFBV3ZtkaMequijJhcBlDJaUz2vtXgL8Z5Lrq+olSRYBX0ry6Gb/+xkkjF9Psi6D6uGhK3W2kiRJK2GNTQar6lvAt4bs2nJI3+OA41pN51XVoRP1qarXtLYXjRPDJ4FPtt5/B9hhSNcdh42XJEmzb6T6dTXWGpsMSpIkrYx+pYImg48wXpVPkiRpTWQyKEmS1DLSs9rgGns3sSRJkiZnZVCSJKllLj8geiaYDEqSJLXM5WcCzgSXiSVJknrMyqAkSVKLN5BIkiSpN0wGJUmSWmqG/lsVSR6f5IwkP2u+Pm5In6cn+XGSZUkuS3LQVOY2GZQkSWoZmaHXKjocOLOqtgDObN6PdT3woqpaALwAODzJ5pNNbDIoSZI09+0FHN9sHw/84dgOVXV/Vd3XvH00U8zzvIFkNfSt/5k0yZ9xv5lDv0b8TubGhb43jtzTdQgA/PWZT+g6hFEb5qGuQwBgnXnrdx0CAAuv/EXXIYw6baMNug4BgNO2OaLrEADY89Ijuw5h1N2H/knXIQBw8NnP7TqEzlTNzL8rSRYDi1tNS6pqyRSHb1pV1wNU1fVJnjTOMZ4K/AfwLOC9VXXdZBObDEqSJM2CJvEbN/lL8m3gd4bsmvJvTVX1S2DbZnn4a0lOrqobJhpjMihJktTS1aNlqmqP8fYluSHJZk1VcDPgxknmui7JZcCuwMkT9Z1Di32SJEkaxynAgc32gcDXx3ZI8pQk6zXbjwN2Bq6cbGKTQUmSpJY5ejfxUcDLk/wMeHnzniQLkxzb9Pk94IdJLgLOBj5aVZdMNrHLxJIkSS2r+kzAmVBVNwMvG9K+FHh7s30GsO2Kzm1lUJIkqcesDEqSJLX42cSSJEnqDSuDkiRJLTP10Om5ymRQkiSpZRru/F2tuEwsSZLUY1YGJUmSWubio2VmkpVBSZKkHrMyKEmS1NK3R8uYDEqSJLX07W5il4knkeSaJJusxLjjkuy7Av3nJ7l0RY8jSZK0KqwMSpIktfRtmdjKYEuSryX5cZLLkiwesv8tSS5OclGSE5q2pyc5s2k/M8nTWkN2S/K9JFctrxJm4CNJLk1ySZI3zNLpSZIkPYKVwd/2tqq6Jcl6wPlJvrp8R5LnAEcAO1fVTUke3+z6FPDFqjo+yduATwB/2OzbDNgF2Ao4BTgZ2AdYAGwHbNIc55zJAmuS08UAb954R168/harfraSJOkRfLRMvx2S5CLgB8BTgXbG9VLg5Kq6CaCqbmnadwL+pdk+gUHyt9zXqmqkqi4HNm3adgG+VFUPVdUNwNnADpMFVlVLqmphVS00EZQkSdPFymAjye7AHsBOVXV3krOAddtdYEq/KrT73DdmfPurJEmag0a8m7i3NgJubRLBrYAXjtl/JvD6JE8AaC0Tfw/Yv9k+APivSY5zDvCGJPOSPBHYDfjRdJyAJEladTVDr7nKyuDDTgMOSnIxcCWDpeJRVXVZkiOBs5M8BFwILAIOAT6f5L3Ar4G3TnKcf2ewtHwRgz8bf1lVv0oyf/pORZIkaWpMBhtVdR/wyiG75rf6HA8cP2bcNQyuJxw736Ix7zdovhbw3uY1dp5tViJ0SZI0jXy0jCRJknrDyqAkSVJL3yqDJoOSJEktfjaxJEmSesPKoCRJUkvflomtDEqSJPWYlUFJkqSWvn02scmgJElSizeQSJIkqTesDEqSJLV4A4kkSZJ6w8qgJElSS9+uGTQZXA2NdB0AcCa3dh3CqL8aWa/rEADYsx7XdQgA/JQHug5h1GvunRt/xdy+1ryuQwDgro226DqEUVfev2HXIQCw7hxZjrv70D/pOoRRj/nY57oOAYAdtnlf1yFolsyNv6klSZLmiL5dM2gyKEmS1NK35wx6A4kkSVKPWRmUJElqGenZDSRWBiVJknrMyqAkSVJL364ZNBmUJElqcZlYkiRJvWFlUJIkqaVvy8RWBiVJknrMyqAkSVJL364ZNBmUJElqcZlYkiRJvWFlUJIkqaVvy8RWBmdIkvlJLm22Fyb5RLO9e5IXdRudJEnSgJXBWVBVS4GlzdvdgTuB73UWkCRJGpfXDIokRyS5Msm3k3wpyWFJzkqysNm/SZJrmu35Sc5NckHzekTVr6kGfjPJfOAg4NAky5LsmuTqJOs0/R6b5Jrl7yVJkmaalcExkmwP7A88j8H35wLgxxMMuRF4eVXdm2QL4EvAwmEdq+qaJMcAd1bVR5vjnQW8Gvhac9yvVtUDQ+JaDCwG+KONd2S39bdYuROUJEkTqhrpOoRZZWXwkXYF/r2q7q6q3wCnTNJ/HeBzSS4BTgK2XsHjHQu8tdl+K/CFYZ2qaklVLayqhSaCkiTNnBFqRl5zlZXB4Yb9xB7k4eR53Vb7ocANwHbN/ntX6EBV5zVLzS8G5lXVpSsRryRJ0kqxMvhI5wB7J1kvyYbAHzTt1wDbN9v7tvpvBFxfg5rym4F5k8x/B7DhmLYvMlheHloVlCRJs6eqZuQ1V5kMjlFVFwBfAZYBXwXObXZ9FPizJN8DNmkN+QxwYJIfAFsCd01yiG8wSDaXJdm1aTsReByDhFCSJGnWuEw8RFUdCRwJkOSDTdsVwLatbu9v2n82pv2vmvZrgG2a7bOAs5rtn47pD7ALcHJV3Tad5yFJklbcXL6+byaYDHYsySeBVwKv6joWSZLEnF7SnQkmg5Ooqg/O8PzvnMn5JUmSJmIyKEmS1OJnE0uSJKk3rAxKkiS19O2ziU0GJUmSWvp2A4nLxJIkST1mZVCSJKmlb88ZtDIoSZLUY1YGJUmSWrxmUJIkSb1hZVCSJKmlbw+dTt9KoWuChZvt2vkPbdnNV3Udwqgrt9ym6xAAeMJ2D3QdAgA7nv6brkMYdeE7n911CACs9YKdug4BgBcceGLXIYx62qMe33UIAFx217VdhwDAwes/t+sQRu1w/31dhwDATpce3XUIo9bZ5Hczm8d73AbPmpF/Z2+98+ezeh5T5TKxJElSj7lMLEmS1OKjZSRJkjSnJHl8kjOS/Kz5+rhx+j0tyelJfpLk8iTzJ5vbZFCSJKmlqmbktYoOB86sqi2AM5v3w3wR+EhV/R6wI3DjZBO7TCxJktQyR+8m3gvYvdk+HjgLeF+7Q5KtgbWr6gyAqrpzKhNbGZQkSZoFSRYnWdp6LV6B4ZtW1fUAzdcnDemzJXBbkn9LcmGSjySZN9nEVgYlSZJaaoZuIKmqJcCS8fYn+TbwO0N2HTHFQ6wN7Ao8D/hv4CvAIuCfJhskSZKkjlXVHuPtS3JDks2q6vokmzH8WsBrgQur6qpmzNeAFzJJMugysSRJUstI1Yy8VtEpwIHN9oHA14f0OR94XJInNu9fClw+2cQmg5IkSS1z9G7io4CXJ/kZ8PLmPUkWJjm2ifsh4DDgzCSXAAE+N9nELhNLkiTNcVV1M/CyIe1Lgbe33p8BbLsic5sMSpIktczUDSRzlcvEkiRJPWZlUJIkqWUaru9brVgZXEVJNk7yjq7jkCRJWhkmg6tuY8BkUJKkNcQcvZt4xpgMrrqjgGcmWZbkpCR7Ld+R5MQkr02yKMnXk5yW5Mokf9Pq80dJftSM/8epfGyMJEmaOTVDrzlrprLfvryA+fD/2jvzaLuqKnt/MwiEJqFRREB6BERMIAFBEkFaG0DpAiKgQRRLoAw2aJUdiPpTgiidII0i0gsUSN8jFEEIAZLQCxagIPwoOhPpibP+2PsmN/e9vIDk7X3y7vrGuOPdc85L9hzvnnvOOmvvNRd35/ebARfm90sAD5PWZY4FngDeDiwC3A1sALwXuBhYMP+b44DPzGGcfYFJ+bXvPND9lv+PefT3Cx0N1RI6mqsldDRXS+hotpZ49f6KzOA8xPYNwBqS3gnsDpxv+/V8+Grbz9h+CfgvYDTJL2gkcJukyXl7tTn83yfa3iC/5tjX8E3wZppj9yehoydN0RI6etIULaGjJ03REjp60iQtQS9ENfG85zRgD+BTwOfa9ndmiE1yBj/V9n8W0hYEQRAEQTAbkRl860wHhrRt/wY4EMD2PW37t5a0tKRFgB2ACcC1wC45k0g+vnIR1UEQBEEQBERm8C1j+xlJEyTdDVxu+yBJ9wEXdvzqTaSs4RrAmU7tY5D0HeAqSYOA14D9gUcLSJ8XU83zgtDRk6ZoCR09aYqW0NGTpmgJHT1pkpagF5QXdwbzCEmLAncBI2z/Pe8bC2xg+4Ca2oIgCIIgCDqJaeJ5iKStgPuBY1qBYBAEQRAEQZOJzGAQBEEQBEEXE5nBIAiCIAiCLiaCwaAo0WElCIKgf4jra/CvEsFglyBp3doaMg9JOlzSOrWFNAVJC0g6vLaOJiJpscrjLyvpV5Iuz9vrSNqnkpbtsutAEMyJRlxfldhT0vfy9kqSPlBTU9A3cWHpHn6ZeyDvJ2nJijqGAX8CTpZ0i6R9JQ0tKUDSdEnT5vQqqQXA9gxgpCSVHrsTSeMlDZW0oKRrJT0tac8KOjaRdC9wX94eLum40jpIvqFXAsvn7T+RfUQr8CngwfwZvbf04JLukjS1l9ddkqZW0FP9XG3a34QGXF8zxwEfJHXiguTH+4sKOoI3SBSQdBGS3kPqijIGmAicYvvqino2Bc4ClgTOA35g+6GC4x8KPEnyfxSpc8wQ2+NLaWjTcgTwHuBc4IXWftv/VVjHZNvrSdqRZI7+FeB628ML67gV2AW4yPb6ed/dtotmuCXdZntDSXe26Zhse72SOtr0DCXdYPcmdTE6BTjL9vQCY7cM8QVcCny8/bjtEv6o7Xqqn6tzaxJQ+m/STs3rq6Q7bI/o+N5MKX0dCd44YTrdRdh+MJtcTwKOBtbP2ahvlQo68pqWbUk3s1WAI4AzgA8BlwFrltCR+Yjtjdq2j89BSPFgEFgaeAbYom2fSX2sS7Jg/vlxUpDxbK2Epe2/dow9o4KMFyS9ndxOUtLGQDXbKNvTJJ0PLELKUO4IHCTpaNvH9PPYMwMbSa/UDHQy1c/Vjr/JssCGeXOi7aeKiqFR19fXspbW92YZ4J+Fxg7+BSIY7BIkDSNdILYFrga2t32HpOWBP1Iu6HgQuB443PbNbfvPy0+yJZkhaQ/gbNJFa3fqBBzY3rvGuL1wsaT7gZeA/fJF/OUKOv4qaRPAkhYCvkyeMi7MV4GLgNUlTQCWIWUsiyPpE6Tv8OqkbPYHbD+Vje7vA/o1GGwgTTlXkbQrcDjwB1Lm9BhJB9k+r7CUplxfjwYuAJaV9CPSd+Y7BccP3iQxTdwlSLoROAk4z/ZLHcf2sn1aIR2jbd/UsW+U7Qklxu8YdxXgKGAUKRicABxo+5EKWgYD+wDvAwa39tv+XAUtSwHTbM9QKuAYYvvJwhreQfpstiLdXK8Cxtl+pqSOrOVtwFpZxwO2XyutIev4LXCy7Rt7Obal7Wv7efwRbZtnkJZVzMT2Hf05fm804VzNOqYAW7eygTkwvabC8oomXV/XBrbMm9fZrvEwF7xBIhjsEiQdaPvIjn3jbB9VWMcdtkfMbV+3IelcUveaTwOHkm6099keV1jHoqRs2Eq2983rTNeyfUlJHbWRtFNfx0uv5QSQdJjtb85tXz+Of33HrtbNQ4Btb0FBmnSuSrrL9vvbtgcBU9r3FdLRmOtrfngYTX7QrvGwELxxYpq4e/gMcGTHvrGk7Eu/I+mDwCbAMpK+2nZoKFDUG0vSN2yPl3QMs25oM7H95ZJ6MmvYHiPpk7ZPlXQmqYq1NKcAt5M+K4DHSEUtRW6wc/pMWhT8bLbPP99J+ltcl7c3J00FFg8Gga2BzsDvY73s6xdsbw4gaRFgP2bd6P8bOL6Ehg6qnqsdXCHpSlLBBsBupDV6RWjS9TXr+R6pUPF80sPCKZLOtf3D0lqCN0YEgwMcSbuTsk2rSrqo7dAQUsFCKRYCFiedc0Pa9k+j/Bqs1nTFpMLj9kVr6vF5JU/IJ0kLwEuzuu3d8nmD7ZdUdlV+6zMZBawDnJO3x5Bu/EVoreGUdAmwju0n8vZyFLbIkPQlUvC1eoddyRDS0obSnEr63h6dt3cHfgvsWlhH7XN1JrYPkrQz6bwVcKLtCwpKaNL1FdI5sb7tlwEk/QS4A4hgsKFEMDjwuRl4AngHqbKsxXSgmA+W7RuAGyT9pnYVou2L889Ta+ro4MS8/um7pIKFxYHvVdDxas78tKoAVwdeKTV46zORNBbYvLU+T9IvSesGS7NKKxDM/H/KVrwDnAlcDvwY+I+2/dNtP1tYC6Sp2Pa1cNfnNXOlqXqudmL7fFImrMbYjbm+Zh4hrX1uFfQsDPy5mppgrkQwOMDJF4ZHSQag1ZB0pO0DgWMl9TY1+4kKmq4Gxth+Pm8vBZxt+yOltdg+Ob+9AVit9PhtHAxcAawo6QxSpmNsBR3LkzIcrWBncWYZP5fkD23TfyYZP3eunetvbPsRSft3HpC0dIWA8E5JG9u+JWvYiDoZykPoea5WqcrPa0wPIy0rELPWURYxfG7g9fUV4J58jTVpicNNko7OemosxQn6IApIBjiSbrI9WtJ0Zl+LVfpiNdL27ZI26+14frItinoxD1abSWphLV/tZfffgdttTy6s5e3AxqRz5BbbT5ccP2vYm3SzbwVemwGH1Mjm5hv9h/LmjYWn/5B0ie3tJD1M+g63T4XadtGHB0n3kaqr/5J3rURaevHPrGdYQS3Vz9Ws4yGSXVeVitmmXV8lfbav4w2blQmIYDDoYiTdDuxo+y95e2XggkqVd2cCGwAX513bArcBawPnup+7onTYhvSgkm3Iu4CWKfitNSxDgp6oIV03JF1rkMV64AAAGIlJREFUe8u57SukZYLtUaXHbSqStgMusx1G0/MJEQwOcCQt3dfxUlNMku6i7yrRYtmEFpI+CpxImpoF2BTY13bxKt48Fbmz7X/k7cVJLaR2JGUH+7XxfJttyGBSUDqFlG0ZRgrERvfn+G061rZ9/5yC01JBaVMy6h2aRgGTbb+g1IN3BHBk62GmW1Dy5FyUlDX+MLMypUOBy20X69vcZkG0GfAu4ELa1i2WsiBq2vVV0umkpUnnk9qehsdgw4lgcIDTMbW0EvBcfr8k8BfbqxbS0comtNY9tUyu9wBetH1oCR2dKJkbt6aZ/lhxmuk+YLjtV/P2wqQb/3tLTl1LOhv4ke278va6wNdtjy00/onZM663dXnFveyaRK4kHk4K0E8DfgXsZLvXqcGBiqRxpFZ8ywOPMysYnAacZPvYglpO6eOwXcg0vinZ2nZUsY928OaJYLBLyNWYF9m+LG9/DNjK9tcK6+gxnVJziiUXjbyH2bt+9OjwUEDHd0lZwN/nXduTqoqPINlU7DGnfzuPdfS2jrLHvm5B0la2r+nY99lKaxfvsD0ie7g9bvtX6mLDdkn/7n7uxxy8NfLD9p6k4P0+YA2g3/toB2+eCAa7BEm32x7ZsW+S7Q0K65gMHODcMkmp/+xxNYINSZ8HxgHvBiaTMoR/rJV9kjSSZOQr4CbbxX0QJZ0FvACcTnqa3xNY3PbuhXX8gFQwMiNvDwWOcuEezkptHO8Bvk6qaD4ZeMV2ce82STeQqmf3Ji1p+F9S9rhol4smkTPX6zD7w9xvK+h4N6k3dKu15U2k9omPFdaxcdbxXpL34ALAC6WXNahnH+1T3dZH23afmcygPINqCwiK8bSk70haRdLKkr5NWdPpFvsAv5D0iKRHgOOA4v13M+OADYFHnborrE+6wRZH0kp57AtI3S2eyvtKszcp+GlNxd1LHbuOtwETJQ2TtA2pmKaY6XQbm5H80SaTbvBn1ggEM7uR1qPtk4tpVgAOr6SlOpIOJgU+x5A6w4wHiltUZU4hZfKXJ30uF+d9pTmWNDX7ILAI8HnS36c0ewA/tz3M9uE5EDzM9ovUu94HfRCZwS4hF5IcTMooANwIfL+CR1lLz1DS+ff3GuNnDbfZ3jBnKzey/UqtKdGOBeCLAKsCD9h+X2ktTUHSVqSb6nPAprYfqqBhaeAEkufhu0kZ08McF87q5O/McOBO28MlLQucbHv7ufzT/tDSiOUVrdkeSVNbRSOSbra9ydz+7TzW0VuP5JmaguYRptNdQg76xtUaX9Ketk/v9NNT7h5l+2cVZD0maUlSBeDVkp4D/lZBB51Tfbma9oulxpf0O9u7zqkqsUI14qakvtmHAu8nmel+znbpz+cW4Ce2f63U7eIwksFy0Zsr1Dc2biAv2f6npNfzw+VT1DNsfzpXeLd6E+9OnZmXFyUtBEyWNJ7UfWqxUoNrVuvE1dSM1onBGySCwQGOsjO9pIvp/SZfalqldUEa0udvFcT2jvntIbl6dQnSmqzq2L5D0oYFh2w9KGxXcMy++CmpO8y9MDMQuo7ku1iSrVrWLbZfAr6cA9UajKeisXEDmZQf5k4iLSH4BzCxkpbPkaZof066zt5MnenQvUjrBA8AvgKsCOxccPymtU4M3iAxTTzAUcOc6YPe6ciYDgJGAku7YGs8SQsAV9reqtSYfWlpFY+07Xu77eLZlrwYvhUA3uDc27qCjjA2ngOSVgGG2i7Wbz0IBhKRGRzg2G4tup9EnlaBmTf+hUvrkXQqqcquvR/wEaX8uOaGcuuvCkO3Z0xfBy6hcNN72zMkvShpiZprOTOLSzqEFISZZAxe3ItS0k9IRUZn5F1flrSJ7f8srYWUCTuHSsbGTUTSCsDK5HuZpE0rWUM14rrW5is7Gy7csjCY/4jMYJcg6RbSlFd7h4urKiws7mGgXNJUeW5IWs72E5U1DCLZuUyrMPbvSBY7V5MsZoDyjeUlnQ/cDbT8/PYimXLvNOd/1S86pgLrdTxE3VljIfwcDI6LGRs3DUmHkSqs7wVaWWQXXPrSrqUR1zWlXs0tBgNjSDMM3yupI5j/iMxg9zC4FQgC2P5H9nwqzSBJS9l+DmZWa1Y5DyUtxuzZ0kFAlYyYUm/ifyPd1G4HlpD0M9ulrUMuza/arG67fa3T93PVdw2WBFrrnZaopIHSHovzATsAa9l+Za6/2f804rrWyzKKIyXdBEQwGPRJBIPdwwuSRjj3ds0Gxy9V0HEEcLOk8/L2GOBHFXQAXAtsRVp4Dqnf6VVUqBQF1rE9TdIewGXAN0lBYdFg0PapuRpxzbzrAduvldSQeUnS6DZz8lHUOV9/DNyZC4xEmrauMUWMpDWB44Flba8raRjwCds/rKGnAfwPsCBtU+YVacR1TbP39B5E6jPemKK9oLnENHGXkCtTz2aWdcpywG5tawpLankfySRWwLWtitEKOhrhDZbHvQdYj1SNd6ztGyRNsT28sI4Pk6ZmHyF9PisCny29DkvSelnHElnHs8BY21NK6shaliOtGxRwazZ8Lo5SB5KDgBNa04+S7ra9bg09tZB0DGld3Aokn8FrmX0NZdElDW261gG2oOJ1LT+0tG7qr5O+xz+1/afSWoL5i8gMdgm2b5O0NrAW6WJ1f6WMD7bvkfS/5BZSklZq2XcUpinZUkjGxo8AU4AblRrPF18zSMpwbGP7AZiZjTqLVN1cDNuTgeHZP47S6yclrW37/rZMS6ut2PKSlm+dM4VZ1PbEljdn5vUKOmrTatN4O6nrRzXydHCLJ0kPczOPVbBTuYQUDLZOEgPbVfZzDeYDIhjsLtZiVh/P9SUV7+OZbTqOILVteopUCXgfUKPTxoHAuZJmy5ZW0IHto4GjW9uS/kLKnpZmwVYgmHX9SdKCpQZvkDn514AvkM7VTkzKAJXmaUmr5/GRtAvJVLirsN0qKiIvaVib9Dd5wParheXcTs/gi7xtyptgjyRlsX+fNWxP6jb118I6gvmMCAa7BKU+nh8mBYOXAR8j9Vot3dT9B6Rq1Wtsry9pc5Jbf3GalC3txLbzmrDSGahJkn5Fai4PqcdoyaUEjTAnt/2F/LNGQD4n9gdOBNaW9DjwMOnz6UokfZyUUf8z6fu7qqQv2r68lAbbq7bpWRp4D3nGoxLvAEbYnp41HQKca/vzFTUF8wGxZrBLUEP6eGpW78wpwPq5ndRE2x8oqGEL29flrhY9aIpvm6STWkFJwTEXJgUdo0k32BuB4xpSsVmMOZ0bLUqeI51ZUlLv6kFk659unfqTdD+wnXPP6pw1vdR26S41SPo8qYvPu4HJpAfem21vWVjH/SQLplfy9sLAlBp/k2D+IjKD3UNT+ng+nz0ObwTOkPQU5dc9bUZqbdZbIGygEcFg6UAwj/kK8LP8Ko6ko/s6XrA4oHVuvJNUXX5d3t4c+ANlz5FWlnQtZp8C3Iv0PepWnmoFgpn/IV3XajCO9NncYnvzPOPw/Qo6TgMmSrqAdC3bkVlenUEwRyIY7B6a0sfzk6Qija+QpriWoHBnCdsHZ0/By23/ruTYnXRYQfSgVKFCzhzPcZqgoMlya0p6FGlJwzl5ewwFp6tbnn6SLiHZ/jyRt5cDflFKR9by/Tz2VfQyBVhSS8O4R9JlwO9I5+4Y4LZWVrdwhv9l2y9LQtLCufhorYLjA2D7R5IuBz6Ud+1t+87SOoL5j5gm7kIUfTwBkHSj7U3n/pv9quH6Pg7bdpFChVy93JeQR0voaJH/Ltu01nDmIparSq/h67RuyQ8RU2vYucQU4Oyo944sLYp2ZsmZuL1JRWlbAM+RirE+XkpDELwVIhgMqiPpRNv7Vhj3u6Qs5TnM3nqttB1E0IGkB4APtj4LpV6vt9gumm2RdCypKOAsUvbpU8BDtv+9pI6s5dvArkD7FOA5tn9cWkswZyRtRprxuKJCdXMQ/EtEMNjFSLrDdp/TlIV0jKxkfv1wL7vtSk3dJa3LLOuflpjS1j87AYeR1sopv2x7aGEdewOHAK3M6WbAIe22IgW17EjqPAJwo+0LSmto0zKCWVOAN3bjFGCb6XSv1DKdDoL5mQgGgypIWsz2C3P/zX7VMNj2y3PbV0hLr9Y/tncprOMhYHvb95Ucdw5a3gVslDerdf4ImoWkz/Z1vMYDQxDM70QwGBRF0ibAycDitleSNBz4ou39KmjpkRmtlS1tkPXPBNujSo45v1FrWUMQBEF/EdXEAxxJ0+l7SqXo9B/wc+Aj5DZStqdIKlrEkTNOKwCLSFqfWd0DhgKLltTSRlXrnzZfvUmSzgEuZPZ+r9XtdpqyrIFkdBxURrP34Z1JqaKrIBhIRDA4wLE9BEDSoaTemaeRgp89qNTlwfZfO/qrzigs4SPAWJJBbLuf3nTgW4W1tKht/dPKQBp4Edim7VgjvBdLB4KSFgB+YvugDh3F17cGvfL1tveDgZ3pzl7NQfCWiWniLkHSrbY3mtu+AjrOIwVgx5Jc+r8MbGD7UyV1ZC072z6/9Lhzo6b1j6RTgXG2n8/bSwFHFLbpWAC40vZWpcbsQ8t1wJaOC+V8gaQbbG9WW0cQzG9EZrB7mCFpD+BsUqZnd8pn5AD+DTiKNE37GHAVqf1ZMSTtaft0YJVeWn1Vae/V21S5pE1tl+4wMawVCALYfi5PpRfD9gxJL0pawvbfS47dC3cCv5d0LrPbD1XPlHY7uRdwi0HABsC7KskJgvmaCAa7h0+TgrCjSMHghLyvKLafJk1R12Sx/HPxXo7VygC1T0UOBj5Ami4uvf5pkKSlbD8HM2+4Na4TLwN3Sbqa2YOw0rYhSwPPMPvn0Ihp84DbSZ+FgNeAR4B9agoKgvmVCAa7BNuPkFrBVaFJ3mC2WwUAq9HLlGgpHR2aZqsalrQiML6ClCOAm/N0vkkmxz+qoOPS/KpKqy1d0Ei+STJ2npYN5EeQ1rsGQfAmiTWDXYKkNYHjgWVtrytpGPAJ2z8sNH7jvMEk3Wl7/bntq4FShc1U2++vMPY6pEyYgGtt31taQ1PILc96q1gttoYy6B1JU20PkzQa+H+kB5lvlV4HHQQDgcgMdg8nkaYiTwCwPVXSmUCRYLChRrBNmRLtzJwOAtYDptTQkoO/qgFg7g7TWxBWujvMJW3vB5NawP2tsIagd1prnrcFfmn795IOqagnCOZbIhjsHha1PbHD0qWYDYOkI20fKOlier/Jf6KUljaaMiUKMKnt/evAWbYnVNLSBDZoez8YGENav1eUzmpzSWcB15TWEfTK45JOALYCDpO0MOlBKgiCN0lME3cJki4HDgDOtT1C0i7APrY/Vmj8kbZvz03ce2D7hhI6Ookp0fkHSTfZHl1Zw1rApbbXqKkjAEmLAh8F7rL9oKTlgPfbvqqytCCY74jMYPewP3AisLakx4GHgT1LDd5m1Lue7aPaj0kaB1QJBmtPieY2dH0V1gwrKKcxSGo3mG7ZhhQ3Se+lg8+TpMKFoDK2X6Stqtv2E8AT9RQFwfxLZAa7DEmLAYNsT680fm/9gBtRtFEDSSvnty2vxdPyzz2AF20fWl5VfXKrsRavkx5ejrD9QKHxR9meIGmw7ZdLjBkEQVCLCAa7BEmnAQe0THxzEPJr21sWGn93kq/haOC/2w4NAWY0odtETSRNsD1qbvsGOpLG2T5K0mjbN1XUcbvtkQ3qhxwEQdBvxDRx93ATcGvuuLECqbL4awXHv5k0hfMOZvfymw4Ub7vWQBZrD4AkbcIsc+xuYm+SMfrRJN+4WryWbWXeLenozoMVzK+DIAj6jQgGuwTbJ0i6B7geeBpY3/aTBcd/FHgU+GCpMecz9gF+LWmJvP080I1edvdJegRYRlL7Q4IAF1xDuR2pSnULUqeLIAiCAUtME3cJkvYCvgscDAwDPgLsbbuol52kjYFjgPcCCwELAC/YHlpSR1ORNJT0vazdk7cakt4FXAn0sBvKDxUltQwv/R0JgiAoTQSDXYKkC4F9bT+Vtz8AnGh7vcI6JgGfAs4lVYh+BljD9rdL6mgKkva0fXqevu+B7Z+V1hQ0q31iEARBfxPTxF2C7R06tifmgLCGlockLWB7BnCKpJtr6GgIrXWBxW1TmkgfVjulp4knzf1XgiAIBgaRGRzgSPqG7fFzynSUznBIupG0FutkkmfbE8BY28NL6giaSZvVTq+UniYOgiDoBiIYHOBIesb22yUdCDzXebx0z+B8s38KWBD4CrAEcJzth0rqaBqSxpP6RL8EXAEMBw60fXpVYV1O9jvs7SFqiwpygiAI+oUIBgc4ku4FPgZcBGzeedz2s8VFBT2QNNn2epJ2BHYgBcrXd2vGtKPzx0Kkh4fihUaSRrZtDgZ2Bl63/Y2SOoIgCPqTWDM48DmelGlajdnXQYl0s12thIhouzZXFsw/Pw6cZftZSTX1VMX2bGsoJe0AFF/j2tZGscUESVVaJwZBEPQXkRnsEiQdb/tLFcePtWB9IOknpIzgS6SgZ0ngEtsbVRXWICTdYnvjwmMu3bY5CBgJHG17rZI6giAI+pMIBoPiSFoW2DBvTmzZ3XQ7kpYCptmekXtIDylpDN4kJO3UtjmIZEO0me2ipuWSHiZltMWsHsmH1myVFwRBMK+JaeKgKJJ2BQ4H/kC6wR4j6SDb51UVVhlJiwL7AysB+wLLA2sBl9TUVZHt296/DjwCfLK0CNurlh4zCIKgNJEZDIoiaQqwdZv59TLANd1aKNFC0jmktmefsb2upEWAP5Y2BQ9mR9IY4Arb0yV9h9Qv+Ye276gsLQiCYJ4xqLaAoOsY1DEt/AxxHgKsbns88BqA7ZdImdOuRNJ4SUMlLSjpWklPS9qzgpTv5kBwNKmF46mkoqwgCIIBQ9yEg9JcLulKSWMljQUuBS6rrKkJvJqzgQaQtDrwSl1JVdnG9jRgO+AxYE3goAo6ZuSf2wLH2/49yeomCIJgwBBrBoPSPAncBaxHynydaPuCupLqouQh80uSBdCKks4ARgFja+qqTFOsdh6XdAKpa85hkhYmHqKDIBhgRDAYlGYIsA/wLHA20M19iYHUcFfSOGAbYGNSkDzO9tN1lVXlYkn3k6x29strS1+uoGNX4KPAT20/L2k56mQogyAI+o0oIAmqIGkYsBupo8NjtreqLKkqkn4B/Mb2bbW1NIWw2gmCIChDTHcEtXiKNGX8DPDOylqawObAHyX9WdJUSXdJmlpbVE1sP2d7Rn7/Qm09LSR1q91PEAQDlMgMBkWR9CVSRnAZ4DzgHNv31lVVnzl1aOn2ziztSLrU9rYN0LGc7Sdq6wiCIJhXRDAYFCW3XTvb9uTaWoIgCIIgiGAwCIKGkr393mP7lFxAsrjthwuNfRfZ5qfzEKnmZ1gJHUEQBCWIYDAIgsYh6WBSP+K1bK8paXngXNujCo3f67R9i5i+D4JgIBHWMkEQNJEdgfWBOwBs/03SkFKDR7AXBEE3EdXEQRA0kVedpi1aHVkWqyFC0saSbpP0D0mvSpohaVoNLUEQBP1FBINBEDSR3+XOH0tK+gJwDXBSBR3HArsDDwKLAJ8HjqmgIwiCoN+INYNBEDQSSVuTurIIuNL21RU0TLK9gaSpraIRSTfb3qS0liAIgv4i1gwGQdBU/kSq3L1G0qKShtieXljDi5IWAiZLGg88AVSZsg6CIOgvYpo4CILGkaeGzwNOyLtWAC6sIGUv0nXyAOAFYEVgpwo6giAI+o0IBoMgaCL7A6OAaQC2H6RO28IdbL9se5rt79v+KrBdBR1BEAT9RgSDQRA0kVdsv9rakPQ2ejeB7m8+28u+saVFBEEQ9CexZjAIgiZyg6RvAYvkQpL9gItLDS5pd+DTwKqSLmo7NBR4ppSOIAiCEkQ1cRAEjUPSIGAf2qqJgZNd6IKVO5CsCvwY+I+2Q9OBqbZfL6EjCIKgBBEMBkHQKCQtAJxqe8/aWgAkLQtsmDcn2n6qpp4gCIJ5TawZDIKgUdieASyTLV2qImkMMBEYA+wK3Cppl7qqgiAI5i2RGQyCoHHk7iMjgItIli4A2P5ZYR1TgK1b2UBJywDX2B5eUkcQBEF/EpnBIAgag6TT8tvdgEtI16ghba/SDOqYFn6GuG4GQTDAiGriIAiaxMhcvPEXmtED+HJJVwJn5e3dgMsq6gmCIJjnRDAYBEGT+CVwBamSd1LbfpF8BlcrrMekLiijs4YTgY0LawiCIOhXYs1gEASNQ9Lxtr/UAB132B7RsW+q7WG1NAVBEMxrIhgMgiDoQNKXSEbXqwF/bjs0BJjQFNubIAiCeUEEg0EQBB1IWgJYil5Mp20/W0dVEARB/xDBYBAEQRAEQRcTFglBEARBEARdTASDQRAEQRAEXUwEg0EQBEEQBF1MBINBEARBEARdTASDQRAEQRAEXcz/AdIzBxziyGKqAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 720x576 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure(figsize=(10,8))\n",
    "sns.heatmap(df.corr())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "volatile.acidity       -0.646461\n",
       "sulphates              -0.506999\n",
       "chlorides              -0.483965\n",
       "density                -0.453016\n",
       "fixed.acidity          -0.427140\n",
       "pH                     -0.318459\n",
       "alcohol                -0.062780\n",
       "quality                 0.135458\n",
       "citric.acid             0.264041\n",
       "residual.sugar          0.431283\n",
       "free.sulfur.dioxide     0.568385\n",
       "total.sulfur.dioxide    0.780594\n",
       "type                    1.000000\n",
       "Name: type, dtype: float64"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.corr()['type'].sort_values()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x1a1e009590>"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAFNCAYAAAANRGjoAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3dabhkZXnu8f/dCKLIKK0i0gyKA1HGFokYnABRFHBAwQmJSec4RI0nBoyKSuIJYIjihKCCOAQQPMZmMMikHECURhoQ1NAiagsGAQ0oKIL3+fCukuqienfvrnfV7l3r/l1XXbvWUOtZe++qetZ6R9kmIiK6a85Mn0BERMysJIKIiI5LIoiI6LgkgoiIjksiiIjouCSCiIiOe9BMn8Cq2Hjjjb3FFlvM9GlERMwqV1xxxa225w6un5WJYIsttmDRokUzfRoREbOKpJ8MW5+ioYiIjksiiIjouCSCiIiOSyKIiOi4KolA0gmSbpH0veVsl6SPSFoi6WpJO/ZtO0jS9c3joBrnExERK6/WHcFngb2m2P58YOvmsQA4FkDSRsB7gacBOwPvlbRhpXOKiIiVUCUR2L4IuH2KXfYFPufiMmADSZsAzwPOtX277V8B5zJ1QomIiMrGVUewKfCzvuWlzbrlrY+IiDEZV4cyDVnnKdY/8ADSAkqxEvPmzat3ZhERq6ktDj1r2q+58Yi9p/2acd0RLAU261t+DHDTFOsfwPbxtufbnj937gN6SEdExCoaVyJYCLy2aT20C/A/tm8GzgH2lLRhU0m8Z7MuIiLGpErRkKSTgWcBG0taSmkJtCaA7U8CZwMvAJYAdwEHN9tul/RPwOXNoQ63PVWlc0REVFYlEdg+cAXbDbxpOdtOAE6ocR4RETF96VkcEdFxSQQRER2XRBAR0XFJBBERHZdEEBHRcUkEEREdl0QQEdFxSQQRER2XRBAR0XFJBBERHZdEEBHRcUkEEREdl0QQEdFxSQQRER2XRBAR0XFJBBERHZdEEBHRcUkEEREdl0QQEdFxSQQRER2XRBAR0XEPqnEQSXsBxwBrAJ+2fcTA9g8Bz24WHwo8wvYGzbb7gGuabT+1vU+Nc4qIaNMWh541rf1vPGLvls5kdCMnAklrAB8H9gCWApdLWmj7ut4+tv+ub/+/BXboO8Tdtrcf9TwiImLV1Cga2hlYYvsG2/cApwD7TrH/gcDJFeJGREQFNRLBpsDP+paXNuseQNLmwJbABX2r15a0SNJlkvarcD4RETENNeoINGSdl7PvAcDptu/rWzfP9k2StgIukHSN7R89IIi0AFgAMG/evFHPOSIiGjXuCJYCm/UtPwa4aTn7HsBAsZDtm5qfNwDfYNn6g/79jrc93/b8uXPnjnrOERHRqJEILge2lrSlpLUoX/YLB3eS9ARgQ+Bbfes2lPTg5vnGwK7AdYOvjYiI9oxcNGT7XklvBs6hNB89wfa1kg4HFtnuJYUDgVNs9xcbPQk4TtIfKUnpiP7WRhER0b4q/Qhsnw2cPbDusIHl9w153aXAU2qcQ0RErJr0LI6I6LgkgoiIjksiiIjouCSCiIiOSyKIiOi4JIKIiI5LIoiI6LgkgoiIjksiiIjouCSCiIiOSyKIiOi4JIKIiI5LIoiI6LgkgoiIjksiiIjouCSCiIiOSyKIiOi4KjOURUSsLrY49Kxpv+bGI/Zu4Uxmj9wRRER0XBJBRETHJRFERHRclUQgaS9JP5S0RNKhQ7a/TtIvJS1uHn/Vt+0gSdc3j4NqnE9ERKy8kSuLJa0BfBzYA1gKXC5poe3rBnY91fabB167EfBeYD5g4Irmtb8a9bwiImLl1Lgj2BlYYvsG2/cApwD7ruRrnweca/v25sv/XGCvCucUERErqUYi2BT4Wd/y0mbdoJdKulrS6ZI2m+ZrIyKiJTUSgYas88DyGcAWtrcFzgNOmsZry47SAkmLJC365S9/uconGxERy6qRCJYCm/UtPwa4qX8H27fZ/n2z+Clgp5V9bd8xjrc93/b8uXPnVjjtiIiAOongcmBrSVtKWgs4AFjYv4OkTfoW9wG+3zw/B9hT0oaSNgT2bNZFRMSYjNxqyPa9kt5M+QJfAzjB9rWSDgcW2V4IvEXSPsC9wO3A65rX3i7pnyjJBOBw27ePek4REbHyqow1ZPts4OyBdYf1PX8n8M7lvPYE4IQa5xEREdOXnsURER2XRBAR0XFJBBERHZf5CCJiLDJPwOordwQRER2XRBAR0XFJBBERHZdEEBHRcUkEEREdl0QQEdFxSQQRER2XRBAR0XFJBBERHZdEEBHRcUkEEREdl0QQEdFxSQQRER2XRBAR0XFJBBERHZdEEBHRcUkEEREdVyURSNpL0g8lLZF06JDtb5d0naSrJZ0vafO+bfdJWtw8FtY4n4iIWHkjT1UpaQ3g48AewFLgckkLbV/Xt9uVwHzbd0l6A3AU8Ipm2922tx/1PCIiYtXUmLN4Z2CJ7RsAJJ0C7Av8KRHYvrBv/8uAV1eIGxEVZC7hqFE0tCnws77lpc265Xk98LW+5bUlLZJ0maT9KpxPRERMQ407Ag1Z56E7Sq8G5gPP7Fs9z/ZNkrYCLpB0je0fDXntAmABwLx580Y/64iIAOrcESwFNutbfgxw0+BOknYH3gXsY/v3vfW2b2p+3gB8A9hhWBDbx9ueb3v+3LlzK5x2RERAnURwObC1pC0lrQUcACzT+kfSDsBxlCRwS9/6DSU9uHm+MbArfXULERHRvpGLhmzfK+nNwDnAGsAJtq+VdDiwyPZC4IPAw4DTJAH81PY+wJOA4yT9kZKUjhhobRQRES2rUUeA7bOBswfWHdb3fPflvO5S4Ck1ziEiIlZNehZHRHRcEkFERMclEUREdFwSQURExyURRER0XBJBRETHJRFERHRcEkFERMclEUREdFyVnsUR0Y7pzhWQeQJiVeSOICKi45IIIiI6LokgIqLjkggiIjouiSAiouOSCCIiOi6JICKi45IIIiI6LokgIqLjkggiIjouiSAiouOqJAJJe0n6oaQlkg4dsv3Bkk5ttn9b0hZ9297ZrP+hpOfVOJ+IiFh5IycCSWsAHweeD2wDHChpm4HdXg/8yvbjgA8BRzav3QY4APgzYC/gE83xIiJiTGrcEewMLLF9g+17gFOAfQf22Rc4qXl+OvBcSWrWn2L797Z/DCxpjhcREWNSIxFsCvysb3lps27oPrbvBf4HePhKvjYiIlpUYz4CDVnnldxnZV5bDiAtABYAzJs3b+iJTHfsdli18du7HmeSfpfVPc445hcY1xwGibN6xoA6dwRLgc36lh8D3LS8fSQ9CFgfuH0lXwuA7eNtz7c9f+7cuRVOOyIioE4iuBzYWtKWktaiVP4uHNhnIXBQ8/xlwAW23aw/oGlVtCWwNfCdCucUEREraeSiIdv3SnozcA6wBnCC7WslHQ4ssr0Q+AzweUlLKHcCBzSvvVbSl4DrgHuBN9m+b9RzioiIlVdlzmLbZwNnD6w7rO/574D9l/PaDwAfqHEeERExfelZHBHRcUkEEREdV6VoKKJrxtWsL2IcckcQEdFxSQQRER2XRBAR0XFJBBERHZdEEBHRcUkEEREdl0QQEdFxSQQRER2XRBAR0XFJBBERHZdEEBHRcUkEEREdl0QQEdFxSQQRER2XRBAR0XFJBBERHZdEEBHRcUkEEREdN1IikLSRpHMlXd/83HDIPttL+pakayVdLekVfds+K+nHkhY3j+1HOZ+IiJi+Ue8IDgXOt701cH6zPOgu4LW2/wzYC/iwpA36tr/D9vbNY/GI5xMREdM0aiLYFzipeX4SsN/gDrb/y/b1zfObgFuAuSPGjYiISkZNBI+0fTNA8/MRU+0saWdgLeBHfas/0BQZfUjSg0c8n4iImKYHrWgHSecBjxqy6V3TCSRpE+DzwEG2/9isfifwC0pyOB44BDh8Oa9fACwAmDdv3nRCR0TEFFaYCGzvvrxtkv5b0ia2b26+6G9Zzn7rAWcB77Z9Wd+xb26e/l7SicDfT3Eex1OSBfPnz/eKzjsiIlbOqEVDC4GDmucHAV8d3EHSWsBXgM/ZPm1g2ybNT1HqF7434vlERMQ0jZoIjgD2kHQ9sEezjKT5kj7d7PNyYDfgdUOaiX5R0jXANcDGwD+PeD4RETFNKywamort24DnDlm/CPir5vkXgC8s5/XPGSV+RESMLj2LIyI6LokgIqLjkggiIjpupDqCiNXNjUfsPdOnEDHr5I4gIqLjkggiIjouiSAiouOSCCIiOi6JICKi45IIIiI6LokgIqLjkggiIjouiSAiouOSCCIiOi6JICKi45IIIiI6LoPOxVhkMLiI1VfuCCIiOi6JICKi41I0FCm2iei43BFERHTcSIlA0kaSzpV0ffNzw+Xsd5+kxc1jYd/6LSV9u3n9qZLWGuV8IiJi+ka9IzgUON/21sD5zfIwd9vevnns07f+SOBDzet/Bbx+xPOJiIhpGrWOYF/gWc3zk4BvAIeszAslCXgO8Mq+178POHbEc5oYKbuPiHEY9Y7gkbZvBmh+PmI5+60taZGkyyTt16x7OPBr2/c2y0uBTUc8n4iImKYV3hFIOg941JBN75pGnHm2b5K0FXCBpGuAO4bs5ynOYwGwAGDevHnTCB0REVNZYSKwvfvytkn6b0mb2L5Z0ibALcs5xk3NzxskfQPYAfgysIGkBzV3BY8BbpriPI4HjgeYP3/+chPGOKTIJiImyahFQwuBg5rnBwFfHdxB0oaSHtw83xjYFbjOtoELgZdN9fqIiGjXqIngCGAPSdcDezTLSJov6dPNPk8CFkm6ivLFf4Tt65pthwBvl7SEUmfwmRHPJyIipmmkVkO2bwOeO2T9IuCvmueXAk9ZzutvAHYe5RwiImI06VkcEdFxSQQRER2XRBAR0XFJBBERHZdEEBHRcUkEEREdl0QQEdFxSQQRER2XRBAR0XFJBBERHZdEEBHRcUkEEREdl0QQEdFxo85ZvFrJhDEREdOXO4KIiI5LIoiI6LgkgoiIjksiiIjouCSCiIiOSyKIiOi4JIKIiI5LIoiI6DjZnulzmDZJvwR+Ms2XbQzc2sLpjDtG4qy+MRJn9Y2ROMXmtucOrpyViWBVSFpke/5sj5E4q2+MxFl9YyTO1FI0FBHRcUkEEREd16VEcPyExEic1TdG4qy+MRJnCp2pI4iIiOG6dEcQERFDJBFERHRcEkFERMdNbCKQtMZMn8NsI+mFklp/T0h6ctsxJs0kvZ8lrSHpg2OOuc444802E5sIgCWSPihpmzaDSDpK0nqS1pR0vqRbJb16lsY5ALi+ifWkysfu90lJ35H0Rkkb1D64pGskXT3kcY2kq2dbnMZY3s/jYPs+YCdJajuWpKdLug74frO8naRPVI7xSEmfkfS1ZnkbSa+vGaM5riS9WtJhzfI8STtXOfakthqStC7li+1gSsI7ATjF9h2V4yy2vb2kFwP7AX8HXGh7u1kaZz3gQMrfzcCJwMm276wcZ2vgL4H9ge8AJ9o+t9KxN+89Bc4CXtC/3fZ0hydZUZyhasVpYrX6fpZ0J+X/PZTt9WrE6Yt3NLA1cBrw2744/7dynG8DLwMW2t6hWfc929XuSpsEcCLwLtvbSXoQcKXtp9SK0cQ5Fvgj8BzbT5K0IfB1208d9dgTe0dg+07bn7L9dOAfgPcCN0s6SdLjKoZas/n5AsoX5u0Vjz32OM0Xy5eBU4BNgBcD35X0t5XjXA+8GzgEeCbwEUk/kPSSCsf+SfO4Efh93/JPan45Dxzzd8BTmsfdNeM0sVp9P9tet/my/zBwKLAp8BjK/+efRz3+EBsBtwHPAV7UPF7YQhxs/2xg1X2VQ2xs+0uUL2ls39tCDICn2X4T5b2G7V8Ba9U48INqHGR11JSp7k25gtoCOBr4IvAXwNnA4yuFOkPSD4C7gTdKmkvzj6qs9TiS9qH8vR4LfB7Y2fYtkh5KubX+aKU42zZx9gbOBV5k+7uSHg18C6h6Vdg2SS8HPgh8g3IX8lFJ77B9esUY43o/P8/20/qWj22uqo+qdHwAbB9c83hT+JmkpwOWtBbwFppioop+K+nhNHdUknYB/qdyDIA/NO+DXpy5NMlnZLYn8gHcAHwGePqQbR+pHGtDYI3m+TrAo1r6nVqNA3wO2G05255bMc5FwGuAhwzZ9poKx9+x7/H9geUdW/i/XAU8om95LnBV5RhjeT8DlwKvAtaglBi8Cri0hb/Z2sCbgE9QirlOAE5oIc7GlIT538AtwBeAh1eOsSNwCeXL/xLgv4BtW/hdXgUsBH4OfAD4IbB/jWNPch3BM2xfPLBuV9uXVI7zUODtwDzbC5qy7yfYPnO2xZF0pO1DVrSuQpy32f7wwLq32j6m0vEvHFjVe5MLsO3n1IjTF+8a95UHNy2vrnLFMuIxvp+3AI4BdqX83S4B3uZSzFYzzmnAD4BXAodTvuS+b/utNeOMS1Mv8ATKe+yHtv/QUpwnAs9tFi+wXeXuZpITwXdt77iidRXinApcAbzW9pMlPQT4lu3tZ1uc5fzNrra9ba0YU8S50k1lXsU4DwHeCDyD8qX2/4BjbdcuUvsgsC1wcrPqFcDVNRPouN7P49L7f/feX5LWBM6plaQlfZSpK7/fUiHGlPVZrlzx3cTckfvfz5fY/m6N405cHYGkPweeDsyV9Pa+TetRbndre6ztV0g6EMD23S01i2stjqQ3UL4wHzvQ7HFdyhVhFc25vxLYUtLCgTi31YrT5yTgDuAjzfKBlOKvl9cMYvsdkl5KuYoWcLztr9Q49rjez5L+wfZRy/sCrfHFOaB3xfxrlX4lv6DUfdSyqPm5K7ANcGqzvD/lgqqGFzU/H0H5H13QLD+bUl9UuwXUYZTz/zLlfXaipNNsj1yZP3GJgFKL/jDK77Zu3/o7KM3IarunufLsVeA8Fvj9LIvz78DXgH+htBjpudN1WyddCtxMKbc9uj8OULvdPZSis/7mtRdKuqqFONj+MuUDWtu43s+9IoZFU+5Vz/FN88f3UMq9HwYcVuvgtk8CkPQ64Nm9ohpJnwS+XinGwc0xzwS2sX1zs7wJ8PEaMQYcCOzQu6OVdATwXWq06qpdobG6PChTso0jzp7AN4FfUiqlbqS88WrH2WNInGdVOvZ6zc+Nhj1m+n85wu/1WWCXvuWnAZ9oIc5LgOsplYV3UBLbHZVjbD7Tf8/Z+KBUqG7Ut7whpQy/ZozvDSzPGVxXKc7XgA36ljcAzqxx7ImrI5D0Ydtvk3QGw29x92kh5sOBXSi3a5fZbmW+0rbiSDrT9gsl/ZjyN+svcrLtrSrFudj2M4Z0XupV4tbutPR9SgXeT5tV8yhXvn9s4lWp+5C0hNIEtnazxLG/nyWdS2mJ8utmeUNKx7XnVY7z9iGr/we4wvbiinEOBt4H9BoQPBN4n5s7hkoxPkbpHHcy5X90ALDEdtW+N5L+A3gqpcm1KReHF1NaQ+ERiu8mMRHsZPsKSc8ctt32NyvHO9/2c1e0boTjT1kZ6EqVRZNIY+r5K+kS27vWONaQY4/7/bzYAw0QWqrI/3dgPnBGs2pv4HLgicBptqv1W5D0KMrdIMC3bf+i1rH7YryE0qcD4CJXqiMaiHHQVNtHSW4TlwjGRdLawEMpVxrP4v6r6PWAr9muMlZPX1PItSkfnKuaWNtS3tTPqBGnibUrsNj2b1XGMdoR+LDtn67gpSt7/I2m2u72emW3oq/VyDOBRwH/QV+9jVtoNdI2SVcAL+79z5tk+hXXb213DvBS279plh8GnE7pyX6F7ZHGVJL0RNs/WN6F1Gy8gJL0QuBs23U6kfWZuMpiSdcwdbOxWk0h/wZ4G/BoSiuEXiK4g4oVRbafDSDpFGCB7Wua5ScDf18rTuNYYDtJ21GGMfgMpYfx0KvRVXAF9xc9zQN+1TzfgFJ8s2WlOOPyor7nd1Hqi3pMhVYjY3w/97wLuFhS705jN2BB5RhQ/v/39C3/gVIPcrekGo0g3k4576OHbDNlaIuRjLuok1LkdIykL1PG5qpWFDlxdwR9xQFvan5+vvn5KuAu24dXjve3tqsMvbCCOMNu2R+wbsQY37W9Y9NM7ee2P9NS34tPUgYBO7tZfj6wu+3/XTPOJBhX8dZAzI25vy7qW23UeUl6D+Xq/6vNqhdRWg8dTWl++6raMSeBWhoUcuISQc+wctu2ynKbq/NtKMU3ANj+XOUYJ1NGafwC5Q3wauBhtg+sGOObwH9S3mS7UVooLXb9URSvsL3TwLpFtufXjDMukh5DGYep1xv3YuCttpfO6ImtoqaCeGuWfT9f1EKcnSidowRcbLt601VJ/0SpHL6vWV4POMYVxzqStLvt8wbWHVSzQnrg2BtTPv9vozR+eBxlmJFVviCd2NFHgXUk/an8XGXgqeqTU0h6L+VL4KOUjiRHAdVbJlG+nK8F3kp5A1zXrKvpFZQy7tc3FWqbUgZTq+1WSe+WtIWkzSW9i3Y6lI3LiZSr2UdT/mZnNOuqkbSLpMsl/UbSPZLuk1R1SPUmzl9RxoI6B3h/8/N9LcSZR7nQ+AqlCO2WZl1tDwK+I2lbSXtSKqRrdSjrOUzSsZLWUZmb4AyWLTasQtI+kr5C6bi2JmVQyOcD2zFqMXGNNqir4wPYiVKxemPzWEw7A45dQ0moVzXLjwTOmOnff3V+UPonHANc2TyOYXb3V1i8MutGjLGIcuV3JaVH8cHAB1r4Xa6h3AksbpafCJzaUpyrm8f1wL3AtS39f3anjNp7E/C4Fo6v5ov4+uZxYEu/x6kMDAoJHNn8HGlQyImrLO6xfQWl4nM9ShFYG8PCQhl7/o+S7m1i3QJUaXcPIOlLtl++vEpDV6wsbFrBHEnpMi9aqvRyaR00KwcXW47ebHG9sYYOpIU7HNtLJK3hUsxxoqRLa8cAfmf7d5KQ9GCXljdPqB3EA8WNTeuev6kdR9JulAuNwylzRXxM0l/avqlimA0pzVN/RJnDYXNJcvMNXdHWfmAR3fOBQ2yfP8qBJy4RSHq17S8MdlhRMyyP7X+rHHKRynSLn6Lccv6GMuNWLb0vzFYm7RhwFC11jIKZ6ew3Jn8JfAz4EOX3urRZV9NdKuPpL5Z0FGWojjbm4V3avJ//AzhX0q8oV9KtcpmPYuSZtob4V0oHuevgTxc7F1DudGq5DDjC9gkqw8AcSRmj6+k1Dq77xwLbSi2NBTZxlcWS/sb2cU3Z/QPYfn+LsbegDNdQddwclckozrG9e83jDonTWseo5vhj7Rw1SZrWQ7dQyob/DlifMlzGkhZjPrOJ85+271nR/tM8dv+F2hxKUe5Grt+DuXcH1b/u4bar3bFJmueBvjaSdhty9b6qx1+fctfR2lhgE5cIZoKkTYHN6bvDqvUm6IuxkDJpS1tFXEg6hjF0jJK0Dk2RWrO8BvBg23fVjDMukk6itBLqH5bhaNu17womxsCF2r2Uerwvu/4Q4etTKrt3o9ytfRM4vPbnSGV2v92axW/aPmOq/Vc3E1c01DOuD6ekIymtba7j/nlKTWl5UdPvgGtUxoLpn+y75vDA69FSx6gB51Mq8H7TLD+EMiJklVvpGbBt730GZS5ZSbWHZOiNA7UMVxoHagWxz7RdtWiy/85cZSKfh9VOAo0TgO9Rhm+GMjPeiZSBAqtQGQX0qZTBIAHeIunptt9ZK0bbJjYRMIYPZ2M/ynDHbQw93e+s5tEaj28e2bXdDC3QxP2Nygxss9UcSRu6TCbeG0qj9merv4/F2pQvtimH7Kjor2sfUGWsof9FuXi6Alhf0r/Zrt1c+bG2X9q3/H5J1Qa1a7wA2L7vDvckSuuuWZMIJrkfwZzmLgBo7cMJZS7ZNVs47jJcOqecTPnQXAH8uyt3WJH0eEnnS/pes7ytpHfXjNH4rfrGgGk6Ft3dQpxxORq4VNI/NR2YLqX+ZO+39T1+7jLVZ9UpN6EU2zVX6L3lObQzEfs2tu+gXEidTRly4jUtxLl7oD/RrrTzXtug7/n6LRy/VZN8R9D7cJ7eLO9PmfC5Ct0/k9NdlJYc57NsuXrVGZ0kPYsy49aNlGadmzW9F2sWQX0KeAdwHIDtq5srt9EnvljW24DTJPVao2xCKV6blWx/TtIiyhezgJf0WqnUomUHT5tDuUNYdzm7j2Kw2O6htFNst6bK9JT7AR+z/QdJbVRYvgE4qakrEHA78LrKMf4FuFJlgEhR6gpmzd0ATHAiaD6cV1B6+7bx4ex1h7+C0qu0bUcDe9r+IZSrd8odwk5Tvmp6Hmr7O1p2Bsx7Kx4fANuXq0zC3Zvs+wduabLvNmnZ0VR/QZnp7U/barXoaBzN/XUEvcrV/Ze796obV7HdcZTf4SrgoqZVVPWe0i5zG/T6E9HchdSOcbKkb1DqCURp1199qOs2TWwiALB9raRf0oyZMqyZ1wjH/lOxTNO++4mUD+oPaze1a6zZSwJN/P9qrqhqulVlCszedJgvo7RXb8MTuH98ph0k4crjM41B/2iqcP8XtZrnNStyzxwS64Ut9I/5raQd3QzT3Faxne2PcP9c0kj6KeWirYpx9CfSA4e67o0t9WhJj/YsGup6YhNB05zraMr4L7dQmnd+H/izynFeQLm6+RHlQ7pl05fhazXjUDqu9YaFhjKaau0xU94EHA88UdLPgR83capqmg4+i5IIzqb0jryYMrH8rGH7T8NmN3cHywzUVtlOlCvOr1LeZy+itEz7WeU4M1JsZ9uStqXMwVtDr7NdG8VnPf+bUpHe2lDX4zKx/QhUJil/DnCe7R0kPZsyBkjVsdUl/QB4Ya9jT3NFfZbtmj0XkfRgyhd1b7TGiygdikZurTR41URpzjmHpplq7d7YKsNlbAdcaXs7SY8EPm27+kBd46AyUNtbKcMLLKYM4XypK81S18T4OmUilzub5XUpM3ntVStGX6w1mYFiO0mfsl29hVKs2MTeEQB/sH2bpDmS5ti+sGnzX9stA707b6CZQ7Sm5gv/35pHbb2rpiew7FXna6jfHwJaHp9pBryV8ne7zPazm/qP2j3YBydyuQfYotbBJT3H9gW6f9a1nq2bYrvWZ1urmQQkfWSq7TUacwz5Ww3GmDUz1Ei3I0cAAA03SURBVE1yIvi1yvR3FwFflHQLLVR8AtdKOhv4EuV2cH/g8t6bZNQ3g8YwQ1Wvc09z1blj31Xn+4DTRj3+EG2PzzRu4xio7fOU4ZS/Qnk/vJjSiqyWZ1LG4Bl2V1atU6HGNwd3r9h0V0oR5KnN8v7UK1Lt/a0eQWlVdUGz/GzgG9TviNmaSS4aWodSyTWHUs69PvDFmmOMNHGmGnfeo/Zk1hhnqGqKubbrFTc1xVFX1S7mGoi5BS2MzzROzZfzwZTy9edQpuBc0/YLKsfZkWUnSL+y8vHnAC+z/aWaxx2IceEUm227arl6E2/PXvFWU+z1dTdTwFaKcSbw17ZvbpY3AT5uu1rv5bZNbCKI6VOZIObllMlCeledp9r+lxk9sVlELQ7UNg6SLrK924r3nB0k/RD4815T3qaT6WW2q92xSfqe7Sf3Lc8Bru5ft7rrVCKQdHytyuK+DmVDtdChbCxzBbR91TlF3OpzI8f0qcwlfDelKKV/TKuafSJ6scYxxevBlEHnenciz6RMXVmtWE3Sxygtxk6mfCccACyx/be1YrSta4lgJ5cJa2oc66Cptrcw/MMSWpwrIAL+NLjdILvy4HbLa0Js+2U14zSxHkWZOAbg22109pL0Yu4fffQi21+pHaNNE58IJK1j+7cr3nP1ppbnCogAkLS2B0YBHbauQpyJakI8201sqyGVyeo/DTwMmCdpO+BvbL+xcpwLGT48cJVKr74maosknUrLcwW0SdKdTF2cVrWYK1bJpcBgEd2wdaOasSbE4yiGrFkMPQ4Tmwgo0wY+j2YcINtXqcxfWtvf9z1fG3gpdZup9q6QegPctT1XQGtsrwsg6XDK2Dyfp9R1vIp2e4DGCjTFJ5sCD1EZrr03lMV6lIHnapuxJsRjqos6bgwxqpnYoiFJ37b9NElX2t6hWXeV7e3GEPubtodOxzjCMSdmFqze/2ZF62J8mjqv11FGNV3Ut+lO4LNt3nm21YRYY5jitYlxhO13tBVjHCb5juBnTfGQVQaFewtlrKGqtOwIlL3hgR9VOw7jm2hnHO6T9CrgFMpdzYHcP7tbzICmccNJkl5q+8ttxxt2d66K8/wC2L5P0l2S1ndLU7w2MXaSJM/iq+pJTgT/CziGcru7lDKm+ptaiNM/AuUfKEPrvr6FOOOYBWtcXkn53xxD+dtd0qyLGaJmtE5giyFjT1Ufb4oy70XP2sDOlM9S7YHaxjHF65XAVyWdNhBj1hTbztYvkhWyfSstjJw5xCGUzkN3NG2wd6SU5dfWP9GOKR2/qk20M062bwT2nenziGX0Rut82JBt1a90B1sHSdqMyrO6NVqf4pUyZehtLJvEZlX93cTVEcxAR6+rbW+rMh3e/6F8Yf9jG+Xdkrbh/lmwznflWbDGRWVSnWOBR9p+ssrww/vYrj0TWkzTTNVFSRKlN+5T2owTw03iHcGiFe9SVa9se2/gk7a/2gzWVl3zxT8rv/wHjGtKzJi+sdRFDVywzQG2p8xWVjvOjxlyYVizg1wz3tiwGLOmIcfEJYLaPXpXws8lHUeZ5/XIZqC2OSt4TdeNZUrMWCXjqovqv2C7FzjZ9iUtxJnf93xtyuijGy1n31V15kCMFwM3LWff1dLEJQJJH7b9NklnMDxL71M55MuBvYB/tf3rZuTBWd2UbAzGOSVmTM9Y6qLGdcE2ZLThD0u6GDisYoxlWllJOhk4r9bxx2ES6wh2sn1FMwrkA9j+5rjPKZYlaSvKlJhPpwzZ/GPg1U0lcsywNuuiNIb5NQbi9Xce6zXvfkOb/YlU5qI4y/bj2opR28TdEfQNKre97WP6t0l6K5BEMMNs3wDsrjJnxBw3E+HE6qHluqgXNj97Tbn75+Buq7Vdz72Ui46X1wwwZOiUX1BaE84aE3dH0DNsPJH+XsYxcyR9Hnhzr5OPyuQ7J7jiHL+xehs2iGLNgRUlvdX2MZKeYfviGsccEmNX25e0MSjfuE1cpaakA5v6gS0lLex7XEhp6xsz72Lg25JeIOmvgXOBD8/wOcV4rdM0uQb+NEjkOlPsP10HNz+nnLt4RL1jX9pijLGYuKIhyj/lZmBjlr0tvBOYtdMhThLbx0m6ljJZyK3ADm2MER+rtdcDJ0hav1n+NVCzueX3Jd0IzJXU/7nvTehUoy7iD03T0cdIekDCqd1nqU0TWzQUqy9JrwHeA7wX2JYySuzBtqu3I4/VWzMEtdoYC6gZUfUc4AEtBV1hrm9JG9M0G2dIK6QZaMq+yiY2EUjaBfgo8CRgLWAN4LcZ837mSfoPYIHtW5rlnYHjbW8/s2cWbeuNaTRsPCNoZUyj1knabrZfxExi0VDPxyhzh55GaTL2WmDWNOeaZLb3G1j+TpMMYvL16gFanX9iimaq1YqG+ntHD3SOBGZX0dAkJwJsL5G0hu37gBMlzfpKndlM0j/YPmqK8aBmzQcnVo3t3rAi72851AtXvMvIxj2cTWsmORHc1cxDsFjSUZQK5JqtEmL6DqGMMPkjSkey6KjmM/nPwN3Af1LmL35bMxT2yGrUAaxEjFlTB7Aik1xHsDllHtQ1gb8D1gc+YXvJjJ5Yh0m6Dng+ZfrQZw9ut3372E8qZoSkxba3l/RiYD/KZ/TC2j1+Bzp7rUX5PqhaV6iW5y0fh4m9I+i7IrgbaPs2NFbOsZSrv61Y9rZalA/SWCYvj9XCms3PF1AGnLt9WDn7qHrzZPdI2o8yCU5Nbc9b3rqJuyMY91gmMX2SjrX9hpk+j5g5ko6g3AncTfli3gA4cxzzVku6zPYuLceoPm95myYxEWw+1fZxlB1GxIo1k97c0cz7uw6wbu2OhZJe0rfYG3Tumbb/vGKMwXnLdwI+YvsJtWK0beKKhvq/6CU9Enhqs/idXrv1iJhZkh5KGXhuHrAAeDTwBJYd27+G/ikx76XMKV57mtT+ect7A9u1MW95aybujqBH0suBDwLfoPyD/gJ4h+3TZ/K8IgIknUr5An1tM13pQ4BvpVPhzJi4Qef6vAt4qu2DbL+WUg75nhk+p4goHmv7KOAPALbvplywVSXpKEnrSVpT0vmSbpX06sox9pe0bvP83ZL+78A8CKu9SU4EcwaKgm5jsn/fiNnknuYuoNcz97HA71uIs6ftOygdzJYCj6f+DILvsX1nM5rq84CTKC3kZo1J/mL8mqRzJL1O0uuAs4CzZ/icIjpPpZ3oJylNiTeT9EXgfOAfWgj3gGaqLcS4r/m5N3Cs7a9S+izMGhNXWdznF8A1wPaUW87jbX9lZk8pImy7mS1wT2AXyufzrbZvbSHcGZJ+QGmm+kZJc4Hak8j8XNJxNCORSnows+wie5Iri99LmZLuduAU4HTb/z2zZxURAJI+DnzW9uVjiNVqM9WmBdRewDW2r5e0CfAU21+vFaNtE5sIeiRtC7yC0ttvqe3dZ/iUIjqvGW7k8cBPgN9Sd8KYFcV+VCZCWtYkFw313EIpJroNeMQMn0tEFM+fwdifoZTnt0bSmbbHMQJqFRN7RyDpDZQ7gbnA6cCptq+b2bOKiC6QtIntm2f6PFbWJN8RbE4Z1nbxTJ9IRMycplnn1rZPbCqLH2b7x23GnE1JACb4jiAiomk0Mh94gu3HS3o0cJrtXSscu/VZ0MZlku8IIiJeDOwAfBfA9k29XsAVzJo6gBVJIoiISXZP02+h14O52iyFkzSS8azq9BARMU1fajp7bSDpr4HzgE/VDCBpF0mXS/qNpHsk3Sfpjpox2pY6goiYaJL2oPRiFnCO7XMrH38RcABwGqU+4rXA42y/q2acNqVoKCIm3X9RKm/Pk/RQSevavrNmANtLJK1h+z7gREmX1jx+25IIImJiNcVBC4CNgMcCm1IGvHtuxTB3SVoLWCzpKOBmoFpdxDikjiAiJtmbgF2BOwBsX0/9EQZeQ/kufTNluIzNgJdM+YrVTBJBREyy39u+p7cg6UEMb/s/iv1s/872Hbbfb/vtzLKmpUkEETHJvinpH4GHNJXGpwFnVI5x0JB1r6sco1VpNRQRE0vSHMpE8n9qNQR82hW++CQdCLwSeAbw//o2rQfcO5tGOk4iiIiJJGkN4CTbVeco7jv+5sCWwL8Ah/ZtuhO42va9bcRtQ1oNRcREaiaimStprf56gorH/wllPoU/l/RI4KnNpu/PpiQASQQRMdluBC6RtJDSogcA2/9WK4Ck/YF/Bb5BKX76qKR32D69Voy2pWgoIiaOpM/bfo2kXwMfGtxu+/0VY10F7GH7lmZ5LnCe7e1qxWhb7ggiYhLt1JTh/xT4aMux5vSSQOM2ZlmLzCSCiJhEnwT+k1KZu6hvvSj9CLaqGOtrks4BTm6WXwGcXfH4rUvRUERMLEnH2n5DyzGOBL5NaUYq4CJgF9uHtBm3piSCiIgRSPqu7R0H1l2dGcoiIiacpDcAbwS2knR136Z1gUtm5qxWTe4IIiJWgaT1gQ0Z0qHM9u0zc1arJokgIqLjZlUTp4iIqC+JICKi45IIIiI6LokgIqLjkggiIjru/wOkWArlz90Y2wAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "df.corr()['type'].sort_values().plot(kind='bar')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id='split'></a>\n",
    "## Split the Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "X = df.drop('type',axis=1).values\n",
    "y = df['type'].values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train, X_test, y_train, y_test = train_test_split(X,y,test_size=0.25,random_state=42)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(2398, 12)"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_train.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(2398,)"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_train.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id='normalize'></a>\n",
    "## Normalize the Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.preprocessing import MinMaxScaler"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "scaler = MinMaxScaler()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train = scaler.fit_transform(X_train)\n",
    "X_test = scaler.transform(X_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id='model'></a>\n",
    "## Create the Model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "from tensorflow.keras.models import Sequential\n",
    "from tensorflow.keras.layers import Dense"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = Sequential()\n",
    "\n",
    "model.add(Dense(40,activation='relu'))\n",
    "model.add(Dense(20,activation='relu'))\n",
    "model.add(Dense(units=1,activation='sigmoid'))\n",
    "\n",
    "# For a binary classification problem\n",
    "model.compile(loss='binary_crossentropy', optimizer='adam')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id='training'></a>\n",
    "## Train the Model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id='overfit'></a>\n",
    "### 1. Choosing too many epochs and overfit"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 2398 samples, validate on 800 samples\n",
      "Epoch 1/500\n",
      "2398/2398 [==============================] - 1s 291us/sample - loss: 0.5837 - val_loss: 0.4379\n",
      "Epoch 2/500\n",
      "2398/2398 [==============================] - 0s 64us/sample - loss: 0.2896 - val_loss: 0.1654\n",
      "Epoch 3/500\n",
      "2398/2398 [==============================] - 0s 64us/sample - loss: 0.1272 - val_loss: 0.0962\n",
      "Epoch 4/500\n",
      "2398/2398 [==============================] - 0s 63us/sample - loss: 0.0888 - val_loss: 0.0752\n",
      "Epoch 5/500\n",
      "2398/2398 [==============================] - 0s 62us/sample - loss: 0.0742 - val_loss: 0.0659\n",
      "Epoch 6/500\n",
      "2398/2398 [==============================] - 0s 72us/sample - loss: 0.0668 - val_loss: 0.0596\n",
      "Epoch 7/500\n",
      "2398/2398 [==============================] - 0s 59us/sample - loss: 0.0609 - val_loss: 0.0579\n",
      "Epoch 8/500\n",
      "2398/2398 [==============================] - 0s 60us/sample - loss: 0.0574 - val_loss: 0.0519\n",
      "Epoch 9/500\n",
      "2398/2398 [==============================] - 0s 47us/sample - loss: 0.0543 - val_loss: 0.0501\n",
      "Epoch 10/500\n",
      "2398/2398 [==============================] - 0s 51us/sample - loss: 0.0524 - val_loss: 0.0474\n",
      "Epoch 11/500\n",
      "2398/2398 [==============================] - 0s 47us/sample - loss: 0.0498 - val_loss: 0.0481\n",
      "Epoch 12/500\n",
      "2398/2398 [==============================] - 0s 45us/sample - loss: 0.0485 - val_loss: 0.0444\n",
      "Epoch 13/500\n",
      "2398/2398 [==============================] - 0s 57us/sample - loss: 0.0473 - val_loss: 0.0440\n",
      "Epoch 14/500\n",
      "2398/2398 [==============================] - 0s 58us/sample - loss: 0.0453 - val_loss: 0.0424\n",
      "Epoch 15/500\n",
      "2398/2398 [==============================] - 0s 47us/sample - loss: 0.0448 - val_loss: 0.0430\n",
      "Epoch 16/500\n",
      "2398/2398 [==============================] - 0s 51us/sample - loss: 0.0432 - val_loss: 0.0403\n",
      "Epoch 17/500\n",
      "2398/2398 [==============================] - 0s 50us/sample - loss: 0.0416 - val_loss: 0.0418\n",
      "Epoch 18/500\n",
      "2398/2398 [==============================] - 0s 56us/sample - loss: 0.0418 - val_loss: 0.0410\n",
      "Epoch 19/500\n",
      "2398/2398 [==============================] - 0s 48us/sample - loss: 0.0402 - val_loss: 0.0400\n",
      "Epoch 20/500\n",
      "2398/2398 [==============================] - 0s 47us/sample - loss: 0.0395 - val_loss: 0.0404\n",
      "Epoch 21/500\n",
      "2398/2398 [==============================] - 0s 48us/sample - loss: 0.0382 - val_loss: 0.0376\n",
      "Epoch 22/500\n",
      "2398/2398 [==============================] - 0s 69us/sample - loss: 0.0395 - val_loss: 0.0373\n",
      "Epoch 23/500\n",
      "2398/2398 [==============================] - 0s 64us/sample - loss: 0.0371 - val_loss: 0.0368\n",
      "Epoch 24/500\n",
      "2398/2398 [==============================] - 0s 58us/sample - loss: 0.0370 - val_loss: 0.0367\n",
      "Epoch 25/500\n",
      "2398/2398 [==============================] - 0s 47us/sample - loss: 0.0354 - val_loss: 0.0366\n",
      "Epoch 26/500\n",
      "2398/2398 [==============================] - 0s 50us/sample - loss: 0.0362 - val_loss: 0.0361\n",
      "Epoch 27/500\n",
      "2398/2398 [==============================] - 0s 47us/sample - loss: 0.0347 - val_loss: 0.0359\n",
      "Epoch 28/500\n",
      "2398/2398 [==============================] - 0s 46us/sample - loss: 0.0337 - val_loss: 0.0351\n",
      "Epoch 29/500\n",
      "2398/2398 [==============================] - 0s 51us/sample - loss: 0.0338 - val_loss: 0.0348\n",
      "Epoch 30/500\n",
      "2398/2398 [==============================] - 0s 57us/sample - loss: 0.0331 - val_loss: 0.0345\n",
      "Epoch 31/500\n",
      "2398/2398 [==============================] - 0s 47us/sample - loss: 0.0324 - val_loss: 0.0345\n",
      "Epoch 32/500\n",
      "2398/2398 [==============================] - 0s 47us/sample - loss: 0.0314 - val_loss: 0.0342\n",
      "Epoch 33/500\n",
      "2398/2398 [==============================] - 0s 49us/sample - loss: 0.0309 - val_loss: 0.0350\n",
      "Epoch 34/500\n",
      "2398/2398 [==============================] - 0s 67us/sample - loss: 0.0312 - val_loss: 0.0338\n",
      "Epoch 35/500\n",
      "2398/2398 [==============================] - 0s 48us/sample - loss: 0.0305 - val_loss: 0.0339\n",
      "Epoch 36/500\n",
      "2398/2398 [==============================] - 0s 45us/sample - loss: 0.0293 - val_loss: 0.0339\n",
      "Epoch 37/500\n",
      "2398/2398 [==============================] - 0s 47us/sample - loss: 0.0291 - val_loss: 0.0373\n",
      "Epoch 38/500\n",
      "2398/2398 [==============================] - 0s 48us/sample - loss: 0.0291 - val_loss: 0.0332\n",
      "Epoch 39/500\n",
      "2398/2398 [==============================] - 0s 49us/sample - loss: 0.0283 - val_loss: 0.0346\n",
      "Epoch 40/500\n",
      "2398/2398 [==============================] - 0s 47us/sample - loss: 0.0275 - val_loss: 0.0331\n",
      "Epoch 41/500\n",
      "2398/2398 [==============================] - 0s 48us/sample - loss: 0.0270 - val_loss: 0.0333\n",
      "Epoch 42/500\n",
      "2398/2398 [==============================] - 0s 54us/sample - loss: 0.0260 - val_loss: 0.0324\n",
      "Epoch 43/500\n",
      "2398/2398 [==============================] - 0s 61us/sample - loss: 0.0259 - val_loss: 0.0320\n",
      "Epoch 44/500\n",
      "2398/2398 [==============================] - 0s 49us/sample - loss: 0.0256 - val_loss: 0.0330\n",
      "Epoch 45/500\n",
      "2398/2398 [==============================] - 0s 56us/sample - loss: 0.0255 - val_loss: 0.0322\n",
      "Epoch 46/500\n",
      "2398/2398 [==============================] - 0s 46us/sample - loss: 0.0244 - val_loss: 0.0322\n",
      "Epoch 47/500\n",
      "2398/2398 [==============================] - 0s 49us/sample - loss: 0.0247 - val_loss: 0.0316\n",
      "Epoch 48/500\n",
      "2398/2398 [==============================] - 0s 50us/sample - loss: 0.0248 - val_loss: 0.0317\n",
      "Epoch 49/500\n",
      "2398/2398 [==============================] - 0s 50us/sample - loss: 0.0241 - val_loss: 0.0315\n",
      "Epoch 50/500\n",
      "2398/2398 [==============================] - 0s 48us/sample - loss: 0.0235 - val_loss: 0.0315\n",
      "Epoch 51/500\n",
      "2398/2398 [==============================] - 0s 50us/sample - loss: 0.0233 - val_loss: 0.0323\n",
      "Epoch 52/500\n",
      "2398/2398 [==============================] - 0s 47us/sample - loss: 0.0222 - val_loss: 0.0349\n",
      "Epoch 53/500\n",
      "2398/2398 [==============================] - 0s 48us/sample - loss: 0.0225 - val_loss: 0.0319\n",
      "Epoch 54/500\n",
      "2398/2398 [==============================] - 0s 46us/sample - loss: 0.0229 - val_loss: 0.0311\n",
      "Epoch 55/500\n",
      "2398/2398 [==============================] - 0s 61us/sample - loss: 0.0217 - val_loss: 0.0324\n",
      "Epoch 56/500\n",
      "2398/2398 [==============================] - 0s 54us/sample - loss: 0.0222 - val_loss: 0.0326\n",
      "Epoch 57/500\n",
      "2398/2398 [==============================] - 0s 62us/sample - loss: 0.0209 - val_loss: 0.0326\n",
      "Epoch 58/500\n",
      "2398/2398 [==============================] - 0s 50us/sample - loss: 0.0209 - val_loss: 0.0336\n",
      "Epoch 59/500\n",
      "2398/2398 [==============================] - 0s 49us/sample - loss: 0.0209 - val_loss: 0.0316\n",
      "Epoch 60/500\n",
      "2398/2398 [==============================] - 0s 46us/sample - loss: 0.0201 - val_loss: 0.0312\n",
      "Epoch 61/500\n",
      "2398/2398 [==============================] - 0s 51us/sample - loss: 0.0201 - val_loss: 0.0320\n",
      "Epoch 62/500\n",
      "2398/2398 [==============================] - 0s 48us/sample - loss: 0.0198 - val_loss: 0.0334\n",
      "Epoch 63/500\n",
      "2398/2398 [==============================] - 0s 50us/sample - loss: 0.0192 - val_loss: 0.0327\n",
      "Epoch 64/500\n",
      "2398/2398 [==============================] - 0s 46us/sample - loss: 0.0195 - val_loss: 0.0335\n",
      "Epoch 65/500\n",
      "2398/2398 [==============================] - 0s 47us/sample - loss: 0.0197 - val_loss: 0.0320\n",
      "Epoch 66/500\n",
      "2398/2398 [==============================] - 0s 47us/sample - loss: 0.0189 - val_loss: 0.0331\n",
      "Epoch 67/500\n",
      "2398/2398 [==============================] - 0s 49us/sample - loss: 0.0188 - val_loss: 0.0330\n",
      "Epoch 68/500\n",
      "2398/2398 [==============================] - 0s 50us/sample - loss: 0.0184 - val_loss: 0.0320\n",
      "Epoch 69/500\n",
      "2398/2398 [==============================] - 0s 46us/sample - loss: 0.0176 - val_loss: 0.0321\n",
      "Epoch 70/500\n",
      "2398/2398 [==============================] - 0s 51us/sample - loss: 0.0177 - val_loss: 0.0319\n",
      "Epoch 71/500\n",
      "2398/2398 [==============================] - 0s 47us/sample - loss: 0.0172 - val_loss: 0.0332\n",
      "Epoch 72/500\n",
      "2398/2398 [==============================] - 0s 51us/sample - loss: 0.0169 - val_loss: 0.0323\n",
      "Epoch 73/500\n",
      "2398/2398 [==============================] - 0s 52us/sample - loss: 0.0173 - val_loss: 0.0325\n",
      "Epoch 74/500\n",
      "2398/2398 [==============================] - 0s 48us/sample - loss: 0.0173 - val_loss: 0.0323\n",
      "Epoch 75/500\n",
      "2398/2398 [==============================] - 0s 49us/sample - loss: 0.0174 - val_loss: 0.0323\n",
      "Epoch 76/500\n",
      "2398/2398 [==============================] - 0s 47us/sample - loss: 0.0166 - val_loss: 0.0329\n",
      "Epoch 77/500\n",
      "2398/2398 [==============================] - 0s 47us/sample - loss: 0.0165 - val_loss: 0.0326\n",
      "Epoch 78/500\n",
      "2398/2398 [==============================] - 0s 50us/sample - loss: 0.0160 - val_loss: 0.0335\n",
      "Epoch 79/500\n",
      "2398/2398 [==============================] - 0s 47us/sample - loss: 0.0160 - val_loss: 0.0333\n",
      "Epoch 80/500\n",
      "2398/2398 [==============================] - 0s 51us/sample - loss: 0.0161 - val_loss: 0.0347\n",
      "Epoch 81/500\n",
      "2398/2398 [==============================] - 0s 46us/sample - loss: 0.0163 - val_loss: 0.0381\n",
      "Epoch 82/500\n",
      "2398/2398 [==============================] - 0s 45us/sample - loss: 0.0174 - val_loss: 0.0311\n",
      "Epoch 83/500\n",
      "2398/2398 [==============================] - 0s 46us/sample - loss: 0.0156 - val_loss: 0.0328\n",
      "Epoch 84/500\n",
      "2398/2398 [==============================] - 0s 46us/sample - loss: 0.0158 - val_loss: 0.0326\n",
      "Epoch 85/500\n",
      "2398/2398 [==============================] - 0s 53us/sample - loss: 0.0151 - val_loss: 0.0327\n",
      "Epoch 86/500\n",
      "2398/2398 [==============================] - 0s 47us/sample - loss: 0.0153 - val_loss: 0.0339\n",
      "Epoch 87/500\n",
      "2398/2398 [==============================] - 0s 49us/sample - loss: 0.0156 - val_loss: 0.0325\n",
      "Epoch 88/500\n",
      "2398/2398 [==============================] - 0s 46us/sample - loss: 0.0153 - val_loss: 0.0309\n",
      "Epoch 89/500\n",
      "2398/2398 [==============================] - 0s 51us/sample - loss: 0.0153 - val_loss: 0.0323\n",
      "Epoch 90/500\n",
      "2398/2398 [==============================] - 0s 48us/sample - loss: 0.0144 - val_loss: 0.0319\n",
      "Epoch 91/500\n",
      "2398/2398 [==============================] - 0s 51us/sample - loss: 0.0144 - val_loss: 0.0328\n",
      "Epoch 92/500\n",
      "2398/2398 [==============================] - 0s 48us/sample - loss: 0.0142 - val_loss: 0.0331\n",
      "Epoch 93/500\n",
      "2398/2398 [==============================] - 0s 49us/sample - loss: 0.0141 - val_loss: 0.0330\n",
      "Epoch 94/500\n",
      "2398/2398 [==============================] - 0s 61us/sample - loss: 0.0138 - val_loss: 0.0325\n",
      "Epoch 95/500\n",
      "2398/2398 [==============================] - 0s 47us/sample - loss: 0.0143 - val_loss: 0.0329\n",
      "Epoch 96/500\n",
      "2398/2398 [==============================] - 0s 50us/sample - loss: 0.0131 - val_loss: 0.0342\n",
      "Epoch 97/500\n",
      "2398/2398 [==============================] - 0s 52us/sample - loss: 0.0137 - val_loss: 0.0330\n",
      "Epoch 98/500\n",
      "2398/2398 [==============================] - 0s 52us/sample - loss: 0.0136 - val_loss: 0.0333\n",
      "Epoch 99/500\n",
      "2398/2398 [==============================] - 0s 54us/sample - loss: 0.0140 - val_loss: 0.0330\n",
      "Epoch 100/500\n",
      "2398/2398 [==============================] - 0s 55us/sample - loss: 0.0136 - val_loss: 0.0330\n",
      "Epoch 101/500\n",
      "2398/2398 [==============================] - 0s 52us/sample - loss: 0.0133 - val_loss: 0.0340\n",
      "Epoch 102/500\n",
      "2398/2398 [==============================] - 0s 48us/sample - loss: 0.0133 - val_loss: 0.0336\n",
      "Epoch 103/500\n",
      "2398/2398 [==============================] - 0s 51us/sample - loss: 0.0134 - val_loss: 0.0349\n",
      "Epoch 104/500\n",
      "2398/2398 [==============================] - 0s 52us/sample - loss: 0.0127 - val_loss: 0.0346\n",
      "Epoch 105/500\n",
      "2398/2398 [==============================] - 0s 51us/sample - loss: 0.0130 - val_loss: 0.0327\n",
      "Epoch 106/500\n",
      "2398/2398 [==============================] - 0s 45us/sample - loss: 0.0125 - val_loss: 0.0320\n",
      "Epoch 107/500\n",
      "2398/2398 [==============================] - 0s 47us/sample - loss: 0.0127 - val_loss: 0.0362\n",
      "Epoch 108/500\n",
      "2398/2398 [==============================] - 0s 48us/sample - loss: 0.0126 - val_loss: 0.0338\n",
      "Epoch 109/500\n",
      "2398/2398 [==============================] - 0s 47us/sample - loss: 0.0120 - val_loss: 0.0347\n",
      "Epoch 110/500\n",
      "2398/2398 [==============================] - 0s 51us/sample - loss: 0.0132 - val_loss: 0.0315\n",
      "Epoch 111/500\n",
      "2398/2398 [==============================] - 0s 47us/sample - loss: 0.0132 - val_loss: 0.0340\n",
      "Epoch 112/500\n",
      "2398/2398 [==============================] - 0s 53us/sample - loss: 0.0122 - val_loss: 0.0343\n",
      "Epoch 113/500\n",
      "2398/2398 [==============================] - 0s 48us/sample - loss: 0.0126 - val_loss: 0.0364\n",
      "Epoch 114/500\n",
      "2398/2398 [==============================] - 0s 52us/sample - loss: 0.0117 - val_loss: 0.0339\n",
      "Epoch 115/500\n",
      "2398/2398 [==============================] - 0s 46us/sample - loss: 0.0114 - val_loss: 0.0346\n",
      "Epoch 116/500\n",
      "2398/2398 [==============================] - 0s 47us/sample - loss: 0.0116 - val_loss: 0.0334\n",
      "Epoch 117/500\n",
      "2398/2398 [==============================] - 0s 46us/sample - loss: 0.0125 - val_loss: 0.0339\n",
      "Epoch 118/500\n",
      "2398/2398 [==============================] - 0s 54us/sample - loss: 0.0114 - val_loss: 0.0344\n",
      "Epoch 119/500\n",
      "2398/2398 [==============================] - 0s 49us/sample - loss: 0.0118 - val_loss: 0.0347\n",
      "Epoch 120/500\n",
      "2398/2398 [==============================] - 0s 51us/sample - loss: 0.0115 - val_loss: 0.0348\n",
      "Epoch 121/500\n",
      "2398/2398 [==============================] - 0s 49us/sample - loss: 0.0123 - val_loss: 0.0347\n",
      "Epoch 122/500\n",
      "2398/2398 [==============================] - 0s 50us/sample - loss: 0.0116 - val_loss: 0.0349\n",
      "Epoch 123/500\n",
      "2398/2398 [==============================] - 0s 48us/sample - loss: 0.0109 - val_loss: 0.0323\n",
      "Epoch 124/500\n",
      "2398/2398 [==============================] - 0s 47us/sample - loss: 0.0112 - val_loss: 0.0376\n",
      "Epoch 125/500\n",
      "2398/2398 [==============================] - 0s 46us/sample - loss: 0.0111 - val_loss: 0.0336\n",
      "Epoch 126/500\n",
      "2398/2398 [==============================] - 0s 50us/sample - loss: 0.0110 - val_loss: 0.0349\n",
      "Epoch 127/500\n",
      "2398/2398 [==============================] - 0s 48us/sample - loss: 0.0112 - val_loss: 0.0340\n",
      "Epoch 128/500\n",
      "2398/2398 [==============================] - 0s 49us/sample - loss: 0.0112 - val_loss: 0.0336\n",
      "Epoch 129/500\n",
      "2398/2398 [==============================] - 0s 48us/sample - loss: 0.0106 - val_loss: 0.0354\n",
      "Epoch 130/500\n",
      "2398/2398 [==============================] - 0s 45us/sample - loss: 0.0106 - val_loss: 0.0353\n",
      "Epoch 131/500\n",
      "2398/2398 [==============================] - 0s 50us/sample - loss: 0.0107 - val_loss: 0.0346\n",
      "Epoch 132/500\n",
      "2398/2398 [==============================] - 0s 59us/sample - loss: 0.0099 - val_loss: 0.0361\n",
      "Epoch 133/500\n",
      "2398/2398 [==============================] - 0s 56us/sample - loss: 0.0106 - val_loss: 0.0357\n",
      "Epoch 134/500\n",
      "2398/2398 [==============================] - 0s 53us/sample - loss: 0.0103 - val_loss: 0.0355\n",
      "Epoch 135/500\n",
      "2398/2398 [==============================] - 0s 64us/sample - loss: 0.0104 - val_loss: 0.0336\n",
      "Epoch 136/500\n",
      "2398/2398 [==============================] - 0s 51us/sample - loss: 0.0106 - val_loss: 0.0345\n",
      "Epoch 137/500\n",
      "2398/2398 [==============================] - 0s 59us/sample - loss: 0.0097 - val_loss: 0.0343\n",
      "Epoch 138/500\n",
      "2398/2398 [==============================] - 0s 52us/sample - loss: 0.0096 - val_loss: 0.0351\n",
      "Epoch 139/500\n",
      "2398/2398 [==============================] - 0s 50us/sample - loss: 0.0100 - val_loss: 0.0327\n",
      "Epoch 140/500\n",
      "2398/2398 [==============================] - 0s 56us/sample - loss: 0.0106 - val_loss: 0.0354\n",
      "Epoch 141/500\n",
      "2398/2398 [==============================] - 0s 63us/sample - loss: 0.0099 - val_loss: 0.0344\n",
      "Epoch 142/500\n",
      "2398/2398 [==============================] - 0s 59us/sample - loss: 0.0092 - val_loss: 0.0351\n",
      "Epoch 143/500\n",
      "2398/2398 [==============================] - 0s 57us/sample - loss: 0.0094 - val_loss: 0.0343\n",
      "Epoch 144/500\n",
      "2398/2398 [==============================] - 0s 61us/sample - loss: 0.0094 - val_loss: 0.0325\n",
      "Epoch 145/500\n",
      "2398/2398 [==============================] - 0s 57us/sample - loss: 0.0103 - val_loss: 0.0357\n",
      "Epoch 146/500\n",
      "2398/2398 [==============================] - 0s 55us/sample - loss: 0.0090 - val_loss: 0.0342\n",
      "Epoch 147/500\n",
      "2398/2398 [==============================] - 0s 51us/sample - loss: 0.0094 - val_loss: 0.0349\n",
      "Epoch 148/500\n",
      "2398/2398 [==============================] - 0s 49us/sample - loss: 0.0090 - val_loss: 0.0374\n",
      "Epoch 149/500\n",
      "2398/2398 [==============================] - 0s 47us/sample - loss: 0.0089 - val_loss: 0.0376\n",
      "Epoch 150/500\n",
      "2398/2398 [==============================] - 0s 52us/sample - loss: 0.0090 - val_loss: 0.0360\n",
      "Epoch 151/500\n",
      "2398/2398 [==============================] - 0s 75us/sample - loss: 0.0089 - val_loss: 0.0336\n",
      "Epoch 152/500\n",
      "2398/2398 [==============================] - 0s 50us/sample - loss: 0.0088 - val_loss: 0.0340\n",
      "Epoch 153/500\n",
      "2398/2398 [==============================] - 0s 64us/sample - loss: 0.0085 - val_loss: 0.0367\n",
      "Epoch 154/500\n",
      "2398/2398 [==============================] - 0s 57us/sample - loss: 0.0088 - val_loss: 0.0344\n",
      "Epoch 155/500\n",
      "2398/2398 [==============================] - 0s 60us/sample - loss: 0.0087 - val_loss: 0.0363\n",
      "Epoch 156/500\n",
      "2398/2398 [==============================] - 0s 60us/sample - loss: 0.0082 - val_loss: 0.0351\n",
      "Epoch 157/500\n",
      "2398/2398 [==============================] - 0s 174us/sample - loss: 0.0088 - val_loss: 0.0395\n",
      "Epoch 158/500\n",
      "2398/2398 [==============================] - 0s 154us/sample - loss: 0.0087 - val_loss: 0.0342\n",
      "Epoch 159/500\n",
      "2398/2398 [==============================] - 0s 122us/sample - loss: 0.0087 - val_loss: 0.0396\n",
      "Epoch 160/500\n",
      "2398/2398 [==============================] - 0s 67us/sample - loss: 0.0085 - val_loss: 0.0362\n",
      "Epoch 161/500\n",
      "2398/2398 [==============================] - 0s 50us/sample - loss: 0.0081 - val_loss: 0.0349\n",
      "Epoch 162/500\n",
      "2398/2398 [==============================] - 0s 47us/sample - loss: 0.0101 - val_loss: 0.0328\n",
      "Epoch 163/500\n",
      "2398/2398 [==============================] - 0s 48us/sample - loss: 0.0089 - val_loss: 0.0364\n",
      "Epoch 164/500\n",
      "2398/2398 [==============================] - 0s 47us/sample - loss: 0.0080 - val_loss: 0.0380\n",
      "Epoch 165/500\n",
      "2398/2398 [==============================] - 0s 45us/sample - loss: 0.0073 - val_loss: 0.0382\n",
      "Epoch 166/500\n",
      "2398/2398 [==============================] - 0s 46us/sample - loss: 0.0078 - val_loss: 0.0355\n",
      "Epoch 167/500\n",
      "2398/2398 [==============================] - 0s 46us/sample - loss: 0.0075 - val_loss: 0.0370\n",
      "Epoch 168/500\n",
      "2398/2398 [==============================] - 0s 47us/sample - loss: 0.0078 - val_loss: 0.0374\n",
      "Epoch 169/500\n",
      "2398/2398 [==============================] - 0s 44us/sample - loss: 0.0074 - val_loss: 0.0380\n",
      "Epoch 170/500\n",
      "2398/2398 [==============================] - 0s 48us/sample - loss: 0.0072 - val_loss: 0.0372\n",
      "Epoch 171/500\n",
      "2398/2398 [==============================] - 0s 47us/sample - loss: 0.0073 - val_loss: 0.0383\n",
      "Epoch 172/500\n",
      "2398/2398 [==============================] - 0s 47us/sample - loss: 0.0073 - val_loss: 0.0374\n",
      "Epoch 173/500\n",
      "2398/2398 [==============================] - 0s 46us/sample - loss: 0.0080 - val_loss: 0.0393\n",
      "Epoch 174/500\n",
      "2398/2398 [==============================] - 0s 45us/sample - loss: 0.0071 - val_loss: 0.0391\n",
      "Epoch 175/500\n",
      "2398/2398 [==============================] - 0s 49us/sample - loss: 0.0076 - val_loss: 0.0374\n",
      "Epoch 176/500\n",
      "2398/2398 [==============================] - 0s 47us/sample - loss: 0.0068 - val_loss: 0.0385\n",
      "Epoch 177/500\n",
      "2398/2398 [==============================] - 0s 47us/sample - loss: 0.0067 - val_loss: 0.0377\n",
      "Epoch 178/500\n",
      "2398/2398 [==============================] - 0s 43us/sample - loss: 0.0068 - val_loss: 0.0400\n",
      "Epoch 179/500\n",
      "2398/2398 [==============================] - 0s 43us/sample - loss: 0.0067 - val_loss: 0.0432\n",
      "Epoch 180/500\n",
      "2398/2398 [==============================] - 0s 47us/sample - loss: 0.0070 - val_loss: 0.0365\n",
      "Epoch 181/500\n",
      "2398/2398 [==============================] - 0s 58us/sample - loss: 0.0066 - val_loss: 0.0372\n",
      "Epoch 182/500\n",
      "2398/2398 [==============================] - 0s 51us/sample - loss: 0.0062 - val_loss: 0.0355\n",
      "Epoch 183/500\n",
      "2398/2398 [==============================] - 0s 49us/sample - loss: 0.0067 - val_loss: 0.0373\n",
      "Epoch 184/500\n",
      "2398/2398 [==============================] - 0s 49us/sample - loss: 0.0064 - val_loss: 0.0413\n",
      "Epoch 185/500\n",
      "2398/2398 [==============================] - 0s 52us/sample - loss: 0.0065 - val_loss: 0.0382\n",
      "Epoch 186/500\n",
      "2398/2398 [==============================] - 0s 48us/sample - loss: 0.0064 - val_loss: 0.0378\n",
      "Epoch 187/500\n",
      "2398/2398 [==============================] - 0s 45us/sample - loss: 0.0061 - val_loss: 0.0393\n",
      "Epoch 188/500\n",
      "2398/2398 [==============================] - 0s 42us/sample - loss: 0.0066 - val_loss: 0.0387\n",
      "Epoch 189/500\n",
      "2398/2398 [==============================] - 0s 44us/sample - loss: 0.0061 - val_loss: 0.0388\n",
      "Epoch 190/500\n",
      "2398/2398 [==============================] - 0s 55us/sample - loss: 0.0059 - val_loss: 0.0367\n",
      "Epoch 191/500\n",
      "2398/2398 [==============================] - 0s 46us/sample - loss: 0.0059 - val_loss: 0.0367\n",
      "Epoch 192/500\n",
      "2398/2398 [==============================] - 0s 45us/sample - loss: 0.0059 - val_loss: 0.0387\n",
      "Epoch 193/500\n",
      "2398/2398 [==============================] - 0s 47us/sample - loss: 0.0056 - val_loss: 0.0386\n",
      "Epoch 194/500\n",
      "2398/2398 [==============================] - 0s 49us/sample - loss: 0.0057 - val_loss: 0.0391\n",
      "Epoch 195/500\n",
      "2398/2398 [==============================] - 0s 46us/sample - loss: 0.0057 - val_loss: 0.0399\n",
      "Epoch 196/500\n",
      "2398/2398 [==============================] - 0s 51us/sample - loss: 0.0070 - val_loss: 0.0359\n",
      "Epoch 197/500\n",
      "2398/2398 [==============================] - 0s 43us/sample - loss: 0.0059 - val_loss: 0.0382\n",
      "Epoch 198/500\n",
      "2398/2398 [==============================] - 0s 44us/sample - loss: 0.0056 - val_loss: 0.0422\n",
      "Epoch 199/500\n",
      "2398/2398 [==============================] - 0s 81us/sample - loss: 0.0054 - val_loss: 0.0398\n",
      "Epoch 200/500\n",
      "2398/2398 [==============================] - 0s 55us/sample - loss: 0.0057 - val_loss: 0.0397\n",
      "Epoch 201/500\n",
      "2398/2398 [==============================] - 0s 52us/sample - loss: 0.0047 - val_loss: 0.0443\n",
      "Epoch 202/500\n",
      "2398/2398 [==============================] - 0s 66us/sample - loss: 0.0056 - val_loss: 0.0402\n",
      "Epoch 203/500\n",
      "2398/2398 [==============================] - 0s 79us/sample - loss: 0.0063 - val_loss: 0.0397\n",
      "Epoch 204/500\n",
      "2398/2398 [==============================] - 0s 54us/sample - loss: 0.0056 - val_loss: 0.0448\n",
      "Epoch 205/500\n",
      "2398/2398 [==============================] - 0s 60us/sample - loss: 0.0051 - val_loss: 0.0458\n",
      "Epoch 206/500\n",
      "2398/2398 [==============================] - 0s 65us/sample - loss: 0.0056 - val_loss: 0.0459\n",
      "Epoch 207/500\n",
      "2398/2398 [==============================] - 0s 62us/sample - loss: 0.0061 - val_loss: 0.0426\n",
      "Epoch 208/500\n",
      "2398/2398 [==============================] - 0s 73us/sample - loss: 0.0049 - val_loss: 0.0452\n",
      "Epoch 209/500\n",
      "2398/2398 [==============================] - 0s 59us/sample - loss: 0.0056 - val_loss: 0.0404\n",
      "Epoch 210/500\n",
      "2398/2398 [==============================] - 0s 59us/sample - loss: 0.0057 - val_loss: 0.0389\n",
      "Epoch 211/500\n",
      "2398/2398 [==============================] - 0s 50us/sample - loss: 0.0047 - val_loss: 0.0443\n",
      "Epoch 212/500\n",
      "2398/2398 [==============================] - 0s 55us/sample - loss: 0.0052 - val_loss: 0.0409\n",
      "Epoch 213/500\n",
      "2398/2398 [==============================] - 0s 55us/sample - loss: 0.0046 - val_loss: 0.0422\n",
      "Epoch 214/500\n",
      "2398/2398 [==============================] - 0s 57us/sample - loss: 0.0046 - val_loss: 0.0428\n",
      "Epoch 215/500\n",
      "2398/2398 [==============================] - 0s 81us/sample - loss: 0.0047 - val_loss: 0.0439\n",
      "Epoch 216/500\n",
      "2398/2398 [==============================] - 0s 58us/sample - loss: 0.0042 - val_loss: 0.0422\n",
      "Epoch 217/500\n",
      "2398/2398 [==============================] - 0s 55us/sample - loss: 0.0043 - val_loss: 0.0443\n",
      "Epoch 218/500\n",
      "2398/2398 [==============================] - 0s 88us/sample - loss: 0.0045 - val_loss: 0.0426\n",
      "Epoch 219/500\n",
      "2398/2398 [==============================] - 0s 96us/sample - loss: 0.0044 - val_loss: 0.0423\n",
      "Epoch 220/500\n",
      "2398/2398 [==============================] - 0s 62us/sample - loss: 0.0042 - val_loss: 0.0434\n",
      "Epoch 221/500\n",
      "2398/2398 [==============================] - 0s 58us/sample - loss: 0.0040 - val_loss: 0.0425\n",
      "Epoch 222/500\n",
      "2398/2398 [==============================] - 0s 53us/sample - loss: 0.0042 - val_loss: 0.0452\n",
      "Epoch 223/500\n",
      "2398/2398 [==============================] - 0s 65us/sample - loss: 0.0044 - val_loss: 0.0436\n",
      "Epoch 224/500\n",
      "2398/2398 [==============================] - 0s 59us/sample - loss: 0.0045 - val_loss: 0.0542\n",
      "Epoch 225/500\n",
      "2398/2398 [==============================] - 0s 71us/sample - loss: 0.0087 - val_loss: 0.0438\n",
      "Epoch 226/500\n",
      "2398/2398 [==============================] - 0s 67us/sample - loss: 0.0052 - val_loss: 0.0423\n",
      "Epoch 227/500\n",
      "2398/2398 [==============================] - 0s 61us/sample - loss: 0.0048 - val_loss: 0.0525\n",
      "Epoch 228/500\n",
      "2398/2398 [==============================] - 0s 70us/sample - loss: 0.0048 - val_loss: 0.0487\n",
      "Epoch 229/500\n",
      "2398/2398 [==============================] - 0s 69us/sample - loss: 0.0036 - val_loss: 0.0508\n",
      "Epoch 230/500\n",
      "2398/2398 [==============================] - 0s 129us/sample - loss: 0.0039 - val_loss: 0.0478\n",
      "Epoch 231/500\n",
      "2398/2398 [==============================] - 0s 73us/sample - loss: 0.0039 - val_loss: 0.0489\n",
      "Epoch 232/500\n",
      "2398/2398 [==============================] - 0s 63us/sample - loss: 0.0033 - val_loss: 0.0488\n",
      "Epoch 233/500\n",
      "2398/2398 [==============================] - 0s 79us/sample - loss: 0.0034 - val_loss: 0.0486\n",
      "Epoch 234/500\n",
      "2398/2398 [==============================] - 0s 66us/sample - loss: 0.0041 - val_loss: 0.0488\n",
      "Epoch 235/500\n",
      "2398/2398 [==============================] - 0s 65us/sample - loss: 0.0037 - val_loss: 0.0471\n",
      "Epoch 236/500\n",
      "2398/2398 [==============================] - 0s 68us/sample - loss: 0.0033 - val_loss: 0.0499\n",
      "Epoch 237/500\n",
      "2398/2398 [==============================] - 0s 65us/sample - loss: 0.0034 - val_loss: 0.0483\n",
      "Epoch 238/500\n",
      "2398/2398 [==============================] - 0s 64us/sample - loss: 0.0037 - val_loss: 0.0513\n",
      "Epoch 239/500\n",
      "2398/2398 [==============================] - 0s 69us/sample - loss: 0.0034 - val_loss: 0.0471\n",
      "Epoch 240/500\n",
      "2398/2398 [==============================] - 0s 60us/sample - loss: 0.0032 - val_loss: 0.0504\n",
      "Epoch 241/500\n",
      "2398/2398 [==============================] - 0s 68us/sample - loss: 0.0033 - val_loss: 0.0515\n",
      "Epoch 242/500\n",
      "2398/2398 [==============================] - 0s 64us/sample - loss: 0.0030 - val_loss: 0.0490\n",
      "Epoch 243/500\n",
      "2398/2398 [==============================] - 0s 66us/sample - loss: 0.0029 - val_loss: 0.0500\n",
      "Epoch 244/500\n",
      "2398/2398 [==============================] - 0s 66us/sample - loss: 0.0027 - val_loss: 0.0532\n",
      "Epoch 245/500\n",
      "2398/2398 [==============================] - 0s 66us/sample - loss: 0.0031 - val_loss: 0.0482\n",
      "Epoch 246/500\n",
      "2398/2398 [==============================] - 0s 64us/sample - loss: 0.0029 - val_loss: 0.0511\n",
      "Epoch 247/500\n",
      "2398/2398 [==============================] - 0s 63us/sample - loss: 0.0057 - val_loss: 0.0440\n",
      "Epoch 248/500\n",
      "2398/2398 [==============================] - 0s 63us/sample - loss: 0.0029 - val_loss: 0.0511\n",
      "Epoch 249/500\n",
      "2398/2398 [==============================] - 0s 59us/sample - loss: 0.0029 - val_loss: 0.0481\n",
      "Epoch 250/500\n",
      "2398/2398 [==============================] - 0s 85us/sample - loss: 0.0029 - val_loss: 0.0495\n",
      "Epoch 251/500\n",
      "2398/2398 [==============================] - 0s 67us/sample - loss: 0.0024 - val_loss: 0.0498\n",
      "Epoch 252/500\n",
      "2398/2398 [==============================] - 0s 71us/sample - loss: 0.0028 - val_loss: 0.0498\n",
      "Epoch 253/500\n",
      "2398/2398 [==============================] - 0s 63us/sample - loss: 0.0028 - val_loss: 0.0516\n",
      "Epoch 254/500\n",
      "2398/2398 [==============================] - 0s 56us/sample - loss: 0.0024 - val_loss: 0.0519\n",
      "Epoch 255/500\n",
      "2398/2398 [==============================] - 0s 63us/sample - loss: 0.0026 - val_loss: 0.0518\n",
      "Epoch 256/500\n",
      "2398/2398 [==============================] - 0s 59us/sample - loss: 0.0024 - val_loss: 0.0529\n",
      "Epoch 257/500\n",
      "2398/2398 [==============================] - 0s 72us/sample - loss: 0.0022 - val_loss: 0.0525\n",
      "Epoch 258/500\n",
      "2398/2398 [==============================] - 0s 80us/sample - loss: 0.0028 - val_loss: 0.0518\n",
      "Epoch 259/500\n",
      "2398/2398 [==============================] - 0s 63us/sample - loss: 0.0034 - val_loss: 0.0519\n",
      "Epoch 260/500\n",
      "2398/2398 [==============================] - 0s 66us/sample - loss: 0.0026 - val_loss: 0.0533\n",
      "Epoch 261/500\n",
      "2398/2398 [==============================] - 0s 68us/sample - loss: 0.0023 - val_loss: 0.0538\n",
      "Epoch 262/500\n",
      "2398/2398 [==============================] - 0s 62us/sample - loss: 0.0022 - val_loss: 0.0551\n",
      "Epoch 263/500\n",
      "2398/2398 [==============================] - 0s 60us/sample - loss: 0.0027 - val_loss: 0.0499\n",
      "Epoch 264/500\n",
      "2398/2398 [==============================] - 0s 63us/sample - loss: 0.0023 - val_loss: 0.0555\n",
      "Epoch 265/500\n",
      "2398/2398 [==============================] - 0s 73us/sample - loss: 0.0023 - val_loss: 0.0541\n",
      "Epoch 266/500\n",
      "2398/2398 [==============================] - 0s 64us/sample - loss: 0.0032 - val_loss: 0.0451\n",
      "Epoch 267/500\n",
      "2398/2398 [==============================] - 0s 69us/sample - loss: 0.0029 - val_loss: 0.0535\n",
      "Epoch 268/500\n",
      "2398/2398 [==============================] - 0s 68us/sample - loss: 0.0018 - val_loss: 0.0545\n",
      "Epoch 269/500\n",
      "2398/2398 [==============================] - 0s 64us/sample - loss: 0.0020 - val_loss: 0.0515\n",
      "Epoch 270/500\n",
      "2398/2398 [==============================] - 0s 61us/sample - loss: 0.0017 - val_loss: 0.0537\n",
      "Epoch 271/500\n",
      "2398/2398 [==============================] - 0s 72us/sample - loss: 0.0026 - val_loss: 0.0546\n",
      "Epoch 272/500\n",
      "2398/2398 [==============================] - 0s 61us/sample - loss: 0.0018 - val_loss: 0.0553\n",
      "Epoch 273/500\n",
      "2398/2398 [==============================] - 0s 58us/sample - loss: 0.0019 - val_loss: 0.0563\n",
      "Epoch 274/500\n",
      "2398/2398 [==============================] - 0s 57us/sample - loss: 0.0021 - val_loss: 0.0568\n",
      "Epoch 275/500\n",
      "2398/2398 [==============================] - 0s 54us/sample - loss: 0.0018 - val_loss: 0.0567\n",
      "Epoch 276/500\n",
      "2398/2398 [==============================] - 0s 62us/sample - loss: 0.0017 - val_loss: 0.0529\n",
      "Epoch 277/500\n",
      "2398/2398 [==============================] - 0s 65us/sample - loss: 0.0018 - val_loss: 0.0537\n",
      "Epoch 278/500\n",
      "2398/2398 [==============================] - 0s 78us/sample - loss: 0.0019 - val_loss: 0.0605\n",
      "Epoch 279/500\n",
      "2398/2398 [==============================] - 0s 58us/sample - loss: 0.0086 - val_loss: 0.0740\n",
      "Epoch 280/500\n",
      "2398/2398 [==============================] - 0s 68us/sample - loss: 0.0064 - val_loss: 0.0587\n",
      "Epoch 281/500\n",
      "2398/2398 [==============================] - 0s 65us/sample - loss: 0.0019 - val_loss: 0.0623\n",
      "Epoch 282/500\n",
      "2398/2398 [==============================] - 0s 59us/sample - loss: 0.0018 - val_loss: 0.0617\n",
      "Epoch 283/500\n",
      "2398/2398 [==============================] - 0s 67us/sample - loss: 0.0017 - val_loss: 0.0608\n",
      "Epoch 284/500\n",
      "2398/2398 [==============================] - 0s 72us/sample - loss: 0.0016 - val_loss: 0.0645\n",
      "Epoch 285/500\n",
      "2398/2398 [==============================] - 0s 54us/sample - loss: 0.0018 - val_loss: 0.0594\n",
      "Epoch 286/500\n",
      "2398/2398 [==============================] - 0s 56us/sample - loss: 0.0014 - val_loss: 0.0587\n",
      "Epoch 287/500\n",
      "2398/2398 [==============================] - 0s 55us/sample - loss: 0.0017 - val_loss: 0.0596\n",
      "Epoch 288/500\n",
      "2398/2398 [==============================] - 0s 69us/sample - loss: 0.0015 - val_loss: 0.0600\n",
      "Epoch 289/500\n",
      "2398/2398 [==============================] - 0s 65us/sample - loss: 0.0013 - val_loss: 0.0597\n",
      "Epoch 290/500\n",
      "2398/2398 [==============================] - 0s 62us/sample - loss: 0.0014 - val_loss: 0.0613\n",
      "Epoch 291/500\n",
      "2398/2398 [==============================] - 0s 58us/sample - loss: 0.0015 - val_loss: 0.0592\n",
      "Epoch 292/500\n",
      "2398/2398 [==============================] - 0s 65us/sample - loss: 0.0012 - val_loss: 0.0613\n",
      "Epoch 293/500\n",
      "2398/2398 [==============================] - 0s 78us/sample - loss: 0.0014 - val_loss: 0.0589\n",
      "Epoch 294/500\n",
      "2398/2398 [==============================] - 0s 82us/sample - loss: 0.0015 - val_loss: 0.0586\n",
      "Epoch 295/500\n",
      "2398/2398 [==============================] - 0s 55us/sample - loss: 0.0013 - val_loss: 0.0615\n",
      "Epoch 296/500\n",
      "2398/2398 [==============================] - 0s 74us/sample - loss: 0.0018 - val_loss: 0.0610\n",
      "Epoch 297/500\n",
      "2398/2398 [==============================] - 0s 59us/sample - loss: 0.0013 - val_loss: 0.0625\n",
      "Epoch 298/500\n",
      "2398/2398 [==============================] - 0s 64us/sample - loss: 0.0012 - val_loss: 0.0643\n",
      "Epoch 299/500\n",
      "2398/2398 [==============================] - 0s 54us/sample - loss: 0.0020 - val_loss: 0.0605\n",
      "Epoch 300/500\n",
      "2398/2398 [==============================] - 0s 60us/sample - loss: 0.0017 - val_loss: 0.0594\n",
      "Epoch 301/500\n",
      "2398/2398 [==============================] - 0s 70us/sample - loss: 0.0024 - val_loss: 0.0638\n",
      "Epoch 302/500\n",
      "2398/2398 [==============================] - 0s 57us/sample - loss: 0.0011 - val_loss: 0.0657\n",
      "Epoch 303/500\n",
      "2398/2398 [==============================] - 0s 57us/sample - loss: 0.0011 - val_loss: 0.0617\n",
      "Epoch 304/500\n",
      "2398/2398 [==============================] - 0s 53us/sample - loss: 0.0013 - val_loss: 0.0635\n",
      "Epoch 305/500\n",
      "2398/2398 [==============================] - 0s 59us/sample - loss: 0.0011 - val_loss: 0.0604\n",
      "Epoch 306/500\n",
      "2398/2398 [==============================] - 0s 57us/sample - loss: 9.9811e-04 - val_loss: 0.0612\n",
      "Epoch 307/500\n",
      "2398/2398 [==============================] - 0s 57us/sample - loss: 0.0011 - val_loss: 0.0606\n",
      "Epoch 308/500\n",
      "2398/2398 [==============================] - 0s 67us/sample - loss: 9.1690e-04 - val_loss: 0.0638\n",
      "Epoch 309/500\n",
      "2398/2398 [==============================] - 0s 55us/sample - loss: 0.0012 - val_loss: 0.0671\n",
      "Epoch 310/500\n",
      "2398/2398 [==============================] - 0s 59us/sample - loss: 0.0011 - val_loss: 0.0663\n",
      "Epoch 311/500\n",
      "2398/2398 [==============================] - 0s 59us/sample - loss: 8.7178e-04 - val_loss: 0.0662\n",
      "Epoch 312/500\n",
      "2398/2398 [==============================] - 0s 56us/sample - loss: 0.0012 - val_loss: 0.0644\n",
      "Epoch 313/500\n",
      "2398/2398 [==============================] - 0s 53us/sample - loss: 0.0019 - val_loss: 0.0629\n",
      "Epoch 314/500\n",
      "2398/2398 [==============================] - 0s 63us/sample - loss: 9.9002e-04 - val_loss: 0.0662\n",
      "Epoch 315/500\n",
      "2398/2398 [==============================] - 0s 53us/sample - loss: 7.7857e-04 - val_loss: 0.0674\n",
      "Epoch 316/500\n",
      "2398/2398 [==============================] - 0s 50us/sample - loss: 0.0010 - val_loss: 0.0668\n",
      "Epoch 317/500\n",
      "2398/2398 [==============================] - 0s 53us/sample - loss: 9.4531e-04 - val_loss: 0.0667\n",
      "Epoch 318/500\n",
      "2398/2398 [==============================] - 0s 54us/sample - loss: 9.7070e-04 - val_loss: 0.0669\n",
      "Epoch 319/500\n",
      "2398/2398 [==============================] - 0s 56us/sample - loss: 8.8912e-04 - val_loss: 0.0668\n",
      "Epoch 320/500\n",
      "2398/2398 [==============================] - 0s 54us/sample - loss: 9.2587e-04 - val_loss: 0.0685\n",
      "Epoch 321/500\n",
      "2398/2398 [==============================] - 0s 53us/sample - loss: 0.0010 - val_loss: 0.0668\n",
      "Epoch 322/500\n",
      "2398/2398 [==============================] - 0s 54us/sample - loss: 0.0011 - val_loss: 0.0712\n",
      "Epoch 323/500\n",
      "2398/2398 [==============================] - 0s 51us/sample - loss: 0.0030 - val_loss: 0.0839\n",
      "Epoch 324/500\n",
      "2398/2398 [==============================] - 0s 68us/sample - loss: 0.0089 - val_loss: 0.0531\n",
      "Epoch 325/500\n",
      "2398/2398 [==============================] - 0s 52us/sample - loss: 0.0022 - val_loss: 0.0534\n",
      "Epoch 326/500\n",
      "2398/2398 [==============================] - 0s 56us/sample - loss: 0.0012 - val_loss: 0.0562\n",
      "Epoch 327/500\n",
      "2398/2398 [==============================] - 0s 58us/sample - loss: 8.1898e-04 - val_loss: 0.0575\n",
      "Epoch 328/500\n",
      "2398/2398 [==============================] - 0s 51us/sample - loss: 8.1884e-04 - val_loss: 0.0587\n",
      "Epoch 329/500\n",
      "2398/2398 [==============================] - 0s 52us/sample - loss: 7.8875e-04 - val_loss: 0.0602\n",
      "Epoch 330/500\n",
      "2398/2398 [==============================] - 0s 51us/sample - loss: 7.6719e-04 - val_loss: 0.0597\n",
      "Epoch 331/500\n",
      "2398/2398 [==============================] - 0s 54us/sample - loss: 7.1065e-04 - val_loss: 0.0629\n",
      "Epoch 332/500\n",
      "2398/2398 [==============================] - 0s 51us/sample - loss: 6.8110e-04 - val_loss: 0.0619\n",
      "Epoch 333/500\n",
      "2398/2398 [==============================] - 0s 50us/sample - loss: 7.0721e-04 - val_loss: 0.0631\n",
      "Epoch 334/500\n",
      "2398/2398 [==============================] - 0s 53us/sample - loss: 6.2826e-04 - val_loss: 0.0640\n",
      "Epoch 335/500\n",
      "2398/2398 [==============================] - 0s 63us/sample - loss: 7.0860e-04 - val_loss: 0.0629\n",
      "Epoch 336/500\n",
      "2398/2398 [==============================] - 0s 63us/sample - loss: 6.3585e-04 - val_loss: 0.0630\n",
      "Epoch 337/500\n",
      "2398/2398 [==============================] - 0s 70us/sample - loss: 6.9561e-04 - val_loss: 0.0646\n",
      "Epoch 338/500\n",
      "2398/2398 [==============================] - 0s 62us/sample - loss: 6.7098e-04 - val_loss: 0.0652\n",
      "Epoch 339/500\n",
      "2398/2398 [==============================] - 0s 52us/sample - loss: 6.1275e-04 - val_loss: 0.0644\n",
      "Epoch 340/500\n",
      "2398/2398 [==============================] - 0s 52us/sample - loss: 5.5765e-04 - val_loss: 0.0639\n",
      "Epoch 341/500\n",
      "2398/2398 [==============================] - 0s 54us/sample - loss: 7.8856e-04 - val_loss: 0.0652\n",
      "Epoch 342/500\n",
      "2398/2398 [==============================] - 0s 52us/sample - loss: 6.6041e-04 - val_loss: 0.0672\n",
      "Epoch 343/500\n",
      "2398/2398 [==============================] - 0s 52us/sample - loss: 6.6010e-04 - val_loss: 0.0700\n",
      "Epoch 344/500\n",
      "2398/2398 [==============================] - 0s 52us/sample - loss: 6.0051e-04 - val_loss: 0.0687\n",
      "Epoch 345/500\n",
      "2398/2398 [==============================] - 0s 60us/sample - loss: 6.7944e-04 - val_loss: 0.0669\n",
      "Epoch 346/500\n",
      "2398/2398 [==============================] - 0s 53us/sample - loss: 5.7388e-04 - val_loss: 0.0690\n",
      "Epoch 347/500\n",
      "2398/2398 [==============================] - 0s 54us/sample - loss: 4.9813e-04 - val_loss: 0.0685\n",
      "Epoch 348/500\n",
      "2398/2398 [==============================] - 0s 61us/sample - loss: 5.5164e-04 - val_loss: 0.0661\n",
      "Epoch 349/500\n",
      "2398/2398 [==============================] - 0s 65us/sample - loss: 5.9327e-04 - val_loss: 0.0681\n",
      "Epoch 350/500\n",
      "2398/2398 [==============================] - 0s 56us/sample - loss: 5.2653e-04 - val_loss: 0.0711\n",
      "Epoch 351/500\n",
      "2398/2398 [==============================] - 0s 54us/sample - loss: 6.6667e-04 - val_loss: 0.0679\n",
      "Epoch 352/500\n",
      "2398/2398 [==============================] - 0s 55us/sample - loss: 4.9917e-04 - val_loss: 0.0702\n",
      "Epoch 353/500\n",
      "2398/2398 [==============================] - 0s 72us/sample - loss: 4.9106e-04 - val_loss: 0.0691\n",
      "Epoch 354/500\n",
      "2398/2398 [==============================] - 0s 52us/sample - loss: 4.1983e-04 - val_loss: 0.0692\n",
      "Epoch 355/500\n",
      "2398/2398 [==============================] - 0s 59us/sample - loss: 4.6166e-04 - val_loss: 0.0691\n",
      "Epoch 356/500\n",
      "2398/2398 [==============================] - 0s 59us/sample - loss: 5.8620e-04 - val_loss: 0.0706\n",
      "Epoch 357/500\n",
      "2398/2398 [==============================] - 0s 61us/sample - loss: 3.8669e-04 - val_loss: 0.0726\n",
      "Epoch 358/500\n",
      "2398/2398 [==============================] - 0s 72us/sample - loss: 6.1172e-04 - val_loss: 0.0712\n",
      "Epoch 359/500\n",
      "2398/2398 [==============================] - 0s 71us/sample - loss: 3.9147e-04 - val_loss: 0.0726\n",
      "Epoch 360/500\n",
      "2398/2398 [==============================] - 0s 68us/sample - loss: 4.5677e-04 - val_loss: 0.0712\n",
      "Epoch 361/500\n",
      "2398/2398 [==============================] - 0s 61us/sample - loss: 3.9627e-04 - val_loss: 0.0734\n",
      "Epoch 362/500\n",
      "2398/2398 [==============================] - 0s 51us/sample - loss: 4.8855e-04 - val_loss: 0.0713\n",
      "Epoch 363/500\n",
      "2398/2398 [==============================] - 0s 51us/sample - loss: 4.6296e-04 - val_loss: 0.0720\n",
      "Epoch 364/500\n",
      "2398/2398 [==============================] - 1s 223us/sample - loss: 4.4421e-04 - val_loss: 0.0726\n",
      "Epoch 365/500\n",
      "2398/2398 [==============================] - 0s 194us/sample - loss: 3.3279e-04 - val_loss: 0.0756\n",
      "Epoch 366/500\n",
      "2398/2398 [==============================] - 0s 120us/sample - loss: 4.8845e-04 - val_loss: 0.0723\n",
      "Epoch 367/500\n",
      "2398/2398 [==============================] - 0s 141us/sample - loss: 7.2468e-04 - val_loss: 0.0701\n",
      "Epoch 368/500\n",
      "2398/2398 [==============================] - 0s 97us/sample - loss: 4.9499e-04 - val_loss: 0.0727\n",
      "Epoch 369/500\n",
      "2398/2398 [==============================] - 0s 96us/sample - loss: 3.1480e-04 - val_loss: 0.0726\n",
      "Epoch 370/500\n",
      "2398/2398 [==============================] - 0s 124us/sample - loss: 2.9155e-04 - val_loss: 0.0751\n",
      "Epoch 371/500\n",
      "2398/2398 [==============================] - 0s 146us/sample - loss: 3.6290e-04 - val_loss: 0.0742\n",
      "Epoch 372/500\n",
      "2398/2398 [==============================] - 0s 64us/sample - loss: 0.0015 - val_loss: 0.1660\n",
      "Epoch 373/500\n",
      "2398/2398 [==============================] - 0s 53us/sample - loss: 0.0238 - val_loss: 0.0970\n",
      "Epoch 374/500\n",
      "2398/2398 [==============================] - 0s 51us/sample - loss: 0.0028 - val_loss: 0.0694\n",
      "Epoch 375/500\n",
      "2398/2398 [==============================] - 0s 53us/sample - loss: 6.5443e-04 - val_loss: 0.0702\n",
      "Epoch 376/500\n",
      "2398/2398 [==============================] - 0s 52us/sample - loss: 5.1085e-04 - val_loss: 0.0718\n",
      "Epoch 377/500\n",
      "2398/2398 [==============================] - 0s 54us/sample - loss: 4.9340e-04 - val_loss: 0.0727\n",
      "Epoch 378/500\n",
      "2398/2398 [==============================] - 0s 53us/sample - loss: 4.2970e-04 - val_loss: 0.0725\n",
      "Epoch 379/500\n",
      "2398/2398 [==============================] - 0s 66us/sample - loss: 4.0590e-04 - val_loss: 0.0738\n",
      "Epoch 380/500\n",
      "2398/2398 [==============================] - 0s 54us/sample - loss: 3.7044e-04 - val_loss: 0.0746\n",
      "Epoch 381/500\n",
      "2398/2398 [==============================] - 0s 54us/sample - loss: 3.8545e-04 - val_loss: 0.0741\n",
      "Epoch 382/500\n",
      "2398/2398 [==============================] - 0s 61us/sample - loss: 3.5069e-04 - val_loss: 0.0742\n",
      "Epoch 383/500\n",
      "2398/2398 [==============================] - 0s 50us/sample - loss: 3.4958e-04 - val_loss: 0.0753\n",
      "Epoch 384/500\n",
      "2398/2398 [==============================] - 0s 63us/sample - loss: 2.9588e-04 - val_loss: 0.0745\n",
      "Epoch 385/500\n",
      "2398/2398 [==============================] - 0s 53us/sample - loss: 2.8778e-04 - val_loss: 0.0756\n",
      "Epoch 386/500\n",
      "2398/2398 [==============================] - 0s 53us/sample - loss: 2.9745e-04 - val_loss: 0.0757\n",
      "Epoch 387/500\n",
      "2398/2398 [==============================] - 0s 108us/sample - loss: 2.9443e-04 - val_loss: 0.0759\n",
      "Epoch 388/500\n",
      "2398/2398 [==============================] - 0s 52us/sample - loss: 3.0773e-04 - val_loss: 0.0751\n",
      "Epoch 389/500\n",
      "2398/2398 [==============================] - 0s 53us/sample - loss: 2.7722e-04 - val_loss: 0.0743\n",
      "Epoch 390/500\n",
      "2398/2398 [==============================] - 0s 53us/sample - loss: 3.2315e-04 - val_loss: 0.0759\n",
      "Epoch 391/500\n",
      "2398/2398 [==============================] - 0s 67us/sample - loss: 2.5998e-04 - val_loss: 0.0761\n",
      "Epoch 392/500\n",
      "2398/2398 [==============================] - 0s 52us/sample - loss: 2.9076e-04 - val_loss: 0.0748\n",
      "Epoch 393/500\n",
      "2398/2398 [==============================] - 0s 50us/sample - loss: 3.2259e-04 - val_loss: 0.0749\n",
      "Epoch 394/500\n",
      "2398/2398 [==============================] - 0s 53us/sample - loss: 2.6228e-04 - val_loss: 0.0755\n",
      "Epoch 395/500\n",
      "2398/2398 [==============================] - 0s 54us/sample - loss: 2.8522e-04 - val_loss: 0.0753\n",
      "Epoch 396/500\n",
      "2398/2398 [==============================] - 0s 52us/sample - loss: 2.8975e-04 - val_loss: 0.0747\n",
      "Epoch 397/500\n",
      "2398/2398 [==============================] - 0s 52us/sample - loss: 2.8895e-04 - val_loss: 0.0760\n",
      "Epoch 398/500\n",
      "2398/2398 [==============================] - 0s 50us/sample - loss: 2.4340e-04 - val_loss: 0.0744\n",
      "Epoch 399/500\n",
      "2398/2398 [==============================] - 0s 60us/sample - loss: 2.2633e-04 - val_loss: 0.0769\n",
      "Epoch 400/500\n",
      "2398/2398 [==============================] - 0s 51us/sample - loss: 2.5875e-04 - val_loss: 0.0750\n",
      "Epoch 401/500\n",
      "2398/2398 [==============================] - 0s 50us/sample - loss: 2.5175e-04 - val_loss: 0.0758\n",
      "Epoch 402/500\n",
      "2398/2398 [==============================] - 0s 59us/sample - loss: 2.5735e-04 - val_loss: 0.0762\n",
      "Epoch 403/500\n",
      "2398/2398 [==============================] - 0s 49us/sample - loss: 2.8248e-04 - val_loss: 0.0754\n",
      "Epoch 404/500\n",
      "2398/2398 [==============================] - 0s 51us/sample - loss: 2.4801e-04 - val_loss: 0.0750\n",
      "Epoch 405/500\n",
      "2398/2398 [==============================] - 0s 54us/sample - loss: 2.3159e-04 - val_loss: 0.0750\n",
      "Epoch 406/500\n",
      "2398/2398 [==============================] - 0s 53us/sample - loss: 2.4450e-04 - val_loss: 0.0763\n",
      "Epoch 407/500\n",
      "2398/2398 [==============================] - 0s 50us/sample - loss: 1.9924e-04 - val_loss: 0.0780\n",
      "Epoch 408/500\n",
      "2398/2398 [==============================] - 0s 53us/sample - loss: 2.5219e-04 - val_loss: 0.0754\n",
      "Epoch 409/500\n",
      "2398/2398 [==============================] - 0s 54us/sample - loss: 2.1705e-04 - val_loss: 0.0775\n",
      "Epoch 410/500\n",
      "2398/2398 [==============================] - 0s 54us/sample - loss: 2.9532e-04 - val_loss: 0.0767\n",
      "Epoch 411/500\n",
      "2398/2398 [==============================] - 0s 51us/sample - loss: 2.3793e-04 - val_loss: 0.0770\n",
      "Epoch 412/500\n",
      "2398/2398 [==============================] - 0s 48us/sample - loss: 2.0528e-04 - val_loss: 0.0792\n",
      "Epoch 413/500\n",
      "2398/2398 [==============================] - 0s 49us/sample - loss: 4.4938e-04 - val_loss: 0.0768\n",
      "Epoch 414/500\n",
      "2398/2398 [==============================] - 0s 48us/sample - loss: 2.9247e-04 - val_loss: 0.0777\n",
      "Epoch 415/500\n",
      "2398/2398 [==============================] - 0s 55us/sample - loss: 2.5408e-04 - val_loss: 0.0767\n",
      "Epoch 416/500\n",
      "2398/2398 [==============================] - 0s 52us/sample - loss: 2.3295e-04 - val_loss: 0.0780\n",
      "Epoch 417/500\n",
      "2398/2398 [==============================] - 0s 53us/sample - loss: 1.9149e-04 - val_loss: 0.0779\n",
      "Epoch 418/500\n",
      "2398/2398 [==============================] - 0s 155us/sample - loss: 1.9037e-04 - val_loss: 0.0778\n",
      "Epoch 419/500\n",
      "2398/2398 [==============================] - 0s 106us/sample - loss: 1.5279e-04 - val_loss: 0.0800\n",
      "Epoch 420/500\n",
      "2398/2398 [==============================] - 0s 94us/sample - loss: 2.7955e-04 - val_loss: 0.0758\n",
      "Epoch 421/500\n",
      "2398/2398 [==============================] - 0s 114us/sample - loss: 2.6277e-04 - val_loss: 0.0761\n",
      "Epoch 422/500\n",
      "2398/2398 [==============================] - 0s 81us/sample - loss: 2.7713e-04 - val_loss: 0.0746\n",
      "Epoch 423/500\n",
      "2398/2398 [==============================] - 0s 77us/sample - loss: 1.9592e-04 - val_loss: 0.0788\n",
      "Epoch 424/500\n",
      "2398/2398 [==============================] - 0s 142us/sample - loss: 1.9078e-04 - val_loss: 0.0786\n",
      "Epoch 425/500\n",
      "2398/2398 [==============================] - 0s 95us/sample - loss: 2.1844e-04 - val_loss: 0.0782\n",
      "Epoch 426/500\n",
      "2398/2398 [==============================] - 0s 96us/sample - loss: 2.0229e-04 - val_loss: 0.0768\n",
      "Epoch 427/500\n",
      "2398/2398 [==============================] - 0s 57us/sample - loss: 2.5354e-04 - val_loss: 0.0786\n",
      "Epoch 428/500\n",
      "2398/2398 [==============================] - 0s 57us/sample - loss: 1.8388e-04 - val_loss: 0.0797\n",
      "Epoch 429/500\n",
      "2398/2398 [==============================] - 0s 59us/sample - loss: 2.3731e-04 - val_loss: 0.0785\n",
      "Epoch 430/500\n",
      "2398/2398 [==============================] - 0s 65us/sample - loss: 2.3230e-04 - val_loss: 0.0786\n",
      "Epoch 431/500\n",
      "2398/2398 [==============================] - 0s 52us/sample - loss: 2.0223e-04 - val_loss: 0.0789\n",
      "Epoch 432/500\n",
      "2398/2398 [==============================] - 0s 52us/sample - loss: 2.3837e-04 - val_loss: 0.0787\n",
      "Epoch 433/500\n",
      "2398/2398 [==============================] - 0s 55us/sample - loss: 1.5382e-04 - val_loss: 0.0796\n",
      "Epoch 434/500\n",
      "2398/2398 [==============================] - 0s 52us/sample - loss: 1.7096e-04 - val_loss: 0.0780\n",
      "Epoch 435/500\n",
      "2398/2398 [==============================] - 0s 51us/sample - loss: 1.3523e-04 - val_loss: 0.0825\n",
      "Epoch 436/500\n",
      "2398/2398 [==============================] - 0s 52us/sample - loss: 1.7076e-04 - val_loss: 0.0819\n",
      "Epoch 437/500\n",
      "2398/2398 [==============================] - 0s 64us/sample - loss: 1.4787e-04 - val_loss: 0.0815\n",
      "Epoch 438/500\n",
      "2398/2398 [==============================] - 0s 53us/sample - loss: 0.0139 - val_loss: 0.1649\n",
      "Epoch 439/500\n",
      "2398/2398 [==============================] - 0s 55us/sample - loss: 0.0048 - val_loss: 0.0837\n",
      "Epoch 440/500\n",
      "2398/2398 [==============================] - 0s 112us/sample - loss: 9.4442e-04 - val_loss: 0.0834\n",
      "Epoch 441/500\n",
      "2398/2398 [==============================] - 0s 79us/sample - loss: 2.5702e-04 - val_loss: 0.0820\n",
      "Epoch 442/500\n",
      "2398/2398 [==============================] - 0s 65us/sample - loss: 2.1876e-04 - val_loss: 0.0825\n",
      "Epoch 443/500\n",
      "2398/2398 [==============================] - 0s 75us/sample - loss: 1.9820e-04 - val_loss: 0.0826\n",
      "Epoch 444/500\n",
      "2398/2398 [==============================] - 0s 58us/sample - loss: 1.9788e-04 - val_loss: 0.0827\n",
      "Epoch 445/500\n",
      "2398/2398 [==============================] - 0s 53us/sample - loss: 1.7465e-04 - val_loss: 0.0833\n",
      "Epoch 446/500\n",
      "2398/2398 [==============================] - 0s 57us/sample - loss: 1.6719e-04 - val_loss: 0.0831\n",
      "Epoch 447/500\n",
      "2398/2398 [==============================] - 0s 51us/sample - loss: 1.6498e-04 - val_loss: 0.0833\n",
      "Epoch 448/500\n",
      "2398/2398 [==============================] - 0s 64us/sample - loss: 1.6550e-04 - val_loss: 0.0834\n",
      "Epoch 449/500\n",
      "2398/2398 [==============================] - 0s 73us/sample - loss: 1.5496e-04 - val_loss: 0.0835\n",
      "Epoch 450/500\n",
      "2398/2398 [==============================] - 0s 59us/sample - loss: 1.5583e-04 - val_loss: 0.0835\n",
      "Epoch 451/500\n",
      "2398/2398 [==============================] - 0s 55us/sample - loss: 1.5960e-04 - val_loss: 0.0837\n",
      "Epoch 452/500\n",
      "2398/2398 [==============================] - 0s 77us/sample - loss: 1.4318e-04 - val_loss: 0.0838\n",
      "Epoch 453/500\n",
      "2398/2398 [==============================] - 0s 66us/sample - loss: 1.3630e-04 - val_loss: 0.0843\n",
      "Epoch 454/500\n",
      "2398/2398 [==============================] - 0s 69us/sample - loss: 1.3300e-04 - val_loss: 0.0844\n",
      "Epoch 455/500\n",
      "2398/2398 [==============================] - 0s 70us/sample - loss: 1.5560e-04 - val_loss: 0.0841\n",
      "Epoch 456/500\n",
      "2398/2398 [==============================] - 0s 70us/sample - loss: 1.3752e-04 - val_loss: 0.0848\n",
      "Epoch 457/500\n",
      "2398/2398 [==============================] - 0s 67us/sample - loss: 1.2853e-04 - val_loss: 0.0846\n",
      "Epoch 458/500\n",
      "2398/2398 [==============================] - 0s 60us/sample - loss: 1.2635e-04 - val_loss: 0.0845\n",
      "Epoch 459/500\n",
      "2398/2398 [==============================] - 0s 67us/sample - loss: 1.1578e-04 - val_loss: 0.0854\n",
      "Epoch 460/500\n",
      "2398/2398 [==============================] - 0s 59us/sample - loss: 1.2031e-04 - val_loss: 0.0847\n",
      "Epoch 461/500\n",
      "2398/2398 [==============================] - 0s 60us/sample - loss: 1.1360e-04 - val_loss: 0.0850\n",
      "Epoch 462/500\n",
      "2398/2398 [==============================] - 0s 56us/sample - loss: 1.1678e-04 - val_loss: 0.0849\n",
      "Epoch 463/500\n",
      "2398/2398 [==============================] - 0s 53us/sample - loss: 1.0905e-04 - val_loss: 0.0847\n",
      "Epoch 464/500\n",
      "2398/2398 [==============================] - 0s 52us/sample - loss: 1.0803e-04 - val_loss: 0.0854\n",
      "Epoch 465/500\n",
      "2398/2398 [==============================] - 0s 51us/sample - loss: 1.0524e-04 - val_loss: 0.0848\n",
      "Epoch 466/500\n",
      "2398/2398 [==============================] - 0s 55us/sample - loss: 1.1815e-04 - val_loss: 0.0854\n",
      "Epoch 467/500\n",
      "2398/2398 [==============================] - 0s 54us/sample - loss: 1.0141e-04 - val_loss: 0.0857\n",
      "Epoch 468/500\n",
      "2398/2398 [==============================] - 0s 66us/sample - loss: 1.0794e-04 - val_loss: 0.0855\n",
      "Epoch 469/500\n",
      "2398/2398 [==============================] - 0s 72us/sample - loss: 1.0376e-04 - val_loss: 0.0860\n",
      "Epoch 470/500\n",
      "2398/2398 [==============================] - 0s 60us/sample - loss: 9.2373e-05 - val_loss: 0.0869\n",
      "Epoch 471/500\n",
      "2398/2398 [==============================] - 0s 60us/sample - loss: 1.0075e-04 - val_loss: 0.0866\n",
      "Epoch 472/500\n",
      "2398/2398 [==============================] - 0s 61us/sample - loss: 1.1880e-04 - val_loss: 0.0875\n",
      "Epoch 473/500\n",
      "2398/2398 [==============================] - 0s 72us/sample - loss: 1.1364e-04 - val_loss: 0.08700\n",
      "Epoch 474/500\n",
      "2398/2398 [==============================] - 0s 57us/sample - loss: 9.7314e-05 - val_loss: 0.0866\n",
      "Epoch 475/500\n",
      "2398/2398 [==============================] - 0s 52us/sample - loss: 9.4008e-05 - val_loss: 0.0857\n",
      "Epoch 476/500\n",
      "2398/2398 [==============================] - 0s 53us/sample - loss: 1.0527e-04 - val_loss: 0.0865\n",
      "Epoch 477/500\n",
      "2398/2398 [==============================] - 0s 61us/sample - loss: 9.2868e-05 - val_loss: 0.0870\n",
      "Epoch 478/500\n",
      "2398/2398 [==============================] - 0s 54us/sample - loss: 9.9993e-05 - val_loss: 0.0874\n",
      "Epoch 479/500\n",
      "2398/2398 [==============================] - 0s 56us/sample - loss: 1.0155e-04 - val_loss: 0.0881\n",
      "Epoch 480/500\n",
      "2398/2398 [==============================] - 0s 84us/sample - loss: 1.0485e-04 - val_loss: 0.0873\n",
      "Epoch 481/500\n",
      "2398/2398 [==============================] - 0s 68us/sample - loss: 1.0254e-04 - val_loss: 0.0871\n",
      "Epoch 482/500\n",
      "2398/2398 [==============================] - 0s 63us/sample - loss: 8.1915e-05 - val_loss: 0.0891\n",
      "Epoch 483/500\n",
      "2398/2398 [==============================] - 0s 111us/sample - loss: 1.5693e-04 - val_loss: 0.0888\n",
      "Epoch 484/500\n",
      "2398/2398 [==============================] - 0s 61us/sample - loss: 8.6330e-05 - val_loss: 0.0890\n",
      "Epoch 485/500\n",
      "2398/2398 [==============================] - 0s 64us/sample - loss: 8.3237e-05 - val_loss: 0.0888\n",
      "Epoch 486/500\n",
      "2398/2398 [==============================] - 0s 83us/sample - loss: 7.8451e-05 - val_loss: 0.0891\n",
      "Epoch 487/500\n",
      "2398/2398 [==============================] - 0s 75us/sample - loss: 8.2770e-05 - val_loss: 0.0881\n",
      "Epoch 488/500\n",
      "2398/2398 [==============================] - 0s 68us/sample - loss: 9.4810e-05 - val_loss: 0.0892\n",
      "Epoch 489/500\n",
      "2398/2398 [==============================] - 0s 75us/sample - loss: 9.5240e-05 - val_loss: 0.0893\n",
      "Epoch 490/500\n",
      "2398/2398 [==============================] - 0s 62us/sample - loss: 8.2290e-05 - val_loss: 0.0894\n",
      "Epoch 491/500\n",
      "2398/2398 [==============================] - 0s 53us/sample - loss: 7.9097e-05 - val_loss: 0.0893\n",
      "Epoch 492/500\n",
      "2398/2398 [==============================] - 0s 61us/sample - loss: 7.5005e-05 - val_loss: 0.0896\n",
      "Epoch 493/500\n",
      "2398/2398 [==============================] - 0s 54us/sample - loss: 7.9101e-05 - val_loss: 0.0885\n",
      "Epoch 494/500\n",
      "2398/2398 [==============================] - 0s 58us/sample - loss: 1.1148e-04 - val_loss: 0.0890\n",
      "Epoch 495/500\n",
      "2398/2398 [==============================] - 0s 73us/sample - loss: 9.2554e-05 - val_loss: 0.0887\n",
      "Epoch 496/500\n",
      "2398/2398 [==============================] - 0s 77us/sample - loss: 7.9321e-05 - val_loss: 0.0892\n",
      "Epoch 497/500\n",
      "2398/2398 [==============================] - 0s 55us/sample - loss: 7.1711e-05 - val_loss: 0.0896\n",
      "Epoch 498/500\n",
      "2398/2398 [==============================] - 0s 54us/sample - loss: 7.3217e-05 - val_loss: 0.0883\n",
      "Epoch 499/500\n",
      "2398/2398 [==============================] - 0s 54us/sample - loss: 7.3298e-05 - val_loss: 0.0893\n",
      "Epoch 500/500\n",
      "2398/2398 [==============================] - 0s 58us/sample - loss: 7.1688e-05 - val_loss: 0.0897\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<tensorflow.python.keras.callbacks.History at 0x1a3b209710>"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.fit(x=X_train, y=y_train, \n",
    "          validation_data=(X_test,y_test),\n",
    "          epochs=500)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Check if the model overfits**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x1a3b7bc450>"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD4CAYAAAD8Zh1EAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3deXxcdb3/8ddntixN0jZN93SFlq2FgqGAaFlkVZbrXlREL5f+FFnUK1dwQQT94ZUreP3dXhUVQQWhgl6rVKos3gIidqG0lNJSSpd0Tds0XbJNZr6/P74zyUw6badt0slJ3s/HI2TOmZMz3xPS93znc77fc8w5h4iIBF+o0A0QEZGuoUAXEeklFOgiIr2EAl1EpJdQoIuI9BKRQr1wVVWVGzt2bKFeXkQkkBYuXLjNOTc413MFC/SxY8eyYMGCQr28iEggmdna/T2XV8nFzC4xsxVmtsrMbt3PNh8xs9fNbJmZPXK4jRURkcNz0B66mYWBmcCFQC0w38xmO+dez9hmAnAbcLZzrt7MhnRXg0VEJLd8euhTgVXOudXOuVbgUeDKTttcB8x0ztUDOOe2dm0zRUTkYPKpoY8E1mcs1wJndNpmIoCZvQiEgTucc0913pGZzQBmAIwePfpw2isiARePx6mtraW5ubnQTenRiouLqa6uJhqN5v0z+QS65VjX+QIwEWACcC5QDTxvZpOcczuzfsi5+4H7AWpqanQRGZE+qLa2lvLycsaOHYtZrngR5xzbt2+ntraWcePG5f1z+ZRcaoFRGcvVwMYc2/zeORd3zr0NrMAHvIhIlubmZgYNGqQwPwAzY9CgQYf8KSafQJ8PTDCzcWYWA6YDsztt8z/AeamGVOFLMKsPqSUi0mcozA/ucH5HBw1051wbcAMwF1gOzHLOLTOzO83sitRmc4HtZvY68Bxwi3Nu+yG3Jg/z1+zg3j+voLUt2R27FxEJrLwmFjnn5gBzOq27PeOxA76Y+upWi9bW84NnV/GZc48hpisXiMhhKCsrY8+ePYVuRpcLXCKmP4UkdUpVRCRL4AI9lEp03WlJRI6Uc45bbrmFSZMmMXnyZB577DEANm3axLRp05gyZQqTJk3i+eefJ5FI8KlPfap92/vuu6/Ard9Xwa7lcrjSJwrUQxcJvm/+YRmvb9zVpfs8cUQF37j8pLy2/e1vf8vixYt59dVX2bZtG6effjrTpk3jkUce4eKLL+arX/0qiUSCxsZGFi9ezIYNG3jttdcA2Llz50H2fvQFroeePu+rHrqIHKkXXniBq666inA4zNChQznnnHOYP38+p59+Oj//+c+54447WLp0KeXl5YwfP57Vq1dz44038tRTT1FRUVHo5u8jcD30UCrRleciwZdvT7q77K9jOG3aNObNm8eTTz7J1VdfzS233MInP/lJXn31VebOncvMmTOZNWsWDzzwwFFu8YEFr4feXnJRoovIkZk2bRqPPfYYiUSCuro65s2bx9SpU1m7di1Dhgzhuuuu49prr2XRokVs27aNZDLJBz/4Qe666y4WLVpU6ObvI7g99MI2Q0R6gfe///289NJLnHLKKZgZ3/3udxk2bBgPPfQQ99xzD9FolLKyMn7xi1+wYcMGPv3pT5NM+jkwd999d4Fbv6/ABTrqoYvIEUqPQTcz7rnnHu65556s56+55hquueaafX6uJ/bKMwWu5BJqPyta0GaIiPQ4gQt0Q8MWRURyCVygh9pniirRRUQyBTDQUzNFC9wOEZGeJnCBnp5ZlFTNRUQkS+ACPaTrKIuI5BS4QE/HuWroIiLZAhfooVSLlecicjSUlZXt97k1a9YwadKko9iaAwtcoHcMW1Sii4hkCtxMUdPUf5He40+3wualXbvPYZPh0u/s9+kvf/nLjBkzhuuvvx6AO+64AzNj3rx51NfXE4/H+da3vsWVV155SC/b3NzMZz/7WRYsWEAkEuHee+/lvPPOY9myZXz605+mtbWVZDLJE088wYgRI/jIRz5CbW0tiUSCr3/963z0ox89osOGAAa6bnAhIkdi+vTpfP7zn28P9FmzZvHUU0/xhS98gYqKCrZt28aZZ57JFVdccUg3ap45cyYAS5cu5Y033uCiiy5i5cqV/OhHP+Lmm2/m4x//OK2trSQSCebMmcOIESN48sknAWhoaOiSYwtcoOsWdCK9yAF60t3l1FNPZevWrWzcuJG6ujoGDhzI8OHD+cIXvsC8efMIhUJs2LCBLVu2MGzYsLz3+8ILL3DjjTcCcPzxxzNmzBhWrlzJWWedxbe//W1qa2v5wAc+wIQJE5g8eTJf+tKX+PKXv8xll13Gu9/97i45tsDV0Dt66AVuiIgE1oc+9CEef/xxHnvsMaZPn87DDz9MXV0dCxcuZPHixQwdOpTm5uZD2uf+qgYf+9jHmD17NiUlJVx88cU8++yzTJw4kYULFzJ58mRuu+027rzzzq44rAD20FPfdVJURA7X9OnTue6669i2bRv/+7//y6xZsxgyZAjRaJTnnnuOtWvXHvI+p02bxsMPP8z555/PypUrWbduHccddxyrV69m/Pjx3HTTTaxevZolS5Zw/PHHU1lZySc+8QnKysp48MEHu+S4ghfo6qGLyBE66aST2L17NyNHjmT48OF8/OMf5/LLL6empoYpU6Zw/PHHH/I+r7/+ej7zmc8wefJkIpEIDz74IEVFRTz22GP86le/IhqNMmzYMG6//Xbmz5/PLbfcQigUIhqN8sMf/rBLjssKdXKxpqbGLViw4JB/bu6yzfyfXy7kjze+i0kj+3dDy0SkOy1fvpwTTjih0M0IhFy/KzNb6JyrybV9XjV0M7vEzFaY2SozuzXH858yszozW5z6+pfDan0eNPVfRCS3g5ZczCwMzAQuBGqB+WY22zn3eqdNH3PO3dANbcyiy+eKyNG2dOlSrr766qx1RUVFvPzyywVqUW751NCnAqucc6sBzOxR4Eqgc6AfFRq2KBJ8zrlDGuNdaJMnT2bx4sVH9TUPpxyeT8llJLA+Y7k2ta6zD5rZEjN73MxGHXJL8mSaWCQSaMXFxWzfvl3/hg/AOcf27dspLi4+pJ/Lp4ee62208/+JPwC/ds61mNlngIeA8/fZkdkMYAbA6NGjD6mhnRujHrpIMFVXV1NbW0tdXV2hm9KjFRcXU11dfUg/k0+g1wKZPe5qYGPmBs657RmLPwH+PdeOnHP3A/eDH+VySC1N6TgpqkQXCaJoNMq4ceMK3YxeKZ+Sy3xggpmNM7MYMB2YnbmBmQ3PWLwCWN51TcymGrqISG4H7aE759rM7AZgLhAGHnDOLTOzO4EFzrnZwE1mdgXQBuwAPtVdDdbUfxGR3PKaKeqcmwPM6bTu9ozHtwG3dW3TctPUfxGR3AJ3cS5N/RcRyS1wgZ6eWKQhTyIi2QIX6Okeuk6KiohkC1ygt/fQNWxRRCRL4AJdwxZFRHILYKBr6r+ISC7BC/TUd+W5iEi2wAV6+8Qi1dBFRLIENtCTyQI3RESkhwlcoJtucCEiklNgA11xLiKSLXiBjka5iIjkErhAD6VarDwXEckWuEBP99A1sUhEJFvgAj3S2sBo24JziUI3RUSkRwlcoPdf/jDzir6AtTUXuikiIj1K4AIdCwPgnAaii4hkClygm6WarCK6iEiWAAZ6etiiaugiIpkCGOi+yU49dBGRLIEL9PRAdNXQRUSyBS/Q0zV0BbqISJbABbpq6CIiuQUw0NM99MK2Q0SkpwleoKdr6LoguohIlrwC3cwuMbMVZrbKzG49wHYfMjNnZjVd18R9XgVQyUVEpLODBrqZhYGZwKXAicBVZnZiju3KgZuAl7u6kVmvE0rPFO3OVxERCZ58euhTgVXOudXOuVbgUeDKHNvdBXwX6N6LrLTX0NVDFxHJlE+gjwTWZyzXpta1M7NTgVHOuT8eaEdmNsPMFpjZgrq6ukNuLICF0vegUw1dRCRTPoFuOda1FzzMDzu5D/jXg+3IOXe/c67GOVczePDg/FuZ1RjfZN1TVEQkWz6BXguMyliuBjZmLJcDk4C/mtka4ExgdnedGLWQJhaJiOSST6DPByaY2TgziwHTgdnpJ51zDc65KufcWOfcWODvwBXOuQXd0+L0KBcFuohIpoMGunOuDbgBmAssB2Y555aZ2Z1mdkV3N7AzXT5XRCS3SD4bOefmAHM6rbt9P9uee+TN2j9L3eACNMpFRCRT8GaKtl/LRT10EZFMwQv0kEouIiK5BC7Q0xOLdFJURCRb4AI9PfXfUKCLiGQKXqCna+iaKSoikiWAgZ4uuaiGLiKSKXCBHgrp4lwiIrkELtDVQxcRyS1wgZ4e5WIa5SIikiWAga6JRSIiuQQ20FVDFxHJFsBAVw1dRCSXwAa6bioqIpIteIGevoGSToqKiGQJXqDrWi4iIjkFNtA1bFFEJFsAA13DFkVEcglgoOsm0SIiuQQ40NVDFxHJFMBA1ygXEZFcghfoGrYoIpJT8AJdM0VFRHIKbKCjW9CJiGQJYKD7kovGoYuIZAtgoGuUi4hILnkFupldYmYrzGyVmd2a4/nPmNlSM1tsZi+Y2Yld39T0i2nqv4hILgcNdDMLAzOBS4ETgatyBPYjzrnJzrkpwHeBe7u8pR0t8v9VoIuIZMmnhz4VWOWcW+2cawUeBa7M3MA5tytjsR/QffWQ9h56t72CiEggRfLYZiSwPmO5Fjij80Zm9jngi0AMOD/XjsxsBjADYPTo0Yfa1tROUhfnQncsEhHJlE8P3XKs26d/7Jyb6Zw7Bvgy8LVcO3LO3e+cq3HO1QwePPjQWtremvTEInXRRUQy5RPotcCojOVqYOMBtn8U+KcjadQB6WqLIiI55RPo84EJZjbOzGLAdGB25gZmNiFj8X3Am13XxE50tUURkZwOWkN3zrWZ2Q3AXCAMPOCcW2ZmdwILnHOzgRvM7AIgDtQD13Rfk3UtFxGRXPI5KYpzbg4wp9O62zMe39zF7do/XctFRCSnAM8U1SgXEZFMAQz0VMklqR66iEimAAa6ToqKiOQS2EBXDV1EJFtgA109dBGRbMELdNITixToIiKZghfoukm0iEhOAQx03eBCRCSXAAa6eugiIrkEMNA1ykVEJJfABrp66CIi2YIX6Lo4l4hITsEL9HTJpRvvciciEkSBDXRLqocuIpIpgIGeviOeAl1EJFMAA12jXEREcglsoOukqIhItgAGuka5iIjkErxAB5IYIZVcRESyBDLQHSHV0EVEOglmoJuhUS4iItkCGehgmGroIiJZAhnoDlPJRUSkk2AGuoVQyUVEJFsgA10lFxGRfeUV6GZ2iZmtMLNVZnZrjue/aGavm9kSM3vGzMZ0fVM7OAxdm0tEJNtBA93MwsBM4FLgROAqMzux02avADXOuZOBx4HvdnVDM6nkIiKyr3x66FOBVc651c65VuBR4MrMDZxzzznnGlOLfwequ7aZnankIiLSWT6BPhJYn7Fcm1q3P9cCf8r1hJnNMLMFZragrq4u/1Z24sehq+YiIpIpn0C3HOtypqmZfQKoAe7J9bxz7n7nXI1zrmbw4MH5t3KfFw9hzmnooohIhkge29QCozKWq4GNnTcyswuArwLnOOdauqZ5+2FGiCSJpCMSzvV+IyLS9+TTQ58PTDCzcWYWA6YDszM3MLNTgR8DVzjntnZ9M7M5QhiQVAddRKTdQQPdOdcG3ADMBZYDs5xzy8zsTjO7IrXZPUAZ8BszW2xms/ezu65hhpEkqZKLiEi7fEouOOfmAHM6rbs94/EFXdyugzBCOBLqoouItAvkTFFnIR/o6qGLiLQLZKBjIQxHUj10EZF2gQx0hxEylVxERDIFMtD9SVGnUS4iIhkCGuipkotq6CIi7QIZ6I4Q4dTEIhEpgJ3r4Y7+sOqZQrdk/xb9EjYsLHQrjqpgBnooTJiEAl2kUGrn+++LflHYdhzI7BvgJ+cXuhVHVUADPUaMhEouIoViqejQVU97lEAGejIUJUZcPXSRQrHUNZQU6D1KMAM9HCNGm0a5iBSKBTI6er1A/l9x4RhRa1PJRaRQenrJJZkodAsKIpiBHoqp5CJSUD285JKIF7oFBRHMQE+VXBToIgXWUwM9qUAPDBeKEUUlF5GCSQdmT/03qB56cLhwjCJTyUWkYNKB2VN76Ar04HAa5SJSWMk2/73HBnproVtQEIEMdMIquYgUVHsPuIf+G1QNPTh8D10lF5GCSark0hMFMtBJl1wU6CKFkUiXXHrov0GVXIIjFC0magla4m2FbopI36RRLj1SIAM9GisCoKm5qcAtEemjNMqlRwpmoBcVA9DSpEAXKYieXkPXSdHgiMZ8oKuHLlIgCQ1b7IkCGeixohIAWloU6CIFke4B99TgTPTN82uBDPRI1NfQW1uaC9wSkT4qXaPuqbXqzDeannrithvkFehmdomZrTCzVWZ2a47np5nZIjNrM7MPdX0zO4nEAAW6SMGkZ4omWgrbjv3JrKEn+05v/aCBbmZhYCZwKXAicJWZndhps3XAp4BHurqBOYV9Dz2uQBcpjHTPvK2nllziuR/3cvn00KcCq5xzq51zrcCjwJWZGzjn1jjnlgBH5wxJ2PfQ2+IKdJGC6PE19MxA76Ft7Ab5BPpIYH3Gcm1q3SEzsxlmtsDMFtTV1R3OLrxUySXe0kM/7on0dokeXnLJDHH10LNYjnWHdZbBOXe/c67GOVczePDgw9mFFy313+N7D38fInL40j301sbCtmN/Muvm6qFnqQVGZSxXAxu7pzl5Kh0EQFFLfUGbIdJntY9yaYF4Dxw+nBnihZhkFG+CPXVQv7bj/qaNO/w5h61vwN5t3fKykTy2mQ9MMLNxwAZgOvCxbmlNvvpVAVDStqOgzRDpszJDsmknREsK15Zcurrk0tYCm5bA9lUw/GSoPAbengern4NQGF6fDS27wSUg3pxdigpFoagcmjLy6rL7oOafj7xdnRw00J1zbWZ2AzAXCAMPOOeWmdmdwALn3GwzOx34HTAQuNzMvumcO6nLW5tWVEGbRSlq3oFzDrNcVSGRPmL3ZiitgnA+/bMukjlxp7kBKoYfvdfOR2b74hllIed8OO/ZAnu2wp7NvufcssuXj7at8OEbKfHb4GDbKv9479YcL2R+m1gZnPhP/vzejtUw+AQYdIx/rY2L/Mi8koH+DeGEy2H8ud1y2Hn9BTjn5gBzOq27PePxfHwp5ugwozlWScXeBuob41T2ix21lxbpURp3wPeOg3feBBfddfReN7OH3rzz6L1uPl7+MbyaMYL6x9Ogeirs2gi7N/le9P6EYxAp9pc0CEd9EA89EaomwDHnwcCxsG0lbH4Njn8fjH03NG6DkkqIlXb7oR3MUXxL71qJkioGNe6itr5RgS591463/fe3ngWOYqAn4r6UkIz7kktX2bvd93Kj/WDZb32oTrjIh+jav/nj/OjDsOQxwMGA0fD3H8Lg430pdvVfYdXTEIpA/9Fw3lfg5R/5oB41FQaMgqIKKBua+hoCJQPAwv5xKAqhg5xaHDcte7n/0evLHkxgAz1UPphBO9axob6Jk6sHFLo5IoVRnwr01HmlbtO8y5cOxp/rT/JtXupfc/em7B56Iu5LHBtf8aWHPVthxKmwZRlEimDICf4NoH4NrHnelySGnQz9R/qA/cPn/eg1C++/J/2tHCPkVj6VvfzPc/3rhsIw5aou+iX0fIEN9KL+wxlui3l5Rw8dNiV92+ybYPRZ3R8mO1b77yWVXbfPeJPvCTvng3nYJN8LXv8yXDkTlj7uT/ANHOe337URFj8CS38Dbz3ne8eHMrJk/cv7Ts+vHA9j3unfIJobYPQZMGgCPHsXbH3db3PtX/xrnvU5X8MOR33tu2G9r1/3QYEN9NiISQx97desXrMazumb//Okh0omYNFD/qu7A71uhf9+KGOt33waispgwBhfblj8iP+acKF/E3rxP+GtZ3L/7O8/578PGANX/hc8+D545pt+XaTYj9wIR30pZNQZvke+7HdQMcKXUZoboLg/7K2DpvpUffo4X4fe8bb/GRzE+uV+/eMu9ScW401+tMmoqftu00fDHAIc6Aw/BYDGta/g3AUa6SI9x64jmKaRTIKZ/wJf2lj7Eow925+Q+/sPfc851s/Xhneu9du17Mpv/5tehYc/mPu5DQv8dwvBsRfAoGNTpRzzJ/9CYf/J44I7YOJFftsBo2HnOvjYb3wvurj/vvs99RMHb1dRmT++gzHzJyglpwAH+sk4jHEtK3hj825OGF5R6BaJePVrspcX/NyfjDv2gtzbN9T6k3wnfQAe+QgMnwLnf833amd9sqOsMuhY3zvtrHy4r3Fnijf5yStrXvC16Jd/7IfjbX3dh+7QSb5337jND6c76QO+d14y0Nezh5yQu63X/y17+bN/S40MKTror0W6X3ADvbg/bUNP4d2blvD061sU6NJzZAb6D8+GLa/5x1/fDjveghV/8iWSY98DS2b5MG+qh2fu9NttWwlLZ0H5CNi9ES68EzYsgtf/B86+Gabd4mvO0X6+Xv3b6/wJS/A9/OW/h8f/OftuQgPH+SGO/YbARx7yge2c/zQRKTr8k6pF5Yf3c9ItghvoQPS4C3nHlu/xk5f/QvO08RRHw4VukvRliTZfrqh7o2NdOswB7hqUvf1z3/bfy4b5XvmmxXDMe3wZZfsqH+bjz/VjzJ2D1Z/0pY9Ip2G6xRW+h75rkw/3Nc/79adeDcdfBq17fO25c13azPfGpdcIdKBz+nW0LnyE63f/lF++dAnXTRtf6BZJoSWTPsCKu+ETW3MDrPyzH6o35myoHOeneyda4YXv+3HTsTJf7hg0Aaomwvu+5yecPP1NP7xv1Bl+5MaWZX7I4ch3wMXf9kP5XvklTP4wlA/z1wCJFPvySLqmfux7crerqMKXTn50tn/ts2/2JZQRU7r+dyA9WrADvXwoxWd8minPfoubnv4Hp4+rZMoojUnv057/D9/z/cRv/XTtKTkuO5S+9sjO9T4on/xXeO89/gSfhfwojYYNvqe79sXUJJqID9z9iRT7mYOvPeGXT/k8TPtSx/OX3XvgdpcMgHfe2LE8cEz+x5x+84o3wYy/wuDj8v9Z6VXMFeh+ezU1NW7BggVHvqOtb+D++0z+ETqFr8Ru5cEZ5zCqsvBTcKVA/mNi6hocGS75jr9CZ6TY96KX/e7A+zj1anjjST/W2kJ+jHfjNphwMbw5d9/tx5wNH37QzzT8x09g8xK46Nvd8ykhl63L4fl74bSr953FKL2OmS10ztXkfC7wgQ4w/2fw5BcBuMLdy5c+fgXTJh7B9dal53DOTzyJN0F1zYFPwiUTcM+x2Ve1O5hwbN8x3OEYjDgNLvymL5uUVkLrXl+DfuVXULsAFv7cb/vF5f5E49G8MJb0ab0/0J2Dp78BL/4nm20IP229kLYTPsBXpp9PLJLXfbClO619yQ/By7eM0NYCtfP9G/WAUX6iS9qFd/lrepzzb768se4lP8V7xGnw1K3Z47HPvN6fDGzZ7a/18cYf/fWoR57mR5ZcfLevbzfugPtSt8k9+2Y458v7n9iSdkdqvPUdDfn/HkS6QO8P9LSlj5Oc+1VCezYD8FpyLA1jLmbyeR/FDT+Z/iXRrn29vmj3Fmhrhv6j9n8RI+f8MLzSSj+G+gen+tmAN/zDnxzsXw2nfdKXM0KpkUnxZh/irXv92OvM60mPPssHd76K+kNLg5/skp4AczCblsAL98H7f5TfmOoty3zo5zMZRqQL9Z1ABz8S4YXvk3jpv2lKRihze9joKnko9EEuPe8cppw61dc6O88s3b3Fz6LLNwCOxLL/8dewPvMzh/fzyWR2mLbs8TPtOtu1CVb+Cd7x6ezjdW7f4+9s7Uv+hOCpV3cE3LLfwh+/4B+f8Vk4/6s+oJvq/aVF173kL8a05kV/gaX01fjSLv8B/OEm/9jC/uTjmLN9D3rzUmjbz51vPvGEH9b3wn3w0n/5daddA+/6PNSt9GOom+qheIAvyyQTftbjqDMOfpwiAdO3Aj0tmcTheP3F2VQ/exP9XcdH8Z1Fw0kccyGVIydgjdt9L+vF7/sJITXX+okb5cP2DYOGWlj8a3jnDX6UROteHx7FFb5M0LK74yJJoVDq1lPWEb6JuL/Y0c8u9Mtfq/MlgqadUHVse7sx84G/9fXUMLdWf7Gk9S/75Vd+5XuIF3zD/+zT3/Drp87wQ+O2LvdXvFv0C7/PoZPhxCv99Tl2b/KTSSrH+xNoyTbY9qYP7YoRviSx5TW/D5fwJxPbWvxQwM4615/TAR6KwugzfY+7tArGvguWPNqx3ZATfS28qLzjUqcnf9SPKtm5zg/1G36qH62xa0PHqI1k0r+pjHmnb6tIH9Q3Az2Tc+zcspZnnn+eutVLmLTnRU4OrabCDnAvxKqJPiyHnuQfx/r5q8w1rPOBFIr4iwkVlflgXPJYx8+GYz7E6lb6wOtf7cO/fs2+d0mPlPhthpzgxzBvWux70HD4d1SPlfn2db7xQGmVP47qGn/d6MbtBzj+4/y1pJ+4tuNKeOlSRvEAqBgJW5f5enP5MH/Mp6SGCKbHTac/CSST/vKmbz0Lp1wF1e/oeJ3Nr/mx1gNG+bHXa/8Gp0xXz1pkPxTonby2oYFlG3byx/kreXV9PcfYRqLF/agZHmFq/12cOaCB4g1/81OnW/fC9rd86JYM8AG8q9afiCsd5AOorcWHerTUD28bMNqXEJzz28RK/b76j4Jhk/1kkp+/1+9v3DRf6ql7w4fkcZf6Hm6ixQdwpCh1zejJPohfmul77md/3pc/TrgMzv2K79nWveHfGAYd6+vcc77kZx4OP8Vf3W7YyR13Vdm53vfiiypg8cO+d9+yy+9/6gx/Nb1QyNeWo6W+R5/+1OGc3/bVX8M7PnXwE4gi0mUU6Afw9ra9bKhv4qcvrGbdjkZW1+2lJBrmkknDGDWwhOOGVTBxaBlV/WIMLE3dzaRlt+8Fm/np3vG9ua8ydyCZ+0gmfDnnUCaTQH61cBHpVQ4U6H1+8Oy4qn6Mq+rHuyb4ixMt29jAQ39bw7NvbGX73tb26kckZNSMHcjUsb5GftywCs4cX8mgsiIIH2KYQ4d0p+kAAAyuSURBVPZ46lD40MMcFOZSUH94dSNzl23mvz52WqGbIil9vod+IPFEkmeWb6WlLcFrGxp4bkUdq7Z2nByMho0h5cWcXN2foRXFjBxQQiwSYtrEwYyuLCUcUuBK7zX21icBeOv/vld/60eReuiHKRoOccmkYQBcOWUkt13q2NjQRFEkzN9Xb2f+mh3U7W5h6YYG5q2sY29rxz0QB5RGGV/Vj4lDy2lpS9LSluCMcYM4ZnAZzfEETy/fwrsnDOb844cQTybpF4voH4UE0s7GVv9JVQpOgX4IQiGjeqA/qXj5KSO4/JSOoXPOORqa4mzd3cLidTt58a1tbG5o5s+vb6E0FqY5nmTO0s1Z+3t0/vqOfRtMHFrOxKHlDCqLkUw6hvUvweFoSzjGVfUjZMaoyhKqyoooiYbZvKuZ5niCk0b014xYKZjte3tmoD+zfAtjq/pxzOAcczR6KQV6FzEzBpTGGFAaY+LQcj5y+qis55NJx+ZdzazZvpeiSJgh5UX84+0dvLl1DwNKo+xsjLNgzQ6W1O6kbndLVm8/H1NGDWBXc5ym1gRVZUVsamhm6riBDCyNtb+hVA8soSgSorw4SiwSojmeoGZsJbX1jSSSjncdW8WOva20JR0jBpQc8PWcc7xVt4fxVWWE9MmiT9u2p4WJQ3vWjS6cc1z7kC/prvnO+wrcmqNHgX6UhELGiAElWUF5oKtCNra2UVvfxNDyYsJhY0N9E3ta4tTWN9HQFKexNYFzUFES4c0te3i1dicDS2NUFDsiIeP4YeX87a3tGNAUTxAJhdjT0rbf1wOIRUK0tvm73BRFQowYUEJpLExJNExpUQQD9ra0URwNs21PC29s3s1powdwxSkjCIdDhFOfIOob42zb3cKIAcWcMmoAsXCIXc1tVA8sIZF07TciSSZd1ptBWyLJzqY4VUfY25v53CqGVhTzoXdUH9F+jsT/e+ZNjhlSxnsnD+/211qwZgc/e+FtfnDVqUTDR/+T2vY9h3CD6qNk+96e16ajIa9AN7NLgP8EwsBPnXPf6fR8EfAL4B3AduCjzrk1XdvUvqU0Fsnq9Rw3zD9+x2EMhgHfY9nZGG8vC5UXR0gkHYvW1VM9sITWtiQL1tRTGguzo7GVSChE3Z4WWuIJdje30dAUZ3dznIriKHtb29jd3MbYQaWs29HIHX94Pa82pOcahUNGLBwinkgyurKUkliY0pgvIW3a2cxZxwyiKBLGDJbWNjC5uj9bd7cQDRklsTCjKksZNbCUipIIIweUUJJ6gyiNRdi8q5l75q4AYPueFs45bjCV/WJUlsZYt6ORUZWlOUNv2cYGHv3Her7y3hMoiR3Zna8aGuN87y8rAXj6i+dw7JDu/cg/45cL2bG3lVfW7WTquMpufa20eKLj9nbb9xzmBLhutG5HY/vjXam/277goIFuZmFgJnAhUAvMN7PZzrnMf8XXAvXOuWPNbDrw78BHu6PBcnjMjIH9YgzsF2NsVcdEoEkjO4ZcXjLp0HuTLW0JauubKIqEcA427myiKBqmKBJiU0MTG3c205ZIUhILs2Z7I0URH+S7mtow8z2pptYEja1tjOhfwpjKfuxsjFO3ew879rYybWIVr2/cRVlRhLW7mxlYGuP5N7fl1ba7//QGd//J3w4uEjLako5YJEQsHMI5/0mhoiRKRXGEV2v9VRPnvVnH8P7FhENGcSRMWXGEpIP5b+9gwtAyBpcV0diaYNueFloTSaZNGExTPEHIoKwoSllxhA31HTOQ756znEsmDSOecJTEQpTGIhRFQjTHkzTHEzTHE5TEwoRDxtt1e0k6OOe4wURChnNQFPW/r6JIiCW1DZxcPYCyogjN8QTxRJK9rQl2pHqjsxasZ1xVP8Ihw4CQmb/+mRkhA8Mwy1jO+N6Zcw7n2G85LbNXXlvfhHMu534KZX1GoP952ZaCflo7mg46bNHMzgLucM5dnFq+DcA5d3fGNnNT27xkZhFgMzDYHWDnQRi2KIXVuSST1hxP0Njqw3DdjkaSSYfDl4PKi6OMHlTK8Ipi1mzfy7yVdSQcbG5oYsSAEjbubCKRBIejOZ5kd3Oc+sZWyooixCJh9ra0Ube7hXDIaG1LsqeljcbWBCeNqGBTQxM7G+OUF0fawy5zGGumEf2LOeuYKp5YVNvNvyUoiYapKo+xfscBLmVxEOlgD5sRDhlJ52hp828kkZARSr1JtKZ65m0J598gwyFaE0nMSG0ban/TyHzzIPVmsr/IT78XWMYWHevSy/v+dPs2GU8Zlvr/Gmd0pf8UObSiyLcptZ/01SnSb3JZ+9znNXK87kFW5DrOzP3c/J4JWYMqDsWRDlscCazPWK4FztjfNs65NjNrAAYBWV0pM5sBzAAYPXp0Xo2Xvmt/vcPiaLi9Dn+gk7fjB5cxvhtHODjnaIonSDr/CSAcMnY3t9GWTFJZGiMcMr72vhPY3ezPXbS0JWiOJ2lNJCmOhihJHceOva00xROMr+rHtj2tvL1tb3vANra2EQuHaG5LMKS8mPU7GnGkwjMcIhoyThhewajKUua9Weefd5B0jqTr6Gm3L5NaTnYsp7dLpNYlU/2w4miYlniCRNKHN/jzLM75N8OasQM57/ghzJq/nl1NcZrbkrQlXMdrdHptyN2/S3f7Mrt/LrVt9rpO27Pvk5mvcOyQMi4/eQS/eGkNu5vbMo614/fQuc/ZuYW5uqT7bnPgfeRa2V2X8s6nh/5h4GLn3L+klq8GpjrnbszYZllqm9rU8lupbfZ79Sf10EVEDt2Beuj5nBKvBTLH4FUDG/e3Tark0h84hPuAiYjIkcon0OcDE8xsnJnFgOnA7E7bzAauST3+EPDsgernIiLS9Q5aQ0/VxG8A5uKHLT7gnFtmZncCC5xzs4GfAb80s1X4nvn07my0iIjsK69x6M65OcCcTutuz3jcDHy4a5smIiKHQhcAERHpJRToIiK9hAJdRKSXUKCLiPQSBbtjkZnVAWsP88er6DQLtQ/QMfcNOua+4UiOeYxzbnCuJwoW6EfCzBbsb6ZUb6Vj7ht0zH1Ddx2zSi4iIr2EAl1EpJcIaqDfX+gGFICOuW/QMfcN3XLMgayhi4jIvoLaQxcRkU4U6CIivUTgAt3MLjGzFWa2ysxuLXR7uoqZPWBmW83stYx1lWb2FzN7M/V9YGq9mdkPUr+DJWZ2WuFafvjMbJSZPWdmy81smZndnFrfa4/bzIrN7B9m9mrqmL+ZWj/OzF5OHfNjqUtVY2ZFqeVVqefHFrL9h8vMwmb2ipn9MbXcq48XwMzWmNlSM1tsZgtS67r1bztQgZ5xw+pLgROBq8zsxMK2qss8CFzSad2twDPOuQnAM6ll8Mc/IfU1A/jhUWpjV2sD/tU5dwJwJvC51P/P3nzcLcD5zrlTgCnAJWZ2Jv7G6veljrkef+N1yLgBO3BfarsguhlYnrHc24837Tzn3JSMMefd+7ft7zkYjC/gLGBuxvJtwG2FblcXHt9Y4LWM5RXA8NTj4cCK1OMfA1fl2i7IX8DvgQv7ynEDpcAi/D16twGR1Pr2v3P8fQjOSj2OpLazQrf9EI+zOhVe5wN/xN9Dudceb8ZxrwGqOq3r1r/tQPXQyX3D6pEFasvRMNQ5twkg9X1Ian2v+z2kPlqfCrxMLz/uVPlhMbAV+AvwFrDTOdeW2iTzuLJuwA6kb8AeJN8H/g1IppYH0buPN80BfzazhWY2I7WuW/+287rBRQ+S6zbwfXHcZa/6PZhZGfAE8Hnn3C6zXIfnN82xLnDH7ZxLAFPMbADwO+CEXJulvgf6mM3sMmCrc26hmZ2bXp1j015xvJ2c7ZzbaGZDgL+Y2RsH2LZLjjtoPfR8bljdm2wxs+EAqe9bU+t7ze/BzKL4MH/YOffb1Opef9wAzrmdwF/x5w8GpG6wDtnHFfQbsJ8NXGFma4BH8WWX79N7j7edc25j6vtW/Bv3VLr5bztogZ7PDat7k8ybb1+DrzGn138ydWb8TKAh/TEuSMx3xX8GLHfO3ZvxVK89bjMbnOqZY2YlwAX4k4XP4W+wDvsec2BvwO6cu805V+2cG4v/9/qsc+7j9NLjTTOzfmZWnn4MXAS8Rnf/bRf6xMFhnGh4L7ASX3f8aqHb04XH9WtgExDHv1tfi68dPgO8mfpemdrW8KN93gKWAjWFbv9hHvO78B8rlwCLU1/v7c3HDZwMvJI65teA21PrxwP/AFYBvwGKUuuLU8urUs+PL/QxHMGxnwv8sS8cb+r4Xk19LUtnVXf/bWvqv4hILxG0kouIiOyHAl1EpJdQoIuI9BIKdBGRXkKBLiLSSyjQRUR6CQW6iEgv8f8B8rmipeHf7+4AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "model_loss = pd.DataFrame(model.history.history)\n",
    "model_loss.plot()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id='dropout'></a>\n",
    "### 2. Add Dropout layers"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "from tensorflow.keras.layers import Dropout"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = Sequential()\n",
    "model.add(Dense(40,activation='relu'))\n",
    "model.add(Dropout(0.5))\n",
    "\n",
    "model.add(Dense(20,activation='relu'))\n",
    "model.add(Dropout(0.5))\n",
    "\n",
    "model.add(Dense(1,activation='sigmoid'))\n",
    "model.compile(loss='binary_crossentropy', optimizer='adam')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 2398 samples, validate on 800 samples\n",
      "Epoch 1/500\n",
      "2398/2398 [==============================] - 1s 292us/sample - loss: 0.6558 - val_loss: 0.5574\n",
      "Epoch 2/500\n",
      "2398/2398 [==============================] - 0s 70us/sample - loss: 0.5087 - val_loss: 0.3408\n",
      "Epoch 3/500\n",
      "2398/2398 [==============================] - 0s 70us/sample - loss: 0.3321 - val_loss: 0.1732\n",
      "Epoch 4/500\n",
      "2398/2398 [==============================] - 0s 65us/sample - loss: 0.2201 - val_loss: 0.1134\n",
      "Epoch 5/500\n",
      "2398/2398 [==============================] - 0s 64us/sample - loss: 0.1733 - val_loss: 0.0887\n",
      "Epoch 6/500\n",
      "2398/2398 [==============================] - 0s 70us/sample - loss: 0.1516 - val_loss: 0.0794\n",
      "Epoch 7/500\n",
      "2398/2398 [==============================] - 0s 79us/sample - loss: 0.1328 - val_loss: 0.0728\n",
      "Epoch 8/500\n",
      "2398/2398 [==============================] - 0s 63us/sample - loss: 0.1130 - val_loss: 0.0661\n",
      "Epoch 9/500\n",
      "2398/2398 [==============================] - 0s 55us/sample - loss: 0.1118 - val_loss: 0.0642\n",
      "Epoch 10/500\n",
      "2398/2398 [==============================] - 0s 59us/sample - loss: 0.1056 - val_loss: 0.0590\n",
      "Epoch 11/500\n",
      "2398/2398 [==============================] - 0s 50us/sample - loss: 0.0924 - val_loss: 0.0579\n",
      "Epoch 12/500\n",
      "2398/2398 [==============================] - 0s 52us/sample - loss: 0.1047 - val_loss: 0.0550\n",
      "Epoch 13/500\n",
      "2398/2398 [==============================] - 0s 53us/sample - loss: 0.0922 - val_loss: 0.0528\n",
      "Epoch 14/500\n",
      "2398/2398 [==============================] - 0s 50us/sample - loss: 0.0974 - val_loss: 0.0520\n",
      "Epoch 15/500\n",
      "2398/2398 [==============================] - 0s 64us/sample - loss: 0.0809 - val_loss: 0.0499\n",
      "Epoch 16/500\n",
      "2398/2398 [==============================] - 0s 68us/sample - loss: 0.0859 - val_loss: 0.0491\n",
      "Epoch 17/500\n",
      "2398/2398 [==============================] - 0s 55us/sample - loss: 0.0792 - val_loss: 0.0485\n",
      "Epoch 18/500\n",
      "2398/2398 [==============================] - 0s 54us/sample - loss: 0.0805 - val_loss: 0.0465\n",
      "Epoch 19/500\n",
      "2398/2398 [==============================] - 0s 59us/sample - loss: 0.0734 - val_loss: 0.0454\n",
      "Epoch 20/500\n",
      "2398/2398 [==============================] - 0s 50us/sample - loss: 0.0747 - val_loss: 0.0464\n",
      "Epoch 21/500\n",
      "2398/2398 [==============================] - 0s 57us/sample - loss: 0.0823 - val_loss: 0.0436\n",
      "Epoch 22/500\n",
      "2398/2398 [==============================] - 0s 53us/sample - loss: 0.0738 - val_loss: 0.0446\n",
      "Epoch 23/500\n",
      "2398/2398 [==============================] - 0s 52us/sample - loss: 0.0710 - val_loss: 0.0431\n",
      "Epoch 24/500\n",
      "2398/2398 [==============================] - 0s 52us/sample - loss: 0.0807 - val_loss: 0.0429\n",
      "Epoch 25/500\n",
      "2398/2398 [==============================] - 0s 53us/sample - loss: 0.0666 - val_loss: 0.0411\n",
      "Epoch 26/500\n",
      "2398/2398 [==============================] - 0s 52us/sample - loss: 0.0700 - val_loss: 0.0404\n",
      "Epoch 27/500\n",
      "2398/2398 [==============================] - 0s 52us/sample - loss: 0.0748 - val_loss: 0.0410\n",
      "Epoch 28/500\n",
      "2398/2398 [==============================] - 0s 57us/sample - loss: 0.0629 - val_loss: 0.0402\n",
      "Epoch 29/500\n",
      "2398/2398 [==============================] - 0s 55us/sample - loss: 0.0662 - val_loss: 0.0390\n",
      "Epoch 30/500\n",
      "2398/2398 [==============================] - ETA: 0s - loss: 0.054 - 0s 51us/sample - loss: 0.0651 - val_loss: 0.0386\n",
      "Epoch 31/500\n",
      "2398/2398 [==============================] - 0s 55us/sample - loss: 0.0635 - val_loss: 0.0408\n",
      "Epoch 32/500\n",
      "2398/2398 [==============================] - 0s 65us/sample - loss: 0.0580 - val_loss: 0.0384\n",
      "Epoch 33/500\n",
      "2398/2398 [==============================] - 0s 50us/sample - loss: 0.0567 - val_loss: 0.0387\n",
      "Epoch 34/500\n",
      "2398/2398 [==============================] - 0s 54us/sample - loss: 0.0656 - val_loss: 0.0377\n",
      "Epoch 35/500\n",
      "2398/2398 [==============================] - 0s 51us/sample - loss: 0.0650 - val_loss: 0.0372\n",
      "Epoch 36/500\n",
      "2398/2398 [==============================] - 0s 57us/sample - loss: 0.0616 - val_loss: 0.0373\n",
      "Epoch 37/500\n",
      "2398/2398 [==============================] - 0s 52us/sample - loss: 0.0583 - val_loss: 0.0368\n",
      "Epoch 38/500\n",
      "2398/2398 [==============================] - 0s 63us/sample - loss: 0.0628 - val_loss: 0.0366\n",
      "Epoch 39/500\n",
      "2398/2398 [==============================] - 0s 59us/sample - loss: 0.0561 - val_loss: 0.0406\n",
      "Epoch 40/500\n",
      "2398/2398 [==============================] - 0s 53us/sample - loss: 0.0550 - val_loss: 0.0363\n",
      "Epoch 41/500\n",
      "2398/2398 [==============================] - 0s 79us/sample - loss: 0.0572 - val_loss: 0.0359\n",
      "Epoch 42/500\n",
      "2398/2398 [==============================] - 0s 81us/sample - loss: 0.0559 - val_loss: 0.0368\n",
      "Epoch 43/500\n",
      "2398/2398 [==============================] - 0s 83us/sample - loss: 0.0571 - val_loss: 0.0358\n",
      "Epoch 44/500\n",
      "2398/2398 [==============================] - 0s 82us/sample - loss: 0.0560 - val_loss: 0.0357\n",
      "Epoch 45/500\n",
      "2398/2398 [==============================] - 0s 69us/sample - loss: 0.0520 - val_loss: 0.0352\n",
      "Epoch 46/500\n",
      "2398/2398 [==============================] - 0s 56us/sample - loss: 0.0524 - val_loss: 0.0381\n",
      "Epoch 47/500\n",
      "2398/2398 [==============================] - 0s 58us/sample - loss: 0.0557 - val_loss: 0.0353\n",
      "Epoch 48/500\n",
      "2398/2398 [==============================] - 0s 63us/sample - loss: 0.0515 - val_loss: 0.0351\n",
      "Epoch 49/500\n",
      "2398/2398 [==============================] - 0s 68us/sample - loss: 0.0581 - val_loss: 0.0343\n",
      "Epoch 50/500\n",
      "2398/2398 [==============================] - 0s 59us/sample - loss: 0.0519 - val_loss: 0.0346\n",
      "Epoch 51/500\n",
      "2398/2398 [==============================] - 0s 53us/sample - loss: 0.0535 - val_loss: 0.0354\n",
      "Epoch 52/500\n",
      "2398/2398 [==============================] - 0s 53us/sample - loss: 0.0541 - val_loss: 0.0341\n",
      "Epoch 53/500\n",
      "2398/2398 [==============================] - 0s 50us/sample - loss: 0.0556 - val_loss: 0.0346\n",
      "Epoch 54/500\n",
      "2398/2398 [==============================] - 0s 53us/sample - loss: 0.0498 - val_loss: 0.0351\n",
      "Epoch 55/500\n",
      "2398/2398 [==============================] - 0s 49us/sample - loss: 0.0539 - val_loss: 0.0344\n",
      "Epoch 56/500\n",
      "2398/2398 [==============================] - 0s 50us/sample - loss: 0.0495 - val_loss: 0.0349\n",
      "Epoch 57/500\n",
      "2398/2398 [==============================] - 0s 50us/sample - loss: 0.0532 - val_loss: 0.0345\n",
      "Epoch 58/500\n",
      "2398/2398 [==============================] - 0s 47us/sample - loss: 0.0476 - val_loss: 0.0344\n",
      "Epoch 59/500\n",
      "2398/2398 [==============================] - 0s 49us/sample - loss: 0.0479 - val_loss: 0.0341\n",
      "Epoch 60/500\n",
      "2398/2398 [==============================] - 0s 56us/sample - loss: 0.0501 - val_loss: 0.0337\n",
      "Epoch 61/500\n",
      "2398/2398 [==============================] - 0s 52us/sample - loss: 0.0440 - val_loss: 0.0344\n",
      "Epoch 62/500\n",
      "2398/2398 [==============================] - 0s 51us/sample - loss: 0.0525 - val_loss: 0.0341\n",
      "Epoch 63/500\n",
      "2398/2398 [==============================] - 0s 51us/sample - loss: 0.0434 - val_loss: 0.0334\n",
      "Epoch 64/500\n",
      "2398/2398 [==============================] - 0s 52us/sample - loss: 0.0456 - val_loss: 0.0328\n",
      "Epoch 65/500\n",
      "2398/2398 [==============================] - 0s 50us/sample - loss: 0.0485 - val_loss: 0.0333\n",
      "Epoch 66/500\n",
      "2398/2398 [==============================] - 0s 51us/sample - loss: 0.0534 - val_loss: 0.0334\n",
      "Epoch 67/500\n",
      "2398/2398 [==============================] - 0s 51us/sample - loss: 0.0476 - val_loss: 0.0333\n",
      "Epoch 68/500\n",
      "2398/2398 [==============================] - 0s 69us/sample - loss: 0.0488 - val_loss: 0.0335\n",
      "Epoch 69/500\n",
      "2398/2398 [==============================] - 0s 53us/sample - loss: 0.0448 - val_loss: 0.0337\n",
      "Epoch 70/500\n",
      "2398/2398 [==============================] - 0s 51us/sample - loss: 0.0513 - val_loss: 0.0344\n",
      "Epoch 71/500\n",
      "2398/2398 [==============================] - 0s 50us/sample - loss: 0.0504 - val_loss: 0.0340\n",
      "Epoch 72/500\n",
      "2398/2398 [==============================] - 0s 53us/sample - loss: 0.0490 - val_loss: 0.0335\n",
      "Epoch 73/500\n",
      "2398/2398 [==============================] - 0s 51us/sample - loss: 0.0467 - val_loss: 0.0334\n",
      "Epoch 74/500\n",
      "2398/2398 [==============================] - 0s 51us/sample - loss: 0.0456 - val_loss: 0.0333\n",
      "Epoch 75/500\n",
      "2398/2398 [==============================] - 0s 55us/sample - loss: 0.0468 - val_loss: 0.0323\n",
      "Epoch 76/500\n",
      "2398/2398 [==============================] - 0s 61us/sample - loss: 0.0431 - val_loss: 0.0326\n",
      "Epoch 77/500\n",
      "2398/2398 [==============================] - 0s 50us/sample - loss: 0.0415 - val_loss: 0.0331\n",
      "Epoch 78/500\n",
      "2398/2398 [==============================] - 0s 51us/sample - loss: 0.0448 - val_loss: 0.0325\n",
      "Epoch 79/500\n",
      "2398/2398 [==============================] - 0s 51us/sample - loss: 0.0436 - val_loss: 0.0324\n",
      "Epoch 80/500\n",
      "2398/2398 [==============================] - 0s 53us/sample - loss: 0.0445 - val_loss: 0.0324\n",
      "Epoch 81/500\n",
      "2398/2398 [==============================] - 0s 54us/sample - loss: 0.0444 - val_loss: 0.0333\n",
      "Epoch 82/500\n",
      "2398/2398 [==============================] - 0s 50us/sample - loss: 0.0386 - val_loss: 0.0339\n",
      "Epoch 83/500\n",
      "2398/2398 [==============================] - 0s 53us/sample - loss: 0.0407 - val_loss: 0.0326\n",
      "Epoch 84/500\n",
      "2398/2398 [==============================] - 0s 56us/sample - loss: 0.0449 - val_loss: 0.0326\n",
      "Epoch 85/500\n",
      "2398/2398 [==============================] - 0s 51us/sample - loss: 0.0423 - val_loss: 0.0325\n",
      "Epoch 86/500\n",
      "2398/2398 [==============================] - 0s 52us/sample - loss: 0.0364 - val_loss: 0.0325\n",
      "Epoch 87/500\n",
      "2398/2398 [==============================] - 0s 51us/sample - loss: 0.0429 - val_loss: 0.0326\n",
      "Epoch 88/500\n",
      "2398/2398 [==============================] - 0s 54us/sample - loss: 0.0433 - val_loss: 0.0335\n",
      "Epoch 89/500\n",
      "2398/2398 [==============================] - 0s 50us/sample - loss: 0.0397 - val_loss: 0.0346\n",
      "Epoch 90/500\n",
      "2398/2398 [==============================] - 0s 55us/sample - loss: 0.0430 - val_loss: 0.0340\n",
      "Epoch 91/500\n",
      "2398/2398 [==============================] - 0s 61us/sample - loss: 0.0382 - val_loss: 0.0340\n",
      "Epoch 92/500\n",
      "2398/2398 [==============================] - 0s 54us/sample - loss: 0.0418 - val_loss: 0.0338\n",
      "Epoch 93/500\n",
      "2398/2398 [==============================] - 0s 50us/sample - loss: 0.0412 - val_loss: 0.0326\n",
      "Epoch 94/500\n",
      "2398/2398 [==============================] - 0s 51us/sample - loss: 0.0408 - val_loss: 0.0323\n",
      "Epoch 95/500\n",
      "2398/2398 [==============================] - 0s 52us/sample - loss: 0.0448 - val_loss: 0.0331\n",
      "Epoch 96/500\n",
      "2398/2398 [==============================] - 0s 54us/sample - loss: 0.0465 - val_loss: 0.0328\n",
      "Epoch 97/500\n",
      "2398/2398 [==============================] - 0s 55us/sample - loss: 0.0361 - val_loss: 0.0330\n",
      "Epoch 98/500\n",
      "2398/2398 [==============================] - 0s 57us/sample - loss: 0.0391 - val_loss: 0.0321\n",
      "Epoch 99/500\n",
      "2398/2398 [==============================] - 0s 76us/sample - loss: 0.0355 - val_loss: 0.0345\n",
      "Epoch 100/500\n",
      "2398/2398 [==============================] - 0s 66us/sample - loss: 0.0399 - val_loss: 0.0336\n",
      "Epoch 101/500\n",
      "2398/2398 [==============================] - 0s 67us/sample - loss: 0.0414 - val_loss: 0.0331\n",
      "Epoch 102/500\n",
      "2398/2398 [==============================] - 0s 97us/sample - loss: 0.0377 - val_loss: 0.0328\n",
      "Epoch 103/500\n",
      "2398/2398 [==============================] - 0s 61us/sample - loss: 0.0409 - val_loss: 0.0331\n",
      "Epoch 104/500\n",
      "2398/2398 [==============================] - 0s 60us/sample - loss: 0.0350 - val_loss: 0.0337\n",
      "Epoch 105/500\n",
      "2398/2398 [==============================] - 0s 55us/sample - loss: 0.0357 - val_loss: 0.0336\n",
      "Epoch 106/500\n",
      "2398/2398 [==============================] - 0s 65us/sample - loss: 0.0389 - val_loss: 0.0368\n",
      "Epoch 107/500\n",
      "2398/2398 [==============================] - 0s 59us/sample - loss: 0.0365 - val_loss: 0.0336\n",
      "Epoch 108/500\n",
      "2398/2398 [==============================] - 0s 52us/sample - loss: 0.0443 - val_loss: 0.0338\n",
      "Epoch 109/500\n",
      "2398/2398 [==============================] - 0s 148us/sample - loss: 0.0397 - val_loss: 0.0334\n",
      "Epoch 110/500\n",
      "2398/2398 [==============================] - 0s 73us/sample - loss: 0.0338 - val_loss: 0.0338\n",
      "Epoch 111/500\n",
      "2398/2398 [==============================] - 0s 95us/sample - loss: 0.0350 - val_loss: 0.0316\n",
      "Epoch 112/500\n",
      "2398/2398 [==============================] - 0s 88us/sample - loss: 0.0337 - val_loss: 0.0326\n",
      "Epoch 113/500\n",
      "2398/2398 [==============================] - 0s 146us/sample - loss: 0.0314 - val_loss: 0.0338\n",
      "Epoch 114/500\n",
      "2398/2398 [==============================] - 0s 76us/sample - loss: 0.0437 - val_loss: 0.0339\n",
      "Epoch 115/500\n",
      "2398/2398 [==============================] - 0s 72us/sample - loss: 0.0406 - val_loss: 0.0329\n",
      "Epoch 116/500\n",
      "2398/2398 [==============================] - 0s 69us/sample - loss: 0.0381 - val_loss: 0.0320\n",
      "Epoch 117/500\n",
      "2398/2398 [==============================] - 0s 54us/sample - loss: 0.0327 - val_loss: 0.0319\n",
      "Epoch 118/500\n",
      "2398/2398 [==============================] - 0s 58us/sample - loss: 0.0408 - val_loss: 0.0314\n",
      "Epoch 119/500\n",
      "2398/2398 [==============================] - 0s 57us/sample - loss: 0.0440 - val_loss: 0.0327\n",
      "Epoch 120/500\n",
      "2398/2398 [==============================] - 0s 66us/sample - loss: 0.0371 - val_loss: 0.0323\n",
      "Epoch 121/500\n",
      "2398/2398 [==============================] - 0s 64us/sample - loss: 0.0377 - val_loss: 0.0320\n",
      "Epoch 122/500\n",
      "2398/2398 [==============================] - 0s 54us/sample - loss: 0.0336 - val_loss: 0.0323\n",
      "Epoch 123/500\n",
      "2398/2398 [==============================] - 0s 53us/sample - loss: 0.0361 - val_loss: 0.0341\n",
      "Epoch 124/500\n",
      "2398/2398 [==============================] - 0s 57us/sample - loss: 0.0384 - val_loss: 0.0327\n",
      "Epoch 125/500\n",
      "2398/2398 [==============================] - 0s 56us/sample - loss: 0.0358 - val_loss: 0.0330\n",
      "Epoch 126/500\n",
      "2398/2398 [==============================] - 0s 54us/sample - loss: 0.0338 - val_loss: 0.0330\n",
      "Epoch 127/500\n",
      "2398/2398 [==============================] - 0s 56us/sample - loss: 0.0365 - val_loss: 0.0327\n",
      "Epoch 128/500\n",
      "2398/2398 [==============================] - 0s 59us/sample - loss: 0.0335 - val_loss: 0.0316\n",
      "Epoch 129/500\n",
      "2398/2398 [==============================] - 0s 57us/sample - loss: 0.0349 - val_loss: 0.0346\n",
      "Epoch 130/500\n",
      "2398/2398 [==============================] - 0s 55us/sample - loss: 0.0339 - val_loss: 0.0333\n",
      "Epoch 131/500\n",
      "2398/2398 [==============================] - 0s 54us/sample - loss: 0.0332 - val_loss: 0.0337\n",
      "Epoch 132/500\n",
      "2398/2398 [==============================] - 0s 53us/sample - loss: 0.0266 - val_loss: 0.0334\n",
      "Epoch 133/500\n",
      "2398/2398 [==============================] - 0s 57us/sample - loss: 0.0344 - val_loss: 0.0327\n",
      "Epoch 134/500\n",
      "2398/2398 [==============================] - 0s 54us/sample - loss: 0.0337 - val_loss: 0.0341\n",
      "Epoch 135/500\n",
      "2398/2398 [==============================] - 0s 57us/sample - loss: 0.0362 - val_loss: 0.0329\n",
      "Epoch 136/500\n",
      "2398/2398 [==============================] - 0s 56us/sample - loss: 0.0334 - val_loss: 0.0340\n",
      "Epoch 137/500\n",
      "2398/2398 [==============================] - 0s 54us/sample - loss: 0.0365 - val_loss: 0.0347\n",
      "Epoch 138/500\n",
      "2398/2398 [==============================] - 0s 64us/sample - loss: 0.0321 - val_loss: 0.0320\n",
      "Epoch 139/500\n",
      "2398/2398 [==============================] - 0s 71us/sample - loss: 0.0396 - val_loss: 0.0333\n",
      "Epoch 140/500\n",
      "2398/2398 [==============================] - 0s 72us/sample - loss: 0.0388 - val_loss: 0.0334\n",
      "Epoch 141/500\n",
      "2398/2398 [==============================] - 0s 60us/sample - loss: 0.0359 - val_loss: 0.0329\n",
      "Epoch 142/500\n",
      "2398/2398 [==============================] - 0s 61us/sample - loss: 0.0357 - val_loss: 0.0327\n",
      "Epoch 143/500\n",
      "2398/2398 [==============================] - 0s 63us/sample - loss: 0.0357 - val_loss: 0.0333\n",
      "Epoch 144/500\n",
      "2398/2398 [==============================] - 0s 57us/sample - loss: 0.0313 - val_loss: 0.0342\n",
      "Epoch 145/500\n",
      "2398/2398 [==============================] - 0s 54us/sample - loss: 0.0306 - val_loss: 0.0348\n",
      "Epoch 146/500\n",
      "2398/2398 [==============================] - 0s 53us/sample - loss: 0.0326 - val_loss: 0.0374\n",
      "Epoch 147/500\n",
      "2398/2398 [==============================] - 0s 58us/sample - loss: 0.0334 - val_loss: 0.0347\n",
      "Epoch 148/500\n",
      "2398/2398 [==============================] - 0s 64us/sample - loss: 0.0355 - val_loss: 0.0348\n",
      "Epoch 149/500\n",
      "2398/2398 [==============================] - 0s 64us/sample - loss: 0.0328 - val_loss: 0.0332\n",
      "Epoch 150/500\n",
      "2398/2398 [==============================] - 0s 62us/sample - loss: 0.0358 - val_loss: 0.0369\n",
      "Epoch 151/500\n",
      "2398/2398 [==============================] - 0s 63us/sample - loss: 0.0275 - val_loss: 0.0360\n",
      "Epoch 152/500\n",
      "2398/2398 [==============================] - 0s 54us/sample - loss: 0.0307 - val_loss: 0.0367\n",
      "Epoch 153/500\n",
      "2398/2398 [==============================] - 0s 52us/sample - loss: 0.0304 - val_loss: 0.0369\n",
      "Epoch 154/500\n",
      "2398/2398 [==============================] - 0s 57us/sample - loss: 0.0302 - val_loss: 0.0361\n",
      "Epoch 155/500\n",
      "2398/2398 [==============================] - 0s 61us/sample - loss: 0.0305 - val_loss: 0.0372\n",
      "Epoch 156/500\n",
      "2398/2398 [==============================] - 0s 60us/sample - loss: 0.0332 - val_loss: 0.0366\n",
      "Epoch 157/500\n",
      "2398/2398 [==============================] - 0s 59us/sample - loss: 0.0277 - val_loss: 0.0370\n",
      "Epoch 158/500\n",
      "2398/2398 [==============================] - 0s 62us/sample - loss: 0.0353 - val_loss: 0.0366\n",
      "Epoch 159/500\n",
      "2398/2398 [==============================] - 0s 55us/sample - loss: 0.0265 - val_loss: 0.0358\n",
      "Epoch 160/500\n",
      "2398/2398 [==============================] - 0s 84us/sample - loss: 0.0327 - val_loss: 0.0361\n",
      "Epoch 161/500\n",
      "2398/2398 [==============================] - 0s 68us/sample - loss: 0.0328 - val_loss: 0.0363\n",
      "Epoch 162/500\n",
      "2398/2398 [==============================] - 0s 77us/sample - loss: 0.0267 - val_loss: 0.0358\n",
      "Epoch 163/500\n",
      "2398/2398 [==============================] - 0s 72us/sample - loss: 0.0285 - val_loss: 0.0372\n",
      "Epoch 164/500\n",
      "2398/2398 [==============================] - 0s 66us/sample - loss: 0.0348 - val_loss: 0.0366\n",
      "Epoch 165/500\n",
      "2398/2398 [==============================] - 0s 67us/sample - loss: 0.0351 - val_loss: 0.0380\n",
      "Epoch 166/500\n",
      "2398/2398 [==============================] - 0s 65us/sample - loss: 0.0304 - val_loss: 0.0365\n",
      "Epoch 167/500\n",
      "2398/2398 [==============================] - 0s 61us/sample - loss: 0.0288 - val_loss: 0.0367\n",
      "Epoch 168/500\n",
      "2398/2398 [==============================] - 0s 66us/sample - loss: 0.0328 - val_loss: 0.0361\n",
      "Epoch 169/500\n",
      "2398/2398 [==============================] - 0s 56us/sample - loss: 0.0334 - val_loss: 0.0379\n",
      "Epoch 170/500\n",
      "2398/2398 [==============================] - 0s 62us/sample - loss: 0.0289 - val_loss: 0.0392\n",
      "Epoch 171/500\n",
      "2398/2398 [==============================] - 0s 67us/sample - loss: 0.0315 - val_loss: 0.0372\n",
      "Epoch 172/500\n",
      "2398/2398 [==============================] - 0s 55us/sample - loss: 0.0287 - val_loss: 0.0375\n",
      "Epoch 173/500\n",
      "2398/2398 [==============================] - 0s 61us/sample - loss: 0.0311 - val_loss: 0.0381\n",
      "Epoch 174/500\n",
      "2398/2398 [==============================] - 0s 54us/sample - loss: 0.0350 - val_loss: 0.0370\n",
      "Epoch 175/500\n",
      "2398/2398 [==============================] - 0s 58us/sample - loss: 0.0312 - val_loss: 0.0382\n",
      "Epoch 176/500\n",
      "2398/2398 [==============================] - 0s 63us/sample - loss: 0.0299 - val_loss: 0.0386\n",
      "Epoch 177/500\n",
      "2398/2398 [==============================] - 0s 65us/sample - loss: 0.0337 - val_loss: 0.0363\n",
      "Epoch 178/500\n",
      "2398/2398 [==============================] - 0s 63us/sample - loss: 0.0304 - val_loss: 0.0378\n",
      "Epoch 179/500\n",
      "2398/2398 [==============================] - 0s 59us/sample - loss: 0.0304 - val_loss: 0.0382\n",
      "Epoch 180/500\n",
      "2398/2398 [==============================] - 0s 190us/sample - loss: 0.0302 - val_loss: 0.0392\n",
      "Epoch 181/500\n",
      "2398/2398 [==============================] - 0s 99us/sample - loss: 0.0325 - val_loss: 0.0375\n",
      "Epoch 182/500\n",
      "2398/2398 [==============================] - 0s 103us/sample - loss: 0.0301 - val_loss: 0.0394\n",
      "Epoch 183/500\n",
      "2398/2398 [==============================] - 0s 111us/sample - loss: 0.0257 - val_loss: 0.0401\n",
      "Epoch 184/500\n",
      "2398/2398 [==============================] - 0s 121us/sample - loss: 0.0281 - val_loss: 0.0400\n",
      "Epoch 185/500\n",
      "2398/2398 [==============================] - 0s 94us/sample - loss: 0.0311 - val_loss: 0.0381\n",
      "Epoch 186/500\n",
      "2398/2398 [==============================] - 0s 58us/sample - loss: 0.0337 - val_loss: 0.0382\n",
      "Epoch 187/500\n",
      "2398/2398 [==============================] - 0s 60us/sample - loss: 0.0268 - val_loss: 0.0394\n",
      "Epoch 188/500\n",
      "2398/2398 [==============================] - 0s 60us/sample - loss: 0.0285 - val_loss: 0.0395\n",
      "Epoch 189/500\n",
      "2398/2398 [==============================] - 0s 62us/sample - loss: 0.0308 - val_loss: 0.0384\n",
      "Epoch 190/500\n",
      "2398/2398 [==============================] - 0s 60us/sample - loss: 0.0256 - val_loss: 0.0391\n",
      "Epoch 191/500\n",
      "2398/2398 [==============================] - 0s 65us/sample - loss: 0.0307 - val_loss: 0.0398\n",
      "Epoch 192/500\n",
      "2398/2398 [==============================] - 0s 60us/sample - loss: 0.0219 - val_loss: 0.0406\n",
      "Epoch 193/500\n",
      "2398/2398 [==============================] - 0s 55us/sample - loss: 0.0272 - val_loss: 0.0388\n",
      "Epoch 194/500\n",
      "2398/2398 [==============================] - 0s 61us/sample - loss: 0.0282 - val_loss: 0.0400\n",
      "Epoch 195/500\n",
      "2398/2398 [==============================] - 0s 61us/sample - loss: 0.0309 - val_loss: 0.0407\n",
      "Epoch 196/500\n",
      "2398/2398 [==============================] - 0s 55us/sample - loss: 0.0311 - val_loss: 0.0372\n",
      "Epoch 197/500\n",
      "2398/2398 [==============================] - 0s 55us/sample - loss: 0.0267 - val_loss: 0.0380\n",
      "Epoch 198/500\n",
      "2398/2398 [==============================] - 0s 57us/sample - loss: 0.0300 - val_loss: 0.0381\n",
      "Epoch 199/500\n",
      "2398/2398 [==============================] - 0s 59us/sample - loss: 0.0302 - val_loss: 0.0380\n",
      "Epoch 200/500\n",
      "2398/2398 [==============================] - 0s 62us/sample - loss: 0.0324 - val_loss: 0.0396\n",
      "Epoch 201/500\n",
      "2398/2398 [==============================] - 0s 60us/sample - loss: 0.0329 - val_loss: 0.0404\n",
      "Epoch 202/500\n",
      "2398/2398 [==============================] - ETA: 0s - loss: 0.031 - 0s 66us/sample - loss: 0.0326 - val_loss: 0.0412\n",
      "Epoch 203/500\n",
      "2398/2398 [==============================] - 0s 54us/sample - loss: 0.0312 - val_loss: 0.0392\n",
      "Epoch 204/500\n",
      "2398/2398 [==============================] - 0s 56us/sample - loss: 0.0304 - val_loss: 0.0394\n",
      "Epoch 205/500\n",
      "2398/2398 [==============================] - 0s 54us/sample - loss: 0.0277 - val_loss: 0.0400\n",
      "Epoch 206/500\n",
      "2398/2398 [==============================] - 0s 57us/sample - loss: 0.0292 - val_loss: 0.0394\n",
      "Epoch 207/500\n",
      "2398/2398 [==============================] - 0s 57us/sample - loss: 0.0274 - val_loss: 0.0394\n",
      "Epoch 208/500\n",
      "2398/2398 [==============================] - 0s 56us/sample - loss: 0.0310 - val_loss: 0.0407\n",
      "Epoch 209/500\n",
      "2398/2398 [==============================] - 0s 59us/sample - loss: 0.0211 - val_loss: 0.0424\n",
      "Epoch 210/500\n",
      "2398/2398 [==============================] - 0s 58us/sample - loss: 0.0247 - val_loss: 0.0428\n",
      "Epoch 211/500\n",
      "2398/2398 [==============================] - 0s 63us/sample - loss: 0.0253 - val_loss: 0.0434\n",
      "Epoch 212/500\n",
      "2398/2398 [==============================] - 0s 56us/sample - loss: 0.0275 - val_loss: 0.0423\n",
      "Epoch 213/500\n",
      "2398/2398 [==============================] - 0s 57us/sample - loss: 0.0317 - val_loss: 0.0393\n",
      "Epoch 214/500\n",
      "2398/2398 [==============================] - 0s 56us/sample - loss: 0.0334 - val_loss: 0.0412\n",
      "Epoch 215/500\n",
      "2398/2398 [==============================] - 0s 53us/sample - loss: 0.0243 - val_loss: 0.0418\n",
      "Epoch 216/500\n",
      "2398/2398 [==============================] - 0s 74us/sample - loss: 0.0254 - val_loss: 0.0443\n",
      "Epoch 217/500\n",
      "2398/2398 [==============================] - 0s 51us/sample - loss: 0.0289 - val_loss: 0.0422\n",
      "Epoch 218/500\n",
      "2398/2398 [==============================] - 0s 53us/sample - loss: 0.0268 - val_loss: 0.0432\n",
      "Epoch 219/500\n",
      "2398/2398 [==============================] - 0s 58us/sample - loss: 0.0275 - val_loss: 0.0420\n",
      "Epoch 220/500\n",
      "2398/2398 [==============================] - 0s 55us/sample - loss: 0.0248 - val_loss: 0.0441\n",
      "Epoch 221/500\n",
      "2398/2398 [==============================] - 0s 54us/sample - loss: 0.0280 - val_loss: 0.0460\n",
      "Epoch 222/500\n",
      "2398/2398 [==============================] - 0s 60us/sample - loss: 0.0262 - val_loss: 0.0428\n",
      "Epoch 223/500\n",
      "2398/2398 [==============================] - 0s 57us/sample - loss: 0.0262 - val_loss: 0.0422\n",
      "Epoch 224/500\n",
      "2398/2398 [==============================] - 0s 60us/sample - loss: 0.0281 - val_loss: 0.0403\n",
      "Epoch 225/500\n",
      "2398/2398 [==============================] - 0s 52us/sample - loss: 0.0268 - val_loss: 0.0430\n",
      "Epoch 226/500\n",
      "2398/2398 [==============================] - 0s 67us/sample - loss: 0.0299 - val_loss: 0.0428\n",
      "Epoch 227/500\n",
      "2398/2398 [==============================] - 0s 58us/sample - loss: 0.0233 - val_loss: 0.0411\n",
      "Epoch 228/500\n",
      "2398/2398 [==============================] - 0s 53us/sample - loss: 0.0270 - val_loss: 0.0414\n",
      "Epoch 229/500\n",
      "2398/2398 [==============================] - 0s 55us/sample - loss: 0.0299 - val_loss: 0.0429\n",
      "Epoch 230/500\n",
      "2398/2398 [==============================] - 0s 55us/sample - loss: 0.0276 - val_loss: 0.0443\n",
      "Epoch 231/500\n",
      "2398/2398 [==============================] - 0s 59us/sample - loss: 0.0337 - val_loss: 0.0460\n",
      "Epoch 232/500\n",
      "2398/2398 [==============================] - 0s 52us/sample - loss: 0.0229 - val_loss: 0.0453\n",
      "Epoch 233/500\n",
      "2398/2398 [==============================] - 0s 85us/sample - loss: 0.0213 - val_loss: 0.0468\n",
      "Epoch 234/500\n",
      "2398/2398 [==============================] - 0s 116us/sample - loss: 0.0277 - val_loss: 0.0466\n",
      "Epoch 235/500\n",
      "2398/2398 [==============================] - 0s 98us/sample - loss: 0.0276 - val_loss: 0.0433\n",
      "Epoch 236/500\n",
      "2398/2398 [==============================] - 0s 98us/sample - loss: 0.0233 - val_loss: 0.0438\n",
      "Epoch 237/500\n",
      "2398/2398 [==============================] - 0s 103us/sample - loss: 0.0258 - val_loss: 0.0456\n",
      "Epoch 238/500\n",
      "2398/2398 [==============================] - 0s 128us/sample - loss: 0.0268 - val_loss: 0.0461\n",
      "Epoch 239/500\n",
      "2398/2398 [==============================] - 0s 65us/sample - loss: 0.0289 - val_loss: 0.0455\n",
      "Epoch 240/500\n",
      "2398/2398 [==============================] - 0s 53us/sample - loss: 0.0282 - val_loss: 0.0451\n",
      "Epoch 241/500\n",
      "2398/2398 [==============================] - 0s 57us/sample - loss: 0.0306 - val_loss: 0.0415\n",
      "Epoch 242/500\n",
      "2398/2398 [==============================] - 0s 56us/sample - loss: 0.0263 - val_loss: 0.0433\n",
      "Epoch 243/500\n",
      "2398/2398 [==============================] - 0s 52us/sample - loss: 0.0229 - val_loss: 0.0453\n",
      "Epoch 244/500\n",
      "2398/2398 [==============================] - 0s 58us/sample - loss: 0.0327 - val_loss: 0.0433\n",
      "Epoch 245/500\n",
      "2398/2398 [==============================] - 0s 53us/sample - loss: 0.0218 - val_loss: 0.0466\n",
      "Epoch 246/500\n",
      "2398/2398 [==============================] - 0s 53us/sample - loss: 0.0279 - val_loss: 0.0431\n",
      "Epoch 247/500\n",
      "2398/2398 [==============================] - 0s 54us/sample - loss: 0.0279 - val_loss: 0.0421\n",
      "Epoch 248/500\n",
      "2398/2398 [==============================] - 0s 61us/sample - loss: 0.0266 - val_loss: 0.0453\n",
      "Epoch 249/500\n",
      "2398/2398 [==============================] - 0s 56us/sample - loss: 0.0279 - val_loss: 0.0450\n",
      "Epoch 250/500\n",
      "2398/2398 [==============================] - 0s 58us/sample - loss: 0.0243 - val_loss: 0.0425\n",
      "Epoch 251/500\n",
      "2398/2398 [==============================] - 0s 52us/sample - loss: 0.0265 - val_loss: 0.0431\n",
      "Epoch 252/500\n",
      "2398/2398 [==============================] - 0s 61us/sample - loss: 0.0291 - val_loss: 0.0429\n",
      "Epoch 253/500\n",
      "2398/2398 [==============================] - 0s 69us/sample - loss: 0.0327 - val_loss: 0.0439\n",
      "Epoch 254/500\n",
      "2398/2398 [==============================] - 0s 75us/sample - loss: 0.0270 - val_loss: 0.0429\n",
      "Epoch 255/500\n",
      "2398/2398 [==============================] - 0s 73us/sample - loss: 0.0253 - val_loss: 0.0457\n",
      "Epoch 256/500\n",
      "2398/2398 [==============================] - 0s 77us/sample - loss: 0.0253 - val_loss: 0.0456\n",
      "Epoch 257/500\n",
      "2398/2398 [==============================] - 0s 75us/sample - loss: 0.0254 - val_loss: 0.0461\n",
      "Epoch 258/500\n",
      "2398/2398 [==============================] - 0s 78us/sample - loss: 0.0307 - val_loss: 0.0451\n",
      "Epoch 259/500\n",
      "2398/2398 [==============================] - 0s 61us/sample - loss: 0.0276 - val_loss: 0.0458\n",
      "Epoch 260/500\n",
      "2398/2398 [==============================] - 0s 56us/sample - loss: 0.0265 - val_loss: 0.0463\n",
      "Epoch 261/500\n",
      "2398/2398 [==============================] - 0s 60us/sample - loss: 0.0270 - val_loss: 0.0485\n",
      "Epoch 262/500\n",
      "2398/2398 [==============================] - 0s 60us/sample - loss: 0.0247 - val_loss: 0.0452\n",
      "Epoch 263/500\n",
      "2398/2398 [==============================] - 0s 62us/sample - loss: 0.0296 - val_loss: 0.0419\n",
      "Epoch 264/500\n",
      "2398/2398 [==============================] - 0s 76us/sample - loss: 0.0283 - val_loss: 0.0448\n",
      "Epoch 265/500\n",
      "2398/2398 [==============================] - 0s 81us/sample - loss: 0.0258 - val_loss: 0.0469\n",
      "Epoch 266/500\n",
      "2398/2398 [==============================] - 0s 75us/sample - loss: 0.0218 - val_loss: 0.0525\n",
      "Epoch 267/500\n",
      "2398/2398 [==============================] - 0s 78us/sample - loss: 0.0303 - val_loss: 0.0461\n",
      "Epoch 268/500\n",
      "2398/2398 [==============================] - 0s 64us/sample - loss: 0.0260 - val_loss: 0.0465\n",
      "Epoch 269/500\n",
      "2398/2398 [==============================] - 0s 61us/sample - loss: 0.0261 - val_loss: 0.0457\n",
      "Epoch 270/500\n",
      "2398/2398 [==============================] - 0s 58us/sample - loss: 0.0285 - val_loss: 0.0438\n",
      "Epoch 271/500\n",
      "2398/2398 [==============================] - 0s 57us/sample - loss: 0.0248 - val_loss: 0.0462\n",
      "Epoch 272/500\n",
      "2398/2398 [==============================] - 0s 63us/sample - loss: 0.0224 - val_loss: 0.0463\n",
      "Epoch 273/500\n",
      "2398/2398 [==============================] - 0s 67us/sample - loss: 0.0281 - val_loss: 0.0466\n",
      "Epoch 274/500\n",
      "2398/2398 [==============================] - 0s 71us/sample - loss: 0.0228 - val_loss: 0.0469\n",
      "Epoch 275/500\n",
      "2398/2398 [==============================] - 0s 59us/sample - loss: 0.0251 - val_loss: 0.0439\n",
      "Epoch 276/500\n",
      "2398/2398 [==============================] - 0s 58us/sample - loss: 0.0253 - val_loss: 0.0473\n",
      "Epoch 277/500\n",
      "2398/2398 [==============================] - 0s 60us/sample - loss: 0.0238 - val_loss: 0.0488\n",
      "Epoch 278/500\n",
      "2398/2398 [==============================] - 0s 64us/sample - loss: 0.0245 - val_loss: 0.0474\n",
      "Epoch 279/500\n",
      "2398/2398 [==============================] - 0s 79us/sample - loss: 0.0275 - val_loss: 0.0485\n",
      "Epoch 280/500\n",
      "2398/2398 [==============================] - 0s 60us/sample - loss: 0.0228 - val_loss: 0.0504\n",
      "Epoch 281/500\n",
      "2398/2398 [==============================] - 0s 61us/sample - loss: 0.0242 - val_loss: 0.0470\n",
      "Epoch 282/500\n",
      "2398/2398 [==============================] - 0s 59us/sample - loss: 0.0266 - val_loss: 0.0442\n",
      "Epoch 283/500\n",
      "2398/2398 [==============================] - 0s 69us/sample - loss: 0.0254 - val_loss: 0.0470\n",
      "Epoch 284/500\n",
      "2398/2398 [==============================] - 0s 58us/sample - loss: 0.0298 - val_loss: 0.0471\n",
      "Epoch 285/500\n",
      "2398/2398 [==============================] - 0s 70us/sample - loss: 0.0299 - val_loss: 0.0479\n",
      "Epoch 286/500\n",
      "2398/2398 [==============================] - 0s 65us/sample - loss: 0.0271 - val_loss: 0.0473\n",
      "Epoch 287/500\n",
      "2398/2398 [==============================] - 0s 64us/sample - loss: 0.0234 - val_loss: 0.0460\n",
      "Epoch 288/500\n",
      "2398/2398 [==============================] - 0s 59us/sample - loss: 0.0252 - val_loss: 0.0489\n",
      "Epoch 289/500\n",
      "2398/2398 [==============================] - 0s 63us/sample - loss: 0.0228 - val_loss: 0.0494\n",
      "Epoch 290/500\n",
      "2398/2398 [==============================] - 0s 60us/sample - loss: 0.0237 - val_loss: 0.0471\n",
      "Epoch 291/500\n",
      "2398/2398 [==============================] - 0s 56us/sample - loss: 0.0256 - val_loss: 0.0444\n",
      "Epoch 292/500\n",
      "2398/2398 [==============================] - 0s 55us/sample - loss: 0.0241 - val_loss: 0.0488\n",
      "Epoch 293/500\n",
      "2398/2398 [==============================] - 0s 76us/sample - loss: 0.0279 - val_loss: 0.0512\n",
      "Epoch 294/500\n",
      "2398/2398 [==============================] - 0s 63us/sample - loss: 0.0278 - val_loss: 0.0521\n",
      "Epoch 295/500\n",
      "2398/2398 [==============================] - 0s 55us/sample - loss: 0.0266 - val_loss: 0.0473\n",
      "Epoch 296/500\n",
      "2398/2398 [==============================] - 0s 89us/sample - loss: 0.0220 - val_loss: 0.0496\n",
      "Epoch 297/500\n",
      "2398/2398 [==============================] - 0s 71us/sample - loss: 0.0270 - val_loss: 0.0510\n",
      "Epoch 298/500\n",
      "2398/2398 [==============================] - 0s 66us/sample - loss: 0.0246 - val_loss: 0.0508\n",
      "Epoch 299/500\n",
      "2398/2398 [==============================] - 0s 60us/sample - loss: 0.0258 - val_loss: 0.0479\n",
      "Epoch 300/500\n",
      "2398/2398 [==============================] - 0s 58us/sample - loss: 0.0240 - val_loss: 0.0489\n",
      "Epoch 301/500\n",
      "2398/2398 [==============================] - 0s 58us/sample - loss: 0.0243 - val_loss: 0.0495\n",
      "Epoch 302/500\n",
      "2398/2398 [==============================] - 0s 61us/sample - loss: 0.0261 - val_loss: 0.0508\n",
      "Epoch 303/500\n",
      "2398/2398 [==============================] - 0s 61us/sample - loss: 0.0266 - val_loss: 0.0505\n",
      "Epoch 304/500\n",
      "2398/2398 [==============================] - 0s 57us/sample - loss: 0.0210 - val_loss: 0.0539\n",
      "Epoch 305/500\n",
      "2398/2398 [==============================] - 0s 56us/sample - loss: 0.0295 - val_loss: 0.0502\n",
      "Epoch 306/500\n",
      "2398/2398 [==============================] - 0s 81us/sample - loss: 0.0234 - val_loss: 0.0465\n",
      "Epoch 307/500\n",
      "2398/2398 [==============================] - 0s 82us/sample - loss: 0.0208 - val_loss: 0.0489\n",
      "Epoch 308/500\n",
      "2398/2398 [==============================] - 0s 70us/sample - loss: 0.0322 - val_loss: 0.0444\n",
      "Epoch 309/500\n",
      "2398/2398 [==============================] - 0s 75us/sample - loss: 0.0254 - val_loss: 0.0458\n",
      "Epoch 310/500\n",
      "2398/2398 [==============================] - 0s 66us/sample - loss: 0.0266 - val_loss: 0.0488\n",
      "Epoch 311/500\n",
      "2398/2398 [==============================] - 0s 58us/sample - loss: 0.0241 - val_loss: 0.0510\n",
      "Epoch 312/500\n",
      "2398/2398 [==============================] - 0s 61us/sample - loss: 0.0247 - val_loss: 0.0479\n",
      "Epoch 313/500\n",
      "2398/2398 [==============================] - 0s 74us/sample - loss: 0.0224 - val_loss: 0.0495\n",
      "Epoch 314/500\n",
      "2398/2398 [==============================] - 0s 70us/sample - loss: 0.0219 - val_loss: 0.0493\n",
      "Epoch 315/500\n",
      "2398/2398 [==============================] - 0s 74us/sample - loss: 0.0212 - val_loss: 0.0530\n",
      "Epoch 316/500\n",
      "2398/2398 [==============================] - 0s 71us/sample - loss: 0.0278 - val_loss: 0.0461\n",
      "Epoch 317/500\n",
      "2398/2398 [==============================] - 0s 72us/sample - loss: 0.0276 - val_loss: 0.0414\n",
      "Epoch 318/500\n",
      "2398/2398 [==============================] - 0s 82us/sample - loss: 0.0261 - val_loss: 0.0481\n",
      "Epoch 319/500\n",
      "2398/2398 [==============================] - 0s 73us/sample - loss: 0.0261 - val_loss: 0.0482\n",
      "Epoch 320/500\n",
      "2398/2398 [==============================] - 0s 60us/sample - loss: 0.0241 - val_loss: 0.0466\n",
      "Epoch 321/500\n",
      "2398/2398 [==============================] - 0s 60us/sample - loss: 0.0246 - val_loss: 0.0462\n",
      "Epoch 322/500\n",
      "2398/2398 [==============================] - 0s 59us/sample - loss: 0.0263 - val_loss: 0.0462\n",
      "Epoch 323/500\n",
      "2398/2398 [==============================] - 0s 71us/sample - loss: 0.0258 - val_loss: 0.0459\n",
      "Epoch 324/500\n",
      "2398/2398 [==============================] - 0s 79us/sample - loss: 0.0213 - val_loss: 0.0468\n",
      "Epoch 325/500\n",
      "2398/2398 [==============================] - 0s 62us/sample - loss: 0.0220 - val_loss: 0.0493\n",
      "Epoch 326/500\n",
      "2398/2398 [==============================] - 0s 61us/sample - loss: 0.0247 - val_loss: 0.0480\n",
      "Epoch 327/500\n",
      "2398/2398 [==============================] - 0s 84us/sample - loss: 0.0214 - val_loss: 0.0488\n",
      "Epoch 328/500\n",
      "2398/2398 [==============================] - 0s 66us/sample - loss: 0.0213 - val_loss: 0.0510\n",
      "Epoch 329/500\n",
      "2398/2398 [==============================] - 0s 90us/sample - loss: 0.0309 - val_loss: 0.0458\n",
      "Epoch 330/500\n",
      "2398/2398 [==============================] - 0s 79us/sample - loss: 0.0246 - val_loss: 0.0477\n",
      "Epoch 331/500\n",
      "2398/2398 [==============================] - 0s 67us/sample - loss: 0.0209 - val_loss: 0.0515\n",
      "Epoch 332/500\n",
      "2398/2398 [==============================] - 0s 63us/sample - loss: 0.0282 - val_loss: 0.0513\n",
      "Epoch 333/500\n",
      "2398/2398 [==============================] - 0s 64us/sample - loss: 0.0248 - val_loss: 0.0497\n",
      "Epoch 334/500\n",
      "2398/2398 [==============================] - 0s 70us/sample - loss: 0.0253 - val_loss: 0.0500\n",
      "Epoch 335/500\n",
      "2398/2398 [==============================] - 0s 79us/sample - loss: 0.0230 - val_loss: 0.0489\n",
      "Epoch 336/500\n",
      "2398/2398 [==============================] - 0s 65us/sample - loss: 0.0229 - val_loss: 0.0508\n",
      "Epoch 337/500\n",
      "2398/2398 [==============================] - 0s 79us/sample - loss: 0.0231 - val_loss: 0.0495\n",
      "Epoch 338/500\n",
      "2398/2398 [==============================] - 0s 66us/sample - loss: 0.0280 - val_loss: 0.0512\n",
      "Epoch 339/500\n",
      "2398/2398 [==============================] - 0s 61us/sample - loss: 0.0246 - val_loss: 0.0488\n",
      "Epoch 340/500\n",
      "2398/2398 [==============================] - 0s 57us/sample - loss: 0.0189 - val_loss: 0.0506\n",
      "Epoch 341/500\n",
      "2398/2398 [==============================] - 0s 58us/sample - loss: 0.0249 - val_loss: 0.0521\n",
      "Epoch 342/500\n",
      "2398/2398 [==============================] - 0s 71us/sample - loss: 0.0227 - val_loss: 0.0515\n",
      "Epoch 343/500\n",
      "2398/2398 [==============================] - 0s 69us/sample - loss: 0.0277 - val_loss: 0.0518\n",
      "Epoch 344/500\n",
      "2398/2398 [==============================] - 0s 68us/sample - loss: 0.0220 - val_loss: 0.0533\n",
      "Epoch 345/500\n",
      "2398/2398 [==============================] - 0s 62us/sample - loss: 0.0257 - val_loss: 0.0500\n",
      "Epoch 346/500\n",
      "2398/2398 [==============================] - 0s 60us/sample - loss: 0.0223 - val_loss: 0.0479\n",
      "Epoch 347/500\n",
      "2398/2398 [==============================] - 0s 60us/sample - loss: 0.0244 - val_loss: 0.0508\n",
      "Epoch 348/500\n",
      "2398/2398 [==============================] - 0s 57us/sample - loss: 0.0215 - val_loss: 0.0546\n",
      "Epoch 349/500\n",
      "2398/2398 [==============================] - 0s 56us/sample - loss: 0.0269 - val_loss: 0.0474\n",
      "Epoch 350/500\n",
      "2398/2398 [==============================] - 0s 53us/sample - loss: 0.0227 - val_loss: 0.0474\n",
      "Epoch 351/500\n",
      "2398/2398 [==============================] - 0s 54us/sample - loss: 0.0247 - val_loss: 0.0492\n",
      "Epoch 352/500\n",
      "2398/2398 [==============================] - 0s 102us/sample - loss: 0.0198 - val_loss: 0.0506\n",
      "Epoch 353/500\n",
      "2398/2398 [==============================] - 0s 86us/sample - loss: 0.0222 - val_loss: 0.0529\n",
      "Epoch 354/500\n",
      "2398/2398 [==============================] - 0s 61us/sample - loss: 0.0238 - val_loss: 0.0503\n",
      "Epoch 355/500\n",
      "2398/2398 [==============================] - 0s 68us/sample - loss: 0.0251 - val_loss: 0.0490\n",
      "Epoch 356/500\n",
      "2398/2398 [==============================] - 0s 61us/sample - loss: 0.0244 - val_loss: 0.0510\n",
      "Epoch 357/500\n",
      "2398/2398 [==============================] - 0s 55us/sample - loss: 0.0205 - val_loss: 0.0523\n",
      "Epoch 358/500\n",
      "2398/2398 [==============================] - 0s 58us/sample - loss: 0.0285 - val_loss: 0.0486\n",
      "Epoch 359/500\n",
      "2398/2398 [==============================] - 0s 55us/sample - loss: 0.0200 - val_loss: 0.0494\n",
      "Epoch 360/500\n",
      "2398/2398 [==============================] - 0s 58us/sample - loss: 0.0295 - val_loss: 0.0469\n",
      "Epoch 361/500\n",
      "2398/2398 [==============================] - 0s 55us/sample - loss: 0.0284 - val_loss: 0.0527\n",
      "Epoch 362/500\n",
      "2398/2398 [==============================] - 0s 88us/sample - loss: 0.0203 - val_loss: 0.0515\n",
      "Epoch 363/500\n",
      "2398/2398 [==============================] - 0s 58us/sample - loss: 0.0276 - val_loss: 0.0488\n",
      "Epoch 364/500\n",
      "2398/2398 [==============================] - 0s 53us/sample - loss: 0.0222 - val_loss: 0.0528\n",
      "Epoch 365/500\n",
      "2398/2398 [==============================] - 0s 66us/sample - loss: 0.0218 - val_loss: 0.0546\n",
      "Epoch 366/500\n",
      "2398/2398 [==============================] - 0s 62us/sample - loss: 0.0252 - val_loss: 0.0575\n",
      "Epoch 367/500\n",
      "2398/2398 [==============================] - 0s 66us/sample - loss: 0.0235 - val_loss: 0.0511\n",
      "Epoch 368/500\n",
      "2398/2398 [==============================] - 0s 68us/sample - loss: 0.0241 - val_loss: 0.0496\n",
      "Epoch 369/500\n",
      "2398/2398 [==============================] - 0s 58us/sample - loss: 0.0251 - val_loss: 0.0527\n",
      "Epoch 370/500\n",
      "2398/2398 [==============================] - 0s 61us/sample - loss: 0.0231 - val_loss: 0.0544\n",
      "Epoch 371/500\n",
      "2398/2398 [==============================] - 0s 64us/sample - loss: 0.0213 - val_loss: 0.0523\n",
      "Epoch 372/500\n",
      "2398/2398 [==============================] - 0s 70us/sample - loss: 0.0208 - val_loss: 0.0568\n",
      "Epoch 373/500\n",
      "2398/2398 [==============================] - 0s 59us/sample - loss: 0.0223 - val_loss: 0.0547\n",
      "Epoch 374/500\n",
      "2398/2398 [==============================] - 0s 62us/sample - loss: 0.0240 - val_loss: 0.0532\n",
      "Epoch 375/500\n",
      "2398/2398 [==============================] - 0s 77us/sample - loss: 0.0255 - val_loss: 0.0514\n",
      "Epoch 376/500\n",
      "2398/2398 [==============================] - 0s 57us/sample - loss: 0.0222 - val_loss: 0.0551\n",
      "Epoch 377/500\n",
      "2398/2398 [==============================] - 0s 60us/sample - loss: 0.0193 - val_loss: 0.0550\n",
      "Epoch 378/500\n",
      "2398/2398 [==============================] - 0s 62us/sample - loss: 0.0179 - val_loss: 0.0551\n",
      "Epoch 379/500\n",
      "2398/2398 [==============================] - 0s 67us/sample - loss: 0.0211 - val_loss: 0.0548\n",
      "Epoch 380/500\n",
      "2398/2398 [==============================] - 0s 78us/sample - loss: 0.0165 - val_loss: 0.0574\n",
      "Epoch 381/500\n",
      "2398/2398 [==============================] - 0s 66us/sample - loss: 0.0201 - val_loss: 0.0565\n",
      "Epoch 382/500\n",
      "2398/2398 [==============================] - 0s 63us/sample - loss: 0.0225 - val_loss: 0.0505\n",
      "Epoch 383/500\n",
      "2398/2398 [==============================] - 0s 54us/sample - loss: 0.0253 - val_loss: 0.0523\n",
      "Epoch 384/500\n",
      "2398/2398 [==============================] - 0s 58us/sample - loss: 0.0229 - val_loss: 0.0505\n",
      "Epoch 385/500\n",
      "2398/2398 [==============================] - 0s 63us/sample - loss: 0.0271 - val_loss: 0.0509\n",
      "Epoch 386/500\n",
      "2398/2398 [==============================] - 0s 60us/sample - loss: 0.0209 - val_loss: 0.0520\n",
      "Epoch 387/500\n",
      "2398/2398 [==============================] - 0s 55us/sample - loss: 0.0235 - val_loss: 0.0508\n",
      "Epoch 388/500\n",
      "2398/2398 [==============================] - 0s 63us/sample - loss: 0.0234 - val_loss: 0.0508\n",
      "Epoch 389/500\n",
      "2398/2398 [==============================] - 0s 63us/sample - loss: 0.0222 - val_loss: 0.0552\n",
      "Epoch 390/500\n",
      "2398/2398 [==============================] - 0s 62us/sample - loss: 0.0229 - val_loss: 0.0535\n",
      "Epoch 391/500\n",
      "2398/2398 [==============================] - 0s 67us/sample - loss: 0.0247 - val_loss: 0.0511\n",
      "Epoch 392/500\n",
      "2398/2398 [==============================] - 0s 72us/sample - loss: 0.0206 - val_loss: 0.0537\n",
      "Epoch 393/500\n",
      "2398/2398 [==============================] - 0s 67us/sample - loss: 0.0269 - val_loss: 0.0553\n",
      "Epoch 394/500\n",
      "2398/2398 [==============================] - 0s 57us/sample - loss: 0.0198 - val_loss: 0.0559\n",
      "Epoch 395/500\n",
      "2398/2398 [==============================] - 0s 62us/sample - loss: 0.0293 - val_loss: 0.0529\n",
      "Epoch 396/500\n",
      "2398/2398 [==============================] - 0s 59us/sample - loss: 0.0266 - val_loss: 0.0530\n",
      "Epoch 397/500\n",
      "2398/2398 [==============================] - 0s 60us/sample - loss: 0.0230 - val_loss: 0.0553\n",
      "Epoch 398/500\n",
      "2398/2398 [==============================] - 0s 59us/sample - loss: 0.0272 - val_loss: 0.0528\n",
      "Epoch 399/500\n",
      "2398/2398 [==============================] - 0s 59us/sample - loss: 0.0217 - val_loss: 0.0534\n",
      "Epoch 400/500\n",
      "2398/2398 [==============================] - 0s 59us/sample - loss: 0.0205 - val_loss: 0.0550\n",
      "Epoch 401/500\n",
      "2398/2398 [==============================] - 0s 57us/sample - loss: 0.0195 - val_loss: 0.0547\n",
      "Epoch 402/500\n",
      "2398/2398 [==============================] - 0s 65us/sample - loss: 0.0210 - val_loss: 0.0543\n",
      "Epoch 403/500\n",
      "2398/2398 [==============================] - 0s 55us/sample - loss: 0.0233 - val_loss: 0.0491\n",
      "Epoch 404/500\n",
      "2398/2398 [==============================] - 0s 53us/sample - loss: 0.0203 - val_loss: 0.0515\n",
      "Epoch 405/500\n",
      "2398/2398 [==============================] - 0s 55us/sample - loss: 0.0259 - val_loss: 0.0505\n",
      "Epoch 406/500\n",
      "2398/2398 [==============================] - 0s 51us/sample - loss: 0.0220 - val_loss: 0.0559\n",
      "Epoch 407/500\n",
      "2398/2398 [==============================] - 0s 55us/sample - loss: 0.0213 - val_loss: 0.0566\n",
      "Epoch 408/500\n",
      "2398/2398 [==============================] - 0s 53us/sample - loss: 0.0202 - val_loss: 0.0548\n",
      "Epoch 409/500\n",
      "2398/2398 [==============================] - 0s 57us/sample - loss: 0.0260 - val_loss: 0.0534\n",
      "Epoch 410/500\n",
      "2398/2398 [==============================] - 0s 53us/sample - loss: 0.0222 - val_loss: 0.0578\n",
      "Epoch 411/500\n",
      "2398/2398 [==============================] - 0s 55us/sample - loss: 0.0196 - val_loss: 0.0543\n",
      "Epoch 412/500\n",
      "2398/2398 [==============================] - 0s 57us/sample - loss: 0.0230 - val_loss: 0.0562\n",
      "Epoch 413/500\n",
      "2398/2398 [==============================] - 0s 56us/sample - loss: 0.0238 - val_loss: 0.0572\n",
      "Epoch 414/500\n",
      "2398/2398 [==============================] - 0s 53us/sample - loss: 0.0198 - val_loss: 0.0569\n",
      "Epoch 415/500\n",
      "2398/2398 [==============================] - 0s 52us/sample - loss: 0.0180 - val_loss: 0.0579\n",
      "Epoch 416/500\n",
      "2398/2398 [==============================] - 0s 56us/sample - loss: 0.0208 - val_loss: 0.0576\n",
      "Epoch 417/500\n",
      "2398/2398 [==============================] - 0s 79us/sample - loss: 0.0222 - val_loss: 0.0542\n",
      "Epoch 418/500\n",
      "2398/2398 [==============================] - 0s 73us/sample - loss: 0.0247 - val_loss: 0.0534\n",
      "Epoch 419/500\n",
      "2398/2398 [==============================] - 0s 57us/sample - loss: 0.0261 - val_loss: 0.0523\n",
      "Epoch 420/500\n",
      "2398/2398 [==============================] - 0s 69us/sample - loss: 0.0249 - val_loss: 0.0501\n",
      "Epoch 421/500\n",
      "2398/2398 [==============================] - 0s 72us/sample - loss: 0.0203 - val_loss: 0.0485\n",
      "Epoch 422/500\n",
      "2398/2398 [==============================] - 0s 57us/sample - loss: 0.0288 - val_loss: 0.0464\n",
      "Epoch 423/500\n",
      "2398/2398 [==============================] - 0s 83us/sample - loss: 0.0324 - val_loss: 0.0525\n",
      "Epoch 424/500\n",
      "2398/2398 [==============================] - 0s 55us/sample - loss: 0.0252 - val_loss: 0.0538\n",
      "Epoch 425/500\n",
      "2398/2398 [==============================] - 0s 67us/sample - loss: 0.0230 - val_loss: 0.0535\n",
      "Epoch 426/500\n",
      "2398/2398 [==============================] - 0s 75us/sample - loss: 0.0208 - val_loss: 0.0564\n",
      "Epoch 427/500\n",
      "2398/2398 [==============================] - 0s 60us/sample - loss: 0.0202 - val_loss: 0.0574\n",
      "Epoch 428/500\n",
      "2398/2398 [==============================] - 0s 66us/sample - loss: 0.0281 - val_loss: 0.0494\n",
      "Epoch 429/500\n",
      "2398/2398 [==============================] - 0s 78us/sample - loss: 0.0282 - val_loss: 0.0501\n",
      "Epoch 430/500\n",
      "2398/2398 [==============================] - 0s 58us/sample - loss: 0.0195 - val_loss: 0.0529\n",
      "Epoch 431/500\n",
      "2398/2398 [==============================] - 0s 64us/sample - loss: 0.0205 - val_loss: 0.0487\n",
      "Epoch 432/500\n",
      "2398/2398 [==============================] - 0s 80us/sample - loss: 0.0261 - val_loss: 0.0544\n",
      "Epoch 433/500\n",
      "2398/2398 [==============================] - 0s 63us/sample - loss: 0.0195 - val_loss: 0.0549\n",
      "Epoch 434/500\n",
      "2398/2398 [==============================] - 0s 57us/sample - loss: 0.0200 - val_loss: 0.0572\n",
      "Epoch 435/500\n",
      "2398/2398 [==============================] - 0s 80us/sample - loss: 0.0217 - val_loss: 0.0577\n",
      "Epoch 436/500\n",
      "2398/2398 [==============================] - 0s 70us/sample - loss: 0.0230 - val_loss: 0.0529\n",
      "Epoch 437/500\n",
      "2398/2398 [==============================] - 0s 76us/sample - loss: 0.0290 - val_loss: 0.0519\n",
      "Epoch 438/500\n",
      "2398/2398 [==============================] - 0s 67us/sample - loss: 0.0176 - val_loss: 0.0525\n",
      "Epoch 439/500\n",
      "2398/2398 [==============================] - 0s 77us/sample - loss: 0.0225 - val_loss: 0.0519\n",
      "Epoch 440/500\n",
      "2398/2398 [==============================] - 0s 76us/sample - loss: 0.0187 - val_loss: 0.0553\n",
      "Epoch 441/500\n",
      "2398/2398 [==============================] - 0s 77us/sample - loss: 0.0200 - val_loss: 0.0571\n",
      "Epoch 442/500\n",
      "2398/2398 [==============================] - 0s 117us/sample - loss: 0.0220 - val_loss: 0.0599\n",
      "Epoch 443/500\n",
      "2398/2398 [==============================] - 0s 148us/sample - loss: 0.0203 - val_loss: 0.0587\n",
      "Epoch 444/500\n",
      "2398/2398 [==============================] - 0s 70us/sample - loss: 0.0266 - val_loss: 0.0610\n",
      "Epoch 445/500\n",
      "2398/2398 [==============================] - 0s 64us/sample - loss: 0.0248 - val_loss: 0.0581\n",
      "Epoch 446/500\n",
      "2398/2398 [==============================] - 0s 60us/sample - loss: 0.0173 - val_loss: 0.0563\n",
      "Epoch 447/500\n",
      "2398/2398 [==============================] - 0s 59us/sample - loss: 0.0245 - val_loss: 0.0534\n",
      "Epoch 448/500\n",
      "2398/2398 [==============================] - 0s 60us/sample - loss: 0.0238 - val_loss: 0.0557\n",
      "Epoch 449/500\n",
      "2398/2398 [==============================] - 0s 82us/sample - loss: 0.0216 - val_loss: 0.0560\n",
      "Epoch 450/500\n",
      "2398/2398 [==============================] - 0s 60us/sample - loss: 0.0201 - val_loss: 0.0529\n",
      "Epoch 451/500\n",
      "2398/2398 [==============================] - 0s 80us/sample - loss: 0.0184 - val_loss: 0.0536\n",
      "Epoch 452/500\n",
      "2398/2398 [==============================] - 0s 76us/sample - loss: 0.0211 - val_loss: 0.0589\n",
      "Epoch 453/500\n",
      "2398/2398 [==============================] - 0s 73us/sample - loss: 0.0245 - val_loss: 0.0527\n",
      "Epoch 454/500\n",
      "2398/2398 [==============================] - 0s 73us/sample - loss: 0.0198 - val_loss: 0.0565\n",
      "Epoch 455/500\n",
      "2398/2398 [==============================] - 0s 69us/sample - loss: 0.0228 - val_loss: 0.0513\n",
      "Epoch 456/500\n",
      "2398/2398 [==============================] - 0s 70us/sample - loss: 0.0230 - val_loss: 0.0550\n",
      "Epoch 457/500\n",
      "2398/2398 [==============================] - 0s 79us/sample - loss: 0.0237 - val_loss: 0.0589\n",
      "Epoch 458/500\n",
      "2398/2398 [==============================] - 0s 72us/sample - loss: 0.0192 - val_loss: 0.0551\n",
      "Epoch 459/500\n",
      "2398/2398 [==============================] - 0s 85us/sample - loss: 0.0239 - val_loss: 0.0533\n",
      "Epoch 460/500\n",
      "2398/2398 [==============================] - 0s 78us/sample - loss: 0.0227 - val_loss: 0.0577\n",
      "Epoch 461/500\n",
      "2398/2398 [==============================] - 0s 71us/sample - loss: 0.0236 - val_loss: 0.0612\n",
      "Epoch 462/500\n",
      "2398/2398 [==============================] - 0s 65us/sample - loss: 0.0173 - val_loss: 0.0619\n",
      "Epoch 463/500\n",
      "2398/2398 [==============================] - 0s 61us/sample - loss: 0.0209 - val_loss: 0.0555\n",
      "Epoch 464/500\n",
      "2398/2398 [==============================] - 0s 59us/sample - loss: 0.0193 - val_loss: 0.0556\n",
      "Epoch 465/500\n",
      "2398/2398 [==============================] - 0s 60us/sample - loss: 0.0229 - val_loss: 0.0581\n",
      "Epoch 466/500\n",
      "2398/2398 [==============================] - 0s 66us/sample - loss: 0.0229 - val_loss: 0.0561\n",
      "Epoch 467/500\n",
      "2398/2398 [==============================] - 0s 59us/sample - loss: 0.0206 - val_loss: 0.0592\n",
      "Epoch 468/500\n",
      "2398/2398 [==============================] - 0s 65us/sample - loss: 0.0205 - val_loss: 0.0563\n",
      "Epoch 469/500\n",
      "2398/2398 [==============================] - 0s 71us/sample - loss: 0.0180 - val_loss: 0.0627\n",
      "Epoch 470/500\n",
      "2398/2398 [==============================] - 0s 60us/sample - loss: 0.0221 - val_loss: 0.0552\n",
      "Epoch 471/500\n",
      "2398/2398 [==============================] - 0s 79us/sample - loss: 0.0233 - val_loss: 0.0550\n",
      "Epoch 472/500\n",
      "2398/2398 [==============================] - 0s 65us/sample - loss: 0.0215 - val_loss: 0.0552\n",
      "Epoch 473/500\n",
      "2398/2398 [==============================] - 0s 84us/sample - loss: 0.0196 - val_loss: 0.0560\n",
      "Epoch 474/500\n",
      "2398/2398 [==============================] - 0s 63us/sample - loss: 0.0198 - val_loss: 0.0589\n",
      "Epoch 475/500\n",
      "2398/2398 [==============================] - 0s 61us/sample - loss: 0.0220 - val_loss: 0.0601\n",
      "Epoch 476/500\n",
      "2398/2398 [==============================] - 0s 57us/sample - loss: 0.0258 - val_loss: 0.0611\n",
      "Epoch 477/500\n",
      "2398/2398 [==============================] - 0s 59us/sample - loss: 0.0196 - val_loss: 0.0634\n",
      "Epoch 478/500\n",
      "2398/2398 [==============================] - 0s 68us/sample - loss: 0.0227 - val_loss: 0.0590\n",
      "Epoch 479/500\n",
      "2398/2398 [==============================] - 0s 67us/sample - loss: 0.0286 - val_loss: 0.0558\n",
      "Epoch 480/500\n",
      "2398/2398 [==============================] - 0s 59us/sample - loss: 0.0244 - val_loss: 0.0584\n",
      "Epoch 481/500\n",
      "2398/2398 [==============================] - 0s 84us/sample - loss: 0.0219 - val_loss: 0.0609\n",
      "Epoch 482/500\n",
      "2398/2398 [==============================] - 0s 80us/sample - loss: 0.0159 - val_loss: 0.0626\n",
      "Epoch 483/500\n",
      "2398/2398 [==============================] - 0s 68us/sample - loss: 0.0223 - val_loss: 0.0598\n",
      "Epoch 484/500\n",
      "2398/2398 [==============================] - 0s 60us/sample - loss: 0.0213 - val_loss: 0.0567\n",
      "Epoch 485/500\n",
      "2398/2398 [==============================] - 0s 65us/sample - loss: 0.0209 - val_loss: 0.0604\n",
      "Epoch 486/500\n",
      "2398/2398 [==============================] - 0s 59us/sample - loss: 0.0220 - val_loss: 0.0588\n",
      "Epoch 487/500\n",
      "2398/2398 [==============================] - 0s 63us/sample - loss: 0.0236 - val_loss: 0.0473\n",
      "Epoch 488/500\n",
      "2398/2398 [==============================] - 0s 64us/sample - loss: 0.0262 - val_loss: 0.0491\n",
      "Epoch 489/500\n",
      "2398/2398 [==============================] - 0s 56us/sample - loss: 0.0246 - val_loss: 0.0578\n",
      "Epoch 490/500\n",
      "2398/2398 [==============================] - 0s 56us/sample - loss: 0.0283 - val_loss: 0.0603\n",
      "Epoch 491/500\n",
      "2398/2398 [==============================] - 0s 53us/sample - loss: 0.0228 - val_loss: 0.0636\n",
      "Epoch 492/500\n",
      "2398/2398 [==============================] - 0s 60us/sample - loss: 0.0215 - val_loss: 0.0626\n",
      "Epoch 493/500\n",
      "2398/2398 [==============================] - 0s 57us/sample - loss: 0.0255 - val_loss: 0.0501\n",
      "Epoch 494/500\n",
      "2398/2398 [==============================] - 0s 61us/sample - loss: 0.0225 - val_loss: 0.0525\n",
      "Epoch 495/500\n",
      "2398/2398 [==============================] - 0s 60us/sample - loss: 0.0224 - val_loss: 0.0587\n",
      "Epoch 496/500\n",
      "2398/2398 [==============================] - 0s 71us/sample - loss: 0.0195 - val_loss: 0.0603\n",
      "Epoch 497/500\n",
      "2398/2398 [==============================] - 0s 65us/sample - loss: 0.0218 - val_loss: 0.0616\n",
      "Epoch 498/500\n",
      "2398/2398 [==============================] - 0s 63us/sample - loss: 0.0268 - val_loss: 0.0534\n",
      "Epoch 499/500\n",
      "2398/2398 [==============================] - 0s 62us/sample - loss: 0.0204 - val_loss: 0.0560\n",
      "Epoch 500/500\n",
      "2398/2398 [==============================] - 0s 69us/sample - loss: 0.0227 - val_loss: 0.0609\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<tensorflow.python.keras.callbacks.History at 0x1a3ba6b190>"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.fit(x=X_train, y=y_train, \n",
    "          validation_data=(X_test,y_test),\n",
    "          epochs=500)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x1a3c06a3d0>"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD4CAYAAAD8Zh1EAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3deXwU9f3H8ddnd3NBLo4EQgKEG7mRgOKBt+KFN+KJd61Fra3Wq7UebbXSattf0WpbRatWUDxQEbSCInIG5D7DHQLkAHKQe/f7++O7STbJAgskhAmf5+PBI9mZyex3NuE9n/nOd2bEGINSSinnczV1A5RSSjUMDXSllGomNNCVUqqZ0EBXSqlmQgNdKaWaCU9TvXHbtm1NampqU729Uko50uLFi3ONMQnB5jVZoKemppKent5Ub6+UUo4kIlsPNE+7XJRSqpnQQFdKqWZCA10ppZqJJutDV0qdmCoqKsjMzKS0tLSpm3Jci4yMJCUlhbCwsJB/RgNdKXVMZWZmEhMTQ2pqKiLS1M05LhljyMvLIzMzky5duoT8c9rlopQ6pkpLS2nTpo2G+UGICG3atDnsoxgNdKXUMadhfmhH8hk5LtAXbdnDn79aR4XX19RNUUqp44rjAn3J1r3838wMDXSl1BGLjo5u6iY0CscFuttlD0MqffpgDqWUCuTYQPdpoCuljpIxhkceeYR+/frRv39/Jk2aBMDOnTsZMWIEgwYNol+/fnz//fd4vV5uu+226mVffvnlJm59fY4btqgVulLNxzOfrWJ1VkGDrrNPh1h+e3nfkJb96KOPWLp0KcuWLSM3N5ehQ4cyYsQI3nvvPS666CKefPJJvF4vxcXFLF26lB07drBy5UoA9u3b16DtbgiOq9BdohW6UqphzJkzhxtuuAG32027du0466yzWLRoEUOHDuXNN9/k6aefZsWKFcTExNC1a1c2bdrE/fffz/Tp04mNjW3q5tfjuArd46/Qvfpwa6UcL9RKurGYA+TIiBEjmD17Nl988QW33HILjzzyCLfeeivLli1jxowZTJgwgcmTJ/PGG28c4xYfnPMq9KouF68GulLq6IwYMYJJkybh9XrJyclh9uzZDBs2jK1bt5KYmMjdd9/NnXfeyZIlS8jNzcXn83HNNdfw3HPPsWTJkqZufj2OrdB9WqErpY7SVVddxbx58xg4cCAiwosvvkj79u156623GD9+PGFhYURHR/P222+zY8cObr/9dnw+O2T6+eefb+LW1+e4QNeTokqpo1VUVATYqzHHjx/P+PHja80fO3YsY8eOrfdzx2NVHshxXS46bFEppYJzXqCLVuhKKRWM8wK9apSLBrpSStWiga6UUs2E4wLdpePQlVIqqJACXURGisg6EckQkccOsMxoEVktIqtE5L2GbWYNj54UVUqpoA45bFFE3MAE4AIgE1gkIlONMasDlukBPA6cbozZKyKJjdVgPSmqlFLBhVKhDwMyjDGbjDHlwPvAFXWWuRuYYIzZC2CMyW7YZtbQYYtKqWPpYPdO37JlC/369TuGrTm4UAI9Gdge8DrTPy1QT6CniPwgIvNFZGSwFYnIPSKSLiLpOTk5R9RgvbBIKaWCC+VK0WAPtqubph6gB3A2kAJ8LyL9jDG17i9pjHkdeB0gLS3tiBLZrSdFlWo+vnwMdq1o2HW27w8Xv3DA2Y8++iidO3fmvvvuA+Dpp59GRJg9ezZ79+6loqKC3/3ud1xxRd2OiIMrLS3lpz/9Kenp6Xg8Hl566SXOOeccVq1axe233055eTk+n48pU6bQoUMHRo8eTWZmJl6vl9/85jdcf/31R7XZEFqgZwIdA16nAFlBlplvjKkANovIOmzALzrqFtZRHeh6cy6l1BEYM2YMP//5z6sDffLkyUyfPp2HHnqI2NhYcnNzOfXUUxk1atRhPah5woQJAKxYsYK1a9dy4YUXsn79ev7xj3/w4IMPctNNN1FeXo7X62XatGl06NCBL774AoD8/PwG2bZQAn0R0ENEugA7gDHAjXWW+QS4AZgoIm2xXTCbGqSFdVTdD10rdKWagYNU0o1l8ODBZGdnk5WVRU5ODq1atSIpKYmHHnqI2bNn43K52LFjB7t376Z9+/Yhr3fOnDncf//9APTu3ZvOnTuzfv16hg8fzu9//3syMzO5+uqr6dGjB/379+fhhx/m0Ucf5bLLLuPMM89skG07ZB+6MaYSGAfMANYAk40xq0TkWREZ5V9sBpAnIquBWcAjxpi8BmlhHR63XliklDo61157LR9++CGTJk1izJgxvPvuu+Tk5LB48WKWLl1Ku3btKC0tPax1Huje6jfeeCNTp04lKiqKiy66iJkzZ9KzZ08WL15M//79efzxx3n22WcbYrNCu9uiMWYaMK3OtKcCvjfAL/z/GlXVsEUNdKXUkRozZgx33303ubm5fPfdd0yePJnExETCwsKYNWsWW7duPex1jhgxgnfffZdzzz2X9evXs23bNnr16sWmTZvo2rUrDzzwAJs2bWL58uX07t2b1q1bc/PNNxMdHc3EiRMbZLsce/tcvR+6UupI9e3bl8LCQpKTk0lKSuKmm27i8ssvJy0tjUGDBtG7d+/DXud9993HvffeS//+/fF4PEycOJGIiAgmTZrEO++8Q1hYGO3bt+epp55i0aJFPPLII7hcLsLCwnj11VcbZLvkQIcJjS0tLc2kp6cf9s9tzdvPWeO/5c/XDeSaISmN0DKlVGNas2YNJ510UlM3wxGCfVYistgYkxZsecfdy0WHLSqlVHCO7XLRPnSl1LGyYsUKbrnlllrTIiIiWLBgQRO1KDgNdKXUMWeMOawx3k2tf//+LF269Ji+55F0hzuvy0VHuSjlaJGRkeTl5R1RYJ0ojDHk5eURGRl5WD/nvAq9oogO5OL1epu6KUqpI5CSkkJmZiZHej+nE0VkZCQpKYc38MNxgR61bCJzI5/ljcq5Td0UpdQRCAsLo0uXLk3djGbJcV0u4nID4PP5mrglSil1fHFcoLtctsnGV9nELVFKqeOL4wJdXLaXyOfVCl0ppQI5LtCrKnSfT0+KKqVUIMcFelUfutFAV0qpWhwX6EhVha5dLkopFcjBga4VulJKBXJsoGuXi1JK1ea8QK8ah65XiiqlVC3OC3TtQ1dKqaAcG+jGaIWulFKBHBjo/mGL2uWilFK1ODDQq54pql0uSikVyIGB7m+yjnJRSqlaQgp0ERkpIutEJENEHgsy/zYRyRGRpf5/dzV8U/30botKKRXUIe+HLiJuYAJwAZAJLBKRqcaY1XUWnWSMGdcIbazTID0pqpRSwYRSoQ8DMowxm4wx5cD7wBWN26yDqOpy0ZOiSilVSyiBngxsD3id6Z9W1zUislxEPhSRjsFWJCL3iEi6iKQf8eOnqka56ElRpZSqJZRAD/Zo7rpPd/0MSDXGDAD+B7wVbEXGmNeNMWnGmLSEhITDa2l1a/xN1kBXSqlaQgn0TCCw4k4BsgIXMMbkGWPK/C//CQxpmOYFofdyUUqpoEIJ9EVADxHpIiLhwBhgauACIpIU8HIUsKbhmlhH1SPotEJXSqlaDjnKxRhTKSLjgBmAG3jDGLNKRJ4F0o0xU4EHRGQUUAnsAW5rtBZrl4tSSgV1yEAHMMZMA6bVmfZUwPePA483bNMOQC8sUkqpoBx4pWjVI+i0QldKqUAODHTbZNELi5RSqhbHBrpW6EopVZvzAt2lFxYppVQwzgt0/+1zRQNdKaVqcWCg67BFpZQKxoGBXjXKRU+KKqVUIAcGun+UC1qhK6VUIMcGuo5yUUqp2pwX6P5RLnpSVCmlanNeoFefFNU+dKWUCuTYQNdx6EopVZtjA127XJRSqjYNdKWUaiYcG+h6YZFSStXmvED3j3LRQFdKqdqcF+ja5aKUUkE5NtC1QldKqdocGOja5aKUUsE4MND9t89FLyxSSqlAzgv06kv/TRM3RCmlji/OC3TtQ1dKqaBCCnQRGSki60QkQ0QeO8hy14qIEZG0hmti3TfRUS5KKRXMIQNdRNzABOBioA9wg4j0CbJcDPAAsKChG1n7jWyXiwuDz6fdLkopVSWUCn0YkGGM2WSMKQfeB64IstxzwItAaQO2r76AB1x4tR9dKaWqhRLoycD2gNeZ/mnVRGQw0NEY8/nBViQi94hIuoik5+TkHHZj7Upsk9348GqFrpRS1UIJdAkyrTpJRcQFvAz88lArMsa8boxJM8akJSQkhN7KQK6aLhct0JVSqkYogZ4JdAx4nQJkBbyOAfoB34rIFuBUYGqjnRj1j0N3idEuF6WUChBKoC8CeohIFxEJB8YAU6tmGmPyjTFtjTGpxphUYD4wyhiT3igtBny4cGmXi1JK1XLIQDfGVALjgBnAGmCyMWaViDwrIqMau4FB2yQ20HWUi1JK1fCEspAxZhowrc60pw6w7NlH36xDEBcutMtFKaUCOe9KUcCgFbpSStXlzEDXCl0ppepxdqBrha6UUtUcGej4T4pqga6UUjUcGehVo1y0QldKqRqODHTQPnSllKrLkYFuxIVbR7kopVQtjgx0xIVoha6UUrU4MtB1lItSStXnyEBHXLjFh08fWqSUUtUcG+iCD592uSilVDVHBrpeKaqUUvU5MtDRUS5KKVWPQwPdrRcWKaVUHY4NdLc+JFoppWpxZqC7XPqQaKWUqsOhga5dLkopVZczA93f5VLp1UBXSqkqzgx0lz/QtUJXSqlqjgx08Xe5VOqlokopVc2RgW67XPReLkopFciRgS4uNy7xUaF96EopVS2kQBeRkSKyTkQyROSxIPPvFZEVIrJUROaISJ+Gb2rA+1X1oXu1y0UppaocMtBFxA1MAC4G+gA3BAns94wx/Y0xg4AXgZcavKWB9KSoUkrVE0qFPgzIMMZsMsaUA+8DVwQuYIwpCHjZEmjUpK0+KaoVulJKVfOEsEwysD3gdSZwSt2FRORnwC+AcODcYCsSkXuAewA6dep0uG2tWY9W6EopVU8oFboEmVYvSY0xE4wx3YBHgV8HW5Ex5nVjTJoxJi0hIeHwWhrYILcGulJK1RVKoGcCHQNepwBZB1n+feDKo2nUobhcHr30Xyml6ggl0BcBPUSki4iEA2OAqYELiEiPgJeXAhsaron1VXW5VGgfulJKVTtkH7oxplJExgEzADfwhjFmlYg8C6QbY6YC40TkfKAC2AuMbcxGVwW6VuhKKVUjlJOiGGOmAdPqTHsq4PsHG7hdBydu3HphkVJK1eLIK0XtOHSDV+/lopRS1ZwZ6FqhK6VUPc4MdH1ikVJK1ePMQBe9fa5SStXlzEB36ROLlFKqLmcGuuiVokopVZczA736iUUa6EopVcWZgS4uvR+6UkrV4cxA1wpdKaXqcWagi94PXSml6nJmoGuFrpRS9Tgz0MWNC0NlpVboSilVxZmB7nIDYHzeJm6IUkodP5wZ6GKb7fVVNnFDlFLq+OHMQK+u0DXQlVKqijMDXWyg+7za5aKUUlWcGehaoSulVD3ODHR/hW60QldKqWrODHR/ha4nRZVSqoYzA90/yqWyQit0pZSq4sxA91foFRUVTdwQpZQ6fjgz0P196GUVFRijl/8rpRSEGOgiMlJE1olIhog8FmT+L0RktYgsF5FvRKRzwzc1gL9Cx3gp1xt0KaUUEEKgi4gbmABcDPQBbhCRPnUW+xFIM8YMAD4EXmzohtZulA10Nz5KyrUfXSmlILQKfRiQYYzZZIwpB94HrghcwBgzyxhT7H85H0hp2GbW4QoIdD0xqpRSQGiBngxsD3id6Z92IHcCXwabISL3iEi6iKTn5OSE3sp6K7LNdmEo1gpdKaWA0AJdgkwLeiZSRG4G0oDxweYbY143xqQZY9ISEhJCb2VdLu1yUUqpujwhLJMJdAx4nQJk1V1IRM4HngTOMsaUNUzzDiCgD10rdKWUskKp0BcBPUSki4iEA2OAqYELiMhg4DVglDEmu+GbWUd1he6luFyvFlVKKQgh0I0xlcA4YAawBphsjFklIs+KyCj/YuOBaOADEVkqIlMPsLqGoaNclFKqnlC6XDDGTAOm1Zn2VMD35zdwuw7OVXVSVEe5KKVUFUdfKap96EopVcOZgV7Vhy7a5aKUUlUcGuhhAHjwUViqN+hSSilwaqB7IgBoG2nILmzcEZJKKeUUzgz0sCgAEqLQQFdKKT9nBnp1he4ju7C0iRujlFLHB4cGuq3Q20T4yC7QCl0ppcCxgW4r9FbhPnKLyvD69CEXSinlzED396HHhXnxGcjbr1W6Uko5M9DdtkKP89j7uGi3i1JKOTXQXS5wRxDtsRcV5ehIF6WUcmigA3giaemyFxXpSBellHJyoIdFEuXSLhellKri3ED3RODxlRMXFaYXFymlFI4O9CioKCExJkK7XJRSCkcHegRUltEuNlIrdKWUwsmBHhYFlf4KXfvQlVLKwYHur9ATYyPJKSzDGL1aVCl1YnNwoNf0oZd7fewr1vuiK6VObA4O9Jo+dIDdemJUKXWCc26gh0VBRTGJsfY2ANNX7mriBimlVNNybqBHxkFpPgNT4hmYEscrszZSoI+jU0qdwEIKdBEZKSLrRCRDRB4LMn+EiCwRkUoRubbhmxlEi7ZQuo9w8fKby/pQ7vXx8ZId7N1ffkzeXimljjeHDHQRcQMTgIuBPsANItKnzmLbgNuA9xq6gQfUorX9WrKXkzu1IibSw2+nrmLwc18fsyYopdTxJJQKfRiQYYzZZIwpB94HrghcwBizxRizHPA1QhuDa9HGft2fi8sldE2IPmZvrZRSx6NQAj0Z2B7wOtM/7bCJyD0iki4i6Tk5OUeyihot29qvxXkAxEeFVc/q8eQ0HZeulDrhhBLoEmTaEaWlMeZ1Y0yaMSYtISHhSFZRo6pCL84FwBcQ4BVeQ36JniBVSp1YQgn0TKBjwOsUIKtxmnMYqgPdVuh3n9m11uzMvSXHukVKKdWkQgn0RUAPEekiIuHAGGBq4zYrBC3agLghfwcAI3omsOWFS5ny09MAmDArA58+PFopdQI5ZKAbYyqBccAMYA0w2RizSkSeFZFRACIyVEQygeuA10RkVWM2GgB3GLTrC1lLak3u3KYFAF+u3MVXq3djjGHRlj3ap66UavY8oSxkjJkGTKsz7amA7xdhu2KOrZQ0WDEFfD77nFGgTcvw6tn3vrOY205LZeLcLdx2Wiqbc/fzz1vTCPc493oqpZQ6EGcnW8owKMuH3SuqJ4kIL40eSI9EO4xx4twt1V+/W5/DptyipmipUko1OmcHevfzAIH1X9WafPXJKXz54Jmc1zux3o/cOTGdK/4+5xg1UCmljh1nB3p0IiQPgfXT683yuF3celoqw1Jbc9XgmmHzO/aVsCwzn/1llceypUop1eicHegAPUfCjsVQlF1v1lk9E5h873D+dN1AfnlBz1rzXp+9iQqvD2MM2/KKdUSMUsrxnB/ovS8FDHz28wMu4nYJ95/Xg0sHJNEvORaAv36zgate+YEP0jMZMX4WN/5rPhnZRezRm3sppeqOisvdAOXFNa+9FbDkbSjZa1/nrANvnaP+rB9h7t9r/1wjk6YazpeWlmbS09MbZmXfvQizfg8/nWuHMh6E12c46anpuEUorfTW+731S47l8/vPbJh2KaWaXlEOpL9hB09c+DtolVozr6wIIvz3gSrfD2Et4OOfwOqpcM8sSDwJctbDhKHQ7xq49g177cuXv4K1n8OQ26FNd/jqSeh3LbTtCYNvtrf3njAMCnbAWY/anoTPH4KBN8DySTDiYX8xevhEZLExJi3ovGYR6Pvz4KXeEJMEd/3P9q0fREm5l8gwFzf+cwHzNuUxMCWOjOwi9pd7ARh3TnfKvT4Gd4xnZL/2ZOWXct+7S/jjNf3p1S4Gr8/gcTv/4Eapw/LD36BVZ+hzhR0qvOZT6HoORMXb+fmZMPtPMGQsdBgcfB1FOfYaksg4kDp3FfH5YMMMiEuBBf+ApEGQvx12rYCIWLj8rzXvBeDzwsJ/gssNw+620/Ztg2+es+8//D6oKIW3R8H2BXZ+h5NhzHv2CvMdi+GzB+x6w1rAR/fAGT+HOS/bZeM6Qf622m38+Qp4/Wy7I/Ae4OH0iX0h+WRY+i4YH7TrZyv8wOVv+QS6nXPIjzyY5h/oABn/g//eCF3OhOvfsU80OoSFm/dwz3/S+dnZ3Tm/TztenL6WL+s8+ahfciwrdxRUv06Oj8LjFu46owvdEqIZ3q0NIsL+skpaRnhYnrmPFuEeuifq3R+btbIicHkgLLLh171jsa0C+4yqPy9nvS1YAoPtcFWUwqZZ0P0C+PZ5iE2CoXdB4W4QF0Qn2O4Dtwf254InEtZ+AR/fY39+7GfwzrU2oFp1gZShkPG1DdiyAltYPbjMPiYSYNItUFFiq9f5r9gwj4y1V3tXlNr/q+c+aUNver3HLUBkPJTuszuSS/5s27dzOUx/HLb6R6z9Yi1kLrRVcHGe/d3cOwdm/s5W0oNvse//9W8O/fm0SrXbsG1e8PmeKFu9h0XBXwdCbLJ9r3evtb87AFcYDLweygph9ad2Ws+LYf2XENMBHlpVfe3M4ToxAh0g/U34/OcQ1xFu+C+0739YP26M4bPlOykoqaB/chwPf7CMDdkHH7f+8vUDydpXyvgZ6/jPncO45d8LAfj6oRG88OVabjylE+ed1O6IN0kdh4yB51Ps39cd9UdYAVCyD9ZMhQHX1wQb2NAMbwF5G+HDO6BNN3tYX1EKp/7UVnWzx9tlxy2283PW2umpZ8J7o6HHhXDTB7Xfr6wIMPD9S5B6Onz7gg2lHhfYvty8jRDbATbOsuFYWWoDsips7poJb10GETG2ut30HVz9ug3NgswDfxaeSLuurmdDbAp4wm33Btj15GfC/oA7q8Z3slV0lej2Nnyr3qNFW2jd1YZzz4vh4hdswH43Hr79g63Ur3sT3r8JKoqrn1xWLawljHze5oDx38377MfhbP+OYv6r9rNxuavvA1Xt5LGw4gO47C/2osWtcyH1DLvzLN4DS/2Pe+h3DbTzPxIib6PdMUXF27+L1Z/AB7fZeVe9BrtXwdy/2Z3J5X+FNy+Gm6dA9/MP/JkewokT6AAZ38CUO+3Jih4Xwfm/tQEfGXvYq1q7q4D5G/MQEcoqvSTERDAgJZ7z/vxd0OXbRoeTW1T/pOq4c7pzcud4zumViNQ9zDyA/WWV5BWV08l/KwMVgsD+0FrTC22oGQNdRtgRUeEta5Yt2Wv/te4a5GeLbGCEt4Apd0OnU21IvjbCzr/jK0jsDTuW2PDyltnAqAqxIbfZQNq5DPZuse1I7GPXW/dwPpi+V9m/6bKC2tOrAuGMh2yoTh578OCtEptsP4PVU6Fi/6GXD3TffFuZV+yH696C+I72s102Cc55wn6eZUU2tHYtt10WnnAbeAPHwK6VcMl4243iibSfUUJvWPEhfHKvfY9r34R+V9e6+rtazjr4z9U123n3TNt//WI38PnvrnrGL+z/+S0/2J2CuGDYT2ofSfm84C23Rx/uMNj6g+3WOe+3dp47pAvog6vqbwd4YCnkZdjK/eaP7HUzVUc+R+HECnSw/XRLJsLM31N9p9/h4+CUe+0f4VHas7+c/WWV/HfhNt5buI19xTW36k1pFcWOfSUYAwkxEQzuGM9Xq3cDcH1aR/L2lxER5ub0bm3J2ldC2+hwLujbnryiMlq1CCelVRT7y70MfOYrvD7D2udGEhnmrvX+pRVe/j4zgzvO6ELrluFkF5SybnchZ/ZIYPueYuZtymN02tFv53HHGNizyZ5o6nw6FO221WD382HjTNt/e/07ttttf66tBr99vqbiBWiZCPuzbVU4cAwUZNmfBeh0mq3aTrvfhtOcv8DOpfXbIW4w3oO3tV1/KNlj2woQEWevag5rYStLgCtegQWvwmkP2Apw1wo4/UFI6Anf/tFWpFX6Xg3R7WzVufC14O+ZMtRu997Ndj2n3gdZS+0RQlQruzMZMNruzErzYfoTtrKe+ayt5ke/bU8M5mXY7oRpv4LyIhj9lj2CSD3dzi8vtt0eDcUYG6odTz102BXshFdOsb/HcYtsP3zOOgiPhnXT7O80Iqbh2na4vJXwzlWQOgLOesROqyy3O7YGcuIFepXti+x/6I3f2NfihpNvsYd5fa+y/W99roKW/lvxVpTCqo+h/7V2zx2isW8s5Lv1tiL7/lfnkNIqihe+XMsFfdoxsGM8/1u9m/veW1JvRM2B9G4fw9pdhQBMHXc6u/JL+Xr1bjq3acEdZ3Rh1tocfvbeEi7p355XbhrC01NXMXHuFt66Yxh3TFyE12f4/lfn0LF1aNV9TmEZbaPDQz56OChj6p/sqppetNuGSsow+/n6Ku30NVPtibCcdTDoRvt4wey1sOy/9oRSfEcbhN88Y8+VgO2jjE6sCcxgqsK769n28D3935C7HobebQNtxeRDb0/n021F6y23379ztZ1+04d2GzK+se289CX7Xj/8DW6cbKv24j3w/o22f7rfNfYQvn1/e1i/fSFc9Y/gn1XV57VrBcx4wp7w6+N/SFhZoe3uAfjlOhvY5UXQaTjEJdu+6s2zbf/4EfbR1uLz2u6J40llmd3OozmP4GAnbqBXqSixld3M39m9eF39rrH9fetnwJbv4cyHofcl0H6grV56XwYdhx30LcoqvZRX+oiJDL4jyNxbzEtfreeOM7pQXO5l9GsHOOGCPfGaHB/Fwi176s07rUs8O7esYbNJIjrCw49PXcDYNxYyd2Pt/sCkuEieGdWXT5dl8cyovnhcQnyLmirB6zN4fYaZa3dz7ztLOL17G/5zxym4BFg5xXYthLWA3SttmJXv91caEXZoqDvcdl1s+R4Kd9rRDiun2FEK/a6xleL+XBs2Me1hQ8DtGcJa2gq3sjT4B9DrEnv1r6nzREN3uD20L9kHP/zFVpUDrrfzolrZQ/X/PW1/j13Osl0aw35id+JguwMqS+3TripKYfqj0OdKSBpo/z62zbPBvyPdVsOeCP/OJ6Bq3Dzb9gEPvrlmWuBO7EA7tIaU/oatkk8b17jvo45LGuiBKkpsf+nSd2HPZhsOW76v30dZV58rbYXZoo2tEGJA1tAAABEZSURBVDwR9rC0TXdbqYkLup1rq85if5/s7pXQugu07mZDo2QPxCbjKy9mxrQpdBt4Jj/7NJOdObnMvXA723dms740jiuvuRnExanPz6SfazOdJZuSlDNYuS2Hp8PeYohrA69XXkq6ryfrwvvQu3wV7eJjeGdPL3y4aBHuoqS8klGuuaw3HSkwLXCJj0cTF9Gl7zDyfC2ZvSCdyvISWkkh//MOoQIP4nLxXEo6abtDqFwPwrgjMN4KjDsMt7cME9UaiYi2J82G3U35gn9REpFAXIK/0kw90/Yx52+D5R9Aq1RM8mCKTnuM6Ky5SFS87W/teIo9gejzUb5mGvsSTyEx2JOvqn4/fulb9jC4UyvcrkYOWqWOAQ30Q/FW2kPwPRttpbdxFlSWwL7tdkSAtwI2f2crxLJCO1qg0n/yM387h/VEPvEfBtetPhvATtOaYhNBV08uBbQkzrfvsNfhNUIOrfAiJEse6b6ebO37Mzq0NLz8Qx6lhHOyawPlhNE13s30Pe3ZQywDo3LZUhLFhvDejOzTlk9+3IEXF2mynrY9hhLZMpZd+aW8evPJjH5tHtmFZbxx21D2l1US7naRltqa9xduJUy8pLSJ41dTlpO5t4TTurXh15f2oVtiSyYt2s61Q1LwGTjl9/+jwmtY9exFhAW5JiC7sJTEmEimLsvigf/+yFOX9eGOM7rUWiZ9yx5EhGtencvUcaczIMUewhtjuPvtdFpGePjrmMFkZBdRUu6lf0pc9S0iXC5hdVYBJyXFHLKrqqC0gsLSSpLjDz2UNlBuURl3vZXOn64bGPIw2ILSCmatzWbUwA4N04V2HNpXXM5N/1rAwxf24pwgN+Br7jTQG1LdQ+qKEv+QMWz/bkS0PVwPawFte9juiOy19sKGmCQo2mVDvW0v2LfFns0XF3Q/1z9tK6z8yJ6VD2thdyp9r7IjJFxue7FFyzZQuIuKLfOoXPkpfy8YwZi09sRvnIq7cActkvvhrazAG9+F8Lh2VBgXvnXTkB4XMnpqKR0lm7vO6EJy525sXfI1bVq3IbVzKnl5edw8pxV7pTV/vm4AW1cv4Il5EPyxsjXaRkeQW3SAiyxCdH1aRyal1zyLvFWLMEYP7ci787dRFHAjteeu6EtxuZfnv1wLwAV92lFe6ePygR3YlV9C/5R4npm6ik25+zm1a2vmb7LdVlcO6kDfDnEMSW1FaYWX+95dUutkdkyEhxtP6URUuJvEmEie+Njeknnxr89n+AszKa/08Y+bh/Dp0h0s2rKXO8/owh+nr+WyAUnERYVxxaBk1u4q4NbhqQAs2baXlPgowj0urnplLptz9/PitQOqT1bnFpXxxpzN9EuOY0jnVsRFhREZ5sbnM2Tll3DW+G/pkRjN2l2FDO4Uz5R7T8PlEgpLK3h73lYGdYznNP81ED6fweU/+nhsynLeX7SdJy7pDcA9I7oB8EH6dronRjO4UyvmbsylbXQEPdvZk4cZ2UV0S2hZawfwyY87mDh3CxnZRYw7tztpnVuRlto65N+nMYbVOwvokxRbvV5jDMsy8xn33hIeuagXSXFRDE1tddg7nj9OX8ur325kcKd4Pr7vdF75NoN9xRU8OrJ39VHY1GVZfLQkkzdvG1q9/pJyL1HhDXs+4IUv1zIwJY6L+yfVml5QWsGfZ6zjZ+d2JzHGjrAxxpCVX3rYO/a6NNBVtcv/bw4rduSz7ncjifDU/+Ou9Pqo9Bkiw9x4fYYz/ziTs3olcMOwTmQXlHHX2/Z3FhPhobCsklEDO/Doxb2Z+MNmrhiUzJQlmZSUe7nrzK5E+B8kUlRWSZjbxVOfrmTuxjyuHZLCh4vrD7G7dEASXyzfSUyEh1mPnE3b6Agy9xZz/Wvz2bGv5hmxER4Xya2i2JRzmMPuDkN0hIeiskraxUawuyD0nVXv9jGktmnJ9FW7gs6/qG87eiTG8PdZGfXmndKlNdmFZWzOrb9dXdq25MlLTmLrnmKe+3w1YHekbVqGs31vMX+6biA780uZunQHyzJrxmUPSInjmVF9ueqVuQD89OxuvPrtRgDO651Ih/go/jN/K90SWvLPW9NYmVXAws15vDO//pDKRy7qRVFZJfklFVzUtz1n9Uzgq1W7mJy+nT9c3b86uBZu3sNN/5pPhdfw3JX9OP+kRLILyvj1JytZsSO/1jrfumMYZ/VMIL+4ghv/NZ+fnNWNrH0lzFybzdt3DGNVVgGDOsbjM4bHP1rBnA257Cqw517iW4Qx6Z7hXPSX2QBc0r89WftK+fi+0+jyuD1XtuCJ82gXG8mSbXu5+pW5jB3emU5tWnJp/yTax0Uyb2MeHrfQPzmOzL0lFJdX0iLcQ4f4SFqEe6j0+qqvCi+t8CICER43z362mu17i/naP4ItcDTa2/O28NSn9qFtZ3RvS3yLMJ4Z1Zfpq3bx5McrmXj7UM7udeRHFhroqlp+cQW7Ckrp1T60oV1en8El1KqiFmzKY2DHeDKyizgpKTbkvultecV8tjyLe0Z0ZVd+KTGRHn7z6So+W2afOb7lhUvZsLuQNtERtA548pQxBmOg6xP2P+nAlDj+NXYot/x7AS4RfjWyF6/P3kS72Ej6JMVycud4Hnx/KZl7S1j4xHmMn7GODwJ2IIM6xvPGbUMpKKng7D99Wz39zduH8kH6dtbuLOT9n5zKvI15fLg4k/gW4Tx4Xg+mLt3Bd+tzuOOMLjz4vh3O+IsLejJ95S5W7zzwOZgbhnXk1uGpXPzX72tN/8mIrrw2e1OtaREeF2WVtbvj4qLCyC+xRxOJMRFEhrm56ZRO/GvOZkrLvRQ2wa2gPS5hRM8EZq61dzm9dEASo9M64nEJt7+5iHJvaF2K5/ZO5LRubfjdF2sOuMwjF/Vi/e5CPl1a82z6M7q3Ze7GXILdJPXW4Z15e97W6te/v6ofT368stYySXGRvHLTydU7urpO7dqa3u1jeWf+Vkb0TGB0Wkee+HgFrVuG8/CFPbn3ndqPvuyTFEub6HA6tm7Bewvq7wyrCoQqf7ymP9cP7XTAbT4YDXR1XHv8o+Wc17sd5/c5+BW1f/tmAxPnbmHuY+fWG5tfV3ZhKXlF5ZyUFEtGdiH3vbuEW4enclJSDH07xFX//A8ZuXy/IZfySh9PXW6v/gvswjiQP0xbQ8fWLbjl1M4UllbwxpwttI+LYEjn1rSMcNMywsPUpVlcNiCpenTR+t2FrN9diDG2qyjC4+L7Dbl8uXIXf7iqHwWllUR4XDzy4XLmb8pj7/5yLh/YgZevH8TuAns18terd/PTs7tx71ndqkcqfblyJ899vhq3S8gvqWB0WkfenreV/9w5jLioMOZtzKNbQnT10dUD53Zn9c5C4lvYeX07xLJ0+z6yC8sYO7wz7eOi+ON026VVFUQJMRH0bh/Dz87pTkqrKH4xeRkLN+/h+rSOJMZG8H8za444oiM8PDqyF1+s2MnKHQVcOySF/WWVtXaqb98xjE+W7uCjJbWHnR7o4jyAX17Qk5YRHp79fDXv3XUK2/YU89hHK2gbHUFRWQWlFTU7kb4dYlmVVXsn63YJXv8eIDbSQ0FpTcAmx0fVOgo8EJdQayfSLzmW3u1jax1xXjmoA7+/qj/b9xbzqw+X0y42srqSrzL7kXOO+KJBDXTVbBhjmu3JvipV/yfLKn24XRL0pO+hlFZ46+30Pl26g9yicu6sc3K4avlv1+VwYZ92uFzCjn0lbM3bz8mdWpGRXUS/5Lh6bdxf7iU6wkN5pY+HJi/li+U7AXjs4t7VO5wKr6+6HXv2l5O1rwSPW+jd3l65vXjrHlZk5jOyXxLllT46xEfy4/Z9dEuI5vsNOfTtEMfdb6dzaf8kHr6oV3XffN8Otj0Z2YW4RNiZX8qe/eWUVfoQ4MrByZz01HTKK33cdEonxp6WSssID6e/MJPbT09l7PBUfv3JSq4cnEzb6HBO6dKGORm5FJdXEhPp4Y6J6fRuH8Nrtwxhd0EZj05Zzt1ndiUttRX/W7Obywd0YN6mPC44qR1rdhZw478WMLhTPO/edQotwutfHLU5dz9Ltu7lgr7tMMYedR0pDXSlVKMrq/Ty5YpdXDYgqUHvRnqkO/HSCi8el9RqS3ZhKa1bhB+yfZl7i0lpVVNBH6wNXp/hT1+t47ohKXRNaPyb8mmgK6VUM3GwQA9pNyoiI0VknYhkiEi9+1uKSISITPLPXyAiqUfXZKWUUofrkIEuIm5gAnAx0Ae4QUT61FnsTmCvMaY78DLwx4ZuqFJKqYMLpUIfBmQYYzYZY8qB94Er6ixzBfCW//sPgfOkuZ+5Ukqp40wogZ4MbA94nemfFnQZY0wlkA+0qbsiEblHRNJFJD0nJ6fubKWUUkchlEAPVmnXPZMayjIYY143xqQZY9ISgt1USSml1BELJdAzgcCnJaQAWQdaRkQ8QBxQ/96vSimlGk0ogb4I6CEiXUQkHBgDTK2zzFRgrP/7a4GZpqnGQyql1AnqkA+3M8ZUisg4YAbgBt4wxqwSkWeBdGPMVODfwH9EJANbmY9pzEYrpZSqr8kuLBKRHGDrIRcMri2Q24DNcQLd5hODbvOJ4Wi2ubMxJuhJyCYL9KMhIukHulKqudJtPjHoNp8YGmubG+6GC0oppZqUBrpSSjUTTg3015u6AU1At/nEoNt8YmiUbXZkH7pSSqn6nFqhK6WUqkMDXSmlmgnHBfqh7s3uVCLyhohki8jKgGmtReRrEdng/9rKP11E5G/+z2C5iJzcdC0/ciLSUURmicgaEVklIg/6pzfb7RaRSBFZKCLL/Nv8jH96F/+zBDb4ny0Q7p/eLJ41ICJuEflRRD73v27W2wsgIltEZIWILBWRdP+0Rv3bdlSgh3hvdqeaCIysM+0x4BtjTA/gG/9rsNvfw//vHuDVY9TGhlYJ/NIYcxJwKvAz/++zOW93GXCuMWYgMAgYKSKnYp8h8LJ/m/dinzEAzedZAw8CawJeN/ftrXKOMWZQwJjzxv3bNsY45h8wHJgR8Ppx4PGmblcDbl8qsDLg9Togyf99ErDO//1rwA3BlnPyP+BT4IITZbuBFsAS4BTsVYMe//Tqv3PsLTeG+7/3+JeTpm77YW5nij+8zgU+x96dtdlub8B2bwHa1pnWqH/bjqrQCe3e7M1JO2PMTgD/10T/9Gb3OfgPrQcDC2jm2+3vflgKZANfAxuBfcY+SwBqb1dIzxo4zv0F+BXg879uQ/Pe3ioG+EpEFovIPf5pjfq3fcibcx1nQrrv+gmgWX0OIhINTAF+bowpOMjDrprFdhtjvMAgEYkHPgZOCraY/6ujt1lELgOyjTGLReTsqslBFm0W21vH6caYLBFJBL4WkbUHWbZBtttpFXoo92ZvTnaLSBKA/2u2f3qz+RxEJAwb5u8aYz7yT2722w1gjNkHfIs9fxDvf5YA1N4upz9r4HRglIhswT6+8lxsxd5ct7eaMSbL/zUbu+MeRiP/bTst0EO5N3tzEnif+bHYPuaq6bf6z4yfCuRXHcY5idhS/N/AGmPMSwGzmu12i0iCvzJHRKKA87EnC2dhnyUA9bfZsc8aMMY8boxJMcakYv+/zjTG3EQz3d4qItJSRGKqvgcuBFbS2H/bTX3i4AhONFwCrMf2Oz7Z1O1pwO36L7ATqMDure/E9h1+A2zwf23tX1awo302AiuAtKZu/xFu8xnYw8rlwFL/v0ua83YDA4Af/du8EnjKP70rsBDIAD4AIvzTI/2vM/zzuzb1NhzFtp8NfH4ibK9/+5b5/62qyqrG/tvWS/+VUqqZcFqXi1JKqQPQQFdKqWZCA10ppZoJDXSllGomNNCVUqqZ0EBXSqlmQgNdKaWaif8HX/qpCzBr1BIAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "model_loss = pd.DataFrame(model.history.history)\n",
    "model_loss.plot()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id='early_stop'></a>\n",
    "### 3. Early Stopping\n",
    "\n",
    "Let's use early stopping to track the val_loss and stop training once it begins increasing too much!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [],
   "source": [
    "from tensorflow.keras.callbacks import EarlyStopping"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [],
   "source": [
    "early_stop = EarlyStopping(monitor='val_loss', mode='min', verbose=1, patience=25)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = Sequential()\n",
    "model.add(Dense(40,activation='relu'))\n",
    "model.add(Dropout(0.5))\n",
    "\n",
    "model.add(Dense(20,activation='relu'))\n",
    "model.add(Dropout(0.5))\n",
    "\n",
    "model.add(Dense(1,activation='sigmoid'))\n",
    "model.compile(loss='binary_crossentropy', optimizer='adam')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 2398 samples, validate on 800 samples\n",
      "Epoch 1/500\n",
      "2398/2398 [==============================] - 1s 305us/sample - loss: 0.6573 - val_loss: 0.6045\n",
      "Epoch 2/500\n",
      "2398/2398 [==============================] - 0s 63us/sample - loss: 0.5204 - val_loss: 0.3479\n",
      "Epoch 3/500\n",
      "2398/2398 [==============================] - 0s 64us/sample - loss: 0.3338 - val_loss: 0.1702\n",
      "Epoch 4/500\n",
      "2398/2398 [==============================] - 0s 80us/sample - loss: 0.2088 - val_loss: 0.1045\n",
      "Epoch 5/500\n",
      "2398/2398 [==============================] - 0s 61us/sample - loss: 0.1716 - val_loss: 0.0842\n",
      "Epoch 6/500\n",
      "2398/2398 [==============================] - 0s 73us/sample - loss: 0.1296 - val_loss: 0.0746\n",
      "Epoch 7/500\n",
      "2398/2398 [==============================] - 0s 73us/sample - loss: 0.1285 - val_loss: 0.0693\n",
      "Epoch 8/500\n",
      "2398/2398 [==============================] - 0s 67us/sample - loss: 0.1178 - val_loss: 0.0621\n",
      "Epoch 9/500\n",
      "2398/2398 [==============================] - 0s 56us/sample - loss: 0.1077 - val_loss: 0.0588\n",
      "Epoch 10/500\n",
      "2398/2398 [==============================] - 0s 66us/sample - loss: 0.1022 - val_loss: 0.0558\n",
      "Epoch 11/500\n",
      "2398/2398 [==============================] - 0s 52us/sample - loss: 0.0987 - val_loss: 0.0535\n",
      "Epoch 12/500\n",
      "2398/2398 [==============================] - 0s 50us/sample - loss: 0.0994 - val_loss: 0.0527\n",
      "Epoch 13/500\n",
      "2398/2398 [==============================] - 0s 49us/sample - loss: 0.0889 - val_loss: 0.0499\n",
      "Epoch 14/500\n",
      "2398/2398 [==============================] - 0s 72us/sample - loss: 0.0885 - val_loss: 0.0479\n",
      "Epoch 15/500\n",
      "2398/2398 [==============================] - 0s 62us/sample - loss: 0.0888 - val_loss: 0.0474\n",
      "Epoch 16/500\n",
      "2398/2398 [==============================] - 0s 55us/sample - loss: 0.0866 - val_loss: 0.0461\n",
      "Epoch 17/500\n",
      "2398/2398 [==============================] - 0s 53us/sample - loss: 0.0801 - val_loss: 0.0444\n",
      "Epoch 18/500\n",
      "2398/2398 [==============================] - 0s 52us/sample - loss: 0.0878 - val_loss: 0.0438\n",
      "Epoch 19/500\n",
      "2398/2398 [==============================] - 0s 51us/sample - loss: 0.0798 - val_loss: 0.0429\n",
      "Epoch 20/500\n",
      "2398/2398 [==============================] - 0s 53us/sample - loss: 0.0792 - val_loss: 0.0425\n",
      "Epoch 21/500\n",
      "2398/2398 [==============================] - 0s 56us/sample - loss: 0.0749 - val_loss: 0.0423\n",
      "Epoch 22/500\n",
      "2398/2398 [==============================] - 0s 52us/sample - loss: 0.0778 - val_loss: 0.0404\n",
      "Epoch 23/500\n",
      "2398/2398 [==============================] - 0s 50us/sample - loss: 0.0696 - val_loss: 0.0396\n",
      "Epoch 24/500\n",
      "2398/2398 [==============================] - 0s 49us/sample - loss: 0.0734 - val_loss: 0.0394\n",
      "Epoch 25/500\n",
      "2398/2398 [==============================] - 0s 69us/sample - loss: 0.0743 - val_loss: 0.0390\n",
      "Epoch 26/500\n",
      "2398/2398 [==============================] - 0s 73us/sample - loss: 0.0729 - val_loss: 0.0380\n",
      "Epoch 27/500\n",
      "2398/2398 [==============================] - 0s 52us/sample - loss: 0.0735 - val_loss: 0.0377\n",
      "Epoch 28/500\n",
      "2398/2398 [==============================] - 0s 52us/sample - loss: 0.0712 - val_loss: 0.0372\n",
      "Epoch 29/500\n",
      "2398/2398 [==============================] - 0s 53us/sample - loss: 0.0679 - val_loss: 0.0383\n",
      "Epoch 30/500\n",
      "2398/2398 [==============================] - 0s 54us/sample - loss: 0.0720 - val_loss: 0.0369\n",
      "Epoch 31/500\n",
      "2398/2398 [==============================] - 0s 68us/sample - loss: 0.0649 - val_loss: 0.0373\n",
      "Epoch 32/500\n",
      "2398/2398 [==============================] - 0s 53us/sample - loss: 0.0676 - val_loss: 0.0361\n",
      "Epoch 33/500\n",
      "2398/2398 [==============================] - 0s 51us/sample - loss: 0.0539 - val_loss: 0.0368\n",
      "Epoch 34/500\n",
      "2398/2398 [==============================] - 0s 61us/sample - loss: 0.0680 - val_loss: 0.0360\n",
      "Epoch 35/500\n",
      "2398/2398 [==============================] - 0s 67us/sample - loss: 0.0608 - val_loss: 0.0366\n",
      "Epoch 36/500\n",
      "2398/2398 [==============================] - 0s 68us/sample - loss: 0.0590 - val_loss: 0.0359\n",
      "Epoch 37/500\n",
      "2398/2398 [==============================] - ETA: 0s - loss: 0.061 - 0s 59us/sample - loss: 0.0605 - val_loss: 0.0349\n",
      "Epoch 38/500\n",
      "2398/2398 [==============================] - 0s 61us/sample - loss: 0.0640 - val_loss: 0.0364\n",
      "Epoch 39/500\n",
      "2398/2398 [==============================] - 0s 54us/sample - loss: 0.0651 - val_loss: 0.0356\n",
      "Epoch 40/500\n",
      "2398/2398 [==============================] - 0s 55us/sample - loss: 0.0610 - val_loss: 0.0353\n",
      "Epoch 41/500\n",
      "2398/2398 [==============================] - 0s 60us/sample - loss: 0.0578 - val_loss: 0.0346\n",
      "Epoch 42/500\n",
      "2398/2398 [==============================] - 0s 58us/sample - loss: 0.0521 - val_loss: 0.0349\n",
      "Epoch 43/500\n",
      "2398/2398 [==============================] - 0s 66us/sample - loss: 0.0561 - val_loss: 0.0344\n",
      "Epoch 44/500\n",
      "2398/2398 [==============================] - 0s 56us/sample - loss: 0.0573 - val_loss: 0.0342\n",
      "Epoch 45/500\n",
      "2398/2398 [==============================] - 0s 68us/sample - loss: 0.0551 - val_loss: 0.0345\n",
      "Epoch 46/500\n",
      "2398/2398 [==============================] - 0s 145us/sample - loss: 0.0546 - val_loss: 0.0357\n",
      "Epoch 47/500\n",
      "2398/2398 [==============================] - 0s 82us/sample - loss: 0.0599 - val_loss: 0.0347\n",
      "Epoch 48/500\n",
      "2398/2398 [==============================] - 0s 110us/sample - loss: 0.0558 - val_loss: 0.0375\n",
      "Epoch 49/500\n",
      "2398/2398 [==============================] - 0s 100us/sample - loss: 0.0548 - val_loss: 0.0341\n",
      "Epoch 50/500\n",
      "2398/2398 [==============================] - 0s 105us/sample - loss: 0.0524 - val_loss: 0.0358\n",
      "Epoch 51/500\n",
      "2398/2398 [==============================] - 0s 136us/sample - loss: 0.0545 - val_loss: 0.0343\n",
      "Epoch 52/500\n",
      "2398/2398 [==============================] - 0s 86us/sample - loss: 0.0542 - val_loss: 0.0357\n",
      "Epoch 53/500\n",
      "2398/2398 [==============================] - 0s 54us/sample - loss: 0.0542 - val_loss: 0.0355\n",
      "Epoch 54/500\n",
      "2398/2398 [==============================] - 0s 55us/sample - loss: 0.0533 - val_loss: 0.0336\n",
      "Epoch 55/500\n",
      "2398/2398 [==============================] - 0s 65us/sample - loss: 0.0512 - val_loss: 0.0344\n",
      "Epoch 56/500\n",
      "2398/2398 [==============================] - 0s 81us/sample - loss: 0.0552 - val_loss: 0.0330\n",
      "Epoch 57/500\n",
      "2398/2398 [==============================] - 0s 59us/sample - loss: 0.0539 - val_loss: 0.0332\n",
      "Epoch 58/500\n",
      "2398/2398 [==============================] - 0s 62us/sample - loss: 0.0513 - val_loss: 0.0351\n",
      "Epoch 59/500\n",
      "2398/2398 [==============================] - 0s 79us/sample - loss: 0.0546 - val_loss: 0.0327\n",
      "Epoch 60/500\n",
      "2398/2398 [==============================] - 0s 67us/sample - loss: 0.0538 - val_loss: 0.0333\n",
      "Epoch 61/500\n",
      "2398/2398 [==============================] - 0s 62us/sample - loss: 0.0543 - val_loss: 0.0375\n",
      "Epoch 62/500\n",
      "2398/2398 [==============================] - 0s 60us/sample - loss: 0.0545 - val_loss: 0.0324\n",
      "Epoch 63/500\n",
      "2398/2398 [==============================] - 0s 75us/sample - loss: 0.0478 - val_loss: 0.0333\n",
      "Epoch 64/500\n",
      "2398/2398 [==============================] - 0s 73us/sample - loss: 0.0490 - val_loss: 0.0369\n",
      "Epoch 65/500\n",
      "2398/2398 [==============================] - 0s 57us/sample - loss: 0.0562 - val_loss: 0.0354\n",
      "Epoch 66/500\n",
      "2398/2398 [==============================] - 0s 58us/sample - loss: 0.0551 - val_loss: 0.0327\n",
      "Epoch 67/500\n",
      "2398/2398 [==============================] - 0s 58us/sample - loss: 0.0497 - val_loss: 0.0342\n",
      "Epoch 68/500\n",
      "2398/2398 [==============================] - 0s 71us/sample - loss: 0.0516 - val_loss: 0.0330\n",
      "Epoch 69/500\n",
      "2398/2398 [==============================] - 0s 64us/sample - loss: 0.0470 - val_loss: 0.0334\n",
      "Epoch 70/500\n",
      "2398/2398 [==============================] - 0s 62us/sample - loss: 0.0497 - val_loss: 0.0325\n",
      "Epoch 71/500\n",
      "2398/2398 [==============================] - 0s 154us/sample - loss: 0.0490 - val_loss: 0.0324\n",
      "Epoch 72/500\n",
      "2398/2398 [==============================] - 0s 76us/sample - loss: 0.0514 - val_loss: 0.0328\n",
      "Epoch 73/500\n",
      "2398/2398 [==============================] - 0s 59us/sample - loss: 0.0512 - val_loss: 0.0318\n",
      "Epoch 74/500\n",
      "2398/2398 [==============================] - 0s 63us/sample - loss: 0.0514 - val_loss: 0.0327\n",
      "Epoch 75/500\n",
      "2398/2398 [==============================] - 0s 71us/sample - loss: 0.0459 - val_loss: 0.0337\n",
      "Epoch 76/500\n",
      "2398/2398 [==============================] - 0s 61us/sample - loss: 0.0484 - val_loss: 0.0326\n",
      "Epoch 77/500\n",
      "2398/2398 [==============================] - 0s 55us/sample - loss: 0.0444 - val_loss: 0.0343\n",
      "Epoch 78/500\n",
      "2398/2398 [==============================] - 0s 57us/sample - loss: 0.0459 - val_loss: 0.0328\n",
      "Epoch 79/500\n",
      "2398/2398 [==============================] - 0s 72us/sample - loss: 0.0440 - val_loss: 0.0337\n",
      "Epoch 80/500\n",
      "2398/2398 [==============================] - 0s 68us/sample - loss: 0.0511 - val_loss: 0.0332\n",
      "Epoch 81/500\n",
      "2398/2398 [==============================] - 0s 55us/sample - loss: 0.0477 - val_loss: 0.0322\n",
      "Epoch 82/500\n",
      "2398/2398 [==============================] - 0s 65us/sample - loss: 0.0422 - val_loss: 0.0327\n",
      "Epoch 83/500\n",
      "2398/2398 [==============================] - 0s 69us/sample - loss: 0.0434 - val_loss: 0.0330\n",
      "Epoch 84/500\n",
      "2398/2398 [==============================] - 0s 58us/sample - loss: 0.0451 - val_loss: 0.0336\n",
      "Epoch 85/500\n",
      "2398/2398 [==============================] - 0s 75us/sample - loss: 0.0523 - val_loss: 0.0330\n",
      "Epoch 86/500\n",
      "2398/2398 [==============================] - 0s 65us/sample - loss: 0.0445 - val_loss: 0.0340\n",
      "Epoch 87/500\n",
      "2398/2398 [==============================] - 0s 63us/sample - loss: 0.0463 - val_loss: 0.0329\n",
      "Epoch 88/500\n",
      "2398/2398 [==============================] - 0s 77us/sample - loss: 0.0452 - val_loss: 0.0332\n",
      "Epoch 89/500\n",
      "2398/2398 [==============================] - 0s 57us/sample - loss: 0.0436 - val_loss: 0.0329\n",
      "Epoch 90/500\n",
      "2398/2398 [==============================] - 0s 73us/sample - loss: 0.0395 - val_loss: 0.0330\n",
      "Epoch 91/500\n",
      "2398/2398 [==============================] - 0s 85us/sample - loss: 0.0476 - val_loss: 0.0329\n",
      "Epoch 92/500\n",
      "2398/2398 [==============================] - 0s 67us/sample - loss: 0.0395 - val_loss: 0.0330\n",
      "Epoch 93/500\n",
      "2398/2398 [==============================] - 0s 61us/sample - loss: 0.0425 - val_loss: 0.0335\n",
      "Epoch 94/500\n",
      "2398/2398 [==============================] - 0s 60us/sample - loss: 0.0418 - val_loss: 0.0342\n",
      "Epoch 95/500\n",
      "2398/2398 [==============================] - 0s 63us/sample - loss: 0.0423 - val_loss: 0.0346\n",
      "Epoch 96/500\n",
      "2398/2398 [==============================] - 0s 72us/sample - loss: 0.0410 - val_loss: 0.0337\n",
      "Epoch 97/500\n",
      "2398/2398 [==============================] - 0s 62us/sample - loss: 0.0405 - val_loss: 0.0330\n",
      "Epoch 98/500\n",
      "2398/2398 [==============================] - 0s 59us/sample - loss: 0.0471 - val_loss: 0.0329\n",
      "Epoch 00098: early stopping\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<tensorflow.python.keras.callbacks.History at 0x1a3b7a8a50>"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.fit(x=X_train, y=y_train, \n",
    "          validation_data=(X_test, y_test),\n",
    "          epochs=500,\n",
    "          callbacks=[early_stop]\n",
    "          )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x1a3ca78890>"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAD4CAYAAAAXUaZHAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3deXhc1X3/8fd3No1mtC9eZVvyhjE22CAbwmISSlkKgSTQxEAgSSlkKRBoQoAnDT9KkqYlLWnTOmlpSyANBBMgiQMONAWCIWGxjDe8G3mTLFv7rtnP748zkkayZI1tyeMZfV/Po0eaO3funDtX87nnnnvuuWKMQSmlVGZxpLoASimlRp+Gu1JKZSANd6WUykAa7koplYE03JVSKgO5UvXGJSUlpry8PFVvr5RSaWndunWNxpjSkeZLWbiXl5dTVVWVqrdXSqm0JCL7kplPm2WUUioDabgrpVQG0nBXSqkMlLI2d6XU+BQOh6mpqSEQCKS6KKc0r9dLWVkZbrf7uF6v4a6UOqlqamrIzc2lvLwcEUl1cU5JxhiampqoqamhoqLiuJahzTJKqZMqEAhQXFyswX4UIkJxcfEJHd1ouCulTjoN9pGd6GeUduG+dm8zj7y8nVhMhypWSqnhpF24bzzQyo9+/yEdwUiqi6KUSlM5OTmpLsKYS7twL/B5AGjtDqW4JEopdepKu3Av9NluQS3d4RSXRCmV7owx3HvvvSxYsICFCxeycuVKAOrq6li2bBmLFi1iwYIFvPnmm0SjUT7/+c/3zfuDH/wgxaU/urTrCqk1d6Uyx9/+ZgtbD7aP6jLnT8nj/338jKTmfeGFF9iwYQMbN26ksbGRJUuWsGzZMp5++mkuv/xyvvnNbxKNRunu7mbDhg3U1tbywQcfANDa2jqq5R5taVdzL4jX3Fu15q6UOkFvvfUWN9xwA06nk4kTJ3LxxRezdu1alixZwk9+8hMeeughNm/eTG5uLjNnzqS6upo777yTl19+mby8vFQX/6jSruZeqDV3pTJGsjXssWLM0L3uli1bxpo1a3jppZe4+eabuffee7nlllvYuHEjr7zyCitWrODZZ5/l8ccfP8klTl7a1dzzvHZ/pG3uSqkTtWzZMlauXEk0GqWhoYE1a9awdOlS9u3bx4QJE7jtttu49dZbef/992lsbCQWi3Hdddfx7W9/m/fffz/VxT+qtKu5u5wO8rwu2no03JVSJ+aTn/wkb7/9NmeddRYiwiOPPMKkSZN48skn+f73v4/b7SYnJ4ef/vSn1NbW8oUvfIFYLAbA9773vRSX/uhkuMOSsVZZWWmO92Ydyx55ncXTC/iX5YtHuVRKqbG2bds2Tj/99FQXIy0M9VmJyDpjTOVIr027Zhmw3SG1WUYppYaXluFe4PPQpidUlVJqWGka7lpzV0qpo0nLcC/0ebQrpFJKHUVahnt+tpv2QIRINJbqoiil1CkpLcO9d3yZ9oCODKmUUkNJKtxF5AoR2SEiu0Xk/mHm+bSIbBWRLSLy9OgWc6De8WVatGlGKaWGNGK4i4gTWAFcCcwHbhCR+YPmmQM8AFxgjDkDuHsMymqt/xl/+vtrcRPR8WWUUmPuaGO/7927lwULFpzE0iQvmZr7UmC3MabaGBMCngGuHTTPbcAKY0wLgDGmfnSLmSDQjr9tF9kE9aSqUkoNI5nhB6YCBxIe1wDnDppnLoCI/AFwAg8ZY14evCARuR24HWD69OnHU15wZwPgJaTdIZVKd7+9Hw5tHt1lTloIV/79sE/fd999zJgxg6985SsAPPTQQ4gIa9asoaWlhXA4zHe+8x2uvXZwHfboAoEAX/7yl6mqqsLlcvHoo4/ysY99jC1btvCFL3yBUChELBbj+eefZ8qUKXz605+mpqaGaDTKt771LT7zmc+c0GoPlky4D3WX1sFjFriAOcBHgTLgTRFZYIwZMOCxMeYx4DGwww8cc2kB3D4AskVr7kqpY7d8+XLuvvvuvnB/9tlnefnll7nnnnvIy8ujsbGR8847j2uuueaYblK9YsUKADZv3sz27du57LLL2LlzJ//+7//OV7/6VW666SZCoRDRaJTVq1czZcoUXnrpJQDa2tpGfT2TCfcaYFrC4zLg4BDzvGOMCQN7RGQHNuzXjkopE8Vr7n4JaZu7UunuKDXssbJ48WLq6+s5ePAgDQ0NFBYWMnnyZO655x7WrFmDw+GgtraWw4cPM2nSpKSX+9Zbb3HnnXcCMG/ePGbMmMHOnTv5yEc+wne/+11qamr41Kc+xZw5c1i4cCFf//rXue+++7j66qu56KKLRn09k2lzXwvMEZEKEfEAy4FVg+b5FfAxABEpwTbTVI9mQfvEa+6l3hitPVpzV0odu+uvv57nnnuOlStXsnz5cp566ikaGhpYt24dGzZsYOLEiQQCgWNa5nCDMN54442sWrWK7OxsLr/8cl577TXmzp3LunXrWLhwIQ888AAPP/zwaKzWACPW3I0xERG5A3gF257+uDFmi4g8DFQZY1bFn7tMRLYCUeBeY0zTqJcW+mruxZ6otrkrpY7L8uXLue2222hsbOSNN97g2WefZcKECbjdbl5//XX27dt3zMtctmwZTz31FJdccgk7d+5k//79nHbaaVRXVzNz5kzuuusuqqur2bRpE/PmzaOoqIjPfvaz5OTk8MQTT4z6OiY1nrsxZjWwetC0BxP+NsBfx3/GVm+4Z0XZpuGulDoOZ5xxBh0dHUydOpXJkydz00038fGPf5zKykoWLVrEvHnzjnmZX/nKV/jSl77EwoULcblcPPHEE2RlZbFy5Up+9rOf4Xa7mTRpEg8++CBr167l3nvvxeFw4Ha7+fGPfzzq65h+47nXb4cfncuPS7/Fi9Fzeemu0W+rUkqNHR3PPXnjazz3eM09360XMSml1HDS7jZ7veFe4IpoV0il1EmxefNmbr755gHTsrKyePfdd1NUopGlbbjnOcN0haKEIjE8rvQ7AFFqPDPGHFMf8lRbuHAhGzZsOKnveaJN5umXii4b7jlO2ySj3SGVSi9er5empqYTDq9MZoyhqakJr9d73MtIv5q70wVOD36HDfXW7jATco//A1BKnVxlZWXU1NTQ0NCQ6qKc0rxeL2VlZcf9+vQLdwB3Nn7pD3elVPpwu91UVFSkuhgZL/2aZQDcPrwEAR3TXSmlhpKm4Z7dF+7aY0YppY6UpuHuw2N6w12bZZRSarA0DfdsnNEgbqfo+DJKKTWEtA13CfdQ4PPQpl0hlVLqCGka7j4Id1OQ7aalS2vuSik1WHqGu8sL4R4KfR69iEkppYaQnuHu9kG4hwKfW0+oKqXUENI03LNts4yGu1JKDSmNw902y+hFTEopdaQ0DXd7QjU/20UwEqMnFE11iZRS6pSSpuGeDRiK4+OF6UlVpZQaKE3D3QdAsScC6FWqSik1WJqGux3TPddlQ70zGEllaZRS6pSTpuFua+65DhvqnQENd6WUSpSm4T7wbkxac1dKqYGSCncRuUJEdojIbhG5f4jnPy8iDSKyIf7zl6Nf1ATxcPfFb9ih4a6UUgONeCcmEXECK4A/BWqAtSKyyhizddCsK40xd4xBGY80KNy7NNyVUmqAZGruS4HdxphqY0wIeAa4dmyLNYJ4uHvjY7p3aJu7UkoNkEy4TwUOJDyuiU8b7DoR2SQiz4nItKEWJCK3i0iViFSd0M1x4ydUHdEe/B6nNssopdQgyYS7DDHNDHr8G6DcGHMm8H/Ak0MtyBjzmDGm0hhTWVpaemwlTRSvuRPuIcfr0mYZpZQaJJlwrwESa+JlwMHEGYwxTcbE20jgP4FzRqd4w4jX3An34M9y0aHhrpRSAyQT7muBOSJSISIeYDmwKnEGEZmc8PAaYNvoFXEIfTX3bnKzXNrPXSmlBhmxt4wxJiIidwCvAE7gcWPMFhF5GKgyxqwC7hKRa4AI0Ax8fgzLDC5tllFKqaMZMdwBjDGrgdWDpj2Y8PcDwAOjW7SjcDjid2Pqxu9x0dTZfdLeWiml0kF6XqEKfWO653hd2ltGKaUGSeNwt7fay8nScFdKqcHSONyz+8M9EMGYwb0zlVJq/Er/cPe6iMQMwUgs1SVSSqlTRvqGu8veJDsny54T1qYZpZTql77hntAsAzqmu1JKJUrjcLc3yfZrzV0ppY6QxuFua+65Gu5KKXWENA53X98JVdAx3ZVSKlEah3u2NssopdQw0jzc+5tl9IYdSinVL43D3QeRHnKynIA2yyilVKI0Dnc7MmS2hHGINssopVSiNA53e8MOiQTw6/gySik1QBqHe/8NO3L0hh1KKTVABoS7jgyplFKDZUC4d+uY7kopNUgGhLvW3JVSarA0Dnd7QlXb3JVS6khpHO4Da+7az10ppfqlcbj31tx78Ge56NBwV0qpPmkc7v0nVHO9tuaut9pTSikrqXAXkStEZIeI7BaR+48y3/UiYkSkcvSKOIxBNfeYgZ5wdMzfViml0sGI4S4iTmAFcCUwH7hBROYPMV8ucBfw7mgXckiDLmICvRuTUkr1SqbmvhTYbYypNsaEgGeAa4eY79vAI0BgFMs3PJfX/g4HyPXqsL9KKZUomXCfChxIeFwTn9ZHRBYD04wxL45i2Y5OpP9Wex4Nd6WUSpRMuMsQ0/rOXIqIA/gB8LURFyRyu4hUiUhVQ0ND8qUcjss74G5M2iyjlFJWMuFeA0xLeFwGHEx4nAssAH4vInuB84BVQ51UNcY8ZoypNMZUlpaWHn+pe/Xeak/vxqSUUgMkE+5rgTkiUiEiHmA5sKr3SWNMmzGmxBhTbowpB94BrjHGVI1JiRPFb7Wn4a6UUgONGO7GmAhwB/AKsA141hizRUQeFpFrxrqARxW/1Z7eJFsppQZyJTOTMWY1sHrQtAeHmfejJ16sJMVPqPbW3PUqVaWUstL3ClXoq7lnuRy4HKInVJVSKi7Nw92eUBURcrw6eJhSSvVK83C3J1QB/B4dPEwppXplQLj3APQNHqaUUirtw90HERvufr0bk1JK9UnzcO+vuevdmJRSql+ah7sPIgGIxfQm2UoplSDNwz0+MmSkhxyPhrtSSvVK83Dvv2FHjlebZZRSqleah3v/DTv8WS66QlFiMb3VnlJKZUa4h7rJjQ9B0BXS2rtSSqV3uHty7O9wV8LgYXofVaWUSu9wz8q1vwPt+PuG/Q2nsEBKKXVqSPNwz7O/gx19zTIdelJVKaXSPdzjNfdghzbLKKVUggwJ9/a+Md3bA9oso5RSGRLuHRT5PQC0dIdSWCCllDo1pHe4O932QqZgOwU+NwAtXRruSimV3uEOtvYe7CDL5SQny0WThrtSSmVIuAfaASj0u7XmrpRSZES450GwA4AifxbN3XpCVSmlMiDcc/vD3eemuSuY4gIppVTqZUi49zbLeGjp0pq7UkolFe4icoWI7BCR3SJy/xDPf0lENovIBhF5S0Tmj35Rh+HN76u5F/s9NGubu1JKjRzuIuIEVgBXAvOBG4YI76eNMQuNMYuAR4BHR72kwxlUc+8JR+kJ6VWqSqnxLZma+1JgtzGm2hgTAp4Brk2cwRjTnvDQD5y8QdV729yNochnL2Rq1guZlFLjXDLhPhU4kPC4Jj5tABH5KxH5EFtzv2uoBYnI7SJSJSJVDQ0Nx1PeI2XlgolBqKv/KlVtmlFKjXPJhLsMMe2ImrkxZoUxZhZwH/A3Qy3IGPOYMabSGFNZWlp6bCUdTsLIkL3hrhcyKaXGu2TCvQaYlvC4DDh4lPmfAT5xIoU6JgmDhxVqzV0ppYDkwn0tMEdEKkTEAywHViXOICJzEh5eBewavSKOIKHmXhwPd+0xo5Qa71wjzWCMiYjIHcArgBN43BizRUQeBqqMMauAO0TkUiAMtACfG8tCD+DtDfd28rxuHKLhrpRSI4Y7gDFmNbB60LQHE/7+6iiXK3kJw/46HEKhz6O9ZZRS415mXKEKfYOHFfk92uaulBr3Mifc41epFvo92ltGKTXuZUC4959QBSjyac1dKaXSP9wdTnD7+4YgKMrx6K32lFLjXvqHOwwYX6bI56GlO0wsdvJGQFBKqVNNZoS7N29Am3s0ZmgP6NC/SqnxKzPCPeFWe3ohk1JKZVK4J9TcQcNdKTW+ZUi45w3oLQMa7kqp8S2Dwr2/twygPWaUUuNahoR77hE1d72QSSk1nmVWuMdiZHuceN0OvZBJKTWuZUa4e/MAA+EuAIr9WTR3aVdIpdT4lRnhPmjwsEK/m+auYAoLpJRSqZVZ4d7bHdLnoblba+5KqfErQ8I93/6Oh3ux36M1d6XUuJYh4d5bc28D7IVMLdrmrpQaxzIs3Pu7Q3YGIwQj0RQWSimlUicjw713CAKtvSulxqvMCPfem2Tr4GFKKQVkSrh7cuzvwTV3HYJAKTVOZUa4O5w24Hvb3P06BIFSanzLjHCH+OBhtrdMb7g3d2p3SKXU+JRUuIvIFSKyQ0R2i8j9Qzz/1yKyVUQ2icirIjJj9Is6gkGDhxX63GyubT/pxVBKqVPBiOEuIk5gBXAlMB+4QUTmD5ptPVBpjDkTeA54ZLQLOqKEcHc4hPNnlfCH3Y0Yo/dSVUqNP8nU3JcCu40x1caYEPAMcG3iDMaY140x3fGH7wBlo1vMJCTcRxXgwjklHGoP8GFD50kvilJKpVoy4T4VOJDwuCY+bTi3Ar8d6gkRuV1EqkSkqqGhIflSJiPhPqoAF84uAeCtXY2j+z5KKZUGkgl3GWLakG0dIvJZoBL4/lDPG2MeM8ZUGmMqS0tLky9lMhKaZQCmFfmYXuTjrd1No/s+SimVBpIJ9xpgWsLjMuDg4JlE5FLgm8A1xpiT300la2CzDNimmXeqm4hEYye9OEoplUrJhPtaYI6IVIiIB1gOrEqcQUQWA/+BDfb60S9mErLyINQBsf7xZC6cXUJnMMLGmtaUFEkppVJlxHA3xkSAO4BXgG3As8aYLSLysIhcE5/t+0AO8AsR2SAiq4ZZ3NjpHV8m1H8C9SMzixGBt3Zp04xSanxxJTOTMWY1sHrQtAcT/r50lMt17BIHD/Pa8d0L/R4WTs3nD7sb+eqlc1JYOKWUOrky5wrVQYOH9bpgdgnv72+hKxhJQaGUUio1MifcBw372+vC2SVEYoZ392jTjFJq/MigcI/X3AeF+zkzCslyObTdXSk1rmROuPuK7e/OQwMme91OFk8vYO3e5hQUSimlUiNzwr1gBjjc0LT7iKeWlhex5WAbndrurpQaJzIn3J0uKKqAxl1HPFVZXkTMwPr9LSkomFJKnXyZE+4AxXOGrLmfPaMQh8DavRruSqnxIcPCfRY0Vw+4ShUgJ8vF/Cl5rN2j7e5KqfEhs8K9ZA5EQ9C6/4inlpQXsf5AC2EdZ0YpNQ5kVrgXx69CHaJpZkl5EYFwjA9q205yoZRS6uTLrHAviYf7kCdVCwGo0nZ3pdQ4kFnh7iu248oMUXOfkOulvNjHe9rfXSk1DmRWuIvEe8wcWXMH2zRTtbdZ76uqlMp4mRXuYJtmGo+suYMN95busN5XVSmV8TIv3ItnQcdBCB4Z4L3t7trfXSmV6ZIazz2t9PaYaf4QJp814KmKEj8lOR4eW1PNB7VtTMzzct7MYpZWFKWgoEopNXYyr+Z+lB4zIsKXLp5FlsvB6s11PPq7nSx/7G1e2XLoiHmVUiqdZV7NvWgmIND04ZBP/+VFM/nLi2YC0B4Ic8t/v8edT6/nib9YwvmzSk5iQZVSauxkXs3dnQ3504btMZMoz+vmJ59fwoxiH7c9WcUmvZG2UipDZF64A5TMHrJZZiiFfg//c+u5FPo9fO7x99hxqGPkFyml1CkuM8O9eI5tlkmyP/ukfC8/u/VcPC4HN/7nO+w6rAGvlEpvGRrusyHUAZ2Hk35JeYmfp287D4dDuOE/32V3vfaFV0qlr6TCXUSuEJEdIrJbRO4f4vllIvK+iERE5PrRL+YxKpltfyfZNNNrVmkOP7/tXMCw/LF3eHHTQb2aVSmVlkYMdxFxAiuAK4H5wA0iMn/QbPuBzwNPj3YBj0tvX/eG7cf80tkTcvn5bedRkuPhjqfX86kf/5F1+3Q8GqVUekmm5r4U2G2MqTbGhIBngGsTZzDG7DXGbAJOjcHS88ugZC5U/QRix16kORNzeemui3jkujOpbenhuh+/zdX/+ib/uaaaQ20BOoMRalq62VbXruPDK6VOScn0c58KHEh4XAOcezxvJiK3A7cDTJ8+/XgWkewbwUVfh1/eDjtWw+lXH/MinA7h00umcfVZk3nmvQP8ekMt3129je+u3jZgvsoZhfz01qX4PJl3yYBSKn0lk0gyxLTjaog2xjwGPAZQWVk5to3ZC66D338P3vgHmHeVDfzj4PO4+IsLK/iLCyvY09jF77baq1kLsj209oT4+99u54v/s47/+lwlWS7naK6BUkodt2TCvQaYlvC4DDg4NsUZRU4XLPs6/PqvYNf/wtzLT3iRFSV+bl82a8C0Ap+Hbzy3iTufXs+PbjoblzMzOyAppdJLMuG+FpgjIhVALbAcuHFMSzVazvyMrbm/8QjMuey4a+9H8+nKaXQHIzz0m6185O9fIz/bTbbbicMhhCMxwtEYXreThWX5LJ5WwNKKImYU+wcsoysY4al393H5GZOOeE4ppY6HJNPVT0T+DPhnwAk8boz5rog8DFQZY1aJyBLgl0AhEAAOGWPOONoyKysrTVVV1QmvwIiqHocX74HPvgCz/2TM3ub5dTX88cMmAuEoPeEokZjB43TgcQntPRE21rTSEYggArecN4NvXDEPf5aL3fWdfPln69hV30me18W/3Xg2y+aWjlk5lVLpTUTWGWMqR5wvVf24T1q4R4Lwb5UQ6oZbfg2TFoz9ew4hFjNUN3bys3f28+Tbe5lakM0NS6fzo9d3k+V28jdXnc5ja6rZebiD+6+cx20XzUTG4EhDKZXeNNwTNe6GJz8OkR64+VcwZdHJed9hVO1t5hvPb6K6oYvF0wtYcePZTCnIpisY4d7nNrJ68yE8Tgd52S7yst0snlbI588vZ2FZPmB3FJtr2xCBM8sKUrouSqmTS8N9sOZqePIaCLbDDSthxkdO3nsPIRCO8tauRpbNLcXj6j8Ja4zhl+tr2Xm4k/ZAmObOEG/uaqArFOWcGYXMKPLxxs4GmrpCANx1yWzuvnQuDkd/LT8SjdHcHaK5K0Rbd5jS3CymFmYP6M0TjRkcgh4dKJVmNNyH0rrf1uBb9sIZn4JL/sbelu8U1x4I81xVDT99ey9tPWGWzS3lY6dN4O0Pm1hZdYDLz5jI9//8LNbuaWbl2gO8tr2eSGzgdhWBSXlejIGOQJiuUJSSnCyWlBdSWV7EBbOLOW1ibl/Yx2KGNbsaqGnp4fpzyvC6tZunUqcCDffhBNrgj/8Kb6+AaAgqb4VLHwKP7+SX5TgYY/oC2BjDT/6wl++8tBWHCJGYoSTHw8fPmsLMEj9F/izysl00dATZ19TNgZZunCLkZbvJyXKxv7mbqn3NHGjuAWBmqZ+rFk7G53Hx8/f2s7+5G4B5k3L54Q2LmTsxt+99m7pClORkHVG+mpZu8rLd5HndJ+kTUWp80XAfScdh202y6r+hdB5c/zhMPGoHn1PWm7saeHFjHR+bN4E/OX0C7mPsa1/X1sOr2+pZvbmOd6qbiBlYWlHEZ8+bgdfl4IEXNtMZjPDFi2dxqK2HN3c1UtcW4DufWMBnz5vRt5w9jV1c9cM3yfW6+MFnFiV9Z6tAOEpdW4CDrT20doe5YHYxBT7PMa2DUuOFhnuyPnwNXviibYv/kwdh0U2QPX5PUjZ2BukORple3H8kU98R4N5fbOKNnQ3kZ7u5YHYx9e1BNtW2seqOC5g3KY9wNMb1//42exu7KM7xsKexiy9fPIt7/nTukDublq4Qqz+oY9WGg7y3t3nA0Ptet4NrzprCLR8pZ8HU/JOx2kqlDQ33Y9HZAL/6Euz+P3C4YdbHYP61MPcK8Ot9VcE2xdS09DClIBunQ2jsDHLFP79Jkd/Nqjsu5Ee//5AfvrqLFTeezcfmlfLwb7byzNoDFPjczJuUy7xJefiznOxp7KK6oYvd9Z1EYoaZpX6uXDCJmSU5TC7w4nE6eP79Wn61vpaecJQ5E3K4YsEkLj9jEmdMyTvhE8BVe5t56t39LJiaz/Vnl5Hv0+YjlV403I+VMVD7Pmz9JWz5NbTtBwTKlsDcy2DGhTBlMbi9qS7pKWPNzgZuefw9ls0t5a1dDXxi8VQe/XR/N9PXth/md1vr2X6onR2HOghGYkwv8jGzxM+8yblcuWDysIHd1hPm1xtq+e3mQ7y7xzYVnTOjkAevns9Z0+yRVTAS5XdbD/PenmYONHdzoKWHcDTGomkFnDOjkIVT88nPduPPctHYGeQHv9vJ/22rx+dx0h2K9h0hLK0oZkJuFhPysigv9g958jgWMwN6JB2LA83dBCNRygp9w56YNsYQjMTIcjmOugNLPOcynDd3NfDenmYm5HmZnOdl7sTcAUdiKr1puJ8IY6BuA+x8BXa+DAfX2+kON0w+CwpngK8E/KUw+UwovxA843PYgL9bvY3H1lRTVpjNb796EbnDnEiNxQxRY475fABAU2eQlzbX8cNXd9HYGeK6s8soyfHwi3U1NHeFyM1yMa3Ix7SibATh/f0t1HcEj1hObpaLL310Fl+4oJzqhi6eencfv1p/kJ5wtG8ej9PBmWX5LK0owudxsrGmjU01rRxuD+IQcDsduJ0OO5qe2NFDi/0eJuZ5mZjn5fTJuZxVVsBpk3J5c1cjT7+7n7erm/qWPzEvi9Mm5XH+rGIumFWCx+Xgpc11vLjpINUNXWS5HBT7PeRlu+kJR+kMROgKRYhEDZF499VPnV3Gt66eT372wM86EI7y97/dzhN/3Dtgugh8cvFUvnbZaUwtyAbsTqIjGDmmE9+9OyCXQ3DGd3TRmC2X0yHHtW1H2/ZD7RT5PUzIzdxKmIb7aOpqhAPvwv53oHYdtB+E7mYIttnnnR6Ycb4N+Sln2xq+ryi1ZT5JQpEY//S7HXz8zClj3j7eEQiz4vUPefytPUSN4dLTJ3DjuTO4aHbJgFp1bxPS9kMddHt45gwAAA5gSURBVAUjdAYjGGO4+swpFPoHnqgNhKMcbg9Q3xHkUFuADw628d6eZjbXtPU1G51VVsD0Ih8xYwhFYoSjBhMfGDUSNTR2BqnvCHKwtYe6tsCA5ZcV2iuRywqz2d/Uzb7mbjYeaGVXwm0cHQLnVhRz3sxiukIRmjpDtAfC+DxOcrJc+LNcuByCyyE0doVYufYAE3Kz+IfrzuTC2SXUdwT5sKGTb7+4le2HOvjCBeV84/J5tAfC1LUF+O0HdfzkD3sBuPrMydS3B/ngYBut3WHOn1XMVz46mwtmFwOw43AHf9zdRDgao9DnId/npralh/f2NLN2b3Pf9RWDed0Obr2wgi9dPGvADr4nFCXbk1w32o5AmO+8uI31B1q4ZN5Erj5z+CO7wSLRGP/y6i7+7fXdFPo8/OsNi7lgdmY2qWq4nwzhHhv4u//Pnpit39r/XHYRZOWAJweyC6GwHIoqoGim7Z1TPBtcR3YlVCNriNfKS3PH7vPrDkWIxsywRyLDaeoMsqmmja117cyfksfFc0qHbM6pbw/wxw+b6A5FuXT+hGOqaW480MrXfrGR3fWdeJwOQvEbxhT5Pfzjn5/JJfMmHvGa2tYeHv3fnbz8QR0VpX4WTMmnJCeLZ6sOUN8R5LSJubR0h4Y84gGYVpTN0vJiZpb6icYM0ZjBGIPL6cDlFLbVdfCbjQcp9nv4iwsrqGvr4Z3qZnbXd7Jgah43Lp3BNYum4Pc4OdQeYE9DF74sF6dNzCXb42TdvmbuXrmB2pYezplRyPr9rURihlmlfr522WlcuWDSsCFf09LN3c9soGpfC59YNIUtB9v5sKGTey+fx20XVXCgpYcdh9pp74kwIS+LiXlephRkH3Hkk6i6oZP/t2oLVy2czPKlo3fvidrWHr71qw/46z+de9yVIQ33VAi0wcENcPB9e8FUqBtCndDdBM17oPNQ/7zigIIZ9oStN9/++EogZ4L9ycoFtw9cXvvjdNsjBH8p5B755VXjSyAc5fE/7KG9J0JZYTZlhdksmlZwzF1Ig5Eov3y/lmerDjClIJtlc0q5cE4JedluWrtDtHaHKc7xMDk/e8RlbTzQyndXb+O9Pc34PU6WVBQxf3Ier22vZ/uhDnweJwJ0hfqbwRxih9Le09jF1MJs/vkzizhnRhEtXSH+d+shHn9rLzsOd3D29ALuvnQuIlDb0kNNSw+76zvZWd/BvqZust1OvvvJBVy7aCpdwQj3Pb+JFzfV4XYK4ejQGVeam8WcCTnMm5THp86e2he2v99Rz50/X093KEo0Zrj5vBk8+PH5fc1Ota09tHSFKPC5KfR58LgcdAYidAQiGAyT87MHXHUO9mjy5+8d4O9WbyNmDN+//iyuOnPyMW2rXhrup6JQlx0GoWGHvb9r027oabU7hUArdDX1N/UcjX8CTFpojwD88bb/7EJ7JNC7E3Bn252D22fncY/85VTqRPU2iU3O9/bd28AYw/v7W/nl+hpcDgezSv1UlOTQGYywta6drQfbKSvM5muXzT3iSCkaMzy37gD/+L87+47YwO4Uyov9zJmYw9yJufz5OdMGnDQ2xvBs1QF2Hu7ktIm5nDYplyK/h/qOAIfagtS0dLO7vpNd9Z1sq2snGImxeHoBi6YV8OQf9zJ3Yi6P3VzJU+/t4z/eqGZpRRGVMwp5dVs9Ow53HPUzcAhMKchmakE2uV4XPo+Lg609VO1r4fxZxfzDdWcyrej4T3BruKercA90NUCw0/4d7rZX0kbDEA3a9v5Dm6FuEzR/aJ9PRlbekUcEHr9tNvL47aBq7XXQUWffq3AGFFZAzkT7vuEeMDHInQR5UyF3sn1NT6u9RsDtB38x+IohFoWeFvuciN3xZBfa53In9TdH9e7sOg6Bt6D/9a5su5MarXFvjIFgh11XR7xG1dMKe96APW/aJrMzPgn5U/tfE+oGE7WfV6JY1L7W4QCHq/+oaiwYY8u48xU47UqoWDb0fOEee+I/0Gab+4pn2+2WQeMGdQUjvLmrkQKfm6kF2UzK947aCdy2njDPravhZ+/sY09jF1cumMQ/ffqsvltn/mp9Lfc9v4lIzLCkvJBLT59IWaGP1u4QLd1hItEYOV4XOVkuDFDTbM+tHGztoSsYpSsUwRj44sUzuXHp9BPuzqvhPl6EuuwJ354WG8qxsB3mOBKwwR/qsjuLznroPGx3GpGAnR7usb9DHeDMgrzJNrgdTjv+TvPe+JGE2Jq/OGwz04nKLrIB31F39PmcWfHmquL+E9ThHlt+ccbPafjjO6sse8Qijv71D3ZAW439ifTYMPZPsMts3GnD25VtnwOYfr7dCdVvteuPsSFZPNvuGFr22OnRxJOKAiVzbC+qCfPtc73bw1dkz7EUVtiyxiIQjUDbATi0ye6gIwGYttS+98T5EA7Yz7xhp70XQeMO+x4YKL8IPvoAFEyHrnq7M975W9t1NzSoNukrgVmX2HsYlM6zZW/cZXekeZNtk2DuZPv/0vu/4M23FQD/hP7tLQ77fxXutvM4XZA7pb9LcO+Os/OwrXh01NnOBv5Se6P6nAl2feu32SPWrFz7eRXPsduj87D9iYbtc1m59iLC3Mm2IpCVa7dnqMv+74UDdnuF49s30Gp/u7zx5s08O29vWcD+T+dPBW+hrahEgnYn3XuU63SDMcRiMZq6QhTnZOFwOOM7RwMG2ntCOCLd5NBjKzOBtvhRd/z9ez8fY+y69zavZhfaios3v7/CIg7In24rM8dBw12dOGPsly6xFh1oh/ZaGxJuX8IXqhu6G+35BYfbfkG98St9e5rtF76rwX6ROw7ZUCuqsMGZO8V+Wbqb7E8kYEMyErBfoN7XIzZUXF57FBHsiH/he+Lzh+KBHZ/H47MBkz/NftF6WuywE91Ntllr9qVQVgkt+2DLC7D113Y5E+bbH6fbHh01fWh3ikXlNqzzptr3j4ZtGQ5vgbqN0F5j19dbYL/UXY1Hhm6vrHxbBpcHDqwder4pi2HpF22tfePP4c1Hbagn8uTYC+7O/Iw9Amn+0A5xXbPWnuTvbhw4v7fABtKJ8hXbHWtnQ//OcSTegv4j0WSJ027T49JbQx7DjMvK6z8admfb9+pqtP/rscjwr7vqUVhy63G9pYa7UidboN1+wXubaYyJn0yvtjsqhyt+9FBqg7h3hxmNwOEP7DmY3tqrfwKUzB64/FA3fPC8DTt/vGY4Yf7wg97FYvYIoWUPFM2yI6B6/HY5bQds7ba3ec7tszvY3pp0JGh3YCY68BxOJAgdB+1rg53xGupE+5M32e6ofUX2SLG9xu5M86facuZMsDXm1n12hwn9tVxnlt3BBTvsjrzzsK15B9rs+3py7Hr2Nim6vfFgjYdrNNhfm/bk2LLkxDsedNRBW62tcTs99vUOZ7ypM97kKQJI30FS37oj/c95fPHtk2crNVl59khmuM8+0NrfPBlojS8zZj+DiWfYps/joOGulFIZKNlwT/0lZUoppUadhrtSSmUgDXellMpAGu5KKZWBNNyVUioDabgrpVQG0nBXSqkMpOGulFIZKGUXMYlIA7DvOF9eAjSOOFdm0nUfn8bruo/X9Ybh132GMaZ0pBenLNxPhIhUJXOFVibSddd1H0/G63rDia+7NssopVQG0nBXSqkMlK7h/liqC5BCuu7j03hd9/G63nCC656Wbe5KKaWOLl1r7koppY5Cw10ppTJQ2oW7iFwhIjtEZLeI3J/q8owVEZkmIq+LyDYR2SIiX41PLxKR34nIrvjvwlSXdayIiFNE1ovIi/HHFSLybnzdV4qIJ9VlHAsiUiAiz4nI9vj2/8h42e4ick/8//0DEfm5iHgzdbuLyOMiUi8iHyRMG3I7i/XDeO5tEpGzR1p+WoW7iDiBFcCVwHzgBhGZn9pSjZkI8DVjzOnAecBfxdf1fuBVY8wc4NX440z1VWBbwuN/AH4QX/cW4PhuQnnq+xfgZWPMPOAs7GeQ8dtdRKYCdwGVxpgFgBNYTuZu9yeAKwZNG247XwnMif/cDvx4pIWnVbgDS4HdxphqY0wIeAa4NsVlGhPGmDpjzPvxvzuwX/Cp2PV9Mj7bk8AnUlPCsSUiZcBVwH/FHwtwCfBcfJaMXHcRyQOWAf8NYIwJGWNaGSfbHXAB2SLiAnxAHRm63Y0xa4DmQZOH287XAj811jtAgYhMPtry0y3cpwIHEh7XxKdlNBEpBxYD7wITjTF1YHcAwITUlWxM/TPwDSAWf1wMtBpjem8pn6nbfibQAPwk3iT1XyLiZxxsd2NMLfCPwH5sqLcB6xgf273XcNv5mLMv3cJdhpiW0X05RSQHeB642xjTnurynAwicjVQb4xZlzh5iFkzcdu7gLOBHxtjFgNdZGATzFDi7cvXAhXAFMCPbY4YLBO3+0iO+f8/3cK9BpiW8LgMOJiisow5EXFjg/0pY8wL8cmHew/H4r/rU1W+MXQBcI2I7MU2vV2CrckXxA/XIXO3fQ1QY4x5N/74OWzYj4ftfimwxxjTYIwJAy8A5zM+tnuv4bbzMWdfuoX7WmBO/Oy5B3uyZVWKyzQm4m3M/w1sM8Y8mvDUKuBz8b8/B/z6ZJdtrBljHjDGlBljyrHb+DVjzE3A68D18dkydd0PAQdE5LT4pD8BtjIOtju2OeY8EfHF//971z3jt3uC4bbzKuCWeK+Z84C23uabYRlj0uoH+DNgJ/Ah8M1Ul2cM1/NC7GHXJmBD/OfPsG3PrwK74r+LUl3WMf4cPgq8GP97JvAesBv4BZCV6vKN0TovAqri2/5XQOF42e7A3wLbgQ+A/wGyMnW7Az/HnlsIY2vmtw63nbHNMiviubcZ26PoqMvX4QeUUioDpVuzjFJKqSRouCulVAbScFdKqQyk4a6UUhlIw10ppTKQhrtSSmUgDXellMpA/x8NSH8UfMACwwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "model_loss = pd.DataFrame(model.history.history)\n",
    "model_loss.plot()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Save the model**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [],
   "source": [
    "#model.save('classification_model.h5')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Load the model**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#model = load_model('classification_model.h5')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id='evaluation'></a>\n",
    "## Evaluate the Model\n",
    "Check common classification metrics [here](https://scikit-learn.org/stable/modules/model_evaluation.html#classification-metrics)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.metrics import classification_report, confusion_matrix"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [],
   "source": [
    "predictions = model.predict_classes(X_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**[Precision and recall](https://en.wikipedia.org/wiki/Precision_and_recall)**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       1.00      0.99      0.99       418\n",
      "           1       0.99      1.00      0.99       382\n",
      "\n",
      "    accuracy                           0.99       800\n",
      "   macro avg       0.99      0.99      0.99       800\n",
      "weighted avg       0.99      0.99      0.99       800\n",
      "\n"
     ]
    }
   ],
   "source": [
    "print(classification_report(y_test,predictions))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**[Confusion matrix](https://en.wikipedia.org/wiki/Confusion_matrix)**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[414   4]\n",
      " [  1 381]]\n"
     ]
    }
   ],
   "source": [
    "print(confusion_matrix(y_test,predictions))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id='predictions'></a>\n",
    "## Predictions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "True wine type: white\n"
     ]
    }
   ],
   "source": [
    "single_wine = df_white.drop('type',axis=1).iloc[-1]\n",
    "print(\"True wine type:\", df_white['type'].iloc[-1])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [],
   "source": [
    "single_wine = scaler.transform(single_wine.values.reshape(-1, 12))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_key(val):\n",
    "    for key, value in dict_cat.items():\n",
    "         if val == value:\n",
    "                return key\n",
    " \n",
    "    return \"key doesn't exist\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Predicted wine type: white\n"
     ]
    }
   ],
   "source": [
    "print(\"Predicted wine type:\", get_key(model.predict_classes(single_wine)[0][0]))"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}