Chapter-wise code/Code - PyTorch/6. Natural-Language-Processing/10. Text Classification/predicting-DJIA-movement-with-BERT.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "execution_count": null,
   "metadata": {
    "papermill": {
     "duration": 0.020399,
     "end_time": "2020-08-07T04:44:49.226291",
     "exception": false,
     "start_time": "2020-08-07T04:44:49.205892",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# Predicting DJIA Movement with BERT\n",
    "\n",
    "In this notebook, I have tried to perform NLP on daily financial news using BERT. Few things to note:<br>\n",
    "1. Due to computational restriction, I couldnot combine all the news columns while working with BERT. \n",
    "2. Also, BERT can process only 512 tokens at a time, so even if the kernel was computationally strong, I couldnot have used all news articles column. If I combined all news articles, final number of tokens generated was 618. Hence the code was breaking. \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "_cell_guid": "b1076dfc-b9ad-4769-8c92-a6c4dae69d19",
    "_uuid": "8f2839f25d086af736a60e9eeb907d3b93b6e0e5",
    "execution": {
     "iopub.execute_input": "2020-08-07T04:44:49.274001Z",
     "iopub.status.busy": "2020-08-07T04:44:49.272900Z",
     "iopub.status.idle": "2020-08-07T04:44:49.280014Z",
     "shell.execute_reply": "2020-08-07T04:44:49.280861Z"
    },
    "papermill": {
     "duration": 0.035453,
     "end_time": "2020-08-07T04:44:49.281171",
     "exception": false,
     "start_time": "2020-08-07T04:44:49.245718",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/kaggle/input/stocknews/RedditNews.csv\n",
      "/kaggle/input/stocknews/upload_DJIA_table.csv\n",
      "/kaggle/input/stocknews/Combined_News_DJIA.csv\n"
     ]
    }
   ],
   "source": [
    "# This Python 3 environment comes with many helpful analytics libraries installed\n",
    "# It is defined by the kaggle/python Docker image: https://github.com/kaggle/docker-python\n",
    "# For example, here's several helpful packages to load\n",
    "\n",
    "import numpy as np # linear algebra\n",
    "import pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n",
    "\n",
    "# Input data files are available in the read-only \"../input/\" directory\n",
    "# For example, running this (by clicking run or pressing Shift+Enter) will list all files under the input directory\n",
    "\n",
    "import os\n",
    "for dirname, _, filenames in os.walk('/kaggle/input'):\n",
    "    for filename in filenames:\n",
    "        print(os.path.join(dirname, filename))\n",
    "\n",
    "# You can write up to 5GB to the current directory (/kaggle/working/) that gets preserved as output when you create a version using \"Save & Run All\" \n",
    "# You can also write temporary files to /kaggle/temp/, but they won't be saved outside of the current session"
   ]
  },
  {
   "cell_type": "markdown",
   "execution_count": null,
   "metadata": {
    "papermill": {
     "duration": 0.018258,
     "end_time": "2020-08-07T04:44:49.318851",
     "exception": false,
     "start_time": "2020-08-07T04:44:49.300593",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# Reading CSV file\n",
    "\n",
    "In this csv file, Top1 to Top25 are the news articles in decreasing order of their priority.<br>\n",
    "\n",
    "Label \"1\" is when DJIA Adj Close value rose or stayed as the same.<br>\n",
    "Label \"0\" is when DJIA Adj Close value decreased.\n",
    "                                                                                             "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "_cell_guid": "79c7e3d0-c299-4dcb-8224-4455121ee9b0",
    "_uuid": "d629ff2d2480ee46fbb7e2d37f6b5fab8052498a",
    "execution": {
     "iopub.execute_input": "2020-08-07T04:44:49.364284Z",
     "iopub.status.busy": "2020-08-07T04:44:49.363389Z",
     "iopub.status.idle": "2020-08-07T04:44:49.475207Z",
     "shell.execute_reply": "2020-08-07T04:44:49.474292Z"
    },
    "papermill": {
     "duration": 0.137771,
     "end_time": "2020-08-07T04:44:49.475395",
     "exception": false,
     "start_time": "2020-08-07T04:44:49.337624",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    " \n",
    "# Here we are interested in Combined_News_DJIA.csv file\n",
    "df = pd.read_csv('/kaggle/input/stocknews/Combined_News_DJIA.csv')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-08-07T04:44:49.545833Z",
     "iopub.status.busy": "2020-08-07T04:44:49.526793Z",
     "iopub.status.idle": "2020-08-07T04:44:49.566922Z",
     "shell.execute_reply": "2020-08-07T04:44:49.567571Z"
    },
    "papermill": {
     "duration": 0.072867,
     "end_time": "2020-08-07T04:44:49.567782",
     "exception": false,
     "start_time": "2020-08-07T04:44:49.494915",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Date</th>\n",
       "      <th>Label</th>\n",
       "      <th>Top1</th>\n",
       "      <th>Top2</th>\n",
       "      <th>Top3</th>\n",
       "      <th>Top4</th>\n",
       "      <th>Top5</th>\n",
       "      <th>Top6</th>\n",
       "      <th>Top7</th>\n",
       "      <th>Top8</th>\n",
       "      <th>...</th>\n",
       "      <th>Top16</th>\n",
       "      <th>Top17</th>\n",
       "      <th>Top18</th>\n",
       "      <th>Top19</th>\n",
       "      <th>Top20</th>\n",
       "      <th>Top21</th>\n",
       "      <th>Top22</th>\n",
       "      <th>Top23</th>\n",
       "      <th>Top24</th>\n",
       "      <th>Top25</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2008-08-08</td>\n",
       "      <td>0</td>\n",
       "      <td>b\"Georgia 'downs two Russian warplanes' as cou...</td>\n",
       "      <td>b'BREAKING: Musharraf to be impeached.'</td>\n",
       "      <td>b'Russia Today: Columns of troops roll into So...</td>\n",
       "      <td>b'Russian tanks are moving towards the capital...</td>\n",
       "      <td>b\"Afghan children raped with 'impunity,' U.N. ...</td>\n",
       "      <td>b'150 Russian tanks have entered South Ossetia...</td>\n",
       "      <td>b\"Breaking: Georgia invades South Ossetia, Rus...</td>\n",
       "      <td>b\"The 'enemy combatent' trials are nothing but...</td>\n",
       "      <td>...</td>\n",
       "      <td>b'Georgia Invades South Ossetia - if Russia ge...</td>\n",
       "      <td>b'Al-Qaeda Faces Islamist Backlash'</td>\n",
       "      <td>b'Condoleezza Rice: \"The US would not act to p...</td>\n",
       "      <td>b'This is a busy day:  The European Union has ...</td>\n",
       "      <td>b\"Georgia will withdraw 1,000 soldiers from Ir...</td>\n",
       "      <td>b'Why the Pentagon Thinks Attacking Iran is a ...</td>\n",
       "      <td>b'Caucasus in crisis: Georgia invades South Os...</td>\n",
       "      <td>b'Indian shoe manufactory  - And again in a se...</td>\n",
       "      <td>b'Visitors Suffering from Mental Illnesses Ban...</td>\n",
       "      <td>b\"No Help for Mexico's Kidnapping Surge\"</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2008-08-11</td>\n",
       "      <td>1</td>\n",
       "      <td>b'Why wont America and Nato help us? If they w...</td>\n",
       "      <td>b'Bush puts foot down on Georgian conflict'</td>\n",
       "      <td>b\"Jewish Georgian minister: Thanks to Israeli ...</td>\n",
       "      <td>b'Georgian army flees in disarray as Russians ...</td>\n",
       "      <td>b\"Olympic opening ceremony fireworks 'faked'\"</td>\n",
       "      <td>b'What were the Mossad with fraudulent New Zea...</td>\n",
       "      <td>b'Russia angered by Israeli military sale to G...</td>\n",
       "      <td>b'An American citizen living in S.Ossetia blam...</td>\n",
       "      <td>...</td>\n",
       "      <td>b'Israel and the US behind the Georgian aggres...</td>\n",
       "      <td>b'\"Do not believe TV, neither Russian nor Geor...</td>\n",
       "      <td>b'Riots are still going on in Montreal (Canada...</td>\n",
       "      <td>b'China to overtake US as largest manufacturer'</td>\n",
       "      <td>b'War in South Ossetia [PICS]'</td>\n",
       "      <td>b'Israeli Physicians Group Condemns State Tort...</td>\n",
       "      <td>b' Russia has just beaten the United States ov...</td>\n",
       "      <td>b'Perhaps *the* question about the Georgia - R...</td>\n",
       "      <td>b'Russia is so much better at war'</td>\n",
       "      <td>b\"So this is what it's come to: trading sex fo...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2008-08-12</td>\n",
       "      <td>0</td>\n",
       "      <td>b'Remember that adorable 9-year-old who sang a...</td>\n",
       "      <td>b\"Russia 'ends Georgia operation'\"</td>\n",
       "      <td>b'\"If we had no sexual harassment we would hav...</td>\n",
       "      <td>b\"Al-Qa'eda is losing support in Iraq because ...</td>\n",
       "      <td>b'Ceasefire in Georgia: Putin Outmaneuvers the...</td>\n",
       "      <td>b'Why Microsoft and Intel tried to kill the XO...</td>\n",
       "      <td>b'Stratfor: The Russo-Georgian War and the Bal...</td>\n",
       "      <td>b\"I'm Trying to Get a Sense of This Whole Geor...</td>\n",
       "      <td>...</td>\n",
       "      <td>b'U.S. troops still in Georgia (did you know t...</td>\n",
       "      <td>b'Why Russias response to Georgia was right'</td>\n",
       "      <td>b'Gorbachev accuses U.S. of making a \"serious ...</td>\n",
       "      <td>b'Russia, Georgia, and NATO: Cold War Two'</td>\n",
       "      <td>b'Remember that adorable 62-year-old who led y...</td>\n",
       "      <td>b'War in Georgia: The Israeli connection'</td>\n",
       "      <td>b'All signs point to the US encouraging Georgi...</td>\n",
       "      <td>b'Christopher King argues that the US and NATO...</td>\n",
       "      <td>b'America: The New Mexico?'</td>\n",
       "      <td>b\"BBC NEWS | Asia-Pacific | Extinction 'by man...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2008-08-13</td>\n",
       "      <td>0</td>\n",
       "      <td>b' U.S. refuses Israel weapons to attack Iran:...</td>\n",
       "      <td>b\"When the president ordered to attack Tskhinv...</td>\n",
       "      <td>b' Israel clears troops who killed Reuters cam...</td>\n",
       "      <td>b'Britain\\'s policy of being tough on drugs is...</td>\n",
       "      <td>b'Body of 14 year old found in trunk; Latest (...</td>\n",
       "      <td>b'China has moved 10 *million* quake survivors...</td>\n",
       "      <td>b\"Bush announces Operation Get All Up In Russi...</td>\n",
       "      <td>b'Russian forces sink Georgian ships '</td>\n",
       "      <td>...</td>\n",
       "      <td>b'Elephants extinct by 2020?'</td>\n",
       "      <td>b'US humanitarian missions soon in Georgia - i...</td>\n",
       "      <td>b\"Georgia's DDOS came from US sources\"</td>\n",
       "      <td>b'Russian convoy heads into Georgia, violating...</td>\n",
       "      <td>b'Israeli defence minister: US against strike ...</td>\n",
       "      <td>b'Gorbachev: We Had No Choice'</td>\n",
       "      <td>b'Witness: Russian forces head towards Tbilisi...</td>\n",
       "      <td>b' Quarter of Russians blame U.S. for conflict...</td>\n",
       "      <td>b'Georgian president  says US military will ta...</td>\n",
       "      <td>b'2006: Nobel laureate Aleksander Solzhenitsyn...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2008-08-14</td>\n",
       "      <td>1</td>\n",
       "      <td>b'All the experts admit that we should legalis...</td>\n",
       "      <td>b'War in South Osetia - 89 pictures made by a ...</td>\n",
       "      <td>b'Swedish wrestler Ara Abrahamian throws away ...</td>\n",
       "      <td>b'Russia exaggerated the death toll in South O...</td>\n",
       "      <td>b'Missile That Killed 9 Inside Pakistan May Ha...</td>\n",
       "      <td>b\"Rushdie Condemns Random House's Refusal to P...</td>\n",
       "      <td>b'Poland and US agree to missle defense deal. ...</td>\n",
       "      <td>b'Will the Russians conquer Tblisi? Bet on it,...</td>\n",
       "      <td>...</td>\n",
       "      <td>b'Bank analyst forecast Georgian crisis 2 days...</td>\n",
       "      <td>b\"Georgia confict could set back Russia's US r...</td>\n",
       "      <td>b'War in the Caucasus is as much the product o...</td>\n",
       "      <td>b'\"Non-media\" photos of South Ossetia/Georgia ...</td>\n",
       "      <td>b'Georgian TV reporter shot by Russian sniper ...</td>\n",
       "      <td>b'Saudi Arabia: Mother moves to block child ma...</td>\n",
       "      <td>b'Taliban wages war on humanitarian aid workers'</td>\n",
       "      <td>b'Russia: World  \"can forget about\" Georgia\\'s...</td>\n",
       "      <td>b'Darfur rebels accuse Sudan of mounting major...</td>\n",
       "      <td>b'Philippines : Peace Advocate say Muslims nee...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 27 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "         Date  Label                                               Top1  \\\n",
       "0  2008-08-08      0  b\"Georgia 'downs two Russian warplanes' as cou...   \n",
       "1  2008-08-11      1  b'Why wont America and Nato help us? If they w...   \n",
       "2  2008-08-12      0  b'Remember that adorable 9-year-old who sang a...   \n",
       "3  2008-08-13      0  b' U.S. refuses Israel weapons to attack Iran:...   \n",
       "4  2008-08-14      1  b'All the experts admit that we should legalis...   \n",
       "\n",
       "                                                Top2  \\\n",
       "0            b'BREAKING: Musharraf to be impeached.'   \n",
       "1        b'Bush puts foot down on Georgian conflict'   \n",
       "2                 b\"Russia 'ends Georgia operation'\"   \n",
       "3  b\"When the president ordered to attack Tskhinv...   \n",
       "4  b'War in South Osetia - 89 pictures made by a ...   \n",
       "\n",
       "                                                Top3  \\\n",
       "0  b'Russia Today: Columns of troops roll into So...   \n",
       "1  b\"Jewish Georgian minister: Thanks to Israeli ...   \n",
       "2  b'\"If we had no sexual harassment we would hav...   \n",
       "3  b' Israel clears troops who killed Reuters cam...   \n",
       "4  b'Swedish wrestler Ara Abrahamian throws away ...   \n",
       "\n",
       "                                                Top4  \\\n",
       "0  b'Russian tanks are moving towards the capital...   \n",
       "1  b'Georgian army flees in disarray as Russians ...   \n",
       "2  b\"Al-Qa'eda is losing support in Iraq because ...   \n",
       "3  b'Britain\\'s policy of being tough on drugs is...   \n",
       "4  b'Russia exaggerated the death toll in South O...   \n",
       "\n",
       "                                                Top5  \\\n",
       "0  b\"Afghan children raped with 'impunity,' U.N. ...   \n",
       "1      b\"Olympic opening ceremony fireworks 'faked'\"   \n",
       "2  b'Ceasefire in Georgia: Putin Outmaneuvers the...   \n",
       "3  b'Body of 14 year old found in trunk; Latest (...   \n",
       "4  b'Missile That Killed 9 Inside Pakistan May Ha...   \n",
       "\n",
       "                                                Top6  \\\n",
       "0  b'150 Russian tanks have entered South Ossetia...   \n",
       "1  b'What were the Mossad with fraudulent New Zea...   \n",
       "2  b'Why Microsoft and Intel tried to kill the XO...   \n",
       "3  b'China has moved 10 *million* quake survivors...   \n",
       "4  b\"Rushdie Condemns Random House's Refusal to P...   \n",
       "\n",
       "                                                Top7  \\\n",
       "0  b\"Breaking: Georgia invades South Ossetia, Rus...   \n",
       "1  b'Russia angered by Israeli military sale to G...   \n",
       "2  b'Stratfor: The Russo-Georgian War and the Bal...   \n",
       "3  b\"Bush announces Operation Get All Up In Russi...   \n",
       "4  b'Poland and US agree to missle defense deal. ...   \n",
       "\n",
       "                                                Top8  ...  \\\n",
       "0  b\"The 'enemy combatent' trials are nothing but...  ...   \n",
       "1  b'An American citizen living in S.Ossetia blam...  ...   \n",
       "2  b\"I'm Trying to Get a Sense of This Whole Geor...  ...   \n",
       "3             b'Russian forces sink Georgian ships '  ...   \n",
       "4  b'Will the Russians conquer Tblisi? Bet on it,...  ...   \n",
       "\n",
       "                                               Top16  \\\n",
       "0  b'Georgia Invades South Ossetia - if Russia ge...   \n",
       "1  b'Israel and the US behind the Georgian aggres...   \n",
       "2  b'U.S. troops still in Georgia (did you know t...   \n",
       "3                      b'Elephants extinct by 2020?'   \n",
       "4  b'Bank analyst forecast Georgian crisis 2 days...   \n",
       "\n",
       "                                               Top17  \\\n",
       "0                b'Al-Qaeda Faces Islamist Backlash'   \n",
       "1  b'\"Do not believe TV, neither Russian nor Geor...   \n",
       "2       b'Why Russias response to Georgia was right'   \n",
       "3  b'US humanitarian missions soon in Georgia - i...   \n",
       "4  b\"Georgia confict could set back Russia's US r...   \n",
       "\n",
       "                                               Top18  \\\n",
       "0  b'Condoleezza Rice: \"The US would not act to p...   \n",
       "1  b'Riots are still going on in Montreal (Canada...   \n",
       "2  b'Gorbachev accuses U.S. of making a \"serious ...   \n",
       "3             b\"Georgia's DDOS came from US sources\"   \n",
       "4  b'War in the Caucasus is as much the product o...   \n",
       "\n",
       "                                               Top19  \\\n",
       "0  b'This is a busy day:  The European Union has ...   \n",
       "1    b'China to overtake US as largest manufacturer'   \n",
       "2         b'Russia, Georgia, and NATO: Cold War Two'   \n",
       "3  b'Russian convoy heads into Georgia, violating...   \n",
       "4  b'\"Non-media\" photos of South Ossetia/Georgia ...   \n",
       "\n",
       "                                               Top20  \\\n",
       "0  b\"Georgia will withdraw 1,000 soldiers from Ir...   \n",
       "1                     b'War in South Ossetia [PICS]'   \n",
       "2  b'Remember that adorable 62-year-old who led y...   \n",
       "3  b'Israeli defence minister: US against strike ...   \n",
       "4  b'Georgian TV reporter shot by Russian sniper ...   \n",
       "\n",
       "                                               Top21  \\\n",
       "0  b'Why the Pentagon Thinks Attacking Iran is a ...   \n",
       "1  b'Israeli Physicians Group Condemns State Tort...   \n",
       "2          b'War in Georgia: The Israeli connection'   \n",
       "3                     b'Gorbachev: We Had No Choice'   \n",
       "4  b'Saudi Arabia: Mother moves to block child ma...   \n",
       "\n",
       "                                               Top22  \\\n",
       "0  b'Caucasus in crisis: Georgia invades South Os...   \n",
       "1  b' Russia has just beaten the United States ov...   \n",
       "2  b'All signs point to the US encouraging Georgi...   \n",
       "3  b'Witness: Russian forces head towards Tbilisi...   \n",
       "4   b'Taliban wages war on humanitarian aid workers'   \n",
       "\n",
       "                                               Top23  \\\n",
       "0  b'Indian shoe manufactory  - And again in a se...   \n",
       "1  b'Perhaps *the* question about the Georgia - R...   \n",
       "2  b'Christopher King argues that the US and NATO...   \n",
       "3  b' Quarter of Russians blame U.S. for conflict...   \n",
       "4  b'Russia: World  \"can forget about\" Georgia\\'s...   \n",
       "\n",
       "                                               Top24  \\\n",
       "0  b'Visitors Suffering from Mental Illnesses Ban...   \n",
       "1                 b'Russia is so much better at war'   \n",
       "2                        b'America: The New Mexico?'   \n",
       "3  b'Georgian president  says US military will ta...   \n",
       "4  b'Darfur rebels accuse Sudan of mounting major...   \n",
       "\n",
       "                                               Top25  \n",
       "0           b\"No Help for Mexico's Kidnapping Surge\"  \n",
       "1  b\"So this is what it's come to: trading sex fo...  \n",
       "2  b\"BBC NEWS | Asia-Pacific | Extinction 'by man...  \n",
       "3  b'2006: Nobel laureate Aleksander Solzhenitsyn...  \n",
       "4  b'Philippines : Peace Advocate say Muslims nee...  \n",
       "\n",
       "[5 rows x 27 columns]"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-08-07T04:44:49.642865Z",
     "iopub.status.busy": "2020-08-07T04:44:49.641991Z",
     "iopub.status.idle": "2020-08-07T04:44:49.648043Z",
     "shell.execute_reply": "2020-08-07T04:44:49.647231Z"
    },
    "papermill": {
     "duration": 0.060837,
     "end_time": "2020-08-07T04:44:49.648202",
     "exception": false,
     "start_time": "2020-08-07T04:44:49.587365",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Date</th>\n",
       "      <th>Label</th>\n",
       "      <th>Top1</th>\n",
       "      <th>Top2</th>\n",
       "      <th>Top3</th>\n",
       "      <th>Top4</th>\n",
       "      <th>Top5</th>\n",
       "      <th>Top6</th>\n",
       "      <th>Top7</th>\n",
       "      <th>Top8</th>\n",
       "      <th>...</th>\n",
       "      <th>Top16</th>\n",
       "      <th>Top17</th>\n",
       "      <th>Top18</th>\n",
       "      <th>Top19</th>\n",
       "      <th>Top20</th>\n",
       "      <th>Top21</th>\n",
       "      <th>Top22</th>\n",
       "      <th>Top23</th>\n",
       "      <th>Top24</th>\n",
       "      <th>Top25</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1984</th>\n",
       "      <td>2016-06-27</td>\n",
       "      <td>0</td>\n",
       "      <td>Barclays and RBS shares suspended from trading...</td>\n",
       "      <td>Pope says Church should ask forgiveness from g...</td>\n",
       "      <td>Poland 'shocked' by xenophobic abuse of Poles ...</td>\n",
       "      <td>There will be no second referendum, cabinet ag...</td>\n",
       "      <td>Scotland welcome to join EU, Merkel ally says</td>\n",
       "      <td>Sterling dips below Friday's 31-year low amid ...</td>\n",
       "      <td>No negative news about South African President...</td>\n",
       "      <td>Surge in Hate Crimes in the U.K. Following U.K...</td>\n",
       "      <td>...</td>\n",
       "      <td>German lawyers to probe Erdogan over alleged w...</td>\n",
       "      <td>Boris Johnson says the UK will continue to \"in...</td>\n",
       "      <td>Richard Branson is calling on the UK governmen...</td>\n",
       "      <td>Turkey 'sorry for downing Russian jet'</td>\n",
       "      <td>Edward Snowden lawyer vows new push for pardon...</td>\n",
       "      <td>Brexit opinion poll reveals majority don't wan...</td>\n",
       "      <td>Conservative MP Leave Campaigner: \"The leave c...</td>\n",
       "      <td>Economists predict UK recession, further weake...</td>\n",
       "      <td>New EU 'superstate plan by France, Germany: Cr...</td>\n",
       "      <td>Pakistani clerics declare transgender marriage...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1985</th>\n",
       "      <td>2016-06-28</td>\n",
       "      <td>1</td>\n",
       "      <td>2,500 Scientists To Australia: If You Want To ...</td>\n",
       "      <td>The personal details of 112,000 French police ...</td>\n",
       "      <td>S&amp;amp;P cuts United Kingdom sovereign credit r...</td>\n",
       "      <td>Huge helium deposit found in Africa</td>\n",
       "      <td>CEO of the South African state broadcaster qui...</td>\n",
       "      <td>Brexit cost investors $2 trillion, the worst o...</td>\n",
       "      <td>Hong Kong democracy activists call for return ...</td>\n",
       "      <td>Brexit: Iceland president says UK can join 'tr...</td>\n",
       "      <td>...</td>\n",
       "      <td>US, Canada and Mexico pledge 50% of power from...</td>\n",
       "      <td>There is increasing evidence that Australia is...</td>\n",
       "      <td>Richard Branson, the founder of Virgin Group, ...</td>\n",
       "      <td>37,000-yr-old skull from Borneo reveals surpri...</td>\n",
       "      <td>Palestinians stone Western Wall worshipers; po...</td>\n",
       "      <td>Jean-Claude Juncker asks Farage: Why are you h...</td>\n",
       "      <td>\"Romanians for Remainians\" offering a new home...</td>\n",
       "      <td>Brexit: Gibraltar in talks with Scotland to st...</td>\n",
       "      <td>8 Suicide Bombers Strike Lebanon</td>\n",
       "      <td>Mexico's security forces routinely use 'sexual...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1986</th>\n",
       "      <td>2016-06-29</td>\n",
       "      <td>1</td>\n",
       "      <td>Explosion At Airport In Istanbul</td>\n",
       "      <td>Yemeni former president: Terrorism is the offs...</td>\n",
       "      <td>UK must accept freedom of movement to access E...</td>\n",
       "      <td>Devastated: scientists too late to captive bre...</td>\n",
       "      <td>British Labor Party leader Jeremy Corbyn loses...</td>\n",
       "      <td>A Muslim Shop in the UK Was Just Firebombed Wh...</td>\n",
       "      <td>Mexican Authorities Sexually Torture Women in ...</td>\n",
       "      <td>UK shares and pound continue to recover</td>\n",
       "      <td>...</td>\n",
       "      <td>Escape Tunnel, Dug by Hand, Is Found at Holoca...</td>\n",
       "      <td>The land under Beijing is sinking by as much a...</td>\n",
       "      <td>Car bomb and Anti-Islamic attack on Mosque in ...</td>\n",
       "      <td>Emaciated lions in Taiz Zoo are trapped in blo...</td>\n",
       "      <td>Rupert Murdoch describes Brexit as 'wonderful'...</td>\n",
       "      <td>More than 40 killed in Yemen suicide attacks</td>\n",
       "      <td>Google Found Disastrous Symantec and Norton Vu...</td>\n",
       "      <td>Extremist violence on the rise in Germany: Dom...</td>\n",
       "      <td>BBC News: Labour MPs pass Corbyn no-confidence...</td>\n",
       "      <td>Tiny New Zealand town with 'too many jobs' lau...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1987</th>\n",
       "      <td>2016-06-30</td>\n",
       "      <td>1</td>\n",
       "      <td>Jamaica proposes marijuana dispensers for tour...</td>\n",
       "      <td>Stephen Hawking says pollution and 'stupidity'...</td>\n",
       "      <td>Boris Johnson says he will not run for Tory pa...</td>\n",
       "      <td>Six gay men in Ivory Coast were abused and for...</td>\n",
       "      <td>Switzerland denies citizenship to Muslim immig...</td>\n",
       "      <td>Palestinian terrorist stabs israeli teen girl ...</td>\n",
       "      <td>Puerto Rico will default on $1 billion of debt...</td>\n",
       "      <td>Republic of Ireland fans to be awarded medal f...</td>\n",
       "      <td>...</td>\n",
       "      <td>Googles free wifi at Indian railway stations i...</td>\n",
       "      <td>Mounting evidence suggests 'hobbits' were wipe...</td>\n",
       "      <td>The men who carried out Tuesday's terror attac...</td>\n",
       "      <td>Calls to suspend Saudi Arabia from UN Human Ri...</td>\n",
       "      <td>More Than 100 Nobel Laureates Call Out Greenpe...</td>\n",
       "      <td>British pedophile sentenced to 85 years in US ...</td>\n",
       "      <td>US permitted 1,200 offshore fracks in Gulf of ...</td>\n",
       "      <td>We will be swimming in ridicule - French beach...</td>\n",
       "      <td>UEFA says no minutes of silence for Istanbul v...</td>\n",
       "      <td>Law Enforcement Sources: Gun Used in Paris Ter...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1988</th>\n",
       "      <td>2016-07-01</td>\n",
       "      <td>1</td>\n",
       "      <td>A 117-year-old woman in Mexico City finally re...</td>\n",
       "      <td>IMF chief backs Athens as permanent Olympic host</td>\n",
       "      <td>The president of France says if Brexit won, so...</td>\n",
       "      <td>British Man Who Must Give Police 24 Hours' Not...</td>\n",
       "      <td>100+ Nobel laureates urge Greenpeace to stop o...</td>\n",
       "      <td>Brazil: Huge spike in number of police killing...</td>\n",
       "      <td>Austria's highest court annuls presidential el...</td>\n",
       "      <td>Facebook wins privacy case, can track any Belg...</td>\n",
       "      <td>...</td>\n",
       "      <td>The United States has placed Myanmar, Uzbekist...</td>\n",
       "      <td>S&amp;amp;P revises European Union credit rating t...</td>\n",
       "      <td>India gets $1 billion loan from World Bank for...</td>\n",
       "      <td>U.S. sailors detained by Iran spoke too much u...</td>\n",
       "      <td>Mass fish kill in Vietnam solved as Taiwan ste...</td>\n",
       "      <td>Philippines president Rodrigo Duterte urges pe...</td>\n",
       "      <td>Spain arrests three Pakistanis accused of prom...</td>\n",
       "      <td>Venezuela, where anger over food shortages is ...</td>\n",
       "      <td>A Hindu temple worker has been killed by three...</td>\n",
       "      <td>Ozone layer hole seems to be healing - US &amp;amp...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 27 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "            Date  Label                                               Top1  \\\n",
       "1984  2016-06-27      0  Barclays and RBS shares suspended from trading...   \n",
       "1985  2016-06-28      1  2,500 Scientists To Australia: If You Want To ...   \n",
       "1986  2016-06-29      1                   Explosion At Airport In Istanbul   \n",
       "1987  2016-06-30      1  Jamaica proposes marijuana dispensers for tour...   \n",
       "1988  2016-07-01      1  A 117-year-old woman in Mexico City finally re...   \n",
       "\n",
       "                                                   Top2  \\\n",
       "1984  Pope says Church should ask forgiveness from g...   \n",
       "1985  The personal details of 112,000 French police ...   \n",
       "1986  Yemeni former president: Terrorism is the offs...   \n",
       "1987  Stephen Hawking says pollution and 'stupidity'...   \n",
       "1988   IMF chief backs Athens as permanent Olympic host   \n",
       "\n",
       "                                                   Top3  \\\n",
       "1984  Poland 'shocked' by xenophobic abuse of Poles ...   \n",
       "1985  S&amp;P cuts United Kingdom sovereign credit r...   \n",
       "1986  UK must accept freedom of movement to access E...   \n",
       "1987  Boris Johnson says he will not run for Tory pa...   \n",
       "1988  The president of France says if Brexit won, so...   \n",
       "\n",
       "                                                   Top4  \\\n",
       "1984  There will be no second referendum, cabinet ag...   \n",
       "1985                Huge helium deposit found in Africa   \n",
       "1986  Devastated: scientists too late to captive bre...   \n",
       "1987  Six gay men in Ivory Coast were abused and for...   \n",
       "1988  British Man Who Must Give Police 24 Hours' Not...   \n",
       "\n",
       "                                                   Top5  \\\n",
       "1984      Scotland welcome to join EU, Merkel ally says   \n",
       "1985  CEO of the South African state broadcaster qui...   \n",
       "1986  British Labor Party leader Jeremy Corbyn loses...   \n",
       "1987  Switzerland denies citizenship to Muslim immig...   \n",
       "1988  100+ Nobel laureates urge Greenpeace to stop o...   \n",
       "\n",
       "                                                   Top6  \\\n",
       "1984  Sterling dips below Friday's 31-year low amid ...   \n",
       "1985  Brexit cost investors $2 trillion, the worst o...   \n",
       "1986  A Muslim Shop in the UK Was Just Firebombed Wh...   \n",
       "1987  Palestinian terrorist stabs israeli teen girl ...   \n",
       "1988  Brazil: Huge spike in number of police killing...   \n",
       "\n",
       "                                                   Top7  \\\n",
       "1984  No negative news about South African President...   \n",
       "1985  Hong Kong democracy activists call for return ...   \n",
       "1986  Mexican Authorities Sexually Torture Women in ...   \n",
       "1987  Puerto Rico will default on $1 billion of debt...   \n",
       "1988  Austria's highest court annuls presidential el...   \n",
       "\n",
       "                                                   Top8  ...  \\\n",
       "1984  Surge in Hate Crimes in the U.K. Following U.K...  ...   \n",
       "1985  Brexit: Iceland president says UK can join 'tr...  ...   \n",
       "1986            UK shares and pound continue to recover  ...   \n",
       "1987  Republic of Ireland fans to be awarded medal f...  ...   \n",
       "1988  Facebook wins privacy case, can track any Belg...  ...   \n",
       "\n",
       "                                                  Top16  \\\n",
       "1984  German lawyers to probe Erdogan over alleged w...   \n",
       "1985  US, Canada and Mexico pledge 50% of power from...   \n",
       "1986  Escape Tunnel, Dug by Hand, Is Found at Holoca...   \n",
       "1987  Googles free wifi at Indian railway stations i...   \n",
       "1988  The United States has placed Myanmar, Uzbekist...   \n",
       "\n",
       "                                                  Top17  \\\n",
       "1984  Boris Johnson says the UK will continue to \"in...   \n",
       "1985  There is increasing evidence that Australia is...   \n",
       "1986  The land under Beijing is sinking by as much a...   \n",
       "1987  Mounting evidence suggests 'hobbits' were wipe...   \n",
       "1988  S&amp;P revises European Union credit rating t...   \n",
       "\n",
       "                                                  Top18  \\\n",
       "1984  Richard Branson is calling on the UK governmen...   \n",
       "1985  Richard Branson, the founder of Virgin Group, ...   \n",
       "1986  Car bomb and Anti-Islamic attack on Mosque in ...   \n",
       "1987  The men who carried out Tuesday's terror attac...   \n",
       "1988  India gets $1 billion loan from World Bank for...   \n",
       "\n",
       "                                                  Top19  \\\n",
       "1984             Turkey 'sorry for downing Russian jet'   \n",
       "1985  37,000-yr-old skull from Borneo reveals surpri...   \n",
       "1986  Emaciated lions in Taiz Zoo are trapped in blo...   \n",
       "1987  Calls to suspend Saudi Arabia from UN Human Ri...   \n",
       "1988  U.S. sailors detained by Iran spoke too much u...   \n",
       "\n",
       "                                                  Top20  \\\n",
       "1984  Edward Snowden lawyer vows new push for pardon...   \n",
       "1985  Palestinians stone Western Wall worshipers; po...   \n",
       "1986  Rupert Murdoch describes Brexit as 'wonderful'...   \n",
       "1987  More Than 100 Nobel Laureates Call Out Greenpe...   \n",
       "1988  Mass fish kill in Vietnam solved as Taiwan ste...   \n",
       "\n",
       "                                                  Top21  \\\n",
       "1984  Brexit opinion poll reveals majority don't wan...   \n",
       "1985  Jean-Claude Juncker asks Farage: Why are you h...   \n",
       "1986       More than 40 killed in Yemen suicide attacks   \n",
       "1987  British pedophile sentenced to 85 years in US ...   \n",
       "1988  Philippines president Rodrigo Duterte urges pe...   \n",
       "\n",
       "                                                  Top22  \\\n",
       "1984  Conservative MP Leave Campaigner: \"The leave c...   \n",
       "1985  \"Romanians for Remainians\" offering a new home...   \n",
       "1986  Google Found Disastrous Symantec and Norton Vu...   \n",
       "1987  US permitted 1,200 offshore fracks in Gulf of ...   \n",
       "1988  Spain arrests three Pakistanis accused of prom...   \n",
       "\n",
       "                                                  Top23  \\\n",
       "1984  Economists predict UK recession, further weake...   \n",
       "1985  Brexit: Gibraltar in talks with Scotland to st...   \n",
       "1986  Extremist violence on the rise in Germany: Dom...   \n",
       "1987  We will be swimming in ridicule - French beach...   \n",
       "1988  Venezuela, where anger over food shortages is ...   \n",
       "\n",
       "                                                  Top24  \\\n",
       "1984  New EU 'superstate plan by France, Germany: Cr...   \n",
       "1985                   8 Suicide Bombers Strike Lebanon   \n",
       "1986  BBC News: Labour MPs pass Corbyn no-confidence...   \n",
       "1987  UEFA says no minutes of silence for Istanbul v...   \n",
       "1988  A Hindu temple worker has been killed by three...   \n",
       "\n",
       "                                                  Top25  \n",
       "1984  Pakistani clerics declare transgender marriage...  \n",
       "1985  Mexico's security forces routinely use 'sexual...  \n",
       "1986  Tiny New Zealand town with 'too many jobs' lau...  \n",
       "1987  Law Enforcement Sources: Gun Used in Paris Ter...  \n",
       "1988  Ozone layer hole seems to be healing - US &amp...  \n",
       "\n",
       "[5 rows x 27 columns]"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.tail()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-08-07T04:44:49.703482Z",
     "iopub.status.busy": "2020-08-07T04:44:49.702385Z",
     "iopub.status.idle": "2020-08-07T04:44:49.724706Z",
     "shell.execute_reply": "2020-08-07T04:44:49.725547Z"
    },
    "papermill": {
     "duration": 0.057152,
     "end_time": "2020-08-07T04:44:49.725788",
     "exception": false,
     "start_time": "2020-08-07T04:44:49.668636",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 1989 entries, 0 to 1988\n",
      "Data columns (total 27 columns):\n",
      " #   Column  Non-Null Count  Dtype \n",
      "---  ------  --------------  ----- \n",
      " 0   Date    1989 non-null   object\n",
      " 1   Label   1989 non-null   int64 \n",
      " 2   Top1    1989 non-null   object\n",
      " 3   Top2    1989 non-null   object\n",
      " 4   Top3    1989 non-null   object\n",
      " 5   Top4    1989 non-null   object\n",
      " 6   Top5    1989 non-null   object\n",
      " 7   Top6    1989 non-null   object\n",
      " 8   Top7    1989 non-null   object\n",
      " 9   Top8    1989 non-null   object\n",
      " 10  Top9    1989 non-null   object\n",
      " 11  Top10   1989 non-null   object\n",
      " 12  Top11   1989 non-null   object\n",
      " 13  Top12   1989 non-null   object\n",
      " 14  Top13   1989 non-null   object\n",
      " 15  Top14   1989 non-null   object\n",
      " 16  Top15   1989 non-null   object\n",
      " 17  Top16   1989 non-null   object\n",
      " 18  Top17   1989 non-null   object\n",
      " 19  Top18   1989 non-null   object\n",
      " 20  Top19   1989 non-null   object\n",
      " 21  Top20   1989 non-null   object\n",
      " 22  Top21   1989 non-null   object\n",
      " 23  Top22   1989 non-null   object\n",
      " 24  Top23   1988 non-null   object\n",
      " 25  Top24   1986 non-null   object\n",
      " 26  Top25   1986 non-null   object\n",
      "dtypes: int64(1), object(26)\n",
      "memory usage: 419.7+ KB\n"
     ]
    }
   ],
   "source": [
    "df.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-08-07T04:44:49.782193Z",
     "iopub.status.busy": "2020-08-07T04:44:49.781156Z",
     "iopub.status.idle": "2020-08-07T04:44:49.801520Z",
     "shell.execute_reply": "2020-08-07T04:44:49.800707Z"
    },
    "papermill": {
     "duration": 0.055508,
     "end_time": "2020-08-07T04:44:49.801693",
     "exception": false,
     "start_time": "2020-08-07T04:44:49.746185",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>count</th>\n",
       "      <th>mean</th>\n",
       "      <th>std</th>\n",
       "      <th>min</th>\n",
       "      <th>25%</th>\n",
       "      <th>50%</th>\n",
       "      <th>75%</th>\n",
       "      <th>max</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Label</th>\n",
       "      <td>1989.0</td>\n",
       "      <td>0.535445</td>\n",
       "      <td>0.498867</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        count      mean       std  min  25%  50%  75%  max\n",
       "Label  1989.0  0.535445  0.498867  0.0  0.0  1.0  1.0  1.0"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.describe().T"
   ]
  },
  {
   "cell_type": "markdown",
   "execution_count": null,
   "metadata": {
    "papermill": {
     "duration": 0.020324,
     "end_time": "2020-08-07T04:44:49.842856",
     "exception": false,
     "start_time": "2020-08-07T04:44:49.822532",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# Observation\n",
    "\n",
    "Here, after looking at the data, I feel we donot require date column and we can combine all the top 25 news into one single column. Hence, we can drop date column and merge the headline columns together. Also, since I would be using BERT for final classification and BERT allows only 512 tokens at a time, hence I would be taking combination of top 10 news only. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-08-07T04:44:49.924141Z",
     "iopub.status.busy": "2020-08-07T04:44:49.922916Z",
     "iopub.status.idle": "2020-08-07T04:44:49.929821Z",
     "shell.execute_reply": "2020-08-07T04:44:49.929041Z"
    },
    "papermill": {
     "duration": 0.066182,
     "end_time": "2020-08-07T04:44:49.929988",
     "exception": false,
     "start_time": "2020-08-07T04:44:49.863806",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Label</th>\n",
       "      <th>Top1</th>\n",
       "      <th>Top2</th>\n",
       "      <th>Top3</th>\n",
       "      <th>Top4</th>\n",
       "      <th>Top5</th>\n",
       "      <th>Top6</th>\n",
       "      <th>Top7</th>\n",
       "      <th>Top8</th>\n",
       "      <th>Top9</th>\n",
       "      <th>...</th>\n",
       "      <th>Top16</th>\n",
       "      <th>Top17</th>\n",
       "      <th>Top18</th>\n",
       "      <th>Top19</th>\n",
       "      <th>Top20</th>\n",
       "      <th>Top21</th>\n",
       "      <th>Top22</th>\n",
       "      <th>Top23</th>\n",
       "      <th>Top24</th>\n",
       "      <th>Top25</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>b\"Georgia 'downs two Russian warplanes' as cou...</td>\n",
       "      <td>b'BREAKING: Musharraf to be impeached.'</td>\n",
       "      <td>b'Russia Today: Columns of troops roll into So...</td>\n",
       "      <td>b'Russian tanks are moving towards the capital...</td>\n",
       "      <td>b\"Afghan children raped with 'impunity,' U.N. ...</td>\n",
       "      <td>b'150 Russian tanks have entered South Ossetia...</td>\n",
       "      <td>b\"Breaking: Georgia invades South Ossetia, Rus...</td>\n",
       "      <td>b\"The 'enemy combatent' trials are nothing but...</td>\n",
       "      <td>b'Georgian troops retreat from S. Osettain cap...</td>\n",
       "      <td>...</td>\n",
       "      <td>b'Georgia Invades South Ossetia - if Russia ge...</td>\n",
       "      <td>b'Al-Qaeda Faces Islamist Backlash'</td>\n",
       "      <td>b'Condoleezza Rice: \"The US would not act to p...</td>\n",
       "      <td>b'This is a busy day:  The European Union has ...</td>\n",
       "      <td>b\"Georgia will withdraw 1,000 soldiers from Ir...</td>\n",
       "      <td>b'Why the Pentagon Thinks Attacking Iran is a ...</td>\n",
       "      <td>b'Caucasus in crisis: Georgia invades South Os...</td>\n",
       "      <td>b'Indian shoe manufactory  - And again in a se...</td>\n",
       "      <td>b'Visitors Suffering from Mental Illnesses Ban...</td>\n",
       "      <td>b\"No Help for Mexico's Kidnapping Surge\"</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>b'Why wont America and Nato help us? If they w...</td>\n",
       "      <td>b'Bush puts foot down on Georgian conflict'</td>\n",
       "      <td>b\"Jewish Georgian minister: Thanks to Israeli ...</td>\n",
       "      <td>b'Georgian army flees in disarray as Russians ...</td>\n",
       "      <td>b\"Olympic opening ceremony fireworks 'faked'\"</td>\n",
       "      <td>b'What were the Mossad with fraudulent New Zea...</td>\n",
       "      <td>b'Russia angered by Israeli military sale to G...</td>\n",
       "      <td>b'An American citizen living in S.Ossetia blam...</td>\n",
       "      <td>b'Welcome To World War IV! Now In High Definit...</td>\n",
       "      <td>...</td>\n",
       "      <td>b'Israel and the US behind the Georgian aggres...</td>\n",
       "      <td>b'\"Do not believe TV, neither Russian nor Geor...</td>\n",
       "      <td>b'Riots are still going on in Montreal (Canada...</td>\n",
       "      <td>b'China to overtake US as largest manufacturer'</td>\n",
       "      <td>b'War in South Ossetia [PICS]'</td>\n",
       "      <td>b'Israeli Physicians Group Condemns State Tort...</td>\n",
       "      <td>b' Russia has just beaten the United States ov...</td>\n",
       "      <td>b'Perhaps *the* question about the Georgia - R...</td>\n",
       "      <td>b'Russia is so much better at war'</td>\n",
       "      <td>b\"So this is what it's come to: trading sex fo...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0</td>\n",
       "      <td>b'Remember that adorable 9-year-old who sang a...</td>\n",
       "      <td>b\"Russia 'ends Georgia operation'\"</td>\n",
       "      <td>b'\"If we had no sexual harassment we would hav...</td>\n",
       "      <td>b\"Al-Qa'eda is losing support in Iraq because ...</td>\n",
       "      <td>b'Ceasefire in Georgia: Putin Outmaneuvers the...</td>\n",
       "      <td>b'Why Microsoft and Intel tried to kill the XO...</td>\n",
       "      <td>b'Stratfor: The Russo-Georgian War and the Bal...</td>\n",
       "      <td>b\"I'm Trying to Get a Sense of This Whole Geor...</td>\n",
       "      <td>b\"The US military was surprised by the timing ...</td>\n",
       "      <td>...</td>\n",
       "      <td>b'U.S. troops still in Georgia (did you know t...</td>\n",
       "      <td>b'Why Russias response to Georgia was right'</td>\n",
       "      <td>b'Gorbachev accuses U.S. of making a \"serious ...</td>\n",
       "      <td>b'Russia, Georgia, and NATO: Cold War Two'</td>\n",
       "      <td>b'Remember that adorable 62-year-old who led y...</td>\n",
       "      <td>b'War in Georgia: The Israeli connection'</td>\n",
       "      <td>b'All signs point to the US encouraging Georgi...</td>\n",
       "      <td>b'Christopher King argues that the US and NATO...</td>\n",
       "      <td>b'America: The New Mexico?'</td>\n",
       "      <td>b\"BBC NEWS | Asia-Pacific | Extinction 'by man...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0</td>\n",
       "      <td>b' U.S. refuses Israel weapons to attack Iran:...</td>\n",
       "      <td>b\"When the president ordered to attack Tskhinv...</td>\n",
       "      <td>b' Israel clears troops who killed Reuters cam...</td>\n",
       "      <td>b'Britain\\'s policy of being tough on drugs is...</td>\n",
       "      <td>b'Body of 14 year old found in trunk; Latest (...</td>\n",
       "      <td>b'China has moved 10 *million* quake survivors...</td>\n",
       "      <td>b\"Bush announces Operation Get All Up In Russi...</td>\n",
       "      <td>b'Russian forces sink Georgian ships '</td>\n",
       "      <td>b\"The commander of a Navy air reconnaissance s...</td>\n",
       "      <td>...</td>\n",
       "      <td>b'Elephants extinct by 2020?'</td>\n",
       "      <td>b'US humanitarian missions soon in Georgia - i...</td>\n",
       "      <td>b\"Georgia's DDOS came from US sources\"</td>\n",
       "      <td>b'Russian convoy heads into Georgia, violating...</td>\n",
       "      <td>b'Israeli defence minister: US against strike ...</td>\n",
       "      <td>b'Gorbachev: We Had No Choice'</td>\n",
       "      <td>b'Witness: Russian forces head towards Tbilisi...</td>\n",
       "      <td>b' Quarter of Russians blame U.S. for conflict...</td>\n",
       "      <td>b'Georgian president  says US military will ta...</td>\n",
       "      <td>b'2006: Nobel laureate Aleksander Solzhenitsyn...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "      <td>b'All the experts admit that we should legalis...</td>\n",
       "      <td>b'War in South Osetia - 89 pictures made by a ...</td>\n",
       "      <td>b'Swedish wrestler Ara Abrahamian throws away ...</td>\n",
       "      <td>b'Russia exaggerated the death toll in South O...</td>\n",
       "      <td>b'Missile That Killed 9 Inside Pakistan May Ha...</td>\n",
       "      <td>b\"Rushdie Condemns Random House's Refusal to P...</td>\n",
       "      <td>b'Poland and US agree to missle defense deal. ...</td>\n",
       "      <td>b'Will the Russians conquer Tblisi? Bet on it,...</td>\n",
       "      <td>b'Russia exaggerating South Ossetian death tol...</td>\n",
       "      <td>...</td>\n",
       "      <td>b'Bank analyst forecast Georgian crisis 2 days...</td>\n",
       "      <td>b\"Georgia confict could set back Russia's US r...</td>\n",
       "      <td>b'War in the Caucasus is as much the product o...</td>\n",
       "      <td>b'\"Non-media\" photos of South Ossetia/Georgia ...</td>\n",
       "      <td>b'Georgian TV reporter shot by Russian sniper ...</td>\n",
       "      <td>b'Saudi Arabia: Mother moves to block child ma...</td>\n",
       "      <td>b'Taliban wages war on humanitarian aid workers'</td>\n",
       "      <td>b'Russia: World  \"can forget about\" Georgia\\'s...</td>\n",
       "      <td>b'Darfur rebels accuse Sudan of mounting major...</td>\n",
       "      <td>b'Philippines : Peace Advocate say Muslims nee...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 26 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   Label                                               Top1  \\\n",
       "0      0  b\"Georgia 'downs two Russian warplanes' as cou...   \n",
       "1      1  b'Why wont America and Nato help us? If they w...   \n",
       "2      0  b'Remember that adorable 9-year-old who sang a...   \n",
       "3      0  b' U.S. refuses Israel weapons to attack Iran:...   \n",
       "4      1  b'All the experts admit that we should legalis...   \n",
       "\n",
       "                                                Top2  \\\n",
       "0            b'BREAKING: Musharraf to be impeached.'   \n",
       "1        b'Bush puts foot down on Georgian conflict'   \n",
       "2                 b\"Russia 'ends Georgia operation'\"   \n",
       "3  b\"When the president ordered to attack Tskhinv...   \n",
       "4  b'War in South Osetia - 89 pictures made by a ...   \n",
       "\n",
       "                                                Top3  \\\n",
       "0  b'Russia Today: Columns of troops roll into So...   \n",
       "1  b\"Jewish Georgian minister: Thanks to Israeli ...   \n",
       "2  b'\"If we had no sexual harassment we would hav...   \n",
       "3  b' Israel clears troops who killed Reuters cam...   \n",
       "4  b'Swedish wrestler Ara Abrahamian throws away ...   \n",
       "\n",
       "                                                Top4  \\\n",
       "0  b'Russian tanks are moving towards the capital...   \n",
       "1  b'Georgian army flees in disarray as Russians ...   \n",
       "2  b\"Al-Qa'eda is losing support in Iraq because ...   \n",
       "3  b'Britain\\'s policy of being tough on drugs is...   \n",
       "4  b'Russia exaggerated the death toll in South O...   \n",
       "\n",
       "                                                Top5  \\\n",
       "0  b\"Afghan children raped with 'impunity,' U.N. ...   \n",
       "1      b\"Olympic opening ceremony fireworks 'faked'\"   \n",
       "2  b'Ceasefire in Georgia: Putin Outmaneuvers the...   \n",
       "3  b'Body of 14 year old found in trunk; Latest (...   \n",
       "4  b'Missile That Killed 9 Inside Pakistan May Ha...   \n",
       "\n",
       "                                                Top6  \\\n",
       "0  b'150 Russian tanks have entered South Ossetia...   \n",
       "1  b'What were the Mossad with fraudulent New Zea...   \n",
       "2  b'Why Microsoft and Intel tried to kill the XO...   \n",
       "3  b'China has moved 10 *million* quake survivors...   \n",
       "4  b\"Rushdie Condemns Random House's Refusal to P...   \n",
       "\n",
       "                                                Top7  \\\n",
       "0  b\"Breaking: Georgia invades South Ossetia, Rus...   \n",
       "1  b'Russia angered by Israeli military sale to G...   \n",
       "2  b'Stratfor: The Russo-Georgian War and the Bal...   \n",
       "3  b\"Bush announces Operation Get All Up In Russi...   \n",
       "4  b'Poland and US agree to missle defense deal. ...   \n",
       "\n",
       "                                                Top8  \\\n",
       "0  b\"The 'enemy combatent' trials are nothing but...   \n",
       "1  b'An American citizen living in S.Ossetia blam...   \n",
       "2  b\"I'm Trying to Get a Sense of This Whole Geor...   \n",
       "3             b'Russian forces sink Georgian ships '   \n",
       "4  b'Will the Russians conquer Tblisi? Bet on it,...   \n",
       "\n",
       "                                                Top9  ...  \\\n",
       "0  b'Georgian troops retreat from S. Osettain cap...  ...   \n",
       "1  b'Welcome To World War IV! Now In High Definit...  ...   \n",
       "2  b\"The US military was surprised by the timing ...  ...   \n",
       "3  b\"The commander of a Navy air reconnaissance s...  ...   \n",
       "4  b'Russia exaggerating South Ossetian death tol...  ...   \n",
       "\n",
       "                                               Top16  \\\n",
       "0  b'Georgia Invades South Ossetia - if Russia ge...   \n",
       "1  b'Israel and the US behind the Georgian aggres...   \n",
       "2  b'U.S. troops still in Georgia (did you know t...   \n",
       "3                      b'Elephants extinct by 2020?'   \n",
       "4  b'Bank analyst forecast Georgian crisis 2 days...   \n",
       "\n",
       "                                               Top17  \\\n",
       "0                b'Al-Qaeda Faces Islamist Backlash'   \n",
       "1  b'\"Do not believe TV, neither Russian nor Geor...   \n",
       "2       b'Why Russias response to Georgia was right'   \n",
       "3  b'US humanitarian missions soon in Georgia - i...   \n",
       "4  b\"Georgia confict could set back Russia's US r...   \n",
       "\n",
       "                                               Top18  \\\n",
       "0  b'Condoleezza Rice: \"The US would not act to p...   \n",
       "1  b'Riots are still going on in Montreal (Canada...   \n",
       "2  b'Gorbachev accuses U.S. of making a \"serious ...   \n",
       "3             b\"Georgia's DDOS came from US sources\"   \n",
       "4  b'War in the Caucasus is as much the product o...   \n",
       "\n",
       "                                               Top19  \\\n",
       "0  b'This is a busy day:  The European Union has ...   \n",
       "1    b'China to overtake US as largest manufacturer'   \n",
       "2         b'Russia, Georgia, and NATO: Cold War Two'   \n",
       "3  b'Russian convoy heads into Georgia, violating...   \n",
       "4  b'\"Non-media\" photos of South Ossetia/Georgia ...   \n",
       "\n",
       "                                               Top20  \\\n",
       "0  b\"Georgia will withdraw 1,000 soldiers from Ir...   \n",
       "1                     b'War in South Ossetia [PICS]'   \n",
       "2  b'Remember that adorable 62-year-old who led y...   \n",
       "3  b'Israeli defence minister: US against strike ...   \n",
       "4  b'Georgian TV reporter shot by Russian sniper ...   \n",
       "\n",
       "                                               Top21  \\\n",
       "0  b'Why the Pentagon Thinks Attacking Iran is a ...   \n",
       "1  b'Israeli Physicians Group Condemns State Tort...   \n",
       "2          b'War in Georgia: The Israeli connection'   \n",
       "3                     b'Gorbachev: We Had No Choice'   \n",
       "4  b'Saudi Arabia: Mother moves to block child ma...   \n",
       "\n",
       "                                               Top22  \\\n",
       "0  b'Caucasus in crisis: Georgia invades South Os...   \n",
       "1  b' Russia has just beaten the United States ov...   \n",
       "2  b'All signs point to the US encouraging Georgi...   \n",
       "3  b'Witness: Russian forces head towards Tbilisi...   \n",
       "4   b'Taliban wages war on humanitarian aid workers'   \n",
       "\n",
       "                                               Top23  \\\n",
       "0  b'Indian shoe manufactory  - And again in a se...   \n",
       "1  b'Perhaps *the* question about the Georgia - R...   \n",
       "2  b'Christopher King argues that the US and NATO...   \n",
       "3  b' Quarter of Russians blame U.S. for conflict...   \n",
       "4  b'Russia: World  \"can forget about\" Georgia\\'s...   \n",
       "\n",
       "                                               Top24  \\\n",
       "0  b'Visitors Suffering from Mental Illnesses Ban...   \n",
       "1                 b'Russia is so much better at war'   \n",
       "2                        b'America: The New Mexico?'   \n",
       "3  b'Georgian president  says US military will ta...   \n",
       "4  b'Darfur rebels accuse Sudan of mounting major...   \n",
       "\n",
       "                                               Top25  \n",
       "0           b\"No Help for Mexico's Kidnapping Surge\"  \n",
       "1  b\"So this is what it's come to: trading sex fo...  \n",
       "2  b\"BBC NEWS | Asia-Pacific | Extinction 'by man...  \n",
       "3  b'2006: Nobel laureate Aleksander Solzhenitsyn...  \n",
       "4  b'Philippines : Peace Advocate say Muslims nee...  \n",
       "\n",
       "[5 rows x 26 columns]"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = df.drop(['Date'], axis=1)\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-08-07T04:44:49.982031Z",
     "iopub.status.busy": "2020-08-07T04:44:49.980690Z",
     "iopub.status.idle": "2020-08-07T04:44:49.984994Z",
     "shell.execute_reply": "2020-08-07T04:44:49.985630Z"
    },
    "papermill": {
     "duration": 0.03346,
     "end_time": "2020-08-07T04:44:49.985856",
     "exception": false,
     "start_time": "2020-08-07T04:44:49.952396",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Index(['Label', 'Top1', 'Top2', 'Top3', 'Top4', 'Top5', 'Top6', 'Top7', 'Top8',\n",
      "       'Top9', 'Top10', 'Top11', 'Top12', 'Top13', 'Top14', 'Top15', 'Top16',\n",
      "       'Top17', 'Top18', 'Top19', 'Top20', 'Top21', 'Top22', 'Top23', 'Top24',\n",
      "       'Top25'],\n",
      "      dtype='object')\n"
     ]
    }
   ],
   "source": [
    "df_columns = df.columns\n",
    "print(df_columns)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-08-07T04:44:50.036594Z",
     "iopub.status.busy": "2020-08-07T04:44:50.035294Z",
     "iopub.status.idle": "2020-08-07T04:44:50.038738Z",
     "shell.execute_reply": "2020-08-07T04:44:50.037883Z"
    },
    "papermill": {
     "duration": 0.031657,
     "end_time": "2020-08-07T04:44:50.038911",
     "exception": false,
     "start_time": "2020-08-07T04:44:50.007254",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "columns = ['Top1']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-08-07T04:44:50.092284Z",
     "iopub.status.busy": "2020-08-07T04:44:50.091367Z",
     "iopub.status.idle": "2020-08-07T04:44:50.128303Z",
     "shell.execute_reply": "2020-08-07T04:44:50.127490Z"
    },
    "papermill": {
     "duration": 0.067672,
     "end_time": "2020-08-07T04:44:50.128486",
     "exception": false,
     "start_time": "2020-08-07T04:44:50.060814",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "df['combined_news'] = df[columns].apply(lambda row:'.'.join(row.values.astype(str)), axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-08-07T04:44:50.181761Z",
     "iopub.status.busy": "2020-08-07T04:44:50.180668Z",
     "iopub.status.idle": "2020-08-07T04:44:50.191752Z",
     "shell.execute_reply": "2020-08-07T04:44:50.190743Z"
    },
    "papermill": {
     "duration": 0.04135,
     "end_time": "2020-08-07T04:44:50.191922",
     "exception": false,
     "start_time": "2020-08-07T04:44:50.150572",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "df = df.drop(columns, axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-08-07T04:44:50.275512Z",
     "iopub.status.busy": "2020-08-07T04:44:50.274241Z",
     "iopub.status.idle": "2020-08-07T04:44:50.280773Z",
     "shell.execute_reply": "2020-08-07T04:44:50.280045Z"
    },
    "papermill": {
     "duration": 0.066063,
     "end_time": "2020-08-07T04:44:50.280964",
     "exception": false,
     "start_time": "2020-08-07T04:44:50.214901",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Label</th>\n",
       "      <th>Top2</th>\n",
       "      <th>Top3</th>\n",
       "      <th>Top4</th>\n",
       "      <th>Top5</th>\n",
       "      <th>Top6</th>\n",
       "      <th>Top7</th>\n",
       "      <th>Top8</th>\n",
       "      <th>Top9</th>\n",
       "      <th>Top10</th>\n",
       "      <th>...</th>\n",
       "      <th>Top17</th>\n",
       "      <th>Top18</th>\n",
       "      <th>Top19</th>\n",
       "      <th>Top20</th>\n",
       "      <th>Top21</th>\n",
       "      <th>Top22</th>\n",
       "      <th>Top23</th>\n",
       "      <th>Top24</th>\n",
       "      <th>Top25</th>\n",
       "      <th>combined_news</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>b'BREAKING: Musharraf to be impeached.'</td>\n",
       "      <td>b'Russia Today: Columns of troops roll into So...</td>\n",
       "      <td>b'Russian tanks are moving towards the capital...</td>\n",
       "      <td>b\"Afghan children raped with 'impunity,' U.N. ...</td>\n",
       "      <td>b'150 Russian tanks have entered South Ossetia...</td>\n",
       "      <td>b\"Breaking: Georgia invades South Ossetia, Rus...</td>\n",
       "      <td>b\"The 'enemy combatent' trials are nothing but...</td>\n",
       "      <td>b'Georgian troops retreat from S. Osettain cap...</td>\n",
       "      <td>b'Did the U.S. Prep Georgia for War with Russia?'</td>\n",
       "      <td>...</td>\n",
       "      <td>b'Al-Qaeda Faces Islamist Backlash'</td>\n",
       "      <td>b'Condoleezza Rice: \"The US would not act to p...</td>\n",
       "      <td>b'This is a busy day:  The European Union has ...</td>\n",
       "      <td>b\"Georgia will withdraw 1,000 soldiers from Ir...</td>\n",
       "      <td>b'Why the Pentagon Thinks Attacking Iran is a ...</td>\n",
       "      <td>b'Caucasus in crisis: Georgia invades South Os...</td>\n",
       "      <td>b'Indian shoe manufactory  - And again in a se...</td>\n",
       "      <td>b'Visitors Suffering from Mental Illnesses Ban...</td>\n",
       "      <td>b\"No Help for Mexico's Kidnapping Surge\"</td>\n",
       "      <td>b\"Georgia 'downs two Russian warplanes' as cou...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>b'Bush puts foot down on Georgian conflict'</td>\n",
       "      <td>b\"Jewish Georgian minister: Thanks to Israeli ...</td>\n",
       "      <td>b'Georgian army flees in disarray as Russians ...</td>\n",
       "      <td>b\"Olympic opening ceremony fireworks 'faked'\"</td>\n",
       "      <td>b'What were the Mossad with fraudulent New Zea...</td>\n",
       "      <td>b'Russia angered by Israeli military sale to G...</td>\n",
       "      <td>b'An American citizen living in S.Ossetia blam...</td>\n",
       "      <td>b'Welcome To World War IV! Now In High Definit...</td>\n",
       "      <td>b\"Georgia's move, a mistake of monumental prop...</td>\n",
       "      <td>...</td>\n",
       "      <td>b'\"Do not believe TV, neither Russian nor Geor...</td>\n",
       "      <td>b'Riots are still going on in Montreal (Canada...</td>\n",
       "      <td>b'China to overtake US as largest manufacturer'</td>\n",
       "      <td>b'War in South Ossetia [PICS]'</td>\n",
       "      <td>b'Israeli Physicians Group Condemns State Tort...</td>\n",
       "      <td>b' Russia has just beaten the United States ov...</td>\n",
       "      <td>b'Perhaps *the* question about the Georgia - R...</td>\n",
       "      <td>b'Russia is so much better at war'</td>\n",
       "      <td>b\"So this is what it's come to: trading sex fo...</td>\n",
       "      <td>b'Why wont America and Nato help us? If they w...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0</td>\n",
       "      <td>b\"Russia 'ends Georgia operation'\"</td>\n",
       "      <td>b'\"If we had no sexual harassment we would hav...</td>\n",
       "      <td>b\"Al-Qa'eda is losing support in Iraq because ...</td>\n",
       "      <td>b'Ceasefire in Georgia: Putin Outmaneuvers the...</td>\n",
       "      <td>b'Why Microsoft and Intel tried to kill the XO...</td>\n",
       "      <td>b'Stratfor: The Russo-Georgian War and the Bal...</td>\n",
       "      <td>b\"I'm Trying to Get a Sense of This Whole Geor...</td>\n",
       "      <td>b\"The US military was surprised by the timing ...</td>\n",
       "      <td>b'U.S. Beats War Drum as Iran Dumps the Dollar'</td>\n",
       "      <td>...</td>\n",
       "      <td>b'Why Russias response to Georgia was right'</td>\n",
       "      <td>b'Gorbachev accuses U.S. of making a \"serious ...</td>\n",
       "      <td>b'Russia, Georgia, and NATO: Cold War Two'</td>\n",
       "      <td>b'Remember that adorable 62-year-old who led y...</td>\n",
       "      <td>b'War in Georgia: The Israeli connection'</td>\n",
       "      <td>b'All signs point to the US encouraging Georgi...</td>\n",
       "      <td>b'Christopher King argues that the US and NATO...</td>\n",
       "      <td>b'America: The New Mexico?'</td>\n",
       "      <td>b\"BBC NEWS | Asia-Pacific | Extinction 'by man...</td>\n",
       "      <td>b'Remember that adorable 9-year-old who sang a...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0</td>\n",
       "      <td>b\"When the president ordered to attack Tskhinv...</td>\n",
       "      <td>b' Israel clears troops who killed Reuters cam...</td>\n",
       "      <td>b'Britain\\'s policy of being tough on drugs is...</td>\n",
       "      <td>b'Body of 14 year old found in trunk; Latest (...</td>\n",
       "      <td>b'China has moved 10 *million* quake survivors...</td>\n",
       "      <td>b\"Bush announces Operation Get All Up In Russi...</td>\n",
       "      <td>b'Russian forces sink Georgian ships '</td>\n",
       "      <td>b\"The commander of a Navy air reconnaissance s...</td>\n",
       "      <td>b\"92% of CNN readers: Russia's actions in Geor...</td>\n",
       "      <td>...</td>\n",
       "      <td>b'US humanitarian missions soon in Georgia - i...</td>\n",
       "      <td>b\"Georgia's DDOS came from US sources\"</td>\n",
       "      <td>b'Russian convoy heads into Georgia, violating...</td>\n",
       "      <td>b'Israeli defence minister: US against strike ...</td>\n",
       "      <td>b'Gorbachev: We Had No Choice'</td>\n",
       "      <td>b'Witness: Russian forces head towards Tbilisi...</td>\n",
       "      <td>b' Quarter of Russians blame U.S. for conflict...</td>\n",
       "      <td>b'Georgian president  says US military will ta...</td>\n",
       "      <td>b'2006: Nobel laureate Aleksander Solzhenitsyn...</td>\n",
       "      <td>b' U.S. refuses Israel weapons to attack Iran:...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "      <td>b'War in South Osetia - 89 pictures made by a ...</td>\n",
       "      <td>b'Swedish wrestler Ara Abrahamian throws away ...</td>\n",
       "      <td>b'Russia exaggerated the death toll in South O...</td>\n",
       "      <td>b'Missile That Killed 9 Inside Pakistan May Ha...</td>\n",
       "      <td>b\"Rushdie Condemns Random House's Refusal to P...</td>\n",
       "      <td>b'Poland and US agree to missle defense deal. ...</td>\n",
       "      <td>b'Will the Russians conquer Tblisi? Bet on it,...</td>\n",
       "      <td>b'Russia exaggerating South Ossetian death tol...</td>\n",
       "      <td>b' Musharraf expected to resign rather than fa...</td>\n",
       "      <td>...</td>\n",
       "      <td>b\"Georgia confict could set back Russia's US r...</td>\n",
       "      <td>b'War in the Caucasus is as much the product o...</td>\n",
       "      <td>b'\"Non-media\" photos of South Ossetia/Georgia ...</td>\n",
       "      <td>b'Georgian TV reporter shot by Russian sniper ...</td>\n",
       "      <td>b'Saudi Arabia: Mother moves to block child ma...</td>\n",
       "      <td>b'Taliban wages war on humanitarian aid workers'</td>\n",
       "      <td>b'Russia: World  \"can forget about\" Georgia\\'s...</td>\n",
       "      <td>b'Darfur rebels accuse Sudan of mounting major...</td>\n",
       "      <td>b'Philippines : Peace Advocate say Muslims nee...</td>\n",
       "      <td>b'All the experts admit that we should legalis...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 26 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   Label                                               Top2  \\\n",
       "0      0            b'BREAKING: Musharraf to be impeached.'   \n",
       "1      1        b'Bush puts foot down on Georgian conflict'   \n",
       "2      0                 b\"Russia 'ends Georgia operation'\"   \n",
       "3      0  b\"When the president ordered to attack Tskhinv...   \n",
       "4      1  b'War in South Osetia - 89 pictures made by a ...   \n",
       "\n",
       "                                                Top3  \\\n",
       "0  b'Russia Today: Columns of troops roll into So...   \n",
       "1  b\"Jewish Georgian minister: Thanks to Israeli ...   \n",
       "2  b'\"If we had no sexual harassment we would hav...   \n",
       "3  b' Israel clears troops who killed Reuters cam...   \n",
       "4  b'Swedish wrestler Ara Abrahamian throws away ...   \n",
       "\n",
       "                                                Top4  \\\n",
       "0  b'Russian tanks are moving towards the capital...   \n",
       "1  b'Georgian army flees in disarray as Russians ...   \n",
       "2  b\"Al-Qa'eda is losing support in Iraq because ...   \n",
       "3  b'Britain\\'s policy of being tough on drugs is...   \n",
       "4  b'Russia exaggerated the death toll in South O...   \n",
       "\n",
       "                                                Top5  \\\n",
       "0  b\"Afghan children raped with 'impunity,' U.N. ...   \n",
       "1      b\"Olympic opening ceremony fireworks 'faked'\"   \n",
       "2  b'Ceasefire in Georgia: Putin Outmaneuvers the...   \n",
       "3  b'Body of 14 year old found in trunk; Latest (...   \n",
       "4  b'Missile That Killed 9 Inside Pakistan May Ha...   \n",
       "\n",
       "                                                Top6  \\\n",
       "0  b'150 Russian tanks have entered South Ossetia...   \n",
       "1  b'What were the Mossad with fraudulent New Zea...   \n",
       "2  b'Why Microsoft and Intel tried to kill the XO...   \n",
       "3  b'China has moved 10 *million* quake survivors...   \n",
       "4  b\"Rushdie Condemns Random House's Refusal to P...   \n",
       "\n",
       "                                                Top7  \\\n",
       "0  b\"Breaking: Georgia invades South Ossetia, Rus...   \n",
       "1  b'Russia angered by Israeli military sale to G...   \n",
       "2  b'Stratfor: The Russo-Georgian War and the Bal...   \n",
       "3  b\"Bush announces Operation Get All Up In Russi...   \n",
       "4  b'Poland and US agree to missle defense deal. ...   \n",
       "\n",
       "                                                Top8  \\\n",
       "0  b\"The 'enemy combatent' trials are nothing but...   \n",
       "1  b'An American citizen living in S.Ossetia blam...   \n",
       "2  b\"I'm Trying to Get a Sense of This Whole Geor...   \n",
       "3             b'Russian forces sink Georgian ships '   \n",
       "4  b'Will the Russians conquer Tblisi? Bet on it,...   \n",
       "\n",
       "                                                Top9  \\\n",
       "0  b'Georgian troops retreat from S. Osettain cap...   \n",
       "1  b'Welcome To World War IV! Now In High Definit...   \n",
       "2  b\"The US military was surprised by the timing ...   \n",
       "3  b\"The commander of a Navy air reconnaissance s...   \n",
       "4  b'Russia exaggerating South Ossetian death tol...   \n",
       "\n",
       "                                               Top10  ...  \\\n",
       "0  b'Did the U.S. Prep Georgia for War with Russia?'  ...   \n",
       "1  b\"Georgia's move, a mistake of monumental prop...  ...   \n",
       "2    b'U.S. Beats War Drum as Iran Dumps the Dollar'  ...   \n",
       "3  b\"92% of CNN readers: Russia's actions in Geor...  ...   \n",
       "4  b' Musharraf expected to resign rather than fa...  ...   \n",
       "\n",
       "                                               Top17  \\\n",
       "0                b'Al-Qaeda Faces Islamist Backlash'   \n",
       "1  b'\"Do not believe TV, neither Russian nor Geor...   \n",
       "2       b'Why Russias response to Georgia was right'   \n",
       "3  b'US humanitarian missions soon in Georgia - i...   \n",
       "4  b\"Georgia confict could set back Russia's US r...   \n",
       "\n",
       "                                               Top18  \\\n",
       "0  b'Condoleezza Rice: \"The US would not act to p...   \n",
       "1  b'Riots are still going on in Montreal (Canada...   \n",
       "2  b'Gorbachev accuses U.S. of making a \"serious ...   \n",
       "3             b\"Georgia's DDOS came from US sources\"   \n",
       "4  b'War in the Caucasus is as much the product o...   \n",
       "\n",
       "                                               Top19  \\\n",
       "0  b'This is a busy day:  The European Union has ...   \n",
       "1    b'China to overtake US as largest manufacturer'   \n",
       "2         b'Russia, Georgia, and NATO: Cold War Two'   \n",
       "3  b'Russian convoy heads into Georgia, violating...   \n",
       "4  b'\"Non-media\" photos of South Ossetia/Georgia ...   \n",
       "\n",
       "                                               Top20  \\\n",
       "0  b\"Georgia will withdraw 1,000 soldiers from Ir...   \n",
       "1                     b'War in South Ossetia [PICS]'   \n",
       "2  b'Remember that adorable 62-year-old who led y...   \n",
       "3  b'Israeli defence minister: US against strike ...   \n",
       "4  b'Georgian TV reporter shot by Russian sniper ...   \n",
       "\n",
       "                                               Top21  \\\n",
       "0  b'Why the Pentagon Thinks Attacking Iran is a ...   \n",
       "1  b'Israeli Physicians Group Condemns State Tort...   \n",
       "2          b'War in Georgia: The Israeli connection'   \n",
       "3                     b'Gorbachev: We Had No Choice'   \n",
       "4  b'Saudi Arabia: Mother moves to block child ma...   \n",
       "\n",
       "                                               Top22  \\\n",
       "0  b'Caucasus in crisis: Georgia invades South Os...   \n",
       "1  b' Russia has just beaten the United States ov...   \n",
       "2  b'All signs point to the US encouraging Georgi...   \n",
       "3  b'Witness: Russian forces head towards Tbilisi...   \n",
       "4   b'Taliban wages war on humanitarian aid workers'   \n",
       "\n",
       "                                               Top23  \\\n",
       "0  b'Indian shoe manufactory  - And again in a se...   \n",
       "1  b'Perhaps *the* question about the Georgia - R...   \n",
       "2  b'Christopher King argues that the US and NATO...   \n",
       "3  b' Quarter of Russians blame U.S. for conflict...   \n",
       "4  b'Russia: World  \"can forget about\" Georgia\\'s...   \n",
       "\n",
       "                                               Top24  \\\n",
       "0  b'Visitors Suffering from Mental Illnesses Ban...   \n",
       "1                 b'Russia is so much better at war'   \n",
       "2                        b'America: The New Mexico?'   \n",
       "3  b'Georgian president  says US military will ta...   \n",
       "4  b'Darfur rebels accuse Sudan of mounting major...   \n",
       "\n",
       "                                               Top25  \\\n",
       "0           b\"No Help for Mexico's Kidnapping Surge\"   \n",
       "1  b\"So this is what it's come to: trading sex fo...   \n",
       "2  b\"BBC NEWS | Asia-Pacific | Extinction 'by man...   \n",
       "3  b'2006: Nobel laureate Aleksander Solzhenitsyn...   \n",
       "4  b'Philippines : Peace Advocate say Muslims nee...   \n",
       "\n",
       "                                       combined_news  \n",
       "0  b\"Georgia 'downs two Russian warplanes' as cou...  \n",
       "1  b'Why wont America and Nato help us? If they w...  \n",
       "2  b'Remember that adorable 9-year-old who sang a...  \n",
       "3  b' U.S. refuses Israel weapons to attack Iran:...  \n",
       "4  b'All the experts admit that we should legalis...  \n",
       "\n",
       "[5 rows x 26 columns]"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-08-07T04:44:50.337182Z",
     "iopub.status.busy": "2020-08-07T04:44:50.336039Z",
     "iopub.status.idle": "2020-08-07T04:44:50.339451Z",
     "shell.execute_reply": "2020-08-07T04:44:50.338804Z"
    },
    "papermill": {
     "duration": 0.035959,
     "end_time": "2020-08-07T04:44:50.339617",
     "exception": false,
     "start_time": "2020-08-07T04:44:50.303658",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "columns_2 = ['Top2','Top3', 'Top4', 'Top5','Top6', 'Top7', 'Top8', 'Top9', 'Top10', 'Top11', 'Top12', 'Top13', 'Top14', 'Top15', 'Top16', 'Top17', 'Top18', 'Top19', 'Top20', 'Top21', 'Top22', 'Top23','Top24', 'Top25']\n",
    "df = df.drop(columns_2, axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-08-07T04:44:50.398323Z",
     "iopub.status.busy": "2020-08-07T04:44:50.396969Z",
     "iopub.status.idle": "2020-08-07T04:44:50.402561Z",
     "shell.execute_reply": "2020-08-07T04:44:50.401884Z"
    },
    "papermill": {
     "duration": 0.040648,
     "end_time": "2020-08-07T04:44:50.402712",
     "exception": false,
     "start_time": "2020-08-07T04:44:50.362064",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Label</th>\n",
       "      <th>combined_news</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>b\"Georgia 'downs two Russian warplanes' as cou...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>b'Why wont America and Nato help us? If they w...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0</td>\n",
       "      <td>b'Remember that adorable 9-year-old who sang a...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0</td>\n",
       "      <td>b' U.S. refuses Israel weapons to attack Iran:...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "      <td>b'All the experts admit that we should legalis...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Label                                      combined_news\n",
       "0      0  b\"Georgia 'downs two Russian warplanes' as cou...\n",
       "1      1  b'Why wont America and Nato help us? If they w...\n",
       "2      0  b'Remember that adorable 9-year-old who sang a...\n",
       "3      0  b' U.S. refuses Israel weapons to attack Iran:...\n",
       "4      1  b'All the experts admit that we should legalis..."
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "execution_count": null,
   "metadata": {
    "papermill": {
     "duration": 0.022757,
     "end_time": "2020-08-07T04:44:50.448549",
     "exception": false,
     "start_time": "2020-08-07T04:44:50.425792",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "As we can clearly see, there are many html tags which should be removed to avoid any biases. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-08-07T04:44:50.517562Z",
     "iopub.status.busy": "2020-08-07T04:44:50.514841Z",
     "iopub.status.idle": "2020-08-07T04:44:50.522744Z",
     "shell.execute_reply": "2020-08-07T04:44:50.521788Z"
    },
    "papermill": {
     "duration": 0.051784,
     "end_time": "2020-08-07T04:44:50.523018",
     "exception": false,
     "start_time": "2020-08-07T04:44:50.471234",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Label</th>\n",
       "      <th>combined_news</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>Georgia 'downs two Russian warplanes' as count...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>Why wont America and Nato help us? If they won...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Label                                      combined_news\n",
       "0      0  Georgia 'downs two Russian warplanes' as count...\n",
       "1      1  Why wont America and Nato help us? If they won..."
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = df.replace('b\\\"|b\\'|\\\\\\\\|\\\\\\\"', '', regex=True)\n",
    "df.head(2)"
   ]
  },
  {
   "cell_type": "markdown",
   "execution_count": null,
   "metadata": {
    "papermill": {
     "duration": 0.022741,
     "end_time": "2020-08-07T04:44:50.569692",
     "exception": false,
     "start_time": "2020-08-07T04:44:50.546951",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# EDA on final dataframe\n",
    "\n",
    "## Counting Labels\n",
    "\n",
    "Here, I am visualizing the number of times DJIA remanined the same/went up and number of times it went down."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-08-07T04:44:50.631896Z",
     "iopub.status.busy": "2020-08-07T04:44:50.630613Z",
     "iopub.status.idle": "2020-08-07T04:44:51.931372Z",
     "shell.execute_reply": "2020-08-07T04:44:51.930404Z"
    },
    "papermill": {
     "duration": 1.338858,
     "end_time": "2020-08-07T04:44:51.931576",
     "exception": false,
     "start_time": "2020-08-07T04:44:50.592718",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZUAAAEYCAYAAACUdWs9AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAYLUlEQVR4nO3df3BU9b3/8dfmRMLPmGxIwoKp/FK6lou5Q9ro1yq3oc7OOAvxWp11VtvbaVGk5ZbrHQtRbIIQYRa4VzJCBG9RBmvtldsqZe2YXAvDoLVU21qrQVFIwJYlIZsAAWOAs/v9g3G/5IsJm+SzP0KejxlmzH72x3ud2Txzzu6e44hGo1EBAGBARqoHAABcPogKAMAYogIAMIaoAACMISoAAGOICgDAmKRFJRAIqKysTNOmTdP+/fslSe3t7brvvvvk8Xg0Z84cLVy4UG1tbbHbNDY2yufzyePxyOfzqampKa41AEBqJC0qs2fP1vPPP68JEybELnM4HJo3b57q6uq0Y8cOFRUVae3atbH1qqoq+f1+1dXVye/3q7KyMq41AEBqJC0qJSUlcrlc3S7LyclRaWlp7Ofi4mIdOXJEkhQOh9XQ0CCv1ytJ8nq9amhoUFtbW69rAIDUyUz1AJ+LRCJ64YUXVFZWJkkKhUIqLCyUZVmSJMuyVFBQoFAopGg02uOa0+lM2XMAgKEubd6oX7FihUaOHKl777031aMAAPopLbZUAoGADh06pI0bNyoj43znXC6XmpubZdu2LMuSbdtqaWmRy+VSNBrtcQ0AkDopj8oTTzyh9957T08//bSGDRsWuzwvL09ut1vBYFDl5eUKBoNyu92x3Vu9rfVFe/tpRSIcUxMA4pGR4VBu7qge1x3JOkpxdXW16uvr1draqtzcXOXk5GjdunXyer2aOHGihg8fLkm66qqrtGHDBknSgQMHVFFRoZMnTyo7O1uBQECTJ0++5FpfhMOniAoAxCkjw6G8vNE9rictKumKqABA/C4VlbR5ox4AMPil/D0VALjcOZ2jZFmD8294246ore103NcnKgCQYJaVoWPHOlI9Rr/k54/p0/UHZzoBAGmJqAAAjCEqAABjiAoApJGysv8T93X/9Ke39e///q99uv/Nmzfpuee29HGq+BEVAIAxfPoLANLcBx/sU03NGn322WeyrEw99FCFvvzl6yRJnZ2faunSH+uTTw5r8uSpevjhSmVlZenjjz/Sk0/+pzo6OjRixAgtXrxUV189MeGzsqUCAGmuqKhITz75tJ599uf68Y8fUU3N/zuZ4b597+v++3+orVv/W5Zlafv2X+rcuXNau3aVfvKTFXrmmZ9pwYJ/1Zo1K5MyK1sqAJDmPv30U61c+ZgOHz4ky8rU0aNHYmtf/vJ1sS0Qj+c2vfTS/6ik5Gs6ePCAHnroR7HrffZZZ1JmJSoAkOb+67+e0le+MkPV1avV2fmpvN5bY2sOR/frOhwORaNSUdGXtHnzc0melN1fAJD2Tp06pfz8fDkcDgWD27utffDBPh0+fEiS9L//+6qKi/9RV189UadOdehPf3pb0vkz63700YdJmZUtFQBII2fOnNE///NtsZ9vu22O7rnnX1RdXalf/epF3XDDTd2u/5Wv/IOeeeZpHTz4sSZPnqry8m8pMzNTK1eu1bp1a1RT8x86d+6syspu1TXXTEv4/Bz6nkPfA0iw/Pwxg/rYXxfOzqHvAQBJQ1QAAMYQFQCAMUQFAGAMUQEAGENUAADG8D0VAJe93CuHKXNYVqrHiBmTPVzDs64wfr+fdZ1Vx8nPjN9vXxAVAJe9zGFZ+uPqeSl7/JmLf9rt5+FZV8i/+Hnjj/Pz1feoQ/FF5fDhQ3r88WU6ceKErrzySj366GMqKvrSgGdg9xcADEFr167SHXfcpV/84le64467jB3FmKgAwBDT3t6m/fs/0De/6ZEkffObHu3f/4Ha29sHfN9EBQCGmObmZo0dWyDLsiRJlmVp7Nh8tbQ0D/i+iQoAwBiiAgBDTGFhoVpbW2TbtiTJtm21th5TQUHhgO87aVEJBAIqKyvTtGnTtH///tjljY2N8vl88ng88vl8ampqGvAaAKBnublOTZ16rV57rU6S9NprdbrmmmnKzc0d8H0n7dD3b7/9tiZMmKB77rlHGzdu1LXXXitJ+s53vqNvfetbKi8v1/bt2/XLX/5SW7duHdBaX3Doe+Dyl58/JuUfKb7w8PHp8D2VQ4eaVF1dpY6ODo0ZM0Y/+clj+tKXJl50vb4e+j7p51MpKyuLRSUcDsvj8Wjv3r2yLEu2bau0tFT19fWKRqP9WnM6nX2ah6gAl790i8pg0teopPTLj6FQSIWFhd0+gVBQUKBQKKRoNNqvtb5GBQBgDm/UAwCMSemWisvlUnNzs2zbju3GamlpkcvlUjQa7ddaX/W2GQcAOL8LLF4pjUpeXp7cbreCwaDKy8sVDAbldrtju7D6u9YXvKcCXP768ksRF0vLN+qrq6tVX1+v1tZW5ebmKicnR6+88ooOHDigiooKnTx5UtnZ2QoEApo8ebIk9XutL4gKcPnjjfr+S/tPf6UbogJc/ohK/w2qT38BwFCUqPO7nDvTpfYTZy55vfXr12n37p0KhY5o69ZfaPLkqcZmICoAkGSJOr/L+fO2XDoqN9/8T7rrrrv1wx/eZ3wGogIAQ8z11xcn7L75ngoAwBiiAgAwhqgAAIwhKgAAY3ijHgCGmHXr1mj37l1qawvr3/7th8rOvlI/+9mLRu6bLz/y5UfgspduX35M9fdU+oIvPwJAmjv/i9/sL/90QVQGKFFncBuM+nLWOQCXJ6IyQMOzrpB/8fOpHiMt/Hz1PeoQUQGGMj79BQAwhqgAAIxh9xcAJNiZU8eVn5+T6jH6xbYjfbo+UQGABPtr7UOpHiEm0ed2YfcXAMAYogIAMIaoAACMISoAAGOICgDAGKICADCGqAAAjCEqAABjiAoAwBiiAgAwhqgAAIwhKgAAY4gKAMCYtInKrl27dPvtt6u8vFxz5sxRfX29JKmxsVE+n08ej0c+n09NTU2x2/S2BgBIvrSISjQa1eLFi7V69Wpt375da9as0ZIlSxSJRFRVVSW/36+6ujr5/X5VVlbGbtfbGgAg+dIiKpKUkZGhjo7zx/jv6OhQQUGB2tvb1dDQIK/XK0nyer1qaGhQW1ubwuFwj2sAgNRIi5N0ORwOrVu3Tj/4wQ80cuRInT59Wps2bVIoFFJhYaEsy5IkWZalgoIChUIhRaPRHtecTmcqnw4ADFlpEZVz585p06ZNqq2t1cyZM/XHP/5RDz74oFavXp3wx87LG53wxxhK8vPHpHoEAJeQyNdpWkRl3759amlp0cyZMyVJM2fO1IgRI5SVlaXm5mbZti3LsmTbtlpaWuRyuRSNRntc64tw+JQikWi/Z+eXaHeJPE0p0F+8TrsbyOs0I8PR6x/jafGeyrhx43T06FEdPHhQknTgwAG1trbq6quvltvtVjAYlCQFg0G53W45nU7l5eX1uAYASI202FLJz8/XsmXLtGjRIjkcDknSqlWrlJOTo2XLlqmiokK1tbXKzs5WIBCI3a63NQBA8qVFVCRp7ty5mjt37kWXT5kyRdu2bfvC2/S2BgBIvrTY/QUAuDwQFQCAMUQFAGAMUQEAGENUAADGEBUAgDFEBQBgDFEBABhDVAAAxhAVAIAxRAUAYAxRAQAYQ1QAAMYQFQCAMUQFAGAMUQEAGENUAADGEBUAgDFxR2Xz5s1fePmzzz5rbBgAwOAWd1Q2bNjwhZc/9dRTxoYBAAxumZe6wptvvilJikQi+v3vf69oNBpb+9vf/qZRo0YlbjoAwKByyagsXbpUktTV1aVHHnkkdrnD4VB+fr4effTRxE0HABhULhmVnTt3SpIWL16s1atXJ3wgAMDgdcmofO7CoEQikW5rGRl8iAwA0IeovP/++1q+fLk+/PBDdXV1SZKi0agcDof27duXsAEBAINH3FGpqKjQN77xDa1cuVLDhw9P5EwAgEEq7qj8/e9/14MPPiiHw5HIeQAAg1jcb4bceuutev311xM5CwBgkIt7S6Wrq0sLFy7UzJkzNXbs2G5rJj4V1tXVpZUrV+rNN99UVlaWiouLtWLFCjU2NqqiokLHjx9XTk6OAoGAJk6cKEm9rgEAki/uqEydOlVTp05N2CBr1qxRVlaW6urq5HA41NraKkmqqqqS3+9XeXm5tm/frsrKSm3duvWSawCA5Is7KgsXLkzYEKdPn9bLL7+s3bt3x96zGTt2rMLhsBoaGmLHF/N6vVqxYoXa2toUjUZ7XHM6nQmbFQDQs7ij8vnhWr7IjTfeOKAhPvnkE+Xk5Gj9+vXau3evRo0apUWLFmn48OEqLCyUZVmSJMuyVFBQoFAopGg02uMaUQGA1Ig7Kp8fruVz7e3tOnv2rAoLC/Xb3/52QEOcO3dOn3zyia677jotWbJEf/nLX/TAAw+opqZmQPcbj7y80Ql/jKEkP39MqkcAcAmJfJ3GHZXPD9fyOdu29dRTTxk5oOT48eOVmZkpr9crSbr++uuVm5ur4cOHq7m5WbZty7Is2batlpYWuVwuRaPRHtf6Ihw+pUgkeukr9oBfot0dO9aR6hGAi/A67W4gr9OMDEevf4z3+/gqlmXpgQce0E9/+tP+3kWM0+lUaWmp3njjDUnnP9UVDoc1ceJEud1uBYNBSVIwGJTb7ZbT6VReXl6PawCA1Ih7S+WLvPHGG8a+DPnYY4/pkUceUSAQUGZmplavXq3s7GwtW7ZMFRUVqq2tVXZ2tgKBQOw2va0BAJIv7qjMmjWrW0A6Ozt15swZVVVVGRmkqKhIzz333EWXT5kyRdu2bfvC2/S2BgBIvrijsmbNmm4/jxgxQpMmTdLo0bzRDQA4L+6ofO1rX5N0/rD3ra2tGjt2LIe8BwB0E3cVTp06pcWLF2vGjBm65ZZbNGPGDC1ZskQdHXzaBwBwXtxRqa6uVmdnp3bs2KF3331XO3bsUGdnp6qrqxM5HwBgEIl799eePXv02muvacSIEZKkSZMmadWqVbr11lsTNhwAYHCJe0slKytLbW1t3S5rb2/XsGHDjA8FABic4t5SufPOO/W9731P3/3udzV+/HgdOXJEW7Zs0V133ZXI+QAAg0jcUVmwYIEKCwu1Y8cOtbS0qKCgQPPmzSMqAICYuHd/Pf7445o0aZK2bNmi3/zmN9qyZYumTJmixx9/PJHzAQAGkbijEgwGNX369G6XTZ8+PXbsLQAA4o6Kw+FQJBLpdplt2xddBgAYuuKOSklJiWpqamIRiUQievLJJ1VSUpKw4QAAg0ufTtI1f/58ff3rX9f48eMVCoWUn5+vjRs3JnI+AMAgEndUxo0bp5deeknvvvuuQqGQXC6XZsyYwfG/AAAxfTqfSkZGhoqLi1VcXJyoeQAAgxibGQAAY4gKAMAYogIAMIaoAACMISoAAGOICgDAGKICADCGqAAAjCEqAABjiAoAwBiiAgAwhqgAAIwhKgAAY9IuKuvXr9e0adO0f/9+SVJjY6N8Pp88Ho98Pp+amppi1+1tDQCQfGkVlffff1/vvPOOxo8fH7usqqpKfr9fdXV18vv9qqysjGsNAJB8aROVM2fOaPny5aqqqpLD4ZAkhcNhNTQ0yOv1SpK8Xq8aGhrU1tbW6xoAIDX6dJKuRKqpqdHcuXNVVFQUuywUCqmwsFCWZUmSLMtSQUGBQqGQotFoj2tOpzMlzwEAhrq0iMqf//xn/fWvf9VDDz2U9MfOyxud9Me8nOXnj0n1CAAuIZGv07SIyltvvaWDBw9q9uzZkqSjR4/q+9//vh5++GE1NzfLtm1ZliXbttXS0iKXy6VoNNrjWl+Ew6cUiUT7PTu/RLs7dqwj1SMAF+F12t1AXqcZGY5e/xhPi/dU7r//fr3++uvauXOndu7cqXHjxmnz5s267bbb5Ha7FQwGJUnBYFBut1tOp1N5eXk9rgEAUiMttlR6s2zZMlVUVKi2tlbZ2dkKBAJxrQEAki8to7Jz587Yf0+ZMkXbtm37wuv1tgYASL602P0FALg8EBUAgDFEBQBgDFEBABhDVAAAxhAVAIAxRAUAYAxRAQAYQ1QAAMYQFQCAMUQFAGAMUQEAGENUAADGEBUAgDFEBQBgDFEBABhDVAAAxhAVAIAxRAUAYAxRAQAYQ1QAAMYQFQCAMUQFAGAMUQEAGENUAADGEBUAgDFEBQBgDFEBABiTFlFpb2/XfffdJ4/Hozlz5mjhwoVqa2uTJDU2Nsrn88nj8cjn86mpqSl2u97WAADJlxZRcTgcmjdvnurq6rRjxw4VFRVp7dq1kqSqqir5/X7V1dXJ7/ersrIydrve1gAAyZcWUcnJyVFpaWns5+LiYh05ckThcFgNDQ3yer2SJK/Xq4aGBrW1tfW6BgBIjcxUD/D/i0QieuGFF1RWVqZQKKTCwkJZliVJsixLBQUFCoVCikajPa45nc5UPgUAGLLSYkvlQitWrNDIkSN17733pnoUAEAfpdWWSiAQ0KFDh7Rx40ZlZGTI5XKpublZtm3LsizZtq2Wlha5XC5Fo9Ee1/oiL290gp7N0JSfPybVIwC4hES+TtMmKk888YTee+89Pf300xo2bJgkKS8vT263W8FgUOXl5QoGg3K73bHdW72txSscPqVIJNrvufkl2t2xYx2pHgG4CK/T7gbyOs3IcPT6x3haROWjjz7Sxo0bNXHiRN19992SpKuuukobNmzQsmXLVFFRodraWmVnZysQCMRu19saACD50iIq11xzjT788MMvXJsyZYq2bdvW5zUAQPKl3Rv1AIDBi6gAAIwhKgAAY4gKAMAYogIAMIaoAACMISoAAGOICgDAGKICADCGqAAAjCEqAABjiAoAwBiiAgAwhqgAAIwhKgAAY4gKAMAYogIAMIaoAACMISoAAGOICgDAGKICADCGqAAAjCEqAABjiAoAwBiiAgAwhqgAAIwhKgAAY4gKAMCYQR+VxsZG+Xw+eTwe+Xw+NTU1pXokABiyBn1Uqqqq5Pf7VVdXJ7/fr8rKylSPBABD1qCOSjgcVkNDg7xeryTJ6/WqoaFBbW1tKZ4MAIamQR2VUCikwsJCWZYlSbIsSwUFBQqFQimeDACGpsxUD5BqGRmOAd/H2NxRBia5PJj4/wkkwrDsvFSPkDYG8jq91G0d0Wg02u97T7FwOCyPx6O9e/fKsizZtq3S0lLV19fL6XSmejwAGHIG9e6vvLw8ud1uBYNBSVIwGJTb7SYoAJAig3pLRZIOHDigiooKnTx5UtnZ2QoEApo8eXKqxwKAIWnQRwUAkD4G9e4vAEB6ISoAAGOICgDAGKICADCGqAAAjCEqAABjiAoGjNMPIF0FAgGVlZVp2rRp2r9/f6rHGRKICgaM0w8gXc2ePVvPP/+8JkyYkOpRhgyiggHh9ANIZyUlJXK5XKkeY0ghKhgQTj8A4EJEBQBgDFHBgLhcLjU3N8u2bUmSbdtqaWlhlwMwRBEVDAinHwBwIY5SjAHj9ANIV9XV1aqvr1dra6tyc3OVk5OjV155JdVjXdaICgDAGHZ/AQCMISoAAGOICgDAGKICADCGqAAAjCEqAABjiAqQRr797W9r27ZtSb8tYApRARKkrKxMv/vd71I9BpBURAUAYAxRAZLoxIkTmj9/vm644QZ99atf1fz583X06NFu1zl8+LDuvPNOzZw5UwsWLNDx48dja++8847uvvtulZSUaO7cudq7d2+ynwLQK6ICJFEkEtEdd9yhXbt2adeuXcrKytLy5cu7Xefll1/WypUrtWfPHmVmZqq6ulqS1NzcrPnz52vBggX6wx/+oCVLluhHP/oRJ0RDWiEqQBLl5ubK4/FoxIgRGj16tBYsWKC33nqr23XKy8t17bXXauTIkVq0aJFeffVV2bat7du365ZbbtGsWbOUkZGhm266SdOnT9fu3btT9GyAi2WmegBgKOns7NSqVau0Z88enThxQpJ0+vRp2bYdO3vmheeiGT9+vM6ePav29nYdOXJEr776qnbt2hVbP3funEpLS5P7JIBeEBUgiZ555hk1NjbqxRdfVH5+vvbt26fbb79dFx4s/MJTMYdCIV1xxRXKzc2Vy+VSeXl5bHcYkI7Y/QUk0NmzZ9XV1RX7d/LkSWVlZSk7O1vHjx/X+vXrL7rNr3/9a3388cfq7OxUTU2NPB6PLMvS3LlztWvXLu3Zs0e2baurq0t79+696I1+IJWICpBA999/v2bMmBH7d/LkSXV1demGG26Qz+fTzTfffNFtysvLVVFRoZtuuklnzpzR0qVLJZ3fLVZbW6tNmzbpxhtv1KxZs7R582ZFIpFkPy2gR5ykCwBgDFsqAABjiAoAwBiiAgAwhqgAAIwhKgAAY4gKAMAYogIAMIaoAACMISoAAGP+L4zPxLAl2EhCAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "plt.style.use('classic')\n",
    "%matplotlib inline\n",
    "import seaborn as sns\n",
    "sns.set()\n",
    "\n",
    "ax = sns.countplot(x='Label', hue='Label', data=df)"
   ]
  },
  {
   "cell_type": "markdown",
   "execution_count": null,
   "metadata": {
    "papermill": {
     "duration": 0.02385,
     "end_time": "2020-08-07T04:44:51.980059",
     "exception": false,
     "start_time": "2020-08-07T04:44:51.956209",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Now, lets visualize the most common terms that occur when DJIA remains the same or goes up."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-08-07T04:44:52.040537Z",
     "iopub.status.busy": "2020-08-07T04:44:52.039435Z",
     "iopub.status.idle": "2020-08-07T04:44:52.044420Z",
     "shell.execute_reply": "2020-08-07T04:44:52.043565Z"
    },
    "papermill": {
     "duration": 0.040722,
     "end_time": "2020-08-07T04:44:52.044606",
     "exception": false,
     "start_time": "2020-08-07T04:44:52.003884",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "from collections import Counter\n",
    "\n",
    "# Copy df enteries in 2 separate lists based on labels\n",
    "\n",
    "data_djia_up = df[df['Label']==1].copy()\n",
    "data_djia_down = df[df['Label']==0].copy()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-08-07T04:44:52.102852Z",
     "iopub.status.busy": "2020-08-07T04:44:52.101583Z",
     "iopub.status.idle": "2020-08-07T04:44:52.105776Z",
     "shell.execute_reply": "2020-08-07T04:44:52.106396Z"
    },
    "papermill": {
     "duration": 0.038391,
     "end_time": "2020-08-07T04:44:52.106621",
     "exception": false,
     "start_time": "2020-08-07T04:44:52.068230",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "   Label                                      combined_news\n",
      "1      1  Why wont America and Nato help us? If they won...\n",
      "4      1  All the experts admit that we should legalise ...\n"
     ]
    }
   ],
   "source": [
    "print(data_djia_up[:2])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-08-07T04:44:52.164912Z",
     "iopub.status.busy": "2020-08-07T04:44:52.163995Z",
     "iopub.status.idle": "2020-08-07T04:44:52.169697Z",
     "shell.execute_reply": "2020-08-07T04:44:52.169003Z"
    },
    "papermill": {
     "duration": 0.039749,
     "end_time": "2020-08-07T04:44:52.169868",
     "exception": false,
     "start_time": "2020-08-07T04:44:52.130119",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "   Label                                      combined_news\n",
      "0      0  Georgia 'downs two Russian warplanes' as count...\n",
      "2      0  Remember that adorable 9-year-old who sang at ...\n"
     ]
    }
   ],
   "source": [
    "print(data_djia_down[:2])"
   ]
  },
  {
   "cell_type": "markdown",
   "execution_count": null,
   "metadata": {
    "papermill": {
     "duration": 0.023278,
     "end_time": "2020-08-07T04:44:52.217557",
     "exception": false,
     "start_time": "2020-08-07T04:44:52.194279",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# Data pre-processing\n",
    "\n",
    "Now that we have separated the contents of dataframe based on their labels, we need to pre-process the `combined_news` coloumn. This will involve the following steps -<br>\n",
    "1. Punctuation removal\n",
    "2. Stopwords removal\n",
    "\n",
    "## Punctuations"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-08-07T04:44:52.272093Z",
     "iopub.status.busy": "2020-08-07T04:44:52.270929Z",
     "iopub.status.idle": "2020-08-07T04:44:52.274682Z",
     "shell.execute_reply": "2020-08-07T04:44:52.275488Z"
    },
    "papermill": {
     "duration": 0.034504,
     "end_time": "2020-08-07T04:44:52.275738",
     "exception": false,
     "start_time": "2020-08-07T04:44:52.241234",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "!\"#$%&'()*+,-./:;<=>?@[\\]^_`{|}~\n"
     ]
    }
   ],
   "source": [
    "import string\n",
    "print(string.punctuation)"
   ]
  },
  {
   "cell_type": "markdown",
   "execution_count": null,
   "metadata": {
    "papermill": {
     "duration": 0.023667,
     "end_time": "2020-08-07T04:44:52.324849",
     "exception": false,
     "start_time": "2020-08-07T04:44:52.301182",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "## Stopwords\n",
    "Stop words are words like “and”, “the”, “him”, which are presumed to be uninformative in representing the content of a text, and which may be removed to avoid them being construed as signal for prediction."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-08-07T04:44:52.380016Z",
     "iopub.status.busy": "2020-08-07T04:44:52.379139Z",
     "iopub.status.idle": "2020-08-07T04:44:53.381804Z",
     "shell.execute_reply": "2020-08-07T04:44:53.380777Z"
    },
    "papermill": {
     "duration": 1.033333,
     "end_time": "2020-08-07T04:44:53.381995",
     "exception": false,
     "start_time": "2020-08-07T04:44:52.348662",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[\"you've\", \"you'll\", \"you'd\", 'your', 'yours']\n"
     ]
    }
   ],
   "source": [
    "from nltk.corpus import stopwords\n",
    "print(stopwords.words('english')[10:15])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-08-07T04:44:53.445760Z",
     "iopub.status.busy": "2020-08-07T04:44:53.440612Z",
     "iopub.status.idle": "2020-08-07T04:44:53.451425Z",
     "shell.execute_reply": "2020-08-07T04:44:53.450549Z"
    },
    "papermill": {
     "duration": 0.045235,
     "end_time": "2020-08-07T04:44:53.451601",
     "exception": false,
     "start_time": "2020-08-07T04:44:53.406366",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "def punctuation_stopwords_removal(news_article):\n",
    "    # filters charecter-by-charecter : ['h', 'e', 'e', 'l', 'o', 'o', ' ', 'm', 'y', ' ', 'n', 'a', 'm', 'e', ' ', 'i', 's', ' ', 'p', 'u', 'r', 'v', 'a']\n",
    "    remove_punctuation = [ch for ch in news_article if ch not in string.punctuation]\n",
    "    # convert them back to sentences and split into words\n",
    "    remove_punctuation = \"\".join(remove_punctuation).split()\n",
    "    filtered_news_article = [word.lower() for word in remove_punctuation if word.lower() not in stopwords.words('english')]\n",
    "    return filtered_news_article"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-08-07T04:44:53.509131Z",
     "iopub.status.busy": "2020-08-07T04:44:53.508152Z",
     "iopub.status.idle": "2020-08-07T04:44:56.655211Z",
     "shell.execute_reply": "2020-08-07T04:44:56.654526Z"
    },
    "papermill": {
     "duration": 3.178672,
     "end_time": "2020-08-07T04:44:56.655395",
     "exception": false,
     "start_time": "2020-08-07T04:44:53.476723",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "   Label                                      combined_news\n",
      "1      1  [wont, america, nato, help, us, wont, help, us...\n"
     ]
    }
   ],
   "source": [
    "data_djia_up.loc[:, 'combined_news'] = data_djia_up['combined_news'].apply(punctuation_stopwords_removal)\n",
    "print(data_djia_up[:1])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-08-07T04:44:56.712917Z",
     "iopub.status.busy": "2020-08-07T04:44:56.711654Z",
     "iopub.status.idle": "2020-08-07T04:44:56.715233Z",
     "shell.execute_reply": "2020-08-07T04:44:56.714517Z"
    },
    "papermill": {
     "duration": 0.034944,
     "end_time": "2020-08-07T04:44:56.715402",
     "exception": false,
     "start_time": "2020-08-07T04:44:56.680458",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "words_djia_up = data_djia_up['combined_news'].tolist()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-08-07T04:44:56.773436Z",
     "iopub.status.busy": "2020-08-07T04:44:56.772401Z",
     "iopub.status.idle": "2020-08-07T04:44:56.777841Z",
     "shell.execute_reply": "2020-08-07T04:44:56.777179Z"
    },
    "papermill": {
     "duration": 0.037754,
     "end_time": "2020-08-07T04:44:56.777999",
     "exception": false,
     "start_time": "2020-08-07T04:44:56.740245",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[['wont',\n",
       "  'america',\n",
       "  'nato',\n",
       "  'help',\n",
       "  'us',\n",
       "  'wont',\n",
       "  'help',\n",
       "  'us',\n",
       "  'help',\n",
       "  'iraq'],\n",
       " ['experts', 'admit', 'legalise', 'drugs'],\n",
       " ['mom',\n",
       "  'missing',\n",
       "  'gay',\n",
       "  'man',\n",
       "  'bad',\n",
       "  'hes',\n",
       "  '21yearold',\n",
       "  'cheerleader',\n",
       "  'theyd',\n",
       "  'still',\n",
       "  'looking']]"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "words_djia_up[:3]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-08-07T04:44:56.836952Z",
     "iopub.status.busy": "2020-08-07T04:44:56.836000Z",
     "iopub.status.idle": "2020-08-07T04:44:59.627726Z",
     "shell.execute_reply": "2020-08-07T04:44:59.626883Z"
    },
    "papermill": {
     "duration": 2.824896,
     "end_time": "2020-08-07T04:44:59.627882",
     "exception": false,
     "start_time": "2020-08-07T04:44:56.802986",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[['georgia', 'downs', 'two', 'russian', 'warplanes', 'countries', 'move', 'brink', 'war'], ['remember', 'adorable', '9yearold', 'sang', 'opening', 'ceremonies', 'fake']]\n"
     ]
    }
   ],
   "source": [
    "data_djia_down.loc[:, 'combined_news'] = data_djia_down['combined_news'].apply(punctuation_stopwords_removal)\n",
    "words_djia_down = data_djia_down['combined_news'].tolist()\n",
    "print(words_djia_down[:2])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-08-07T04:44:59.691903Z",
     "iopub.status.busy": "2020-08-07T04:44:59.690850Z",
     "iopub.status.idle": "2020-08-07T04:44:59.695163Z",
     "shell.execute_reply": "2020-08-07T04:44:59.694336Z"
    },
    "papermill": {
     "duration": 0.042776,
     "end_time": "2020-08-07T04:44:59.695317",
     "exception": false,
     "start_time": "2020-08-07T04:44:59.652541",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "djia_up_list = []\n",
    "for sublist in words_djia_up:\n",
    "    for words in sublist:\n",
    "        djia_up_list.append(words)\n",
    "\n",
    "djia_down_list = []\n",
    "for sublist in words_djia_down:\n",
    "    for words in sublist:\n",
    "        djia_down_list.append(words)\n",
    "        "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-08-07T04:44:59.752123Z",
     "iopub.status.busy": "2020-08-07T04:44:59.751251Z",
     "iopub.status.idle": "2020-08-07T04:44:59.754732Z",
     "shell.execute_reply": "2020-08-07T04:44:59.755322Z"
    },
    "papermill": {
     "duration": 0.035378,
     "end_time": "2020-08-07T04:44:59.755555",
     "exception": false,
     "start_time": "2020-08-07T04:44:59.720177",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "DJIA up list : 13443\n",
      "DJIA down list : 11955\n"
     ]
    }
   ],
   "source": [
    "print('DJIA up list : {}'.format(len(djia_up_list)))\n",
    "print('DJIA down list : {}'.format(len(djia_down_list)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-08-07T04:44:59.901726Z",
     "iopub.status.busy": "2020-08-07T04:44:59.900431Z",
     "iopub.status.idle": "2020-08-07T04:44:59.904058Z",
     "shell.execute_reply": "2020-08-07T04:44:59.903368Z"
    },
    "papermill": {
     "duration": 0.124018,
     "end_time": "2020-08-07T04:44:59.904212",
     "exception": false,
     "start_time": "2020-08-07T04:44:59.780194",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "djia_up_counter = Counter(djia_up_list)\n",
    "djia_down_counter = Counter(djia_down_list)\n",
    "\n",
    "djia_up_top_30_words = pd.DataFrame(djia_up_counter.most_common(30), columns=['word', 'count'])\n",
    "djia_down_top_30_words = pd.DataFrame(djia_down_counter.most_common(30), columns=['word', 'count'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-08-07T04:44:59.978125Z",
     "iopub.status.busy": "2020-08-07T04:44:59.976841Z",
     "iopub.status.idle": "2020-08-07T04:45:00.565804Z",
     "shell.execute_reply": "2020-08-07T04:45:00.564834Z"
    },
    "papermill": {
     "duration": 0.636928,
     "end_time": "2020-08-07T04:45:00.565983",
     "exception": false,
     "start_time": "2020-08-07T04:44:59.929055",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(array([ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13, 14, 15, 16,\n",
       "        17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29]),\n",
       " <a list of 30 Text major ticklabel objects>)"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAmYAAAHLCAYAAABiRlQnAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nOzdeZhcZZX48W/2EAgBYqKGRXDhuA/GGVHHDfeNcXCbH4Isziio4zKgoyKD4C46jCBxAFEHQeOKKyrjiqIgLuCGHHBUDHsMW4KsIb8/3lukuukkXbduV93ufD/Pk6dTt+uefqu6uurcdznvtHXr1iFJkqThmz7sBkiSJKkwMZMkSWoJEzNJkqSWMDGTJElqCRMzSZKkljAxkyRJagkTM0mTRkQcGRGnTWD8dRFx/4mKL0mbMnPYDZB0dxGxpuvmPOBWYG11+6DM/GQDP+NoYG9gAXAdcFJmvqvr+7sBHwUeBPwO+OfMvKDfn6uSYAJvBW6pDl0J/C/wrsy8srrPzsAfgVmZeUdE/A9wWWYe3hVnS+Bq4AeZ+eyBPQBJE8YeM6mFMnOrzj/gz8CeXcf6TsoqHwUemJlbA48FXhIRzweIiNnAl4HTgG2BU4AvV8cHIiKm+oXjZzJzPrAdsBdwL+DnEXHvHmK8kJK0P73H8yS11FR/45OmlIiYA7wPeHF16LPAmzLz1oh4EiWR+jBwCLAGeOuGErnMzFGH7gQ6w3hPorw/fDAz1wHHRcQbgCcD3xzVpl2A84HtMvPOiDgZ+IfMXFx9/zTgZ5n5wYhYApwAPA64FnhfZn6kut+RwEMpvUj/ABwSEd8B/gdYCpwLZNfPnQucDDwLmAFcAjw3M68e1b4Dgedn5p7V7d8Dv8jMF1e3V1AS305v4FMj4hvAPYBPAf9aPQdExMuAN1KSqPOAV2TmpdX31gGvBA4d69wNyczbgd9GxD8Bv6jOf8PGzumyP+X5fBawD/CBDd0xIp4OfKhq+yeBhwCnZubJETEdOAx4ObAF5Xf8msy8oTr30cAxwIOBS4HXZeb3q+8dABwBLAL+Ahw+1mtudI9f5/WamTtUt/8EnAi8FLg38CXglZl5y+hY0lRmj5k0ubwVeDSwG/A3wKOAw7u+fy9KUrA95UP7pIiIDQWLiDdXw6aXAVtSkgkoH9q/GpVU/Ko6PkJm/hG4EXhEdejxwJqIeFB1+wnAWdX/l1c/awmlt+fdEfGUrnDPAz4PbENJHj4F/Lx6TO+oHlPH/pRh2B2BhcDBwM1jPMyzgMdHxPSqV2kW8PfV478vsFX12DqeC/wd5fl9MfCM6r7/SElenk9JQn5YPR42de54ZOZaSi/l48dz/4jYiZJAf7L6t99G7nsPyvP6FspzlZRe0o4Dqn97AJ3n5Pjq3O2BM4B3Unr33gB8ISIWVUOpxwHPqnr/Hgv0M9y9D+U5ux+wKyNf29JmwcRMmlz2Ad6emddk5krgKEoPQ7f/yMxbM/Msygfqi0cH6cjM9wLzKT1SpwI3VN/aquv/HTdU9x3LWcATI+Je1e3PV7d3AbYGfhkRO1J6yt6UmbdUPVQnj2r/OZn5pcy8k5L8/F3X4/kB8NWu+95OSTLun5lrM/PnmXnjGI/xD8BqSjL7ROBM4PKIeGB1+4fVz+t4b2Zen5l/Br5XnQdwEPCezPxdZt4BvBvYLSLuM45zx+sKSvIzHvtRkucLKQniQyLiERu477OB32bm6VXbjwOu6vr+PsAxmfmHzFxDSeD+XzWcvC/w9cz8embemZnfAn5WxYTS0/rQiNgiM6/MzN/28HhHOz4zV2TmtcC7KHMgpc2KiZk0uSyhDCV1XFod67guM2/ayPfvJjPXZeb5lN6mo6rDaygJVbetKQnOWM6i9N48AfgB8H1K0tOd+CwBrs3M7hiXUnr3OlZ0/X/JBh5Px6mUJOvTEXFFRBwdEbPG0b6zRrXvrFH37U5Y/kpJUgHuAxwbEddHxPWUodhpo9q/oXPHa/sq7njsR+kpIzOvoDyO/Tdw3yV0PbdVT+hlo74/+nU1E7gn5XG/qPO4q8f+OODe1e/mnyi9lVdGxBlVwltX9+9/k69daSpyjpk0uVxB+aDs9ErsVB3r2DYituxKZnYCfjPO2DMpQ0hU8Q+NiGldw5kPB5Zt4NyzgPdTPuzPAs6mzH26hfWJzxXAdhExvys52wm4vCtO99DplRt4POvgrrlZRwFHVSsYv04ZovvoBtq3J7ALpafrekov0WOohuzGYQVl1WRTiy9GqOZ57Ql8exz3fSzwAOAtEXFodXg+pdfsDVWvWLcrgR26zp/WfZv1r6uOnYA7KCs+V1Dmor18rLZk5pnAmRGxBWW48yOMPRx7E2WFcce9xrjPjqPacMUY95GmNBMzaXJZDhweET+lJChHUCb8dzsqIg4DdqfMeXrb6CBVEvByyuKB6ylDhq8G3lPd5fuU8hyvjYgTqvsCfHesRmXmJRFxM2XY672ZeWNEXA28gLIYgcxcERE/Bt5TLSTYFfjn6pyxYl4aET/rejyPoiQuX6kewx6UyeYXUua43c76kiKjnUWZvH51Zl4WETdSetxmUhYujMcJwDsi4oLM/G1ELACenpmfG+f5Y6p6+e4PHElJVo4Zx2n7A99i5LyyLShz5Z7FyCFfKEPax1fz5L5G6eHqToyWA2+qFj2spCSvn6nKdJwG/DQinkFJGmdR5jn+nvKc7w58h9LjuoYN/w4uoCT77wRmA68f4z6vjoivUXobDwM+s4nnQZpyHMqUJpd3Uub3/Ar4NWUV3zu7vn8VpSbZFZRhroMz86INxNoL+D/K8ORplBV7HwLIzNuAf6R88F8PvAz4x+r4hpwFrKrmV3VuT2Nk4rM3sHPVvi8Cb6vmLG3ISygf/NdSEsxPdH3vXpS5bDdS6qydxd2TVKrHczElafhhdftG4A/Aj6pJ95uUmV+krIj9dJXY/YaSBNX1T9XCi+spyeYq4JHVsGS3Eas6q9WoLwY+lJlXdf37IyXZvNtwZmb+BXgRcHT1cx5MeR3dWt3lY9W5P6DUTrsFeE117grKoozDKEnbCsrK1OnVv0Mpv89rKUPDr9rA4z0V+CXwJ0rNtrGSrk9V3/tD9e+dY9xHmtKmrVu30ZXcjYqI51JWVk2j/EEfmZmnR8SulDpJCylvGvtl5iUDa5g0BYwuP6DJLSIeTikcu80ExJ5OGXbeJzO/13T8OqpyGf+SmZscypWmsoH1mFVzGk4FXpqZu1GGL06p3iBOAJZl5q6UOSwnDqpdktQ21fviiym9Wk3FfEZEbFPVwjuMcoF8blPxJTVj0EOZd1LqDkGpU3QlpT7RUtbXA1oOLI2IRQNumyS1xZ+BZwL/1mDMx1CGrv9Cmav3j5k5Vt03SUM06KHMp1DmFdxEWUH0HOA24BOZ+ZCu+10I7JuZvxhH2JmU1UWXUVYRSZIktdVG85aBrcqsChW+BXheZv4oIv6ekqSNLo7Zq/tQVgc9npF1eSRJktpmB8pCpPtTerFHGGS5jN2AJZn5I4AqObuJsvpn+4iYkZlrI2IGo4ohbkJn494fNt5iSZKkiXFvhpyYXQbsEBGRmVnto3cvysbDF1CW0Z9WfT2/2m5mPK4EuO66m9h22y1ZddoXazVu4b57sWrVmlrnSpIkjcf06dPYdtstocpfRhtYYpaZV0XEK4HPR0RnX7oDM/PaiDiYskLzCEoNpg1uxjuGtQB33lnmyt25+qaN3nljOjEkSZIm2Jg1FAda+b/ayuRu25lUBTB3H2RbJEmS2sbK/5IkSS1hYiZJktQSJmaSJEktYWImSZLUEiZmkiRJLWFiJkmS1BImZpIkSS1hYiZJktQSJmaSJEktYWImSZLUEiZmkiRJLWFiJkmS1BImZpIkSS1hYiZJktQSJmaSJEktYWImSZLUEiZmkiRJLWFiJkmS1BImZpIkSS1hYiZJktQSJmaSJEktYWImSZLUEiZmkiRJLWFiJkmS1BImZpIkSS1hYiZJktQSJmaSJEktYWImSZLUEiZmkiRJLWFiJkmS1BImZpIkSS1hYiZJktQSJmaSJEktMXNQPygidga+1HVoG2DrzNwuInYFTgEWAquA/TLzkkG1TZIkqQ0Glphl5p+A3Tq3I+KDXT//BGBZZp4WEfsCJwJPHlTbJEmS2mAoQ5kRMRvYB/hYRCwGlgLLq28vB5ZGxKJhtE2SJGlYhjXH7B+AyzPzF8CO1f/XAlRfr6iOS5IkbTYGNpQ5ysuAjzUZcOHCrfqOsWjR/AZaIkmSVM/AE7OIWAI8EXhpdWgFsH1EzMjMtRExA1hSHR+3VavW9J2crVy5uq/zJUmSNmb69GkbzVeGMZR5AHBGZq4CyMxrgAuAvavv7w2cn5krh9A2SZKkoRnGUOYBwGtHHTsYOCUijgCuA/YbdKMkSZKGbeCJWWbuOsaxi4DdB90WSZKkNrHyvyRJUkuYmEmSJLWEiZkkSVJLmJhJkiS1hImZJElSS5iYSZIktYSJmSRJUkuYmEmSJLWEiZkkSVJLmJhJkiS1hImZJElSS5iYSZIktYSJmSRJUkuYmEmSJLWEiZkkSVJLmJhJkiS1hImZJElSS5iYSZIktYSJmSRJUkuYmEmSJLWEiZkkSVJLmJhJkiS1hImZJElSS5iYSZIktYSJmSRJUkuYmEmSJLWEiZkkSVJLmJhJkiS1hImZJElSS5iYSZIktYSJmSRJUkuYmEmSJLWEiZkkSVJLzBzkD4uIucB/AU8FbgHOycxXRMSuwCnAQmAVsF9mXjLItkmSJA3boHvMjqYkZLtm5sOA/6iOnwAsy8xdgWXAiQNulyRJ0tANLDGLiK2A/YD/yMx1AJl5dUQsBpYCy6u7LgeWRsSiQbVNkiSpDQY5lHk/yjDl2yJiD2ANcDhwM3B5Zq4FyMy1EXEFsCOwcoDtkyRJGqpBJmYzgfsC52fmGyNid+CrwIuaCL5w4VZ9x1i0aH4DLZEkSapnkInZpcAdVEOWmfmTiPgLpcds+4iYUfWWzQCWACt6Cb5q1Zq+k7OVK1f3db4kSdLGTJ8+baP5ysDmmGXmX4DvAU8DqFZiLgYuBi4A9q7uujelV81hTEmStFkZ9KrMg4HDIuLXwKeBl2bm9dXx10TExcBrqtuSJEmblYHWMcvMPwBPGuP4RcDug2yLJElS21j5X5IkqSVMzCRJklrCxEySJKklTMwkSZJawsRMkiSpJUzMJEmSWsLETJIkqSVMzCRJklrCxEySJKklTMwkSZJawsRMkiSpJUzMJEmSWsLETJIkqSVMzCRJklrCxEySJKklTMwkSZJawsRMkiSpJUzMJEmSWsLETJIkqSVMzCRJklrCxEySJKklTMwkSZJawsRMkiSpJUzMJEmSWmLmsBvQVtstmMuM2bNqnbv2ttu59oZbGm6RJEma6kzMNmDG7Flcc8Jxtc5dfPBrARMzSZLUG4cyJUmSWsLETJIkqSVMzCRJklrCxEySJKklTMwkSZJawsRMkiSpJUzMJEmSWmKgdcwi4k+UAl+dIl9vyswzI2JX4BRgIbAK2C8zLxlk2yRJkoZtGAVmX5iZvxl17ARgWWaeFhH7AicCTx580yRJkoZn6EOZEbEYWAosrw4tB5ZGxKLhtUqSJGnwhtFj9smImAacDRwG7AhcnplrATJzbURcUR1fOYT2SZIkDcWgE7PHZ+aKiJgDfBA4HvivJgIvXLhV3zEWLZrfQEtGxlp3x+1Mm1lvM/R+zpUkSZPPQBOzzFxRfb01Ij4MfAU4BNg+ImZUvWUzgCXAil5ir1q1pu/kbOXK1Xf9v98krRNr0aL5XLHskFoxlrz6mBFtkiRJk9v06dM2mq8MbI5ZRGwZEQuq/08D/h9wQWZeA1wA7F3ddW/g/Mx0GFOSJG1WBtljdk/gC1WP2AzgQuBV1fcOBk6JiCOA64D9BtguSZKkVhhYYpaZfwAesYHvXQTsPqi2SJIktdHQy2VIkiSpMDGTJElqCRMzSZKkljAxkyRJagkTM0mSpJYwMZMkSWoJEzNJkqSWMDGTJElqCRMzSZKkljAxkyRJagkTM0mSpJYwMZMkSWoJEzNJkqSWMDGTJElqCRMzSZKkljAxkyRJaomZw26Axm/bBbOZOXtOrXPvuO1WrrvhtoZbJEmSmmRiNonMnD2Hi5Y9r9a5D3z1lwETM0mS2syhTEmSpJYwMZMkSWoJEzNJkqSWMDGTJElqiXEnZhHxhg0cP6S55kiSJG2+eukxO2IDxw9voiGSJEmbu02Wy4iIJ1f/nRERewDTur59X2D1RDRMkiRpczOeOmYfrb7OBT7WdXwdcBXwmqYbJUmStDnaZGKWmbsARMQnMnO/iW+SJEnS5mnclf+7k7KImD7qe3c22ShJkqTN0bgTs4hYCiwDHk4Z1oQy32wdMKP5pkmSJG1eetkr8xTgq8DLgL9OTHM0CG6GLklSO/WSmN0HeGtmrpuoxmgwZs6ewzknPbfWuY95xddwM3RJkiZGL3XMvgg8faIaIkmStLnrpcdsLvDFiDibUibjLq7WlCRJ6l8vidmF1b++RcTbgCOBh2XmbyJiV8octoXAKmC/zLykiZ+libXNgtnMqjlf7fbbbuV656tJknSXXsplHNXED6xWdz4a+HPX4ROAZZl5WkTsC5wIPHms89Uus2bP4cyPPrvWuc/456/jfDVJktbrpVzGBhOlzPzuOGPMoZTceAnwverYYmAp8LTqbsuB4yNiUWauHG/7JEmSJrtehjI/Our2ImA2cBllz8zxeDtwWmb+MSI6x3YELs/MtQCZuTYirqiOm5hJkqTNRi9Dmbt0346IGcDhjHMT84h4DPB3wJt7aeB4LVy4Vd8xFi2a30BLmo1lmyRJ2nz00mM2QtWz9S5Kj9kx4zjlicADgU5v2Q7AmcC/AdtHxIwq5gxgCbCil/asWrWm7+Rs5cr1OWa/CUMnVlNxpnqbJEnaHEyfPm2j+UovdczG8jRgXPtkZuZ7M3NJZu6cmTtTErpnZOZngQuAvau77g2c7/wySZK0uell8v8Kyr6YHfMotc1e1UA7DgZOiYgjgOsA66JJkqTNTi9DmfuOun0TcHFm3ljnB1e9Zp3/XwTsXieOJEnSVNHL5P+zACJiOnBP4OrMHNcwpiRJkjZt3HPMImJ+RHwCuBm4HLg5Ik6JiAUT1jpJkqTNSC+T/z8EbAk8DNii+joPOG4C2iVJkrTZ6WWO2TOB+2bmX6vbF0fEgcD/Nd8sbY7cd1OStLnrJTG7hVLt/9KuY/cAbm20RdpszZo9h89//Jm1zn3hgd/EfTclSZNdL4nZycC3IuIYSnJ2H0px2I9MRMMkSZI2N70kZu+iTPrfh1KZ/wrg6MwcvYemJEmSauhl8v+xQGbmUzPzwZn5VOB3EfHBCWqbJEnSZqWXxGxv4Gejjv0ceElzzZEkSdp89ZKYrQNmjDo2o8cYkiRJ2oBekqofAu+oKv93dgA4sjouSZKkPvUy+f91wNeAKyPiUmAn4Epgz4lomCRJ0uaml70yL4uIpcCjgB2BFcB57peptlmwzSxmz5pb69zbbr+FG66/fUJiSZK0Kb30mFElYedW/6RWmj1rLh8/5em1zj1w//8F1idTs2fN5bhPPqNWrNfuc+aIWJIkbYoT9yVJklrCxEySJKklehrKlFSPc9UkSeNhYiYNwOxZcznys/Xmqh35YueqSdLmwqFMSZKkljAxkyRJagkTM0mSpJYwMZMkSWoJEzNJkqSWMDGTJElqCctlSJPM1tvMZs6sObXOvfX2W7nx+tvuuj1/mznMnTW7Vqxbbr+N1dffWutcSdLYTMykSWbOrDkc+MVn1jr343t9E1ifmM2dNZtnffnVtWJ943nLWI2JmSQ1yaFMSZKkljAxkyRJagmHMiX1rcm5avO3mcvcWbNqxrqd1dffUutcSWoDEzNJfZs7azbP/uI7a5379b0OHzFXbe6sWTzn9ONqxTrj+a9lNSZmkiYvhzIlSZJawsRMkiSpJQY6lBkRXwJ2Ae4E1gCvycwLImJX4BRgIbAK2C8zLxlk2yRJkoZt0HPM9s/MGwAi4nnAx4ClwAnAssw8LSL2BU4EnjzgtkmaQlxEIGkyGmhi1knKKguAOyNiMSU5e1p1fDlwfEQsysyVg2yfpKlj7qxZPOcLJ9c694wX/IuLCCQNxcBXZUbEycDTgWnAM4Edgcszcy1AZq6NiCuq4yZmkiRpszHwyf+Z+S+ZuRNwGPD+Qf98SZKkthpaHbPMPDUiTgIuA7aPiBlVb9kMYAmwopd4Cxdu1XebFi2a33eMpmPZpsHHsk2Dj9X2Nt22di2zZ8yoFaefcyVtfgaWmEXEVsC2mbmiur0ncC1wDXABsDdwWvX1/F7nl61atabv5GzlytV3/b/fN/dOrKbiTPU2NRnLNg0+1ubQpud+/pO14nzthfuMiCVp8zZ9+rSN5iuD7DHbEvhcRGwJrKUkZXtm5rqIOBg4JSKOAK4D9htguyRJklphYIlZZl4NPHoD37sI2H1QbZEkSWojK/9LkiS1hImZJElSS5iYSZIktYSJmSRJUksMrY6ZJG2O5m+zBXNn1XvrveX2O1h9/c0Nt0hSm5iYSdIAzZ01kz0/f3qtc7/6wudjRTRpanMoU5IkqSVMzCRJklrCoUxJmoSanKvmvDepPUzMJGkSmjtrJs/7/DdrnfvlFz5zxFy1ubNmstcXzq4V64sveJzz3qQGOZQpSZLUEiZmkiRJLeFQpiSpEfO3mcfcWTNqnXvL7WtZff1fG26RNPmYmEmSGjF31gxe9IVf1Tr3cy94uHPVJBzKlCRJag0TM0mSpJYwMZMkSWoJ55hJklpn623mMafmQoJbb1/LjV0LCRZssyWzZ9Xrh7jt9ju54fqbANhmmy2ZVTPO7bffyfVVnKZjaWoxMZMktc6cWTN47RdX1Dr3uL12HHF79qzpLPvi1bVivXqve971/1mzpnP65/9SK87zX3iPEbdnzZrO9z65slasPfZZVOs8TQ4OZUqSJLWEiZkkSVJLOJQpSdIkte2CLZk5u14fyx233cl1NzhXrW1MzCRJmqRmzp7O+SdfU+vcR/zL4oZboyY4lClJktQSJmaSJEkt4VCmJEliuwVbMqPmfLW1t93JtV3z1bZbMI8Zs+vVoVt721quveGvjcYpsbZgxux6ac/a2+7g2htubiTO9atv2eh9TMwkSRIzZk/nTx+8qta5O7/+XqNizeCqD/y+Vqx7veH+I+Jc/cGf14pzz9c/clSbZnL1cd+vF+u1TxoR55plX60VZ/Gr99zkfRzKlCRJagkTM0mSpJYwMZMkSWoJEzNJkqSWMDGTJElqCRMzSZKklhhYuYyIWAicCtwPuBX4PXBQZq6MiF2BU4CFwCpgv8y8ZFBtkyRJaoNB9pitA47OzMjMhwP/B7y3+t4JwLLM3BVYBpw4wHZJkiS1wsASs8y8NjO/33XoXOA+EbEYWAosr44vB5ZGxKJBtU2SJKkNhjLHLCKmA68EvgLsCFyemWsBqq9XVMclSZI2G8Oa/P8hYA1w/JB+viRJUusMfK/MiPgA8ABgz8y8MyJWANtHxIzMXBsRM4AlwIpe4i5cuFXfbVu0aH7fMZqOZZsGH8s2DT6WbRp8LNs0+Fi2afCx2timTeUrA03MIuJdwCOB52TmrQCZeU1EXADsDZxWfT0/M1f2EnvVqjV9J2crV66+6//9/gI6sZqKM9Xb1GQs2zT4WLZp8LFs0+Bj2abBx5qKbdpUvjLIchkPAQ4DLgZ+HBEAf8zMvYCDgVMi4gjgOmC/QbVLkiSpLQaWmGXmb4FpG/jeRcDug2qLJElSG1n5X5IkqSVMzCRJklrCxEySJKklTMwkSZJawsRMkiSpJUzMJEmSWsLETJIkqSVMzCRJklrCxEySJKklTMwkSZJawsRMkiSpJUzMJEmSWsLETJIkqSVMzCRJklrCxEySJKklTMwkSZJawsRMkiSpJUzMJEmSWsLETJIkqSVMzCRJklrCxEySJKklTMwkSZJawsRMkiSpJUzMJEmSWsLETJIkqSVMzCRJklrCxEySJKklTMwkSZJawsRMkiSpJUzMJEmSWsLETJIkqSVMzCRJklrCxEySJKklZg7qB0XEB4AXADsDD8vM31THdwVOARYCq4D9MvOSQbVLkiSpLQbZY/Yl4AnApaOOnwAsy8xdgWXAiQNskyRJUmsMLDHLzLMzc0X3sYhYDCwFlleHlgNLI2LRoNolSZLUFsOeY7YjcHlmrgWovl5RHZckSdqsDGyO2URbuHCrvmMsWjS/gZY0G8s2DT6WbRp8LNs0+Fi2afCxbNPgY7WxTZvKV4admK0Ato+IGZm5NiJmAEuq4z1ZtWpN38nZypWr7/p/v7+ATqym4kz1NjUZyzYNPpZtGnws2zT4WLZp8LGmYps2la8MdSgzM68BLgD2rg7tDZyfmSuH1ypJkqThGFhiFhHHRcRlwA7AtyPit9W3DgZeExEXA6+pbkuSJG12BjaUmZmvBV47xvGLgN0H1Q5JkqS2GvaqTEmSJFVMzCRJklrCxEySJKklTMwkSZJawsRMkiSpJUzMJEmSWsLETJIkqSVMzCRJklrCxEySJKklTMwkSZJawsRMkiSpJUzMJEmSWsLETJIkqSVMzCRJklrCxEySJKklTMwkSZJawsRMkiSpJUzMJEmSWsLETJIkqSVMzCRJklrCxEySJKklTMwkSZJawsRMkiSpJUzMJEmSWsLETJIkqSVMzCRJklrCxEySJKklTMwkSZJawsRMkiSpJUzMJEmSWsLETJIkqSVMzCRJklrCxEySJKklZg67AR0RsStwCrAQWAXsl5mXDLdVkiRJg9OmHrMTgGWZuSuwDDhxyO2RJEkaqFYkZhGxGFgKLK8OLQeWRsSi4bVKkiRpsNoylLkjcHlmrgXIzLURcUV1fOUmzp0BMH36NACmz9+ydiM6Me66PX9+I7FmzN+2sTbNmr+4kVhztmomDsDcBmPN2+qejcTaastm4gDMbyjWNvOaa9PCBmMt3mK7RmItnreguTbNa+Zvb/G8rRpsU3PvLYvnzWsk1uJ5WzTWpkXz5jQSa9G8WY21abt5MxqLNX9e/dRemlYAACAASURBVH6I7ljzGooDMHfLZmLN3qq5Ns3curnnfMbW9VOM7ljTt57dWJumz5/bTJvmN/K3N+aTPW3dunW1gzclIh4JfCIzH9J17EJg38z8xSZOfxzww4lsnyRJUsMeD5w9+mBbesxWANtHxIyqt2wGsKQ6vik/pTy4K4G1E9hGSZKkfs0A7k3JX+6mFYlZZl4TERcAewOnVV/Pz8xNDWMC3MoYGackSVJL/d+GvtGKoUyAiHggpVzGtsB1lHIZOdxWSZIkDU5rEjNJkqTNXSvKZUiSJMnETJIkqTVMzCRJklrCxEySJKklTMwkSZJawsRMkiSpJUzMJEmSWsLEbBKrivJu8pg2PxFRf9dfqQERMT0inj3sdmyOImLr8RxTO7ViS6bJICKeCpydmbcMuy1dPgUsHcexcYmIXYEHZeaXI2IrYHZmXlu3cVVysF1mXlU3RttExL9n5tGbOjbOWGOdcwNwTmZ+t4c4H8jMN3Tdngl8AdizRpsae51HxFMor6fjI+KewILMvLjfuFNV9fdy13tyZv51iG35YGa+flPHNiYz74yIw4GvN9y2xp6nlj3n+2bmaZs6Nk7f5+6fA2MdG6iIuB9wP0Y+542+PnoVEQH8DTC3cywzPzG8Fk3xxCwi/hN4O3AT8D3Ki/Kgmi/0fwOWR8Svge8C3wHOy8xxbZy+gQ/hu2Tmv4+3IRFxD2AxMDciHgRMq761ANhyvHFGxdwfeAswG/gysD2wDHhqj3E+DRwE3Ab8ErhHRLw7Mz9Qo00PAD4ObJ+Zu0TEUuAfMvPIHuMcAnw0M2+IiFOBvwNem5n/22ubgP8HjP5djnVsPO4JPB74UnX7eZR9X18cEZ/NzHeNM86OEfGvVQI0nZKcX1SjPdDn67wjIt4MPJuyUe/xwCzgY8Djem1Qk7+/Jl5TEfE5YINbpmTmi3ts017AhyjPFZS/53WUjY57Un3IHM7dP/we1WOoJ4xx7Im9tgf4WUQ8KjPPq3HuCA0/T43EioiHAydSPtjndI5nZs9tAg6h7BW9qWMba89Mynv49IjYgpGfDfNqtImImAvsw91fU+P+zKrivAf4F+B3QOf9ZB01EvemnveIeC3l86qzofjjgbOAcSVmTb8XdEzpxAx4amYeGhHPAS4H/onyIug5McvM51Qv+kcDT6F8+C0Exts9fFP19X6UN7jTq9t7AWf22Jx9gNcDSxj5or6BegkCVby/BX4IkJkZEfeqESeqD9AXUj7YDwHOBXpOzID/Bt4JvLe6fQFwKnBkj3EOyMxjImIPSkL7MuA4YNwf7BHxNODpwJJRSfaCHtvSbQnwyMy8rvoZ76C8ITwO+Akw3sRsP+CbEXEl8A/ANZn5xjoNauB13rE35fV0XhX3sj6GUvr+/XVp4jX1terro6p/nfeTl1B6JXr1fuDFwLmZeWeN87t9GvgcJfnsKZkGiIgXVW3ZOSI+2/WtBUCd3qTHA6+MiEuANZ2DNRJFaPZ5airWf1MS4WOAZwKvBlb3EiAi/hbYnXIR+6quby2gJFm9eCvwNkqycFPX8RuB/+wxVsfnqnb8BLi1ZgyAFwH3y8wb+4jR0ffzXnkF5W/4R5n5jIh4KHBYD+c3/V4ATP3ErOMJwOmZeUVE1NoctOqlejLwNOAxQFJ6E8YlM4+q4nwdWJqZq6rb76Rs3j5umXkscGxEHJaZ7+7l3I24LTPXlAvuu9xRI86s6usTga9n5l8jou4b34LM/GZ1pdUZGrmtRpzOB9QewCcz88dVz1IvbqN8sIx+w7sSeE+NNkHptbmucyMzr4uInTNzdUSM+w0wM2+NiOdTkupzM/Nfa7an79d5l5sz8/ZRr6e6G/M28fvr6Ps1lZmnAETES4EnZObN1e2TgK/UaNO1mfnjGueNZXqf7wkXA2dQPmTO6Dp+I/VeB+Me+hyHJp+npmLNzczvRMT0zLwSODwivg+8r4cY21MuYrak9AZ33Agc0Etjqs+ZoyLi+H7eB0a5f2Y+qIE4VzaUlEEzzzvALZl5UzUfclpm/qYabh2XCXgvAKZ+YnZNRHwEeAbw3qonoO5jvho4hzI0+qrMvL1mnJ06SRlAZq6KiJ3rBMrMd0fEPGAHRnYxX1gj3Kpqjtk6KHMbgMtqxLkwIv4XeCDw5qo7va61ETGrq03bA3WSvJsj4q2Unsa/j4hp9HglmplnAWdFxBcy8zc12jCWC6s/4I9THuMBwMURMYdx9HZExMrqvM4wzBbAvSPiecC6zFxco01Nvc5XRMTjgHVVEnUY8Nuasfr+/XVp6jUF5e+uO4G+DdixRpwvRsQrgc8Ad83tqznf6ZyIeHhm/qrGuWTmL4FfRsRX+plf2hXvrH5jVO9x0MDz1GSsSufi9dqI+BvKe+Z9eglQzen9KvDczKz9YT4q5r9CY3Po/hAR8zOzTo9Ut3MiYjmlB677Oa8zx6zv573y1+r94JfA+yJiBfWGfJt6LwCmfmL2kurfyVVvxH2oN6QGZWjmyZTu4Csi4jvAtzPzFz3G+V1EnAx8tLp9IDXnA1Xd3u8DrmPkmP19a4R7PWXYKiLiT5Rhi54njwP7UxLhX1ZXItsDb64RB+DDwBcpXfxHUobs3lojzgHAq4A3ZuZV1RXRJ2u26ekRsaKh+WovA46gzMGaRpkH+SbK7/JZ4zj/b0fd7vRITRt9xx409Tp/DWVY9qGU19IPgX1rtukAmvv9NfWagjIX5esR0enxfml1rFedIetlXcd6mu8UET+tzpkFHBgRycgPv16HDhdGxFfof37nAsprejdGTq5+cg9hOj3Vndf1MkZekPQyr6jJWACfiYiFlF7zs6vzj+gxRqfn9s300cvSrWsO3RL6e3xQpsj8LCLOZORrqqc5ZqzvDXxN17Fac8xo6HmnvK/MBg4F3k357HxpjThNvRcAMG3durqjC+3X1aMwQs2ehE7MWZQPr6MovV+9TjbcmvIC2oPyx/Id4B11ungj4g/AHpl5aa/nbiDeDGDXql3Z64TvKsabKYnwXxpq0+MoCeI04KuZ+cMez58BnJCZL2+oPb/KzIdX853eTHkdHJeZo5OkgWnj67wrzjzK8NqaTd55wzEePLoXOCKelpnfqhmvr9dUV5xZwMHAk1j/t3xSH72MtUXERifm99pzFRHfplzEvjczd6t6PX+dmQ/pMc4XgAspr6X/oFyM/Dwz616stVb1ephbt2cpIo4DTmtoocTvKRcdfc/Hi4i3jXW8Mz1n2Pp93htsQ2PvBVO9x6z7w7KzsqTWExURh1ImQz+K0u15MjXmXFQJ2Bs2ecfxuarBpOypwE8z83fV7W0iYmn2ULahcm/gN9Vw5rLM/Ek/7crMsylXRHXPX9vLnIFxaGy+U2xgpW6NK9HWvc4j4uzMfFz30EnnWI1mLY+IZ2RVdiUinkDpZYyNnza2fl9TXXFup/RKfKjfWNXV/6MpCfa5vQ4jdhKv2EDJhRpNamp+5/0z8wUR8bzMXB4RpzNy7tq4VRe1a6q2PJTSG3t6ZvbcrmigHEjXeSNKQERE3eG5JhdKNDYfr8kErOpBDUb2nv6gh/N3ycw/RsSDx/jeuKfxRMOrKZt8L4ApnpiNkbQcERHfBd5RI9xCyvDOj7JGjacYudrmbjLzwzXa9K3qw/3TjOxirjPH7P2MrHFzI+WKuae6N5n5uoh4C+Vq7aSIuJ0yXLB8vM9b0380wHcj4njK0Fr3G16d56nJ+U7diwjmAs8FftZrkDa9zruMmKdR9VxuVzPW64EvR6mL9hDKNIDn1gkU0Vg5iU7dv4/R/3DfMyiruS6oDj28SrDq9Aj2XXKh0tRcvM68m9siYjvKtIsdasSBMtT/hIiYT1nJ/hvKirwDasRqpBxINFgCgmYXSjQ5b5GIeDp3H45+e48x/onymbItpUrC/SkXf718xnyI8rc/VnLfyzSeRlZTRsTrMvPYiHg/Y49a9HqRDUzxxGy0iLgvsHOdczPzsOqK7cFAr/NtYORqm9HqjifvV3190ahYdeaYTcvMu9pRXZXWGr7KshLzJOAa4IOU+mjviIhDMvOzGz8bWP9H05R/rr4+p+tY3efpABqa7zT6SjQi3k2NUi6j9fs67/NnvxH4d2BBRFzT9a151H+evhcRx1I+7O4F/GNmZs0m9lVOYpQP00w5l3dRVnR1eqsfSHkdjDsxi2ZLLkBzc/EurhKyT1HK5lzP+gS0V9Oqeat7Ax/JzCOj1Nsbt2i+HEhjJSCaWCjRpXveYl9zzCLivZTPr4dQalw+D/h2jTYdBjwSODMzHxGlBNHzewmQmc+tvu5S4+d3x2lqNWUn6a09VWMsUzoxGzX3ZjplYuzrasZ6FnAS5c185+qN8G2ZOa4J8pl5YDXk9cya3dxjxezrxTnK6ojYvTP0GBG7M7JHZ1yiVHg/iLKo4afAvpn5g4jYhTIZcpOJWeePpilNPU9VovrG7vlqmfl/1C+XMdoaylVkr+1q8nW+I6UW3uhK2ONNYk+iJD7HU2oLddyYXaVBxtmW0b3M84AfAE+MiCfW7GXut5xEt6aG+2Z1krIqzkVVb1UvGiu5ULXhE9Uc1j0pz/v+NefinUV5no6JiPOAbYBv1IgDpaD2HMriouOqY70m102XA2msBESsX8AxQp3e3MxscrvF5wCPoMwNPCgi3s7IhSrjdUdmXhOlOgKZ+a0q6R+3sYYwu9UYBelrNWVmnlh9bXS+3ZROzBg59+YOypysulfJb6e84X0DIDN/1uvcpZyALUpi5LY3i4Ftst62N/8OfCkiOiUNHkyPVzOV84H/AR6fmXeV26jmBXy8l0ARcW/KG/AelDes7wGvy1K3pmfV89OdbPy5l/Obnq82qvt7OuVqsk5PUJOv849RepV2owzXvhL4v/GenJk3UFZx1RpqHGV0L/OvKVf8f0f9XuZzoo9yEqM0Ndy3MiIOyMz/qeLsD6zsJUBOTMmFJubi7QYcGqVsQ2cniXtQyrL06jOU5+Ui4EdRCmD3NNyeDZcDodkSEN1zj+dSFkxcUbdh0dwWe7dk5h0RsS4iZmXm5dUFXK9uraZ+XBIRrwH+BCzqMcYZrO8B3ImSUEPp8bwU6PUivLHVlE0M93ZM6cRsjLk3/ca7KkYWzKxTBbnJLUpGb3szm5rb3mTmOdXVyGMoL/of99rDUbnvhuYmZeaYq3s24lRKD8nrqjYdWB3rdZuoJ1OK+N6TcoU9G1hFqSLfqybnq3V3f99B2WLkC70Gafh1fo/M/GhEvL56TfyEGsMWDfS8kZkH9vpzx2F3RpaTmEap+VZngnVTw30HAZ+MiBMoHzoXUKO0SDZYcqGpuXiZ+eoq3g6Unpd3Uz5Qex5Sy8yjqiHtG6vHugZ4Qa9xKjdGxCu4+wfpy3qM01gJiNFDmVEWUNUpw0NEHEBZNd7XFnuV1VFWV/8YOCXKDiN1io8fTtlB5E2Uyv0LKNNCxq0z+lG9Ds7OzM9Vt19IvX1A/5WymvKFlPeCMyi9/j1pcLgXmOKJWcNWV8N0navjJ1HmS/SqyZU3TW57Q5WI9dWbl5m3NHjlcI9R572z+gPs1fspKw0/Q/nj/RfKh0MdjcxXq4ZFp9dIVidaZyhuTUTsROnZqFO4sa+et9GqRKGJjYY7E6y3qr7WnhvS1HBfNRz+6KpXY1r2t+y/qQu/RubiRcQjKX97T6VcQJ5Jj0OGcfeVeEtGXSBfXqNpJ1I+//agJAkvoVwE9iQz96jxs8dra+rNg4VyMdvEFntQPmfWUnr0DqEMR79oo2eMIdev8L+Beglit90z867pGpn5+YjoudpBNreasqnhXsDErBdvpgxj7hJl64cHUPYl7FWTK2/63vYmIr6TmU+Ju9fC6vQk9NSr1PCVw+8j4v6Z+fsq9v0oK7F6lpkXV93w64CPRETPb8JVnEbmq1XDomOtDBu2H1STtT8M/JzSK/y5GnEa6XkDiD43Gh7lMsoihN2q27+gfuHbvob7xkg4Osc7sev0wjZ14dfUXLyfUnpa3lJzjho0txKv26My82FR6hK+JyI+TLlwG5cN/e466vzuRs0xm055XHX3t2xqiz0ys3vY+Z0120OUXWD2psyjrb0ZemVeRDy+85qKUpuw54r91Xy3l9F/z2lTw72Aidm4ZeZ5UYqKPpb1Q30995jl+npDW1a3e55g36WJbW86H0pNFUht8spha8p8kM4H399TtkX6LPRUNqNT0+vyiNiTMrehpyX7ETEny56UY/7xZ71l6GdUV3mjh0VrLWlvQq7f/PzUiDgL2DrrbUHVVM8b9L/RcLcTKEMVnfmOB1B6T5423gDRXDmXiUg4mrrwa2ou3qMpO0kcUc3x/BFlJ4nTxxsgG1qJN8rN1de1ETEvy04e2/dw/kT87rp7fO4A/piZdeeYNbXFXqe3+q3cPaHqNdk/nTIHs3PB149XU+obdj4/O0lfrxrpOaW54V7AxKwn1R/vd1hfSHBerx+iUUoZfIqSoa+LiPMpKxf/UKNJY217s08vAToT6Rucp9TklcMnGVli4VM14xwbEdtSKo9/ijK34d96jHEtZcXb6C1doP5WJ0eP+tpPrL5sIOH8C/CXOq9zxu55+3zN5vW10fAoizLzY123Px4Rva5gbaScy0QkHA1e+I2ei9eJ3+scs/OA86JsX/YcSumcg6hXtuGzo5PesY6N07XVe8I3gW9ExF+Aq8Z78kT97qoenKC8D1yziVM2pqkt9qD0mJ9KWdTVT4mZnbLHnSM2JDN/WH2WBuW9+KKsUWiYPntOu+xNScQ6w727UOat1WJiNk4R8XzKCsF7V4fq1oU5kT6v2DuyVEJ/evSx7U1sYIl218/o9aqosSuHbKhsRmYur/57HjXKUVQ6+5l+P3vb5+9uupKg+f3EadiGEs5ar/MGe96guY2GAe6MiMgsddCqXoWePmyael1uqPe16+f03HPa4IVfIz1vURbKPJnSo/FdygTwXncT6Rjrb/dBNWM9p5pO8FZKL8k21Bsa7/wed2Bkb1Kdocy/pSz+uZXydzczIl6Qve9T25m6sTt9brFXuSMz31/z3G6/iYh7Z81V9WOYSXmuZgL3jx4q/3fpt+e041jKBcdtlEVAiyhFh2vtzW1iNn5HUwoT9rv3WBNX7HeJru1Auuam9DKBv6ntoToau3KIiHtQVps+hZIcfJtSLqOnUgLVG+dhlBWjL4lSwPOBmfmlHsJsEREvAHaKUtNuxEbhPT7nnSRoLOsYwt9lNlv3aERPRlZlSfro3Whqo2Eor4MfRsQFlOd6t15jxQa20uroYc7MWMlwR92e074v/KIsTPlAZm6sKPZ4/Qr4r2qBQy0R8XLKcPauUWqhdSygXnmZzhzPrSlbRtUu6hwRr6YUGL6OkZX/6wxlHgsc2JkkX02d+RBlCsd427OhOl9RM3EB+GZEPDMzv1nj3G5HAT+p/va6e2F7fk/oet6vZX2JmjrPe189p91NqpK6F1IWtxwC/AQTswnX1N5jfV+xd1QfEPtT3pxqbQeSd1+i3e8QSJNXDidS5swdSvnwenl1rNf6av8NXElZ1QdlrsVyoJfErDMEc09KzbduvT7n0wGqq/XbKB+k0yirRQe+AfYEGat344F1AnX1tN1EeY5qy1IQ9iGUobppwDmZ+Zcew/QzL7S7LY0mw5W+L/yqpOUvETE3+9uWi8zsufTAGP4XuIRykfbGruM3UhK/nkXEsynvJbUKhnc5FHhoQ1NBtsyuvYmz7HixZY8xmq7zBeWC+MsRcSfre/N6XhhG6ZH8CmXBTb+7bjT1vHf3nO5DeZ7q9Jx2CkI/EfhGZt5cPV+1mJiNX1N7j72F9VfsUJKFulf/ewG7NDFZvMEhkCavHO6Xmd11it7W9bz14mGZuX+UPQmpViv19KGYmV+mvDkdk5mH1GjDWJ6VIzf1/kCUhQ7HNhR/4DbRu1Gn8DER8QBKD1Bfe1J2VD2uteeJZcNVvhvW1IXfJZR5gp9n5MKUOrst9KX68L2UMpe2KUfRZ8HwylUNzs/9a0TskZnfA4iIJ9LjNlHZfJ0vKBeOB9J/QjU7M/+1j/O7NfK8d4Z3q1GwU/sIdWGUunMPBN4cZQVqbSZm49fU3mM7UibB3pP6V+wdf2b96rd+NTX3rckrh+kRsTgzr4G7KvfX6WUY8RxFxNyacWgwKQNYGHcvB7KwwfjDMFbvxlxK7aKf1oz53zSzJ2UjIuJFmfm5uPuWUcD4k5e4e4majro9EjByqBbqX/jNp/RWd8/hqrvbQl8i4n2Z+abYwGrYmsPjfRUM7xoy/FY1cvFpRl6w1xkyfC3whYi4lfI451C/gG4jdb4q12Zm3YU73c6NiIdlZk/7m25AI897RDyWMk3pvoycI9jr397+lK3CfpllodL2lBJbtZiYjVODww6PoMy/6mxRslVEfDdH1ooZr0OBr0bEtxj54qxzVdvU3Lcmrxw+AJwfEZ3u+WdTehx79YOIOAyYE6Uw8CGUGmvD9lbKm9XPq9uPoPQ2TVqd3o0oW3tdQEmKf0nZhufd1Os5bWpPyqY8lLJSbaw5WL0kL02VqLlL9Tw9lFJepPaFX07Mrgt1dcrlNLIattJvwfDRZTK6C67WnWO2DeU1tZjyu7ua+r2EjdT5qnwpIg6m7HPcz2jRoygFkPta6VvZr/ra7/P+UeAdwLn00RuYZRP0L3Xdvpx6hY8BE7OBywa3KKFk5PemDD92zzGro6khkMauHLJUVv8F8CTKG9WxNa9E306phL2acnX0Fdb3vgxNZp4eET+k1HrqfIj2tLChxXbtGtL+LiUZPpd6iVlTe1I2Itfv1nBQjlqiHxHb9BCn0S3juuJeU10c9VPWZx7lIui+mblPzQUzjcjMr1YLEu6bze2U8Rb6KBjeNWT4NOCnWdW0rH7/j6zZpvcDS7tGCKZT/l7qDEE2VecL1heV/TD9jRbVXuQ2WjZXpuTmzKxbhmnCmJgNWDSwRUmXR1I+AJsYYmhkCKTpKwfKcO2Ps8aS8S6/p9RD26czbNgWVSL21WG3YwJ0D2l/PTP/2seQdlN7Ujbt43TVDYyI+ZTVXY8ez8kRcWpmvjQ2ULKmTk9CNFfWp7NgprNDQp0FM43JhnbKiPUlSn5NqevVSXp+QVlZ2aujGZk43UiVYNWINa37vbzqGa5V0zCbq/PV2GhRNlBjL5ov9P31iHhWZn6j17ZMJBOzwWtii5KOi1lf9LQvDa1Wa1SDK6ceThki/E5EXEiZ//T1hhJaja2xIe1saE/KCXB1RBydmf9efVB8jZEFkTflg9XXJkvWNFXWp+8FMxOgiZ0yRpeqmTbqdq+JUGPJFGV4dffM/AlAlDpkPSUvYyQunTIlMyNiZhMLxepqaIHZOZSkt6lC3wcBh0XEavpbcdooE7PB63uLki43Aj+PiDMZOWZfZ+8xWP8GNbRholEaWTlVDQ28s5qj9DxKb8DaKMUvl2WfJQE0pkYnw2Yfe1JOoEOBz0bEIcAzgTMyc9ybIWfmz6uvZ23qvj1oqqxPYwtmGtT3Thm58VI1dXqU+k6muvw7ZT5XZ1u9B9N7aaDRiUtH3Z7TJvW9wCwzl1Zfm3otNj7PswnT1q2z02AYqg+qzhYlO2VmnS1KxpxvUWc5fzUEchJl+5zplKHMVwxjTklXm36amX8XEedn5iOqY3f9v8dY8yhDs68CVgEnU/ZHu1/2WclfE2NDq/A66q7G69eoYZQtKBcO36NcSPQ8nBLR2F6ERMRbKJPY+yrrU614u56yl+6rKHMEf5WZh/fapjaKiLNzZKmaMY+NI85jKHtAjkimMvPcmu3aFngM6/djrjO82koRcUFm7rapY7LHbOCiwS1K6iRgG/Eu4LGZeTHcVTvqKwxpTkml35VTVOd9iLLs/CvASzKz8yb6qYi4aMNnasiaXIXXpO5hlM7Xv6WUB6nTK9HUXoTQXFmft1J6cLoXzLynz7b1LcpuII+mPKZzM3NVzVCNlKrJzHOq0hmNJFPVub3sIjKmKLsarKmGVh9KWd15et15Zg1prLh6U6Ls43w0pSNibud4ZtZZVdsYE7PB63uLko5qeGEfqi2ZOsdrDmVe20nKqhiXRETdN72mvJk+Vk51+TPw4M7KqVH2qN88TaRsaE/KpjU4jNLR1F6ETbbtfpn5LtYnelQrM4d2IdPVq/8LShL0NxFRt1e/sVI1TSVTDfse8IRqQcqZwG8ow+0HDLFNTRZXb8rHKLXQdqN8lr6S9fPyhsbEbMCymS1KOj5HqYf2E3oojrgB/1vNu/go5U3vQMpuB/Og3obK/crM86LsF/dY1l+N9txjtrEPvWxuQ101LCJel5nHxgb2puxjLmXbNLUXYZM+xd1XFo51bJAa69Wf4qVqoCxKuCki9gY+kplHRkQThV370WRx9abcIzM/GhGvr3o/f0LZgmqoTMwmt/tn5oM2fbdx6cxXe8eo4+9nuJNGZ1U/eyibe2uoOnOkGtmbssWa2ouwb9VQ4WJgbkQ8iPWr3hZQVoAPU6O9+lO4VA2U398cygKc46pjQx02pNni6k3pDO2uiYidKEV97zPE9gB+0E12f4iI+Zm5ut9AEzA807exFiT0MXShSSYzT6z+e8zo13j1JjpVNLUXYRP2AV4PLGHk8NwNjFwNOTBdiy3G7NUfRpsmgc8AKylDzz+KiHvRtRhkGLLZ4upN+UFEbEeplfhzyoXR54bYHsDEbLK7gbLFRVPlMtqmjQsSNHinV0N9awEi4t6UuYcPGW6zGtPUXoR9y8xjgWMj4rDMfPew21MZXbOqu1d/HfCfA29Ry2XmUVE2Mr+xWgCwhvr7bjYimi2u3pQzgbWZeWpEnEVJFGcPuU0mZpNcVv+mqjYuSNDgfRM4Bdg3IhZR3kz/Y7hNalRTexE2JjPfXfVU7cDIhUV1tkTrty2t681vu4iYRknEdgXeRNmrdgn97cLSryaLqzflrl0aMvPPEXEZsZrVQwAABwhJREFU8DOGO5fSxGyyqqpL75CZLx92WyZQ6xYkaPAy8z8j4tiqB+DvgffWLMjcVk3tRdiYiHgV8D7KNkXd+/AOtYyAxu0YyiT7pZTEbDVlp4k6G4Y3pcni6k1pcueGxpiYTVJZ9o7ruQr+JNPWBQkagKo+VMfJlKXt3wcuiIgHD6P3ZiK0tEfoDcBDc4I2WteE24My2f4XAJm5qiqvNDSZeR5wXkScyvri6gcx3PfxJnduaIyJ2eT23apg7ei94/zA0lRwxhjHXlD9s/dmYl1lUjap3ZKZ68qmEhBln9NpGz9lYjVZXL1BTWyD1TgTs8ntn6uvz+k65geWpoTM3GXYbdiMfauqH/dpRs57mxIXfZuBX0fEPsC0iNiZqrjrcJvUXHH1pjS9c0NT3CtTUitFxJzMvHXU3pR3cY7hxImIP45xeN2wt6rR+FQV/49h/U4pXwFen5lDH6bTptljNslFxFOAB2Xm8dWEym26VzJKk9g5lMnLo8slgHMMJ5S9lZNXNXn9DdXCsKm8OGzKMjGbxCLizcCzKTVhjqfUX/kY8LhhtktqQmZ2lqwfRFm9NVYvjhpkL+XkVy0Me8Kw26H6TMwmt72BvwXOA8jMyyJi6+E2SWrcbsChEdHZyuXbwHcz85rhNmtKspdyajgjIt7A3ReGmVhPAiZmk9vNmXl7Z+VNxUmDmlLG2MrlPQx/K5cpqdNL6YroSe/oUV/BxHrSMDGb3FZExOOAddVy6MOA327iHGlSaelWLlJrmVhPbiZmk9trKF3VDwX+SlkOvc9QWyQ1r41buUitFhH3oFTbXwecm5luZzdJWC5jCqgm6k7PzDWbvLM0yUTEoyiFKZ8CtGUrF6m1IuL5wEmUyv/TgL8BXpGZXxpqwzQuJmaTWER8l7KP5OmZefOw2yNNpIjYnvVbueyUmc6XkcYQEb8DntcpnRQRDwC+kpkPGm7LNB6OQ09uxwB7AZdGxEci4rHDbpDUtIg4PiIuBM4Gdqds5bLDcFsltdq13fUsM/MSwKHMScIesykgIhZSSmfsD8zPzAcOuUlSYyLiFcB32rSVi9RmEfE24A7KiMo04EDgVuC/wbIZbefk/6mhk11Pw15QTTGZedKw2yBNMm+rvr5j1PH3Y9mM1rPHbBKLiD2BA4DHA18GTsnMs4faKEmSVJs9ZpPb64D/AfZ18r8kSZOfPWZTQERsCZCZNw27LZIk/f/27uY1rjIMw/gVqWBLXbUFW1FRMLcofmA2gsTWnWK7KX6AoCiouLKLggv1D3AlVlciVKGgC3VnBHeJoSIFjRvFp2qg1vhFu/ADXDRtXJyTMgaKZNI65yTXb3WeM2cOz2yGe9553/doeAazHktyA/AOzbMEl4A5mtGz+ZE2JkmShuJE8X57g2YTwc3AFuDN9pwkSeoh55j1246qOjxQv5XkwMi6kSRJa+KIWb+dS5LlIsk4cHaE/UiSpDVwxKzfXgBmk3zZ1rcDj4+wH0mStAYGs377BJgAbqPZXHauqhZG25IkSRqWqzJ7LMk5mtWYY+2pJeAMcAx4uqpqVL1JkqTVc8Ss314E/gYO04SzJ4ArgF9pVmfuGVVjkiRp9Qxm/fZgVU0M1IeSzFbVZJKDI+tKkiQNxVWZ/bal3WQWOL/h7Pa2XBxNS5IkaViOmPXbS8CxJJ/TzC+bAJ5NshV4b6SdSZKkVXPyf88l2QHcRTPH7LOq+m3ELUmSpCEZzCRJkjrCOWaSJEkdYTCTJEnqCIOZJElSRxjMJEmSOsJgJkkXSZLpJE+Nug9J/WUwkyRJ6giDmSStUpKxJH5/Srro3Plf0rqX5Elgf1Xta+vvgC+q6uG2PgnsA7YAh4Bx4DhwoKo+ba+ZBo4Ce4A7gVuTXA+8DuwEjtBs9CxJQ/MXn6SNYAaYTHJZkp3A5cDdcP4Zs1uBH4Ap4DVgG/AKMJVk28B9HgOeAa4Efgc+oHk02nbg++V7StKwDGaS1r2qmgf+BO4AdgMfAwtJbmrrWeAB4NuqOlJVi1X1LvANzUjasrer6quqWgTuB76uqver6gzwKvDL//epJK1HBjNJG8UMzd+Q97TH0zShbHdb7wJOrHjPCeDqgfrkwPGuwbqqlla8LkmrZjCTtFEsB7PJ9niGfwezn4DrVrznWmBhoB58uPDPwDXLRZKxwVqShmEwk7RRzAD3Apur6keavy/vo5lPNgd8BIwneTTJpiSPADcDH17gflPALUn2J9kEPAdcdak/hKT1zWAmaUOoquPAXzSBjKr6A5gHjlbV2ao6DewFDgKngeeBvVV16gL3OwU8BLzcXn8jzapNSRra2NLS0n9fJUmSpEvOETNJkqSOMJhJkiR1hMFMkiSpIwxmkiRJHWEwkyRJ6giDmSRJUkcYzCRJkjrCYCZJktQR/wBxyJNJE70A9gAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 720x432 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "%matplotlib inline\n",
    "\n",
    "fig, ax = plt.subplots(figsize=(10, 6))\n",
    "sns.barplot(x='word', y='count', data=djia_up_top_30_words, ax=ax)\n",
    "plt.title('Top 30 words when DJIA goes up')\n",
    "plt.xticks(rotation='vertical')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-08-07T04:45:00.640585Z",
     "iopub.status.busy": "2020-08-07T04:45:00.634257Z",
     "iopub.status.idle": "2020-08-07T04:45:01.222827Z",
     "shell.execute_reply": "2020-08-07T04:45:01.221793Z"
    },
    "papermill": {
     "duration": 0.630886,
     "end_time": "2020-08-07T04:45:01.223009",
     "exception": false,
     "start_time": "2020-08-07T04:45:00.592123",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(array([ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13, 14, 15, 16,\n",
       "        17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29]),\n",
       " <a list of 30 Text major ticklabel objects>)"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAmYAAAHLCAYAAABiRlQnAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nOzdeZhcVZn48W/SSQiBECAmKouCC6/7EmdEHTdwX9BxnR+CKOO4j8uo4ygi4owrOowgjKC4oGhcQdwGF1AExR1mRhlecMOwSQxbUCAh5PfHuUWqm07Sdeuk+ib5fp6nn+q6VffUqe6qe997znvOmbF27VokSZI0/WZOdwUkSZJUGJhJkiR1hIGZJElSRxiYSZIkdYSBmSRJUkcYmEmSJHWEgZmkzoiIwyPipE1Y/tqIuNumKl/rRMSjI+KS6a6HtLmZNd0VkLZGEXF93915wE3Amub+SzPz0xVe4whgf2ABcDXw4cx8Z9/jDwA+CtwT+D/gRZl53rCvqxJgAm8Bbmw2XQ58C3hnZl7ePGcP4HfA7My8OSI+AVySmYf2lbMd8Efg+5n55JG9AUnTxhYzaRpk5va9H+APwH5924YOyhofBe6RmTsADwOeFxHPBIiIOcCpwEnATsCJwKnN9pGIiC39wvBzmTkf2Bl4BnAH4OcRcccByng2JWh//ID7SdpMbekHRmmzEhHbAO8Fntts+jzwL5l5U0Q8mhJI/SfwOuB64C3rC+QyMydsugXodeM9mvL9/0BmrgWOjog3APsCp02o057AucDOmXlLRJwAPC0zFzePnwT8LDM/EBG7AMcBDweuAt6bmR9pnnc4cB9KK9LTgNdFxOnAJ4AlwI+A7HvducAJwJOAMeAi4KmZ+ccJ9TsYeGZm7tfc/zXwi8x8bnN/GSXw7bUGPjYi/gu4HfAZ4B+bvwER8ffAP1OCqJ8AL8nMi5vH1gIvB14/2b7rk5mrgV9FxN8Bv2j2f8OG9unzAsrf80nAAcD71/fEiHg88MGm7p8G7g18KjNPiIiZwCHAi4FtKf/jV2Xmtc2+DwGOBO4FXAy8JjO/1zz2QuAwYBHwJ+DQyT5zEbEt8CHg6ZQWwo9PePyezeMPAC4F3pyZXxnw8/U94CzK5/R+wDnA8zLzT1P5Y0qbA1vMpG55C/AQysnr/sCDgUP7Hr8DJSjYlXLS/nBExPoKi4g3Nd2mlwDbUYIJKCft/5kQVPxPs32czPwdcB3wwGbTI4DrmxMtwCOBM5vflzavtQulteddEfGYvuKeDnwR2JESPHwG+Hnznv6teU89L6B0w+4OLAReBtwwyds8E3hERMxsWpVmA3/TvP+7ANs3763nqcBfU/6+zwWe0Dz3bynByzMpQchZzfthY/tORWauobRSPmIqz4+IO1EC6E83Pwdt4Lm3o/xd30z5WyWllbTnhc3PPkDvb3JMs++uwNeBd1Ba994AfCkiFjVdqUcDT2pa/x4GrK+7+23AXZufJ9D3v4yI2cBXKd25i4FXAZ+OiBjw8wXwPODgppw5TD3IlTYLBmZStxwA/GtmXpmZy4G3A8+f8Jy3ZuZNmXkm5YT63ImF9GTme4D5lBapTwHXNg9t3/d7z7XNcydzJvCoiLhDc/+Lzf09gR2A/46I3SktZf+SmTc2LVQnTKj/OZn55cy8hRL8/HXf+/k+5eTds5oSZNwtM9dk5s8z87pJ3uNvgZWUYPZRwDeBSyPiHs39s5rX63lPZl6TmX8AvtvsB/BS4N2Z+X+ZeTPwLuABEXHnKew7VZdRgp+pOIgSPJ9PCRDvHREPXM9znwz8KjNPbup+NHBF3+MHAEdm5m8z83pKAPf/mu7kA4FvZOY3MvOWzPw28LOmTCgtrfeJiG0z8/LM/NV66vBcSg7dVZm5rKlDz0Mon7n3ZOaqzDwD+BolBxKm8PnqK+vjmXlhZt5AaVEe9H8gdZqBmdQtu1C6knoubrb1XJ2Zf97A47eRmWsz81xKa9Pbm83XU054/XagBDiTOZPSevNI4PvA9yhBT3/gswtwVWb2l3ExpXWvZ1nf77us5/30fIoSZH02Ii6LiCOalpeN1e/MCfU7c8Jz+wOWv1ACBoA7A0dFxDURcQ2lK3bGhPqvb9+p2rUpdyoOorSUkZmXUd7HC9bz3F3o+9s2LaGXTHh84udqFnB7yvt+Tu99N+/94cAdm//N31FaKy+PiK83Ae9G6zDh9XYBlk0IkPs/G1P5fPUM+z+QOs3ATOqWyygnyp47Ndt6dmq6l9b3+IbMonQzAfwKuF9EzOh7/H7N9smcSelienTz+9mU7sL+wOcyYOeI6G91uxMln6inv+v08vW8H6DkZmXm2zPzXpQutKey/u683on9Ec3vZ7L+wGx9llFGxO7Y97NtZv5wivtvUJPntR+li3Rjz30YcHfgzRFxRURcAewN7L+eQROXA7v17T+j/z6Tf65upoz4XEbJRet/39s1ra1k5jcz83HAHYELgI+sp9qXU7qd+1+j//V3b/4G/Y/3PhtT+XxJWwWT/6VuWQocGhE/pQQxh1ES/vu9PSIOoZyon0rJ7RmnOQG+mNLVcw2ly/CVwLubp3yPMj3HqyPiuOa5AGdMVqnMvCgibqB0e70nM6+LiD8Cz6IMRiAzl0XED4F3NwMJ9gJe1OwzWZkXR8TP+t7PgymBy1ea97APJdn8fEoO0mrWTSky0ZmU5PU/ZuYlEXEdpcVtFiWxfCqOA/4tIs7LzF9FxALg8Zn5hSnuP6mmle9uwOGUHMEjp7DbC4BvMz4Q3ZaSK/ckxnf5QunSPqbJk/sapYXrDn2PLwX+pRn0sJzSTfu5ZpqOk4CfRsQTgO9QcvQeAvya8jffGzid0uJ6Pev/H3yeEkj+mJLP+Kq+x34M/Bl4Y0T8OyXo2o/yuZzS50vaWthiJnXLOyj5Pf8D/C9lFN87+h6/gjIn2WWUbq6XZeYF6ynrGcBvKN2TJ1FG7H0QIDNXAX9LOfFfA/w98LfN9vU5E1jR5Ff17s9gfOCzP7BHU79TgLc1OUvr8zzKif8qSoD5yb7H7kDJNbqOMs/amdw2SKV5PxdSgoazmvvXAb8FftAk3W9UZp5CGRH72Saw+yUlCGrr75qBF9dQgs0VwIOabsl+40Z1NqNRnwt8MDOv6Pv5HSXYvE13ZjMq8TnAEc3r3IvyObqpecrHmn2/T5k77UaawKnJB3s6ZeDDckoL2j9Tzg8zKaNIL6P8jx4FvGI97/ftlO7J31GS/D/VV79VlJG4T6IE2/8JHDThszuVz5e0xZuxdu0GR3pXFRFPpYy8mkH5wh+emSdHxF6UeZQWUg4qB2XmRSOrmLQZ6E2XkZm7bey56r6IuB9l4tgdN0HZMyk5Zgdk5ndrly9p0xlZi1mT8/Ap4PmZ+QBKk/WJzQHkOODYzNwLOBY4flT1kqRRa457z6W0atUq8wkRsWMzF94hlAvgH9UqX9JojLor8xbKvERQ5jG6nDJ/0RLWzRe0FFgSEYtGXDdJGpU/AE8E/qlimQ+ldF3/iZK/9bfNlBKSNiOj7sp8DPA5ShLofOApwCrgk5l5777nnQ8cmJm/mEKxsyijjy6hjDKSJEnqqg3GLSMbldkM8X4z8PTM/EFE/A0lSJs4eeag7kwZPfQIxs/bI0mS1DW7UQYq3Y3Syj3OKKfLeACwS2b+AKAJzv5MGR20a0SMZeaaiBjjthMVbkhvYd+Nzg0kSZLUEXdkmgOzS4DdmrXRslkH7Q6UhYnPowyzP6m5PbdZjmYqLge4+uo/c8sto+uWlSRJGtTMmTPYaaftoIlfJhp1jtkBwJsogwCgzHH05WaJjxOBnShzNB2UmTnFYvcAfrdixfUGZpIkqdNmzpzBwoXbA+wJ/H7i4yMNzDaRPTAwkyRJm4GNBWbO/C9JktQRBmaSJEkdYWAmSZLUEQZmkiRJHWFgJkmS1BEGZpIkSR1hYCZJktQRBmaSJEkdYWAmSZLUEQZmkiRJHWFgJkmS1BEGZpIkSR1hYCZJktQRBmaSJEkdYWAmSZLUEQZmkiRJHTFruitQ084L5jI2Z3arfdesWs1V195YuUaSJElTt0UFZmNzZrP8Qye12nfRyw8EDMwkSdL0sStTkiSpIwzMJEmSOsLATJIkqSMMzCRJkjrCwEySJKkjDMwkSZI6wsBMkiSpIwzMJEmSOsLATJIkqSMMzCRJkjrCwEySJKkjDMwkSZI6wsBMkiSpIwzMJEmSOsLATJIkqSNmjeqFImIP4Mt9m3YEdsjMnSNiL+BEYCGwAjgoMy8aVd0kSZK6YGSBWWb+HnhA735EfKDv9Y8Djs3MkyLiQOB4YN9R1U2SJKkLpqUrMyLmAAcAH4uIxcASYGnz8FJgSUQsmo66SZIkTZfpyjF7GnBpZv4C2L35fQ1Ac3tZs12SJGmrMbKuzAn+HvhYzQIXLtx+6DIWLZpfoSaSJEntjDwwi4hdgEcBz282LQN2jYixzFwTEWPALs32KVux4vqhg7Ply1cOtb8kSdKGzJw5Y4PxynR0Zb4Q+HpmrgDIzCuB84D9m8f3B87NzOXTUDdJkqRpMx1dmS8EXj1h28uAEyPiMOBq4KBRV0qSJGm6jTwwy8y9Jtl2AbD3qOsiSZLUJc78L0mS1BEGZpIkSR1hYCZJktQRBmaSJEkdYWAmSZLUEQZmkiRJHWFgJkmS1BEGZpIkSR1hYCZJktQRBmaSJEkdYWAmSZLUEQZmkiRJHWFgJkmS1BEGZpIkSR1hYCZJktQRBmaSJEkdYWAmSZLUEQZmkiRJHWFgJkmS1BEGZpIkSR1hYCZJktQRBmaSJEkdYWAmSZLUEQZmkiRJHWFgJkmS1BEGZpIkSR1hYCZJktQRBmaSJEkdYWAmSZLUEQZmkiRJHWFgJkmS1BEGZpIkSR1hYCZJktQRs0b5YhExF/gP4LHAjcA5mfmSiNgLOBFYCKwADsrMi0ZZN0mSpOk26hazIygB2V6ZeV/grc3244BjM3Mv4Fjg+BHXS5IkadqNLDCLiO2Bg4C3ZuZagMz8Y0QsBpYAS5unLgWWRMSiUdVNkiSpC0bZlXlXSjfl2yJiH+B64FDgBuDSzFwDkJlrIuIyYHdg+QjrJ0mSNK1GGZjNAu4CnJuZ/xwRewNfBZ5To/CFC7cfuoxFi+ZXqIkkSVI7owzMLgZupumyzMwfR8SfKC1mu0bEWNNaNgbsAiwbpPAVK64fOjhbvnzlUPtLkiRtyMyZMzYYr4wsxywz/wR8F3gcQDMSczFwIXAesH/z1P0prWp2Y0qSpK3KqEdlvgw4JCL+F/gs8PzMvKbZ/qqIuBB4VXNfkiRpqzLSecwy87fAoyfZfgGw9yjrIkmS1DXO/C9JktQRBmaSJEkdYWAmSZLUEQZmkiRJHWFgJkmS1BEGZpIkSR1hYCZJktQRBmaSJEkdYWAmSZLUEQZmkiRJHWFgJkmS1BEGZpIkSR1hYCZJktQRBmaSJEkdYWAmSZLUEQZmkiRJHWFgJkmS1BEGZpIkSR1hYCZJktQRBmaSJEkdYWAmSZLUEQZmkiRJHWFgJkmS1BEGZpIkSR0xa7or0FU7L5jL2JzZrfZds2o1V117Y+UaSZKkLZ2B2XqMzZnNlccd3WrfxS97NWBgJkmSBmNXpiRJUkcYmEmSJHWEgZkkSVJHGJhJkiR1hIGZJElSRxiYSZIkdYSBmSRJUkeMdB6ziPg9ZYKv3iRf/5KZ34yIvYATgYXACuCgzLxolHXbVHZesA1jc+a02nfNqlVcde1NlWskSZK6ajommH12Zv5ywrbjgGMz86SIOBA4Hth39FWrb2zOHC479nWt9t3llUcCBmaSJG0tpr0rMyIWA0uApc2mpcCSiFg0fbWSJEkaveloMft0RMwAzgYOAXYHLs3MNQCZuSYiLmu2L5+G+kmSJE2LUQdmj8jMZRGxDfAB4BjgP2oUvHDh9kOXsWjR/Ao1qVtWfzm33LyKmbPa5av171urHEmSVNdIA7PMXNbc3hQR/wl8BXgdsGtEjDWtZWPALsCyQcpeseL6oYOz5ctX3vr7sIFVr6xa5fTKuuDYp7cq5x6vPHVcnc758FNblfPQl3xtXJ0kSdLUzZw5Y4PxyshyzCJiu4hY0Pw+A/h/wHmZeSVwHrB/89T9gXMz025MSZK0VRlli9ntgS81LWJjwPnAK5rHXgacGBGHAVcDB42wXpIkSZ0wssAsM38LPHA9j10A7D2qukiSJHXRtE+XIUmSpMLATJIkqSMMzCRJkjpiOiaY1RZkxwVzmD1nm1b7rl51E9dcu6pyjSRJ2nwZmGkos+dswzc/+uRW+z7hRd8ADMwkSeqxK1OSJKkjDMwkSZI6wsBMkiSpIwzMJEmSOsLATJIkqSMMzCRJkjrCwEySJKkjDMwkSZI6wsBMkiSpIwzMJEmSOsLATJIkqSMMzCRJkjrCwEySJKkjDMwkSZI6wsBMkiSpIwzMJEmSOsLATJIkqSOmHJhFxBvWs/119aojSZK09Rqkxeyw9Ww/tEZFJEmStnazNvaEiNi3+XUsIvYBZvQ9fBdg5aaomCRJ0tZmo4EZ8NHmdi7wsb7ta4ErgFfVrpQkSdLWaKOBWWbuCRARn8zMgzZ9lSRJkrZOU2kxA6A/KIuImRMeu6VmpSRJkrZGUw7MImIJcCxwP0q3JpR8s7XAWP2qSZIkbV2mHJgBJwJfBf4e+MumqY62ZjsumMPsOdu02nf1qpu45tpVlWskSdJoDRKY3Rl4S2au3VSV0dZt9pxt+OLHn9hq32cffBpgYCZJ2rwNMo/ZKcDjN1VFJEmStnaDtJjNBU6JiLMp02TcytGakiRJwxskMDu/+RlaRLwNOBy4b2b+MiL2ouSwLQRWAAdl5kU1XkuSJGlzMch0GW+v8YLN6M6HAH/o23wccGxmnhQRBwLHA/tOtr8kSdKWapDpMtYbKGXmGVMsYxvKlBvPA77bbFsMLAEe1zxtKXBMRCzKzOVTrZ8kSdLmbpCuzI9OuL8ImANcQlkzcyr+FTgpM38XEb1tuwOXZuYagMxcExGXNdsNzCRJ0lZjkK7MPfvvR8QYcChTXMQ8Ih4K/DXwpkEqOFULF24/dBmLFs2vUJO6ZVmn6SlLkqTpMEiL2ThNy9Y7KS1mR05hl0cB9wB6rWW7Ad8E/gnYNSLGmjLHgF2AZYPUZ8WK64cOzpYvXxdjDnuS75VVq5wtvU61y5IkqYtmzpyxwXhlkHnMJvM4YErrZGbmezJzl8zcIzP3oAR0T8jMzwPnAfs3T90fONf8MkmStLUZJPl/GWVdzJ55lLnNXlGhHi8DToyIw4CrAedFkyRJW51BujIPnHD/z8CFmXldmxduWs16v18A7N2mHEmSpC3FIMn/ZwJExEzg9sAfM3NK3ZiSJEnauCnnmEXE/Ij4JHADcClwQ0ScGBELNlntJEmStiKDJP9/ENgOuC+wbXM7Dzh6E9RLkiRpqzNIjtkTgbtk5l+a+xdGxMHAb+pXS5IkaeszSIvZjZTZ/vvdDripXnUkSZK2XoO0mJ0AfDsijgQuBu5MmRz2I5uiYpIkSVubQQKzd1KS/g+gzMx/GXBEZk5cQ1OSJEktDNKVeRSQmfnYzLxXZj4W+L+I+MAmqpskSdJWZZDAbH/gZxO2/Rx4Xr3qSJIkbb0GCczWAmMTto0NWIYkSZLWY5Cg6izg35qZ/3srABzebJckSdKQBkn+fw3wNeDyiLgYuBNwObDfpqiYJEnS1maQtTIviYglwIOB3YFlwE9cL1Nds2DH2cyZPbfVvqtW38i116yuXCNJkqZmkBYzmiDsR82P1ElzZs/l4yc+vtW+B7/gW4CBmSRpepi4L0mS1BEGZpIkSR1hYCZJktQRBmaSJEkdYWAmSZLUEQZmkiRJHWFgJkmS1BEGZpIkSR1hYCZJktQRBmaSJEkdYWAmSZLUEQZmkiRJHWFgJkmS1BEGZpIkSR1hYCZJktQRBmaSJEkdYWAmSZLUEQZmkiRJHTFrlC8WEV8G9gRuAa4HXpWZ50XEXsCJwEJgBXBQZl40yrpJkiRNt5EGZsALMvNagIh4OvAxYAlwHHBsZp4UEQcCxwP7jrhu0m0s2HE2c2bPbbXvqtU3cu01q6uWI0naso00MOsFZY0FwC0RsZgSnD2u2b4UOCYiFmXm8lHWT5pozuy5HP3pJ7Ta99UHfBNYfWs5h3++XTmHP3ddOZKkLduoW8yIiBOAxwMzgCcCuwOXZuYagMxcExGXNdsNzCRJ0lZj5Mn/mfkPmXkn4BDgfaN+fUmSpK4aeYtZT2Z+KiI+DFwC7BoRY01r2RiwC7BskPIWLtx+6DotWjR/6DJql2WdRl9W1+u0es0qZo/NaVXOxH1XrVnNnLHZrcoaZl9J0uRGFphFxPbATpm5rLm/H3AVcCVwHrA/cFJze+6g+WUrVlw/dHC2fPnKW38f9oTaK6tWOVt6nWqWtTXU6eBTntiqnI8/47TblPWkU1/Zqqz/evqx48qSJG3czJkzNhivjLLFbDvgCxGxHbCGEpTtl5lrI+JlwIkRcRhwNXDQCOslSZLUCSMLzDLzj8BD1vPYBcDeo6qLJElSFznzvyRJUkcYmEmSJHWEgZkkSVJHGJhJkiR1xLTNYyZpyzF/x22YO7vd3Go3rl7FymtuqlwjSdo8GZhJGtrc2XN48invaLXvN55xKCsxMJMksCtTkiSpMwzMJEmSOsKuTEmdMn/Hucyd3W4NzhtXr2blNTdWLUeSRsnATFKnzJ09m6ecfHSrfb/+zFezkhvXlfOlE9qV86x/uLUcSRoluzIlSZI6wsBMkiSpI+zKlKSNMF9N0qgYmEnSRsydPZunfvHTrfb92rMPMF9N0pTZlSlJktQRBmaSJEkdYWAmSZLUEeaYSdIIzd9xW+bObnfovXH1zay85oaq5dQuS9JwDMwkaYTmzp7Ffl88udW+X332M1nZV87Tv3haq3JOffYTby2nV9YzvnR2q7JOedbDx5UlaTh2ZUqSJHWEgZkkSVJH2JUpSapi/o7zmDt7rNW+N65ew8pr/lK5RtLmx8BMklTF3NljPOdL/9Nq3y88637mqknYlSlJktQZBmaSJEkdYVemJKlzdthxHtu0zFe7afUaruvLV1uw43bMmd2uHWLV6lu49po/A7Djjtsxu2U5q1ffwjVNOdKGGJhJkjpnm9ljvPqUZa32PfoZu4+7P2f2TI495Y+tynrlM25/6++zZ8/k5C/+qVU5z3z27Vrtp62PXZmSJEkdYWAmSZLUEXZlSpI0YrXy1XZasB2z5rQr5+ZVt3D1tevy3nZesB1jLctas+oWrhpX1jzG5rTLEVyzag1XXbv1zmlnYCZJ0ojNnj2T7356eat99zlg0a2/z5ozk3NPuLJVOQ/8h8Xj7o/NmcnvP3BFq7L2eO0dJpQ1xhXv/3Wrsu7whru12m9LYVemJElSRxiYSZIkdcTIujIjYiHwKeCuwE3Ar4GXZubyiNgLOBFYCKwADsrMi0ZVN0mS1D01c9V2XrAtY3PahT1rVt3MVdfeUKWca1beuMHnjDLHbC1wRGZ+DyAi3ge8B3gRcBxwbGaeFBEHAscD+46wbpIkqWPG5ozxxw/8vNW+t3/tgyaUNYs/Hv29dmW9+tHjyrny2K+2KmfxK/fb6HNG1pWZmVf1grLGj4A7R8RiYAmwtNm+FFgSEYuQJEnaikxLjllEzAReDnwF2B24NDPXADS3lzXbJUmSthrTlfz/QeB64Jhpen1JkqTOGfk8ZhHxfuDuwH6ZeUtELAN2jYixzFwTEWPALsBAi6QtXLj90HVbtGj+0GXULss6jb4s6zT6sqzT6MuyTqMvyzqNvqwu1mlj8cpIA7OIeCfwIOApmXkTQGZeGRHnAfsDJzW352bmQDPvrVhx/dDB2fLlK2/9fdh/QK+sWuVs6XWqWZZ1Gn1Z1mn0ZVmn0ZdlnUZf1pZYp43FK6OcLuPewCHAhcAPIwLgd5n5DOBlwIkRcRhwNXDQqOolSZLUFSMLzDLzV8CM9Tx2AbD3qOoiSZLURc78L0mS1BEGZpIkSR1hYCZJktQRBmaSJEkdYWAmSZLUEQZmkiRJHWFgJkmS1BEGZpIkSR1hYCZJktQRBmaSJEkdYWAmSZLUEQZmkiRJHWFgJkmS1BEGZpIkSR1hYCZJktQRBmaSJEkdYWAmSZLUEQZmkiRJHWFgJkmS1BEGZpIkSR1hYCZJktQRBmaSJEkdYWAmSZLUEQZmkiRJHWFgJkmS1BEGZpIkSR1hYCZJktQRBmaSJEkdYWAmSZLUEQZmkiRJHWFgJkmS1BEGZpIkSR1hYCZJktQRs0b1QhHxfuBZwB7AfTPzl832vYATgYXACuCgzLxoVPWSJEnqilG2mH0ZeCRw8YTtxwHHZuZewLHA8SOskyRJUmeMLDDLzLMzc1n/tohYDCwBljablgJLImLRqOolSZLUFdOdY7Y7cGlmrgFobi9rtkuSJG1VRpZjtqktXLj90GUsWjS/Qk3qlmWdRl+WdRp9WdZp9GVZp9GXZZ1GX1YX67SxeGW6A7NlwK4RMZaZayJiDNil2T6QFSuuHzo4W7585a2/D/sP6JVVq5wtvU41y7JOoy/LOo2+LOs0+rKs0+jL2hLrtLF4ZVq7MjPzSuA8YP9m0/7AuZm5fPpqJUmSND1GFphFxNERcQmwG/CdiPhV89DLgFdFxIXAq5r7kiRJW52RdWVm5quBV0+y/QJg71HVQ5Ikqaume1SmJEmSGgZmkiRJHWFgJkmS1BEGZpIkSR1hYCZJktQRBmaSJEkdYWAmSZLUEQZmkiRJHWFgJkmS1BEGZpIkSR1hYCZJktQRBmaSJEkdYWAmSZLUEQZmkiRJHWFgJkmS1BEGZpIkSR1hYCZJktQRBmaSJEkdYWAmSZLUEQZmkiRJHWFgJkmS1BEGZpIkSR1hYCZJktQRBmaSJEkdYWAmSZLUEQZmkiRJHWFgJkmS1BEGZpIkSR1hYCZJktQRBmaSJEkdYWAmSZLUEQZmkiRJHWFgJkmS1BGzprsCPRGxF3AisBBYARyUmRdNb60kSZJGp0stZscBx2bmXsCxwPHTXB9JkqSR6kRgFhGLgSXA0mbTUmBJRCyavlpJkiSNVle6MncHLogptDYAACAASURBVM3MNQCZuSYiLmu2L9/IvmMAM2fOAGDm/O1aV6JXxq3358+vUtbY/J2q1Wn2/MVVytpm+zrlAMytWNa87W9fpaztt6tTDsD8SmXtOK9enRZWLGvxtjtXKWvxvAX16jSvzndv8bztK9ap3rFl8bx5VcpaPG/banVaNG+bKmUtmje7Wp12njdWraz589q3Q/SXNa9SOQBzt6tT1pzt69Vp1g71/uZjO7QPMfrLmrnDnGp1mjl/bp06za/y3Zv0jz1j7dq1rQuvJSIeBHwyM+/dt+184MDM/MVGdn84cNamrJ8kSVJljwDOnrixKy1my4BdI2KsaS0bA3Zptm/MTylv7nJgzSasoyRJ0rDGgDtS4pfb6ERglplXRsR5wP7ASc3tuZm5sW5MgJuYJOKUJEnqqN+s74FOdGUCRMQ9KNNl7ARcTZkuI6e3VpIkSaPTmcBMkiRpa9eJ6TIkSZJkYCZJktQZBmaSJEkdYWAmSZLUEQZmkiRJHWFgJkmS1BEGZpIkSR1hYLYZaybl3eg21RERMyPiydNdD0FEtF/VWJI6rBNLMm2tmpPLzpl5RcsiPgMsmcK2kYqIxwJnZ+aNFcraC7hnZp4aEdsDczLzqqEr2UJm3hIRhwLfqFFeRHwgM1+7sW0b2P9eG3o8M88fpn5dERHvz8w39N2fBXwJ2G8a6/Qm4COZuWKIMp6TmV+IiFdM9nhm/mfrCg4pIt6YmUdsbJvqHe8iYh5wCHCXzHxec5F9j8z88hBlzqHvPJ+Zf2lZzmOAu04oa0qfz2bt6+My88VtXntTiojTgGOAr2dmZ2bb36IDs4j4d+BfgT8D36UELC/NzJMGKGODB6LMfOOAdfos8FJgFfDfwO0i4l2Z+f4ByrgdsBiYGxH3BGY0Dy0AthukPn1l3h34OLBrZu4ZEUuAp2Xm4S2K+ydgaUT8L3AGcDrwk8wcaJH5iHgB8GZgDnAqsCtwLPDYAct5HfDRzLw2Ij4F/DXw6sz81iDlNH4WEQ/OzJ+02HeiR06y7VED7P/1DTy2FrjLYNWp9zmIiPsBxwP3B7bpbc/MsUHrBOweEf+YmcdExEzKxccFLcohIh4GHEH528yifHfWZubiAYu6I3B+RHwTODYzf9yiOvcBvkD5PE405ZNERHxhQ8/PzOcOXjX+H+XvtLFtG1Tju7cp3l9EBHAotw02HjxoWVQ63gEfAi6nfGcALgGWAgMHZhHxDOCDlM8pNJ9zyuLZg5b1CeCvgF8Avfc05c9nZq6JiLsO+robqVOt7/GHgdcCR0fE8cAJg1xsRcRP2fBns83nacsOzIDHZubrI+IpwKXA31FaO6YcmFGCOihf4EcBJzf3nwF8s0WdojlIPZvyJX4d8CNgyoEZcADlw7QL41tvrmXAA2efDwHvAN7T3D8P+BRw+KAFZeZTmlaNhwCPoZxIFwI7DFjUaykHhLOacjMi7jBofYAXZuaREbEPJaD9e+BooE1g9gjg5RFxEXB9b+MgX8CIeA7wXGCPiPh830MLgClf0WbmnlN97gBqfQ4+RDnxHQk8EXglsLJlnQ4CTouIy4GnAVdm5j+3LOujwL9RvnODnjhvlZmviYg3N3X7cESsplw0LJ1qy0lmvq25PbhtPRpfa24f3Pz0jm/PA743SEER8Tjg8cAuEy5KF7SsW43vXrX31+ezlKD44wzxOYCqx7v7ZuYLIuIJTbnXNxcibbyPcoz5UWbe0rKMnocB987M1UOUcUZEHAN8kvHHzbat+rW+xycDJzeB+iuBX0XEt4CjMvPnUyii15L/FOAeTb0ADqYEsq1s6YFZzyOBkzPzsogYqLkyM98OEBHfAJb0oumIeAdl0fVBzW5uHwV8IzP/EhEDfXEy8yjgqIg4JDPf1aIOk1mQmadFxLub17glIla1Kahp0dsXeBzwUCApV5GDWtUcnPq33dyinN4Xdx/g05n5wyEOeFPqZtyICymtXQ9mfKvXdbT7O/W6Gu7ZtCotBnbMzAtbFFXrczA3M0+PiJmZeTlwaER8D3jvoAVl5k0R8UxKgP6jzPzHFvXpuSEzPzPE/v31+ktEfBi4EvgApXX33yLidZn5+Q3vDevrwuwrf0pdRZl5YlPe84FHZuYNzf0PA1+ZShl9VlFOnGtZd1EKpSXn3QOWBRW+e5XfX8/MWsfOmse7CeXOpX0e+FWZ+cOW+060rEIZL2pun9K3rVWrfqPa93iCVcCNwCcj4rTMfP2GnpyZZwJExOHAvr3u0Ij4GvAdSo/dwLb0wOzKiPgI8ATgPc1VTdv3fKf+Js7MXBERe7Qo5/wmIr8H8KaI2LZlfcjMdzV5Cbsxvjm+zVXImoiYTdMsGxG7Am2vtP4InEP5UL5iiCutFU2OWa9OB1Ka9wd1Q0S8hdLS+DcRMYPSPTqw3hdxGJn538B/R8RXauTLNflOT6Z0WxxDeW8fAx7eorhan4NeAH1VRNyf8n+78yAFRMTyph69bphtgTtGxNNp120B8I2IeFJm/leLffvrdntKSsLBwE+BAzPz+xGxJ3AmsNHAjHVdmLejXKj1TuaPobQoDZpjthtwU9/9VcDugxTQfL7PjIgvZeYvB3z9yVT77lHh/fU5JyLul5n/03L/frWOd9+PiEOAbSLi0ZTelFNblnVKRLwc+Bwl0ABa55hdCJweEV+eUNaUP5+boHW/1vf4mcA/ArentHjfq2kMmAX8GthgYNZnN2AucENzfxtK6k0rW3pg9rzm54TMvDoi7sxgXYb9/i8iTmB8U2WbPJcXUALF/87MPzcnvje1qVBzxf1e4GrG9/23uQr5T+AUSs7b4ZQumre0qRewP+UK8t+ByyLidOA7mTlo0+5rKd0CERG/p3TztUn4fiHwCuCfM/OKJt/h0y3KISIWAP8CPIDyRQQgM/dtUdzCiPgKw+f17U/p8v1JU5dLImLQbpSeWp+Dz0XEQkory9mU3JbDBizjrybc77V2z5j4xAG8FDgkIlZSTvJtc1POBT4BPCIzb71YyMzfRcTHp1JArwszIk4B7p+Zv2vu70m71qkzKSesXkv+85ttbTw+IpZVyMt8IZW+e1R4f305QbOBgyMiGR9stMkJqnW8ewvwRkqX/xGU1sD3bHCP9Xtnc3ts37ZWOWaU49xvgPtOKGtgTWt+/3HzD23Kod73+EXAezNzXFpSZt4cEa8aoJzPUYL9zzX3n9tsa2VLD8zOZ92Vf//2Nl2QL6KcWI6hfAhOZ13/8iBeQwkU/wSQmZdS8t/aeANwn8y8uOX+t8rMT0bEbymBzzzgBZl5VsuyPg98vml52R94O+VEM9BBITMvjIi9gb0of/NsMYBgjHJSuHVEUGb+hnYnPigtUec3dXorJWdmKrkIk6mVz3VDZq6e8BlvdeCs9TnIzCObX0+LiJ0pXZsD5Zj1Ptd9LWcTtWkxmxjstXWX9eWS9XLHBrBHLyhr9v9dTPhnTtE/Ai8Dnk35vnydktzcRq28zFnZN8o4M38TET9rWaca76/NMXuDKh7vVlMCqndu7LlTKKvaVFgVciCJiH0p593bUxoR5gAraPcdhgrf4+bccNnEoKwnM7861bIy89CI+DHwaMpn89DM3NDgrA3a0gOz/n/eXEpzeqtm5sy8jjpf6jsCv2y6M9uO5uq5okZQ1pOZZ1NaN4YSEa+ndMc8mDLy9ARa5FxEGYb+08z8v+b+jhGxJDPPmGoZWX9E0N0y81kR8fTMXBoRJ7Ph0ZEbUiufa1lEPBxY2+TvHAL8qmWdan4O7krfqLeIIDPbTDVS7XsM/IwKQV5m3hgRj+e2LadtckpWRMRbKd8TKEHQwNNwNCf2DzY/w6qVl7k0Ip6QzZRAEfFIysXtwIFnjffXlxN0YE4Ynd+kSgys4vHuaODwXnpD0+L81pzi9DmTlLeQMiBhLSU3s3XaRHOhcH/Gf9Y/OUAR76P8jT5HmR3hH4A7ta1PpcaIqueGJpCbcjC3IVt0YDbJP++wiDiDMppjSmol6PY9f+jRXH2+HWXk1GcZ3xw/5Ryz2DRD7RdSmvV/0OI99Xsf4+dku47SFT3oPG01RwT1clxWNS1BV1PyC9qolc/1Ksp7uw+lu/csSvAysOYAPPQ0Ak2w+Q/A/zG+m33gwKzG97hPlSAvIt5D6d67NyUP6OmUZN82DqK0RvVyuk5vtg1ap8WUz8LE/12b73Ct3LDXAqdGGZxyb0oqyFMHKSAiXpOZR0XE+5jkWJUDTlnUeB23HZ0/2bapqHW8e0R/8NTkMQ8yfc6toozsPInSCg9wvyYY/XaLsl5N6Tq8IyWf8hGULuRBArNeD8jsJkH+IxHx/UHr0len3SndvRODxUHTeIY6N2yi8+eWHZhNFBF3AfYYcLfJ5hjqadtdNNRorj69g/dzJtRpkA/n1zb+lMFk5iFNjtO9GGLIMDAj+yb9a1qU2uRI1BwRdGETkH2GMlT7GtYd/AZVJZ+raY14fJSBIDMz8/qN7bMBtaYReA5w16aluaqW32OgapD3FOCBwM8z86UR8a+Mz+cZpE6XUbrnhvUlSiD8HYacAoJKuWGZ+d2IOIoSkN8B+NvMzAGL6QU7w3yuAYiIvwL2pnzn+i+6F9B+QNAhw9arMdmxbfYk26binZQRrL3ehntQArWBAzPgJZTWwB9k5hMi4j6UVvlB9C5+Lo2I/YDf0/6CFkpKyWcpLdYHAC+n5MENathzQ/XzJ2zhgdmE3JSZlA/5awYpIzMPbprwn9iyG2ZinWqM5urVbeiRLtkMRa8pIp5Eyf1YQ5mr66+At2XmoIn7KyNi7153b5Nv9ueN7HMbNf5Ofc6kdEEeGRE/AXYEWo0MqpnXF2VQQlAmHe6V3+aKtNY0ApfXCspqfI83UHbbIO/GJkF4bdMKcGlzFd+mDrVm/t8pM1/Spg4T6jN0XuYk72ke8H3gURHxqEHeW2Ye39y+far7bMCulFbT7Rh/0X0dJRgdWMXWm582QewRlDylf6acI9qY3QvKmrpc0LTOt3FjloFqMyNiRmb+skUX4FERsRMlL/czlED4n1rWB+B2mfnRiHhtZp7T5HcN3GI97LlhU5w/YQsPzBjfbXEzJSdr4CvJrLsUz9CjufpFpfmrIuKOlO6UfSgnwe8Cr8kyB9Wg/pVy0PsvgMz8Wcu+/DcCX46IXr7UvYBntigHqDYi6AHA66MsddKb5ft2lCHzA6uRzxURf0fp4t2JMpDkbpRclzZLc50TdaYROCcillJa3/q72YfNMWv9PYaqQd7KpoXyh8CJUSa/bTPHHowPEOZSvoM/YvDpMn4ZEbs0LXCtVcq9mdjT8L+UFqG/pmVPA8CweX1Zlnb7KvDUzGw7B9pEtVpv/onSg3Iu5W/0NdrPm7g8Il6YmZ+AW1dRWd6yrL80Qd1/A++NiGWUQHvKMnNp8+tPKMenYfVyca+PiDtRjr8DTccDEBGfn9jdONm2KZQzaZemXZmTqJkYT72leKqN5oq681d9inJF+xrK1drBzbaBlj/qabo/+jfdtL7nbqCMc6KsB/nQpk4/zMyrBy0nKo4IysxXNmXuRmn+fhclibXNUie1loU5BHgQ8M3MfGCU2dvbBrB7M34agd4w9EHr1Dsx9w85r5VjNoxaQd7+zf5voOQm7UnL7sicMOqtuUj6QIuidgL+NyJ+wPhguM3JYajcm4nvqYZaeX3NhfabaD857US1Wm+uowz8qOGlwKcj4jjK9+48oNXgBkqX9hzKnF7vonTzPX+QAqL+OqDfb1JK/pMyKv4mykXgoCYLEu/Zopz+Ls25lGNB67WKt+jArLKhl+Jpnl9zNFfN+atuN6EO74iybFQbK5su215S+6MpuVgDawKxYVsqq40IiogHNWU9lhIQf5OWs/VTL5/r5sy8MsqkiGTmt5uctTZ6V+jbN7et8noyc5+Wr79JVQzyjmLdmrcHAYso+V1t50m8VWZeHmVi5UF9pvmpoVpeZnMBMsyIvp5qeX3UXfO2SusNDLdYeL+m6/khEbE9JVe37XJo5LqJhv9MOXa2UW0d0KZOvSXZPhURZwI75AATIkfEiym5c3s1KSk9CygrNwxan3Fdmk3vV+vA38Bs6mosxVN7NFe1+auAX0fE3TLz100978q6UWKDehOlG3PPKMvw3J2yxuGURMTpmfmYuO38VW0nEaw5IuinlO6rN7fNB+tTK5/rpiij5i6KMini7ymBQhuXUJK8H9Dc/wUDXGlHxJ5Nt/y9Jnt8qi0um4H+NW9Pp7Sa/ZgWgdmEfKyZlOPDwC3DNfNdauVlRqURfY1qeX1UutBuVGm9iSEXC2/KmPT715d3OvD3r0kB+UdK61Lb0b5V1gGNiG2yLNHW35X6J+BPETEvp76ywbeAiyg9Tf3r7l4H1FgNYi2lFb0VA7MpynXz32zX3B84Cb1R86qv5vxVO1CWCerlO/0NZWmWz8NgX8LM/EmUiSkfxrouyEFazHqBQK3JQGuOCHoIZZbvw5oD1g8os3yfvOHdJlUrn+tQyv/vXyhXpgso3Q9tHEcZuNHLd3whcDxlHcCp+CBlOoTJ5nYbZm28rulf8/a/MvOGGHDN2z79+Vg3U1reBk6MrpHnsp4TX39Zgy7pU2NEX0/NvL4qF9owfOtNnxqLhW+K79+plGBxmNG+tdYBPYfS69Fbz7V/FZApr2zQtJxfTJliaGgTvnszgfvRvsHFwGyqooze+gylJWFtRJxLGVH52wGLqnnVV23+KkorSf9w+KG6RJrWhNNZN7nolK9megMOanY7RaURQU3Xx0+iLFPzFMpUJy+l3VInE/O5eq8x5av2KCPo7pZl0t1raZkT2GdRZn6s7/7HI2LKyfGZ+dTmtvbaeF1Tc83bWvlYNfJcrqKMWBzqxNenxoi+npp5fUNfaK8neG3TetMz9GLhm+j7N6+XWzuEKuuAZuaS5rbKygYV83z7v3s3A+/PzB+1rZeB2dQdz3AtCT3Vrvqy4vxVNbtBoiwMezSl+wLWLUI9pYN6rFvPblIt8vqqjQhqEqL3pSyofQblSz3llQgmGPqqvRlB93zaL70z0S0REZllrqkm16ntCMh5lJbJ/gPeltKVWW3NWxh+tGHz/Bp5Lr31f7+X7dZ/nWjoEX19quX1VbrQXl/wOtDxrs/Qi4Wvr6Wzr6w2i5j/OCLum5n/22Lfnn+lDCyrsQ4oTS719c1AjvtQGidOzsxBV06pkufb++5V6FEDDMwGMVRLQp9qV30wftmbvjyCgZPlI+J2lP72x1AOKt+hTJfRZoj1EZRFXH+UmW26d6quZxd1RwT9D/AfTXLtMHUao1xVbWgC46n6TkQ8OzO/WKGsQ4CzIuI8yufgAQw4AgsgIl5JOfBezfh8mS2iKzMzb6AvcTmHWPO2ct5pvzZ5LttGxLOAO0WZj3DcgvEtji1Dj+jrUy2vjwoX2rVabfrUWCx8smCxv6w2LfvHUVq8ltF+wfdfU3pkDujlMQ/pu8AjI2I+ZQDWL4EnMvhcdFXyfPsC/fs399v2qAEGZoOo1ZJQ86rvCMqVezLksjeUg9KvKAfQGcCLm21tpl24KjN/2GI/YF03Q0+Fq5BqI4Iys0rLVNPS9aeImJvDLeMCJTF3YUTcQBk5NcwgidMi4t6UbtYZwDmZ+acWdXo9cJ+K3dFbsip5p5XyXHpd87enzCPYb+BjS6URfT018/pqXWhXU6NLexMEi1BWDHgn4wclDOp+lHzD0yPifEojwDeyb2WXAc1oWqr3Bz6SmYdHRJsWvVp5vrV61AADs0G8mXUtCVBO8m2u/Gpe9T0D2LNl8/REd83MZ/Xdf1vfex3UKRHxcsr0FP1XWAPVs2JeX5URQZvARZQr0S8yfmTYoMPjaw2S6L3+coZfauQKg7Ipq5V3OnSeS2aeSlnb8sjMfF2LOowTEXennKx2zcw9I2IJ8LTMPLxFcdXy+qjYZV9Tk/NUY2qRmm7MzKGmgcnMKylTML2b0iL8IcpawccAx7a4OJ0bEdtQ0gmObra1+f8NnefbqBroG5hN3e6UK9vbM1xLQs2rvj8wYbTLEGZGxOLmC9QbIt02eHlnc3ssw+Vc1LoKqTUiqLb5lFbK/gkNB76C7A+Amv/bXYZJPB1GrBum/+2mRfezjD/gbSk5ZjVVyTutmedSIyhrfAh4B+vyic6jTFx9eIuyaub19XfZQ/sL7WqiwtQicdsphnpat6IDp0XEEzPztBb79tdtHuVv/ApK1+YJlFUuvkHJ2x3E5ygrGVwA/CAi7kDfcWYAtUbnVg30Dcym7oGU3KfeUjzbR8QZmTnoUjw1r/peD3w1Ir5Ny2TRPu8Hzo2Ir1O+2E+mtBIOrGJzeq2rkCojgmqrNRovIs6iDJGfQVnO5ZqI+EauG8Y/ShOH6T+n7/ctJsessip5p7XzXCpZ0HSPvxtunXW/1cVkzby+pk73oUzlMcyFdk01phap2nreeDHlXLWSMkfbwEFeRHwQeBYl6f95mdmb1ukzEXHB+vecXGa+Pcq6otc1n6mVTfmDlnPmxp81JbV61AADsynLekvx1LzqexPl6uoBtJyQsCfLgtq/AB5N+eId1YHWjVpXIVVHBNXSXEG+mdLCdcAQgxK2b7rHD6Qk2L6JMgpu5IFZNsP0oywL9dNs5q+LiB0py0bptmrlnVbNc6lkTZRRmb1VQHYF2vYQVJVltYxv0WJKn01k6KlFNlH6QI1g7w/AvXLy+SwHXiUkIh4J/KIJyl5EGTzz3hblTDoDQIuuzFo9aoCB2ZRFpaV4al71UU50ew2RQDnRHyiTwf6iUnnDqtXdUHtEUC29QQm9WfbbDkrYprndB/hcc7BqO/FmLUcwfhH16yhLY7VZWH1LVyvvtHMJ7ZTZ8E8BbhdlmbCDgLdMa40YfkqfTWToqUUi4lOZ+fyKAUeVYC8z37eBxy5vUeQxwP2bgUqvpwxQ+CiDd4n2zwAwl9J6fVmL+tTqUQMMzAZRcymeWi5k3WSQQ4mIJ1OurtcAe0TEXwFvy8z9hi27rYojBGuPCKql1qCE7zXJqzMpy8zsyPQnMs/o//s2weJ0nvS6rFbeaecS2puW+N8C+1GCjBd05Pg57JQ+m0KNqUU+0NxWm3JofXlrLfPVark5M9dGmdLlQ5n5wYh4zkb3mmBiV2bTgvqtFuXU6lEDDMwGUXMpnlquA34eEd9kfI7ZxGHuU/F2SnPwfzVl/GzQZvRNpHdAaH3w3AQjgmqpNSjhlZTWxN9m5qooky++uEL9hrEyIvbOzB8DRMTelCkTdFtD5Z1GxN0z8yIq57nUkplnA2dv9ImjNdSUPrU1Fy3Pycy3McTUIpn58+a2Vu4UjO/KnEtZXWaYZaNqmBURf0PJxXxRs63Ghd8OtMiDrdWj1mNgNkVZdymeWi5g3UzdQ8vMK2L8gug31Sq7jaa74cOUhYFnUpquX9IiB6v2iKBahhqUEM26hpRVCC5sts2jLM+V9as7kDcCX46IXpLvvWg3J97WYNi8089S0hreSJmkdtgW5qHFetbt7MnBFsDeFKpM6VNLlnkNH1mrvCgH8rdw24XHa3RlHhYRZwD/NlQlh/NWyqj/MzLzV03r8MBpKhO6fGdSgrJ/b1Gfqj1qBmZTFHWX4qkiM99esbiVEXF71iXpPhoYZOHxTeGdwMMysxd03J2SuD9QYFZ7RFBFb6GcTPsHJbx7gP0nW9B3mOlJqsnMc6JMnfFQ1i1kf/V01afLKuSd9mbrvzOldaPXuvzgiGi1EkgFw86Dt6nVmtKnpq9HxBso02P0z2vYJlj8AmVakk9QuTu7Gf27R80yB5XNfHt99y+k3YVff5fvzcDvMrNNjlnVHjUDs6mrshRPTU3X1wHcdgHWNl2Zb6J0Y+4ZEd8D7g48rUI1h3FVLygDyMyLImJFi3Kqjgiq6K6Z+U7WnSRoRmZOKVjMygv61tYEYtMRFGxtqs7WX0NWXHt3U+jSd6avK/qIZtMRfQ+3DRZv3lDC/SAm5JiNUc410zqoJMocibcx6LkvM8+MiFlAUN7jlW3qU7tHzcBsirLSUjyVfYGSLPpjhux2zMyfRMQ+wMNY18Ix3S1m34qIt1BG28wADqZ0QcyDqV9JboIRQbV8htuOUpxsm7ReWXm2/hoi4jWZeVStE+gWrtcVXWvheKg0KWyjP8fsZsqqHtM9uKg/X3UuZR7Hnw1aSDPI7Uusm59tVkQ8a9CZCWr3qBmYbd7ulpn33PjTpmw2JcJfSzc+G29rbifmMryP6e92aC3KgvGLKcuK3JN1Cw4voIyylQbWlaCs0cvbcsDHxtVeOB7KGqmnNqN7W00K2/f6FwNExBzKeWGbpot82uZ8m5jGExHvokyZMaijgIMz84ymnH2ADwJ/M2A5VXvUunDyVXu/jYj5mbly2IJqJtrX0qXuhsoOoCwFsgvju5muZXw3hrRZyszjm1+PnHh8iog7TUOVumxTdEV/mNLDMMzC40Bn53yb6HrKQIdBbdcLygAy87vRLGk2iNo9agZmm7drgZ9Vmi6jSqK9Ni4zjwKOiohDMvNd010faRM6uelSWwMQEXek5LLee3qr1R2bqCv6qsz8YqWyOjfnW0T0ek2gNCQ8iHYj0f8SEftk5nebch9FGdU+rQzMNm9JvWkRaiXaa4oy811NvtxujB+8Md1LYUm1nAacCBwYEYso8zu9dXqr1E2Vu6K/HBEvAz7P8NOBdGrOt0b/pOo3UyZH/1KLcl4NfCkibqIEetvQYs3N2masXTvdk5+rjWZCwuMys8pEohHxNsoHvD/R/ibKZKzTmk+wpYqIV1DWd7uavrVOM9OFvrXFiLLYNJS8nSMz8zPTWZ+twYSVI26dDiQzp9z92BtkRRmBeQ0dmfOtOfcd1kzGO2xZj6Xkhy2m/I3+CNynv3tzOhiYbcaatbiqjOLZyBIwA32hNTXNUjX7TDKBo7RZa+aw6xkDPgac2dzaKrwZaM4JvaCup1WQi703AQAABfRJREFUV1tEfDczh57qKCLOBZZks3xcsyTez3pTEU0XuzI3b2c0w3QnTkg48EFvC06077IrDMq0hfr6JNue1fyspcWyNxqtjp8Tak3G28k1fQ3MNm+9NcKe0rfNg97m49vNPE+fZXwXga0J2qxl5p7TXQdt0fon4x1m5YZOrulrV6Y0TSLid5NsNsdMm73eOq59eUrjmLOqLoiIhwInA+PW9M3MH01frWwx2+xFxGOAe2bmMc0aXTv2j65Ud9mqoC3YZOu49nRtDixtpbq6pq+B2WYsIt4EPJky8d8xlOWZPgY8fDrrpQ2zNUFbur7k6ZdSFnOerHVYmnZdXNPXwGzztj9lHbOfAGTmJRGxw/RWSVNga4K2Fg8AXt8s53MGZamgMzKz1WLR0tbAwGzzdkNmro6I/m0mDXZcrzWh46OepKFl5isBImI3yiCldwN3wosPab0MzDZvyyLi4cDaZv6VQ1iXxChJ0yoiHgQ8BngsJeXim8Dp01opqeMMzDZvr6LM43IfyvpeZ1EWyJakLvgp8EPgzZl51nRXRtocOF3GFqBJIp+Zmddv9MmSNCIR8WBgX0qr2WLgB5TBACdPa8WkDjMw24xFxBmUtS1Pzswbprs+kjSZiNiVkmP2ZuBOLvEmrZ/Jx5u3I4FnABdHxEci4mHTXSFJ6omIYyLifOBsYG/gUGC36a2V1G22mG0BImIhZeqMFwDzM/Me01wlSSIiXgKcnpm/me66SJsLk/+3DL3oega2gkrqiMz88HTXQdrc2GK2GYuI/YAXAo8ATgVOzMyzp7VSkiSpNVvMNm+vAT4BHGjyvyRJmz9bzLYAEbEdQGb+ebrrIkmS2jMw24xFxF2Az1DWo1sLnEtpPfvttFZMkiS1YqL45u144MPAtsA84CPNNkmStBkyx2zztigzP9Z3/+MR8Zppq40kSRrK/2/v7l3kKsMwjF8rETTEKhFMRMXCvUXxA7cRZE3sFJMm+AGCoqBipUXAIvoHWInRXoQUFmrnCna7LhEJaGwUn6iBGDdRSQo/wCKbjMU5G8aFIDNJPGd2rl91npkzh+c0wz3vvO97HDGbbOeTZK1IMguc67AfSZJ0CRwxm2z7geUkX7f1PcAzHfYjSZIugcFssn0GzAF302wue6SqVrptSZIkjctVmRMsyXma1Zgz7UsD4CxwGHihqqqr3iRJ0ugcMZtsrwF/A+/ShLNngWuAX2lWZ+7qqjFJkjQ6g9lke6yq5obqA0mWq2o+yb7OupIkSWNxVeZk29xuMgtc2HB2W1uudtOSJEkalyNmk+114HCSL2nml80BLyXZAnzQaWeSJGlkTv6fcEmuB+6nmWP2RVX91nFLkiRpTAYzSZKknnCOmSRJUk8YzCRJknrCYCZJktQTBjNJkqSeMJhJ0mWSZDHJ8133IWlyGcwkSZJ6wmAmSSNKMpPE709Jl507/0va8JI8B+ytqj1t/QPwVVU90dYngD3AZuAAMAscBV6pqs/bcxaBQ8Au4D7griS3Au8A24GDNBs9S9LY/MUnaRosAfNJrkqyHbgaeAAuPGN2C/ATsAC8DWwF3gQWkmwdus7TwIvAdcDvwEc0j0bbBvy4dk1JGpfBTNKGV1XHgD+Be4GdwKfASpLb23oZeBT4vqoOVtVqVb0PfEczkrbmvar6pqpWgUeAb6vqw6o6C7wF/PL/3ZWkjchgJmlaLNH8Dflge7xIE8p2tvUO4Pi6zxwHbhyqTwwd7xiuq2qw7n1JGpnBTNK0WAtm8+3xEv8OZieBW9Z95mZgZagefrjwKeCmtSLJzHAtSeMwmEmaFkvAQ8C1VfUzzd+XD9PMJzsCfALMJnkqyaYkTwJ3AB9f5HoLwJ1J9ibZBLwM3HClb0LSxmYwkzQVquoo8BdNIKOq/gCOAYeq6lxVnQF2A/uAM8CrwO6qOn2R650GHgfeaM+/jWbVpiSNbWYwGPz3WZIkSbriHDGTJEnqCYOZJElSTxjMJEmSesJgJkmS1BMGM0mSpJ4wmEmSJPWEwUySJKknDGaSJEk98Q8OTGoN6+Uh6wAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 720x432 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "%matplotlib inline\n",
    "\n",
    "fig, ax = plt.subplots(figsize=(10, 6))\n",
    "sns.barplot(x='word', y='count', data=djia_down_top_30_words, ax=ax)\n",
    "plt.title('Top 30 words when DJIA goes down')\n",
    "plt.xticks(rotation='vertical')"
   ]
  },
  {
   "cell_type": "markdown",
   "execution_count": null,
   "metadata": {
    "papermill": {
     "duration": 0.02709,
     "end_time": "2020-08-07T04:45:01.277791",
     "exception": false,
     "start_time": "2020-08-07T04:45:01.250701",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# Using DistilBERT for DJIA movement prediction\n",
    "\n",
    "Here, we would be using DistilBert for predicting the movement of DJIA. <br>\n",
    "You can also view Jalammar's notebook on [using BERT for the first time](https://github.com/jalammar/jalammar.github.io/blob/master/notebooks/bert/A_Visual_Notebook_to_Using_BERT_for_the_First_Time.ipynb)<br>\n",
    "Let's have a look at our pre-processed dataset :"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-08-07T04:45:01.346987Z",
     "iopub.status.busy": "2020-08-07T04:45:01.345949Z",
     "iopub.status.idle": "2020-08-07T04:45:01.351912Z",
     "shell.execute_reply": "2020-08-07T04:45:01.351182Z"
    },
    "papermill": {
     "duration": 0.045809,
     "end_time": "2020-08-07T04:45:01.352070",
     "exception": false,
     "start_time": "2020-08-07T04:45:01.306261",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Label</th>\n",
       "      <th>combined_news</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>Georgia 'downs two Russian warplanes' as count...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>Why wont America and Nato help us? If they won...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0</td>\n",
       "      <td>Remember that adorable 9-year-old who sang at ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0</td>\n",
       "      <td>U.S. refuses Israel weapons to attack Iran: r...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "      <td>All the experts admit that we should legalise ...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Label                                      combined_news\n",
       "0      0  Georgia 'downs two Russian warplanes' as count...\n",
       "1      1  Why wont America and Nato help us? If they won...\n",
       "2      0  Remember that adorable 9-year-old who sang at ...\n",
       "3      0   U.S. refuses Israel weapons to attack Iran: r...\n",
       "4      1  All the experts admit that we should legalise ..."
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-08-07T04:45:01.412934Z",
     "iopub.status.busy": "2020-08-07T04:45:01.411910Z",
     "iopub.status.idle": "2020-08-07T04:45:11.315596Z",
     "shell.execute_reply": "2020-08-07T04:45:11.314635Z"
    },
    "papermill": {
     "duration": 9.937248,
     "end_time": "2020-08-07T04:45:11.315764",
     "exception": false,
     "start_time": "2020-08-07T04:45:01.378516",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: transformers in /opt/conda/lib/python3.7/site-packages (2.11.0)\r\n",
      "Requirement already satisfied: sentencepiece in /opt/conda/lib/python3.7/site-packages (from transformers) (0.1.91)\r\n",
      "Requirement already satisfied: regex!=2019.12.17 in /opt/conda/lib/python3.7/site-packages (from transformers) (2020.4.4)\r\n",
      "Requirement already satisfied: tqdm>=4.27 in /opt/conda/lib/python3.7/site-packages (from transformers) (4.45.0)\r\n",
      "Requirement already satisfied: sacremoses in /opt/conda/lib/python3.7/site-packages (from transformers) (0.0.43)\r\n",
      "Requirement already satisfied: packaging in /opt/conda/lib/python3.7/site-packages (from transformers) (20.1)\r\n",
      "Requirement already satisfied: tokenizers==0.7.0 in /opt/conda/lib/python3.7/site-packages (from transformers) (0.7.0)\r\n",
      "Requirement already satisfied: filelock in /opt/conda/lib/python3.7/site-packages (from transformers) (3.0.10)\r\n",
      "Requirement already satisfied: numpy in /opt/conda/lib/python3.7/site-packages (from transformers) (1.18.5)\r\n",
      "Requirement already satisfied: requests in /opt/conda/lib/python3.7/site-packages (from transformers) (2.23.0)\r\n",
      "Requirement already satisfied: six in /opt/conda/lib/python3.7/site-packages (from sacremoses->transformers) (1.14.0)\r\n",
      "Requirement already satisfied: click in /opt/conda/lib/python3.7/site-packages (from sacremoses->transformers) (7.1.1)\r\n",
      "Requirement already satisfied: joblib in /opt/conda/lib/python3.7/site-packages (from sacremoses->transformers) (0.14.1)\r\n",
      "Requirement already satisfied: pyparsing>=2.0.2 in /opt/conda/lib/python3.7/site-packages (from packaging->transformers) (2.4.7)\r\n",
      "Requirement already satisfied: chardet<4,>=3.0.2 in /opt/conda/lib/python3.7/site-packages (from requests->transformers) (3.0.4)\r\n",
      "Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/lib/python3.7/site-packages (from requests->transformers) (2020.6.20)\r\n",
      "Requirement already satisfied: idna<3,>=2.5 in /opt/conda/lib/python3.7/site-packages (from requests->transformers) (2.9)\r\n",
      "Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /opt/conda/lib/python3.7/site-packages (from requests->transformers) (1.24.3)\r\n",
      "\u001b[33mWARNING: You are using pip version 20.1.1; however, version 20.2.1 is available.\r\n",
      "You should consider upgrading via the '/opt/conda/bin/python3.7 -m pip install --upgrade pip' command.\u001b[0m\r\n"
     ]
    }
   ],
   "source": [
    "!pip install transformers"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-08-07T04:45:11.379636Z",
     "iopub.status.busy": "2020-08-07T04:45:11.378684Z",
     "iopub.status.idle": "2020-08-07T04:45:19.986758Z",
     "shell.execute_reply": "2020-08-07T04:45:19.985908Z"
    },
    "papermill": {
     "duration": 8.643747,
     "end_time": "2020-08-07T04:45:19.986950",
     "exception": false,
     "start_time": "2020-08-07T04:45:11.343203",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[34m\u001b[1mwandb\u001b[0m: \u001b[33mWARNING\u001b[0m W&B installed but not logged in.  Run `wandb login` or set the WANDB_API_KEY env variable.\n"
     ]
    }
   ],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "from sklearn.model_selection import GridSearchCV\n",
    "from sklearn.model_selection import cross_val_score\n",
    "import torch\n",
    "import transformers as ppb\n",
    "import warnings\n",
    "warnings.filterwarnings('ignore')"
   ]
  },
  {
   "cell_type": "markdown",
   "execution_count": null,
   "metadata": {
    "papermill": {
     "duration": 0.027309,
     "end_time": "2020-08-07T04:45:20.043402",
     "exception": false,
     "start_time": "2020-08-07T04:45:20.016093",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "## Load pre-trained DistilBERT model\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-08-07T04:45:20.106368Z",
     "iopub.status.busy": "2020-08-07T04:45:20.105306Z",
     "iopub.status.idle": "2020-08-07T04:45:41.320130Z",
     "shell.execute_reply": "2020-08-07T04:45:41.319271Z"
    },
    "papermill": {
     "duration": 21.249869,
     "end_time": "2020-08-07T04:45:41.320320",
     "exception": false,
     "start_time": "2020-08-07T04:45:20.070451",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "96cc8b0e9bac48a3b0566b290bc352a9",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(FloatProgress(value=0.0, description='Downloading', max=231508.0, style=ProgressStyle(descripti…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "bea62a5d76f9468f8a6e639ef00fb2ca",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(FloatProgress(value=0.0, description='Downloading', max=442.0, style=ProgressStyle(description_…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "db1d2a9ba497421cb2267b317b7b8cfd",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(FloatProgress(value=0.0, description='Downloading', max=267967963.0, style=ProgressStyle(descri…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    }
   ],
   "source": [
    "# For DistilBERT\n",
    "model_class, tokenizer_class, pretrained_weights = (ppb.DistilBertModel, ppb.DistilBertTokenizer, 'distilbert-base-uncased')\n",
    "\n",
    "# load pre-trained model/tokenizer\n",
    "tokenizer = tokenizer_class.from_pretrained(pretrained_weights)\n",
    "model = model_class.from_pretrained(pretrained_weights)"
   ]
  },
  {
   "cell_type": "markdown",
   "execution_count": null,
   "metadata": {
    "papermill": {
     "duration": 0.02714,
     "end_time": "2020-08-07T04:45:41.375842",
     "exception": false,
     "start_time": "2020-08-07T04:45:41.348702",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "## Model 1 : Preparing dataset\n",
    "Before we can hand our sentences to BERT, we need to so some minimal processing to put them in the format it requires.\n",
    "### Tokenization\n",
    "Our first step is to tokenize the sentences -- break them up into word and subwords in the format BERT is comfortable with"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-08-07T04:45:41.460948Z",
     "iopub.status.busy": "2020-08-07T04:45:41.450526Z",
     "iopub.status.idle": "2020-08-07T04:45:43.478977Z",
     "shell.execute_reply": "2020-08-07T04:45:43.479683Z"
    },
    "papermill": {
     "duration": 2.07667,
     "end_time": "2020-08-07T04:45:43.479901",
     "exception": false,
     "start_time": "2020-08-07T04:45:41.403231",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "tokenized = df['combined_news'].apply((lambda x: tokenizer.encode(x, add_special_token=True)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-08-07T04:45:43.542785Z",
     "iopub.status.busy": "2020-08-07T04:45:43.541731Z",
     "iopub.status.idle": "2020-08-07T04:45:43.546816Z",
     "shell.execute_reply": "2020-08-07T04:45:43.545759Z"
    },
    "papermill": {
     "duration": 0.039092,
     "end_time": "2020-08-07T04:45:43.546978",
     "exception": false,
     "start_time": "2020-08-07T04:45:43.507886",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(1989,)\n"
     ]
    }
   ],
   "source": [
    "print(tokenized.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-08-07T04:45:43.612834Z",
     "iopub.status.busy": "2020-08-07T04:45:43.611807Z",
     "iopub.status.idle": "2020-08-07T04:45:43.616300Z",
     "shell.execute_reply": "2020-08-07T04:45:43.616932Z"
    },
    "papermill": {
     "duration": 0.042472,
     "end_time": "2020-08-07T04:45:43.617140",
     "exception": false,
     "start_time": "2020-08-07T04:45:43.574668",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    [101, 4108, 1005, 12482, 2048, 2845, 24136, 20...\n",
       "Name: combined_news, dtype: object"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tokenized[:1]"
   ]
  },
  {
   "cell_type": "markdown",
   "execution_count": null,
   "metadata": {
    "papermill": {
     "duration": 0.027295,
     "end_time": "2020-08-07T04:45:43.672248",
     "exception": false,
     "start_time": "2020-08-07T04:45:43.644953",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "### Padding\n",
    "After tokenization, tokenized is a list of sentences -- each sentences is represented as a list of tokens. We want BERT to process our examples all at once (as one batch). It's just faster that way. For that reason, we need to pad all lists to the same size, so we can represent the input as one 2-d array, rather than a list of lists (of different lengths)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-08-07T04:45:43.780781Z",
     "iopub.status.busy": "2020-08-07T04:45:43.779839Z",
     "iopub.status.idle": "2020-08-07T04:45:43.785227Z",
     "shell.execute_reply": "2020-08-07T04:45:43.784587Z"
    },
    "papermill": {
     "duration": 0.085401,
     "end_time": "2020-08-07T04:45:43.785384",
     "exception": false,
     "start_time": "2020-08-07T04:45:43.699983",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "78\n"
     ]
    }
   ],
   "source": [
    "max_len = 0\n",
    "for i in tokenized.values:\n",
    "    if len(i)>max_len:\n",
    "        max_len = len(i)\n",
    "print(max_len)\n",
    "padded = np.array([i + [0]*(max_len - len(i)) for i in tokenized.values])"
   ]
  },
  {
   "cell_type": "markdown",
   "execution_count": null,
   "metadata": {
    "papermill": {
     "duration": 0.027467,
     "end_time": "2020-08-07T04:45:43.841601",
     "exception": false,
     "start_time": "2020-08-07T04:45:43.814134",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "One thing to note is that BERT model takes only 512 tokens at a time. So in this case, max length exceeded 512 (628 to be exact.) For this purpose, I will be taking top 10 news only. Due to compuational capacity restriction, I cannot use more than one column for training the model. Hence, I will be taking only top1 news into account. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-08-07T04:45:43.903914Z",
     "iopub.status.busy": "2020-08-07T04:45:43.903014Z",
     "iopub.status.idle": "2020-08-07T04:45:43.912116Z",
     "shell.execute_reply": "2020-08-07T04:45:43.911424Z"
    },
    "papermill": {
     "duration": 0.042764,
     "end_time": "2020-08-07T04:45:43.912273",
     "exception": false,
     "start_time": "2020-08-07T04:45:43.869509",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(1989, 78)"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.array(padded).shape"
   ]
  },
  {
   "cell_type": "markdown",
   "execution_count": null,
   "metadata": {
    "papermill": {
     "duration": 0.027856,
     "end_time": "2020-08-07T04:45:43.968808",
     "exception": false,
     "start_time": "2020-08-07T04:45:43.940952",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "### Masking\n",
    "If we directly send padded to BERT, that would slightly confuse it. We need to create another variable to tell it to ignore (mask) the padding we've added when it's processing its input. That's what attention_mask is:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-08-07T04:45:44.034753Z",
     "iopub.status.busy": "2020-08-07T04:45:44.033819Z",
     "iopub.status.idle": "2020-08-07T04:45:44.040354Z",
     "shell.execute_reply": "2020-08-07T04:45:44.039671Z"
    },
    "papermill": {
     "duration": 0.041512,
     "end_time": "2020-08-07T04:45:44.040548",
     "exception": false,
     "start_time": "2020-08-07T04:45:43.999036",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(1989, 78)"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "attention_mask = np.where(padded != 0, 1, 0)\n",
    "attention_mask.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "execution_count": null,
   "metadata": {
    "papermill": {
     "duration": 0.02762,
     "end_time": "2020-08-07T04:45:44.096404",
     "exception": false,
     "start_time": "2020-08-07T04:45:44.068784",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "## Model #1: And Now, Deep Learning!\n",
    "Now that we have our model and inputs ready, let's run our model!<br>\n",
    "The `model()` function runs our sentences through BERT. The results of the processing will be returned into last_hidden_states."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-08-07T04:45:44.161356Z",
     "iopub.status.busy": "2020-08-07T04:45:44.159977Z",
     "iopub.status.idle": "2020-08-07T04:48:34.761703Z",
     "shell.execute_reply": "2020-08-07T04:48:34.762421Z"
    },
    "papermill": {
     "duration": 170.638061,
     "end_time": "2020-08-07T04:48:34.762713",
     "exception": false,
     "start_time": "2020-08-07T04:45:44.124652",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "input_ids = torch.tensor(padded)\n",
    "attention_mask = torch.tensor(attention_mask)\n",
    "with torch.no_grad():\n",
    "    last_hidden_states = model(input_ids, attention_mask=attention_mask)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-08-07T04:48:34.828944Z",
     "iopub.status.busy": "2020-08-07T04:48:34.827689Z",
     "iopub.status.idle": "2020-08-07T04:48:34.832285Z",
     "shell.execute_reply": "2020-08-07T04:48:34.831277Z"
    },
    "papermill": {
     "duration": 0.041092,
     "end_time": "2020-08-07T04:48:34.832502",
     "exception": false,
     "start_time": "2020-08-07T04:48:34.791410",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "features = last_hidden_states[0][:, 0, :].numpy()\n",
    "labels = df['Label']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-08-07T04:48:34.899102Z",
     "iopub.status.busy": "2020-08-07T04:48:34.898035Z",
     "iopub.status.idle": "2020-08-07T04:48:34.903126Z",
     "shell.execute_reply": "2020-08-07T04:48:34.902160Z"
    },
    "papermill": {
     "duration": 0.042034,
     "end_time": "2020-08-07T04:48:34.903392",
     "exception": false,
     "start_time": "2020-08-07T04:48:34.861358",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[-0.36086106 -0.16207744 -0.2084857  ... -0.30507     0.34105778\n",
      "   0.08085383]\n",
      " [ 0.30954662  0.20850469 -0.29937974 ...  0.1224136   0.4723484\n",
      "   0.26783004]\n",
      " [ 0.29998276  0.04621707 -0.04180933 ...  0.08885639  0.3317783\n",
      "   0.3698523 ]\n",
      " ...\n",
      " [-0.19172376 -0.41772428  0.05932816 ... -0.45935822  0.3520215\n",
      "   0.02621031]\n",
      " [ 0.2486663  -0.02985957 -0.1605662  ...  0.04126985  0.18912628\n",
      "   0.28713652]\n",
      " [-0.37453964 -0.14715368 -0.26433668 ... -0.48020387  0.17097425\n",
      "   0.1307473 ]]\n"
     ]
    }
   ],
   "source": [
    "print(features[:10])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-08-07T04:48:34.973195Z",
     "iopub.status.busy": "2020-08-07T04:48:34.971093Z",
     "iopub.status.idle": "2020-08-07T04:48:34.983656Z",
     "shell.execute_reply": "2020-08-07T04:48:34.982828Z"
    },
    "papermill": {
     "duration": 0.050779,
     "end_time": "2020-08-07T04:48:34.983833",
     "exception": false,
     "start_time": "2020-08-07T04:48:34.933054",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "from sklearn.model_selection import GridSearchCV\n",
    "from sklearn.model_selection import cross_val_score\n",
    "\n",
    "train_features, test_features, train_labels, test_labels = train_test_split(features, labels)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-08-07T04:48:35.049432Z",
     "iopub.status.busy": "2020-08-07T04:48:35.048548Z",
     "iopub.status.idle": "2020-08-07T04:48:35.212927Z",
     "shell.execute_reply": "2020-08-07T04:48:35.211647Z"
    },
    "papermill": {
     "duration": 0.199856,
     "end_time": "2020-08-07T04:48:35.213106",
     "exception": false,
     "start_time": "2020-08-07T04:48:35.013250",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "LogisticRegression()"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "lr_clf = LogisticRegression()\n",
    "lr_clf.fit(train_features, train_labels)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-08-07T04:48:35.282315Z",
     "iopub.status.busy": "2020-08-07T04:48:35.281373Z",
     "iopub.status.idle": "2020-08-07T04:48:35.290853Z",
     "shell.execute_reply": "2020-08-07T04:48:35.289946Z"
    },
    "papermill": {
     "duration": 0.048446,
     "end_time": "2020-08-07T04:48:35.291028",
     "exception": false,
     "start_time": "2020-08-07T04:48:35.242582",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.4759036144578313"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "lr_clf.score(test_features, test_labels)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-08-07T04:48:35.365602Z",
     "iopub.status.busy": "2020-08-07T04:48:35.364693Z",
     "iopub.status.idle": "2020-08-07T04:48:35.708298Z",
     "shell.execute_reply": "2020-08-07T04:48:35.707411Z"
    },
    "papermill": {
     "duration": 0.387808,
     "end_time": "2020-08-07T04:48:35.708456",
     "exception": false,
     "start_time": "2020-08-07T04:48:35.320648",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZMAAAEiCAYAAAA8ij+xAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO3dd3hUddbA8W+6QIBAEnq1cFREBEFEEUWxi7J2EBBX7L2tyirr6rqWtSuWl3UVpIggdkQUxYoCUgTUY6VDGjWkZ+b9494Jk5Ayk0wmZc7nefKQuXPLmctkzvx6lNfrxRhjjKmJ6LoOwBhjTMNnycQYY0yNWTIxxhhTY5ZMjDHG1JglE2OMMTVmycQYY0yNWTIxtUJE1ojICXUdR10TkRdF5N4wX/NVEflXOK9ZW0TkEhGZX81j7T0YRlE2zqTxE5G1QFugGMgG5gHXq2p2HYbV6IjIWGCcqg6q4zheBTaq6j11HMd9wIGqOioM13qVevCaI5mVTCLHMFVNBI4A+gB313E8QROR2Ei8dl2ye24CZSWTCOCWTMap6ifu40eBnqp6pvv4aOAJ4FBgHXCTqi50n2sNPA6cCjQBPlfV4e5zZwH/AroBPwJXq+oP/td0t/8OdFTVbe5zfYCPgfaqWigifwXuANoBi4ErVXWdu68XuB64GYhV1e7lvL6zgYeAjsAK4BpV/ckvjpeA0UB74G33+bwAX8MLwCWAAM2A24ErgDbABuDvqvqWiBwCLAfigFygSFWT/L8xu1UuU4EngTtxSorjVfUV93rJwKvA8YACHwEnVFTSEZFBwKPu/9tu4F5VfdW95h73NQ12X9dIVf3dPe5p4FygJfArcLOqfuk+dx9wGJAHnA3cCvwAPA0c4r62N4FbVbXAPaYn8BRwJFDo7rsMeBeIAvKB31W1t4i0xHmvnQF4gFeAf6hqsVuyuwLnPXAp8DzwG25pT0Si3GMvARJw3qsjgWOAiYAXKAA+U9Vh/u97EYlx7/nl7v/dL8BwVd1Q3r01wbOSSYQRkU7A6Th/pIhIR+ADnA/U1jgflm+KSKp7yGtAU6Anzh/hk+5xfYH/AVcByTgf2O+KSIL/9VR1M7AIOM9v80hgtptIhgPjcT7cUoEvgRllwh4ODMD50Cz7enq4+9/sHj8XeE9E4v12uwQnGR4A9ADuCeI1jADOBJJUtQgnMR6H80H8T2CqiLR3k9fVwCJVTVTVpLKxutq5x3bE+WCbKCKt3Ocm4iSBdjgfppdWcA5EpAvwIfCs+7qPwEmk/nH/E2iF83/9oN9zS9z9WwPTgVkisp/f8+cAs4EkYBpO0rsFSAEGAicB17pxNAc+wak67QAcCCxQ1XnAv4GZ7v3o7Z57MlDk7tcHOAXnS4fPAOAPnPeaf8y4+w7G+T9MAi4CslT1/9w4H3WvNaycW3are0/OAFoAfwVyytnPVJMVIyPH2+63/ETgU+Af7vZRwFxVnes+/lhElgJnuA2fpwPJqrrdff5z998rgJdU9Tv38WQRGQ8c7bePz3ScBDLJ/XZ5Mc4HPDgf5A/5lST+DYwXka6+0on7/LYKXtdFwAeq+rF7/GPATTjfVhe6+zzn+wYqIg/ifADfE+BreMb/26uqzvK79kwRuRs4CningvjKKgTudxPTXBHJdsKSJTgJ9zBVzQF+FJHJwAkVnOcS4BNV9SXeLPfHZ46qLnZf8zScb/S+1zDVb7/HReQenJLXSnfbIlV92/09F/jeb/+1IvISTunpKeAsYKuqPu4+nwd8RzlEpC3O+ylJVXOBPSLyJHAlTiIH2Kyqz7q/F4mI/ykKgebAwcBi33smQOOAv6mquo9XVrazCZ4lk8gx3C3uH4/z4Z4C7AC6AheIiP+3uTjgM6AzsM0vkfjrClwqIjf4bYvH+XZa1mzgWRHpAByEUx3xpd95nhaRx/32j8L55u5LJpVVRXTw2w9V9YjIBvd4H//j1/nFGMhrKHVtERmD8y23m7spEedeBirLTSQ+Oe45UnH+Hv2vV9nr7oxTSqrI1nKuAYCI3Ibz4doB5/+iBaVfQ9nX3AMnGfXDKaXGsjfBVBWHv644760tfkkimgBfs6p+KiLP4ZTguojIW8DtqrorgGsHE6epBksmEUZVP3fr1B/DqT7aALymqleU3VdE2gOtRSRJVXeUeXoD8KCqlq2KKO+aO9xSzoU49e4zVNXXWOc7z7RKTlFZw95moJdfzFE4Hxyb/Pbp7Pd7F/eYQF9DybVFpCswCaeaZ5Fbz78CJ/lVFWdVMnCqfzrh1OeXjbusDTgloqCIyHE4bQcnAWvc5Ludva8B9n0dL+C0B41Q1d0icjNwvl8cIyq4XNnzbMBpP0kpk1ArO6YUVX0GeEZE2gBv4LS13VvVce61DwBWV7GfqSZLJpHpKZzqiiNwGoSXiMipOHXfcTjVPL+p6kYR+RB4XkSuw+lWPFBVv8D5UH1LRD7BaTBtilMl84Wq7i7nmtNxPsS64HyQ+bwIPCAiK1R1jdtAe0qZ6qTKvAHcJSInAV/gVHHlA9/47XOdiLyP8w19PDDT3R7sa2iG86GVASAil+E0VvukAZ1EJN7XOB0oNzHNAe4TkXE492kMsL6CQ6bhVAdeCMzBaYfprKorKtjfpzlO0soAYkXkLpySSVXH7AKyReRg4Br3eID3gSfcBPMCTsnuULfqMA04WUSiVdWjqlvcLxWPu2NvsoHuQCdVLVs1ug8R6Y9TklmG07aUh9Oeg3ut/Ss5/L8477MfcdqQegGbVDWrkmNMEKwBPgKpagYwBaf3zwacBtfxOB8QG3C+7fneG6Nx6qp/BtJxGrpR1aU4bQ7PAdtx/kDHVnLZd3GquNJUtaS+WlXfAh4BXheRXTjfHE8P4rUoTrvPs0AmMAynG7T/h/l0YD5Ow+4fOJ0Ngn4NqvojTs+2RTgfXr2Ar/12+RRYA2wVkcxAX4Of63GSwlacjg8zcBJjebGsx2lMvg3YhtP43ru8fcv4CKfh/hecKr88Kq9OA6dTxkicHmOT2JuMcZPuyTj3fStO77Ah7tO+LwRZIrLM/X0MTsL5Eeeez8bpZReIFu71t7uxZ+GUsAFeBg4VkR0i8nY5xz6B88VjPk5ifBmnd6IJEesabBq1st2iGxIReQRop6oV9uoypr6wai5j6gm3CikeWAX0x+k6PK7Sg4ypJyyZGFN/NMep2uqAU6X4OIF3OTamTlk1lzHGmBprjCWTBJwqgi3s7elhjDGmYjE4HSGWUEGnj6o0xmTSn70D4owxxgTuOOCr6hzYGJPJFoDt2/fg8VgVXnJyIllZNtO83Ye97F7sZffCER0dRatWzcD9/KyOxphMigE8Hq8lE5fdB4fdh73sXuxl96KUajcN2KBFY4wxNWbJxBhjTI1ZMjHGGFNjYWszcdeZOA9n6u5eqrrP7J3uamjPAKfhTKj3sKr+N1wxGmOMqZ5wlkzexlklbV0l+1yCswLbQTgrut0nIt1qPzRjjDE1EbZkoqpfBbDe8kXAJHe66gycBHRB7UdnjDGmJupbm0kXSpdc1lP5AkHGGGNqYOGKTUyeuajG52mM40wAZzCScaSmNq/rEOoFuw972b3YK1LvxbxFa/l8+UZW/56FNCus8fnqWzJZj7NO9BL3cdmSSsCysrJtMBLOH0pGRnmLBkYWuw972b3YK5LvxSffrWN9ejbSOYkhR3aq8fnqWzKZBVzhLl+ajLNG+eC6DckYYxqXhSs2oRt2IJ2TuPOSvkRHR9X4nGFrMxGRZ0RkI9AJ+ERE1rjb54pIP3e313CWVf0V+Ba4X1X/CFeMxhjTmGWvXEH6zBl8tyYNgAE924bs3GErmajqjcCN5Ww/w+/3YuCacMVkjDGRoGjnDtJnTCN76RJ2Nktma+dUpHNrTjiiY8iuUd+quYwxxoSI1+Nh51dfkDn7DbwFBazpPpBP9+tBp3YtQ1oqAUsmxhjTaO1Y+CkZ06fSpIfQdsxlzFmwmU7AnZf0Dfm1LJkYY0wj1fLY4/glI5+3CtvBgs2sT8+mS5vaGTZR3wYtGmOMCZHohAQWFrVnfcYeALq0SQx59ZaPlUyMMaaB+/y731mz8g92N0ve5zlfaaQ2qrb8WcnEGGMasN3Lvidp8hP0XfE+eD37PF+bpRF/VjIxxpgGqHD7djKmTyV7+ffsim/Fj71O4c5R/ao+sJZYMjHGmAbmu5kfkPjpO0R7i/kiuS9Lkg5l1MBD6jQmSybGGNNALFyxyRm9/lM6feJb80vvU9jTJIlRPduGdABidVgyMcaYBmDhik1MmacAyMGHE9/zZK7vU/MJGkPFkokxxtRjC5dv5Lsf09ENOwAYc5rUeSmkPJZMjDGmnlm4YhPLVqyn5x9fsy4vDm3dC+mcxIB6UJ1VEUsmxhhTD5S0h3i98PNKTs5YQrPiPPK79qu3pRF/lkyMMaYe+G5NGts2beWMrCW0z/qTgtQOdL36SqRrt7oOLSCWTIwxpo75Fqsas+N7OuzeRMqFF5N00slExcTUdWgBs2RijDF1xFe15WtcjznzXLod3Ia41NQ6jix4lkyMMaaOfLcmrWQd9gE923JcPW8XqYwlE2OMCbOcn38qKX2EYxLGcLBkYowxYVKcnU3GrJns+vpLWg4+HmJ613VIIWPJxBhjapnX62X3ku/ImDGd4j3ZtDr9TNZ07Y9+8gfSOamuwwsJSybGGFOLvF4vW55/juzl35PQrTudbr2dhM5d+G7aMoCwTA8fDpZMjDGmFkVFRbHfAQfQRISkE4cSFb13GSnpnFTvByMGypKJMcbUstannVHXIdQ6W2nRGGNCxFNYWNch1BlLJsYYEwJ71qxm3YTxZC9fVteh1Amr5jLGmBoo3r2b9DdmsHvRN8S1bUdMYuI++5RM4uhnfXo2Xdrsu29DZcnEGGOqwev1svvbb0ifOQNPbi6tzxpG6zOHER0XD5ROIL7pUvy7AXdpk9hoenKBJRNjjKkWb1EhWe++TXybtrS99DISOpZe9dA3VUqXNon1fi2SULBkYowx1RAdF0+n2+8itlWrUt19/TWWqVICYcnEGGOqKS45ua5DqDcsmRhjTCU8eXls//gjWp1yGtEJCZXu699O0tga2KtiycQYYyqwZ9UPpE2dTFFWFvEdO9G875GV7u/fTtLYGtirYsnEGGPKKNq1i4zXp7N78bfEt+9A5zvH0+SgHgEdG0ntJP7CmkxEpAcwGUgGsoAxqvprmX3aAK8AnYF44FPgRlUtCmesxpjIVLRjO2sn3IO3IJ/ks4fT6vQziY6Lq/I439K7jWUW4GCFewT8i8BEVe0BTAReKmef8cBPqno40As4Ejg3fCEaYyJZbFIrWp18Cl0m3E/y2cMDSiRASVtJJFVt+QtbMnFLHH2BGe6mGUBfESm72LEXaC4i0UACTulkU7jiNMaY5GHnkNChQ9DHNaZZgIMVzpJJZ2CTqhYDuP9udrf7ewDoAWwBtgIfqerXYYzTGBMhCnbsqPE5Fq7YxCPTlrE+PTsEETVc9XGixwuAH4D2QEdgsIicX7chGWMaE09eLunTp/L9FdeQv3lzjc7l34MrUqu4ILwN8BuAjiISo6rFIhIDdHC3+7sB+KuqeoCdIvIOMASYHczFkpMjp393VVJTm9d1CPWC3Ye9IvlebFuylLUvTqIgK4t2p59Ku4M6E9u0abXONW/RWnTDDg47IJmHrh0U2kAbmLAlE1VNF5EVwAhgqvvvclXNKLPrn8BpwGIRiQeGAnOCvV5WVjYej7eGUTd8qanNycjYXddh1Dm7D3tF6r0o2rmT9BlTyV66hPgOHel819/pcnQf517sCf5+LFyxiSnzFIC+B6U06HsaHR1V4y/g4R5ncjUwWUQmANuBMQAiMheYoKpLgZuBF0VkFRADfAZMCnOcxphGxltYQM7PP5E8/Fxan3YGUbHV+/jzjXL3zQQ85jSJ2EZ3f1Feb6P79t4N+NNKJo5I/RZalt2HvSL5Xnjy8ojeb7+Sx4Hei4qmk28sMwH7lUy6A2urcw4bAW+MiRj+iaQqFSWQxpREQsmSiTGmUcn97Vd2ffctbUaOIioqqtrnibT1SGoq4GQiIicDFwNtVHWYiPQDWqjqp7UWnTHGBKg4J4fMt2azc+FnxLZqTfGZZxGb1KpG54zUebaqI6BkIiI3ADcB/wV8Yz5ygWeAY2onNGOMCUz28u9Jm/YaxTt3knTSyaQMPzeoKi2fSJ5CvqYCHbR4MzBUVR8GPO62nwGplaiMMSZAu5cuZvPEZ4lt3pwu4++lzcUjq5VIYG/VFjS+NdprW6DVXM3ZO7jQ10UqDigIeUTGGBOExCP60mb0pbQ89rhqd/eF0rP+WtVW8AK9818AdwEP+m27EWcMiDHG1Jmo2FiSjh9S7ePLjhux0kj1BJpMbgDeE5ErcGb0VWAXMKzWIjPGGD+ewkIK09NJ6Bi6HlXzFq0tGcVuPbZqJqBkoqpbRKQ/0B/oilPltdidP8sYY2pVzi9K2pRX8OTm0f2hR4mOj6/2ucobP2Kj2Gsu0N5c76jqOcBi98e3fY6q2sJVxphaUZyzh8zZb7Dzi8+JTUmh3WWX1yiRQOnxI4cdkEzfg1IskYRAoNVcFVVInhCiOIwxppQ9q35g66svU7xrF61OPY3ks/9CdEJCtc/nK5H4Esmdl/SN6KllQq3SZCIi97u/xvv97rM/sK5WojLGRLyouDhik1rR8cZb2K9rt4CP86/G8ld2Ti0TWlWVTHyrIEZTekVEL067yX21EJMxxtD04EPo8vcJREVXPRyuonm0/FkDe+2qNJmo6mUAIvKNqto08MaYsAo0kfj3yLKkUTcC7c01CUBEmgMpQJTfc3/UTmjGmMbOU1DAtvffxevxkHr+hdU6h69EYj2y6lagvbkOAaYDvXGquKLYOxI+pnZCM8Y0Zjk//0TalFcpTE+jxaDBeL3eoGb59W9Ql85JlkjqWKC9uV7AGe0+BGdZ3W7AQ8A3tROWMaaxKs7OJmPWTHZ9/SVxqW3odNvfaHrIoRXubw3qDUOgyaQ3cLKqFopIlKruFJE7gNU467kbY0xAMufMZteir2l1+pkkDzunwnEjZac5sQb1+i3QZJKHM7FjIZApIl1w1nBPrq3AjDGNU/I5fyFpyIkkdO5S6X7+VViWNOq/QJPJl8CFwKvAbOBDIB+whbGMMUGJbdmS2JYtA9rXFqdqOALtzeXfzWI8sAZIBCbXRlDGmIYvb/06ouPiiG/fIajjyo5UNw1DoItjlVBVj6q+BrwMXBb6kIwxDZknP5+M2W+w/l//JOPNWUEf759IrGG94aiyZCIiJwFHAL+p6jsiEgtcC9wJbAMm1m6IxpiGYs+a1aRPnUxhRgYtjhtM6vkXVes8Vr3V8FQ1N9edwL041Vo9ReR5nMkd84ErVfWDWo/QGNMgpE17jZ2fLSCubTs63XEXTeXgug7JhFFVJZOrgONV9XsRORr4GrhdVZ+s/dCMMQ1JQseOtD5rGK3PHEZ0XHDTxPuPJbG2koapqmSSoqrfA6jqtyKSDzxV+2EZYxqapBNOLHd7RYMO/fmPJbG2koYpkDaTKJzpU6JwxpsgIiUN97baojGRxVtcDNHRAU99EkjPLBtL0vBVlUwSgSK/x1F+j33zc9ncXMZEiLy1f5I2+RWShp5My2OPq3L/hSs2oRt2IJ2TrEG9kasqmXQPSxTGmHrNk5dH5jtvseOT+cS0aElMs8DaNHzVW1Zt1fhVtZ6JraRoTITbs+oH0qZOpigri5bHn0DKeRcQ07RZwMfbjL6RIdDpVIwxEcjr8ZA5ZxbR8Ql0vnM8TQ7qUdchmXrKkokxpkJR0dF0uP4mYlq0JDourq7DMfWYJRNjTKXiklOCPsbm14o8QSUTEekMdFTVb2spHmNMHfAWFbF9wce0HDSYmGaBtYdUNn7EFq6KPIEu29sFmIEzR5cXSBSR84HTVHVcoBcTkR44Mw0nA1nAGFX9tZz9LsSZxsXX/XioqlY+6skYUy25f/xO2uRXKNi0keiEhHIHH5aXOCpatMq3zcaNRJZASyYvAR8Ax+EkAYCPgceDvN6LwERVnSoio9zzlnrnikg/4D7gRFXdKiItceYCM8aEkCcvl8w5b7LjswXEJiXR4fqbSDyiD7Bv8igvcVjCMP4CTSZHAWeqqkdEvADu0r2BrXADiEgboC9wsrtpBvCciKSqaobfrrcAj6nqVt91Ar2GMSYwxbm5rPvHPRRt30bLE04k5dzziWnSpOT5su0dljhMVQJNJmnAgcAvvg0iciiwPohrdQY2qWoxgKoWi8hmd7t/MjkU+FNEvsAZgT8HeFBVvUFcyxhTiZgmTUg6YQhN5GCaHHBgufvYNPAmGIEmk8eA90XkISBWREbgrLj4cC3FdDhOCSYemIeTtKYEc5LkZOtB4pOa2ryuQ6gX7D7slZranNRLR1T4fFx8TMl+jV0kvMZwCHTZ3v+JyDbgSmADMAa4V1XfDuJaG4COIhLjlkpigA7udn/rgNmqmg/ki8g7ONVsQSWTrKxsPB4rzKSmNicjY3ddh1HnIvk+FGdnE5O498uV715U1hvLV8XV2O9ZJL8v/EVHR9X4C3igvbli3MQRTPIoRVXTRWQFMAKY6v67vEx7CcB04AwRec2N7yRgdnWva0yk8hYVse3DD9j24Qd0uv0umuy/f6nnKxsHYtPAm2AFWs21VURmAdNU9esaXO9qYLKITAC245RwEJG5wARVXQq8DvQDfgQ8wEc4680bYwKU+9uvpE15hYLNm2ne/yjikluXPDdv0Vo++W5dSSKxdhETCoEmk1NwShIzRMSD0xNruqquCuZiqvozMKCc7Wf4/e4BbnV/jDFBKM7NJXPOLHYu/IzYVq3pcOPNJB5+RKl9Pl++sSSRWOnDhEqgbSbLgeXA30TkeJzEskBEtqrq4bUZoDEmcN7CQrKXLiHppJNJGX4u0fvtV+5+ViIxoVadubkU+Amn4fyg0IZjjKmJ2BYt6PbgI8Q0bVrXoZgIE2gDfBJwHjASOBqYDzwCvFt7oRljqqOyRLJwxSZW/55V7hQoxtREoCWTzcA3OD2tzrVR6cbUrfzNm9jx6QLajBxFVHR0wMfZyoemtgSaTA5Q1S21GokxpkqewkK2zX2fbXPfJ7pJE5JOHEpChw5BneOwA5JtWhQTchUmExEZrKpfuA8PEZFDyttPVT+tlciMMaXk/KKkTXmFwq1baX70QFIvGkFs8xZ1HZYxQOUlk+eBw9zfKxrn4QX2r+A5Y0yI5OjPbPzPw8SmpNDx5ttodlivKo8pb4T7+vRsDugU8PysxgSswmSiqof5/d49POEYY8rT5KAepF58CS2PG0x0QkJAx5Q3wr1Lm0SO79OptsI0ESzQ3lzvqOo55Wyfo6rnhj4sY4y/qOhoWg09ueodXQtXbEI37EA6J+0znsTmozK1IdBuIEMq2H5CiOIwxgBej4f8TZtqfB7rtWXCrdKSiYjc7/4a7/e7z/44M/waY0Igf+MGZz6trVvp/uAjxDQPfmp0XzvJ+vRspHOS9doyYVNVNVdn999ov9/BaXjfgLO8rjGmBjwFBWx7/122ffQhMU2b0uaS0UQnVj4deEXTx/svr2ulEhNOlSYTVb0MQES+UdVJ4QnJmMiR+/tvbH15EoXpabQ4ZhCpF15cau2R8ixcsYkp8xRgn5HstryuqSuVjTPppqpr3YcLRKTcLsCq+kdtBGZMJIiKjYXoKDrd9jeaHnJoQMf4SiRjThNLGqbeqKxksgrwVdr+hlO1FVVmHy8QUwtxGRMR9uvajW73/zuoKVEAaw8x9U5l40ya+/0e3DvdGBOwYBOJMfVRdaagx63yKlZV681lTBW8Hg87FnxMYUY6bUaOrtY5/BvcK1pq15i6FNBXIhGZISLHuL9fBqwBfhSRy2szOGMaurz161j/7wfImDmDwsxMvEVF1TqPr7sv2Prspn4KtGRyEnCp+/utwFBgB/A2tj67Mfvw5OeT9d47bJ8/j5hmibS/8hoS+x9FVFTZZseKlVcasdURTX0VaDKJV9UCEekItFbVrwFExL4eGVOObR++z/Z5c2lx3GBSz7uwyu6+Pv4JxH/MiJVGTH0XaDJZISJ3A12BDwDcxLKrtgIzpiFrdcrpND2kJ03l4ID29yUR/wRiY0ZMQxJoMrkceAAoBP7mbhsITKuNoIxp6GKaNg0qkfgPQrQEYhqigJKJqv6Os/67/7bZwOzaCMqYhqIgIx1vXh4JnbtU+xw2CNE0BgF3DXZ7cY0GOgKbgNdU9ZXaCsyY+sxbXMz2+R+R9d7bJHTuQpe776nR+WwQomnoAl3P5O/AGOBxnJmCuwJ/E5EOqvpgLcZnTL2Tt/ZP0ia/Qv6G9TQ7ok+Nx47YuBHTGARaMhkHnOA/SFFEPgK+ACyZmIiR9f67ZL3zFjEtWtL+mutJ7HtkUN19/fknEuupZRq6QJNJMyCjzLYsoElowzGmfotLTaXl4ONJOe8CYpo2q/H5bOyIaSwCTSbzgGkichewHqea60Hgo9oKzJj6qMWAgbQYMLCuwzCm3gl0hrnrgd3ASiAbWAHsAW6opbiMqVNerxevx1PXYRjTYFRZMhGRJJwleq8DxgIpQKaq2l+aaZQK0tJIe+1VEg/vTatTTgvpuW3CRtNYVbUG/JnAGzhtI7uB4ar6WTgCMybcvEVFbJ8/j6z33iEqNpYWRx8T0vOXHZxoDe+mMamqZPIAcCfwP+AKnHaS0P6FGVMP5P7xO2mTX6Fg00YSj+xHmxGXEJvUKiTnLjtVig1ONI1RVclkf1V9DkBEJgJ/r/2QjAm/zDdn4cnZQ4frbyLxiD4hPbevC7BNlWIas6qSSUkDvaoWiUi1FtPyEZEewGQgGadr8RhV/bWCfQVYDjyvqrfX5LrGVKXd5VcQ3aQpMU1qp7e7dQE2jV1VyaGpiHzh97h5mceo6uAgrvciMFFVp4rIKOAl4MSyO4lIjPvc20Gc25hqi2udXNchGNOgVZVMyq6kWO2FsESkDdAXONndNAN4TkRSVbXsgMi7gPeBRPfHmBrzejzs+upLmh1+eMjaQ8ry763lY722TCSoNJmo6uQQXqszsElVi91zF4vIZnd7SQZUKDMAACAASURBVDIRkcOBU4EhwL0hvL6JYDkbN7Lx6efJ/UVpPewcUs75S61cp7y5tqzXlokENWoDCTURiQMmAZe5yaauQzINnLeoiG0ffsCvH7xHVHw8bS+9jBaDgqmZDUzZSRutfcREmnAmkw1ARxGJcRNFDNDB3e7THjgAmOsmkiQgSkRaqOqVwVwsOdmqFXxSU5vXdQh1wlNUxA+338meP9eSMuhYuo+7jPhWNavemrdoLZ8v37jP9tW/ZwFw2AHJHN+nU72/5/U9vnCyexEaUV6vN2wXE5GFwH/9GuAvV9Uhlex/H5AYZG+ubsCfWVnZeDzhe231VWpqczIydtd1GHVm+/x5xLVrR/eTjqv2fahoXfayGkq330h/T/ize+GIjo7yfQHvDqytzjnCXc11NTBZRCYA23HWSEFE5gITVHVpmOMxjVxNp0MpO2rdxooYU75AF8dKACYAI4BkVW0pIqcAPXyDGgOhqj8DA8rZfkYF+98X6LlNZCvO2ROSKeGh/JKIjVo3pnKBlkyexFmu9xLgQ3fbGnd7wMnEmFDzejzs/HwhmXNm0f7q62jW87Bqn6vstCdWEjEmcIEmk78AB6rqHhHxAKjqJhGxvzBTZ/I3bSJtyivk/f4bTQ85lLjUNkGfo6L2EEsgxgQn0GRSUHZfEUnFmRLFmLDyFBay7YP32PbhB0Q3aUK7v15B84HHBL18rrWHGBM6gSaTWTgN57cAiEh74Cng9doKzJgKFRex65uvaN7/KFIvGkFs8xZBHW6z+BoTeoEmk/HAo8AqoCnwK87gwn/WUlzGVCh6vyZ0nXA/MYnVG0tks/gaE3oBJRNVLQBuBm52q7cyVdUGcZg6U91E4mOj1I0JrUC7Bu9fZlNz31QnqvpHqIMyBqBwWxbb580l5YKLiY6Lq/Z5Fq7YxLJfMyksKAZs4kVjakOg1Vy/AV7Av4XTVzKJCWlEJuJ5PR52fLaAzDlvgtdD86OOpsmBB1V5XHkz9sK+o9Zt4kVjQi/Qaq5o/8ci0g74B/BlbQRlIlf+xg1Od98//qBpz8NoO+pS4lJTKz2mvPEh/qRzEkMHdOXIA23NEmNqS7WmU1HVrSJyM/ALMD20IZlIlb9pI+seuI+YJk1pN+5Kmg8YWG5337IlkEDGh9gcTMbUrprMzSU4PbuMCYn4Dh1JPf9CWgw8ttIG9rJrhlivLGPqXqAN8F+yt40EnCTSE7i/NoIykSkqKopWJ58a0L7WG8uY+iXQksl/yzzeA6xU1V9DHI+JAF6vl8K0rcS3a1/XoRhjQqTKZOIuYnUicKWq5td+SKYxK8zMIG3qa+TqT3R74N/EpVTeuF7WwhWb0A07yl1PxBhTd6pMJu6qiKcAnjDEYxopr8fDjk8+JvPtNyEqipTzLiS2dfC9q3wN79a115j6JZgp6P8pIv9Q1cLaDMg0PvmbN7P1f5PIX/snzQ7vTZtLxhCXXP1uutI5yRrbjalnKk0mIjJCVWcANwDtgFtFJAO/xnhV7VK7IZqGLiommuLdu2h/1bUk9usf9Oy+xpj6r6qSyUvADGBUGGIxjVR823Z0//ejRMXYZAnGNFZVJZMoAFX9PAyxmEbMEokxjVtVySRGRIZQek6uUlT109CGZBoir9fL7m8XkaM/0fbSv4a0Kst/xLtN0mhM/VRVMkkAXqbiZOIFys4obCJMQUY66a9NJufHNey3/wF48vKIadKkRuesaDldm6TRmPqpqmSyR1UtWZhyeYuL2f7xR2S9+zZR0dG0GTmKliecSFR0dNUHl6OiBGLTpRhT/9Vkbi4T4XZ89imZs9+g2RF9aDNyNHGtW9fofP5zblkCMaZhCagB3pjytDz+eOJSU2l2eO+g2kgqWnfEl0hszi1jGp5K6yNUtXm4AjENT3RcPIm9jwi6sd1XAinL2kOMabismstUqWjnToq2b2O/bt1rdB5ficRKIMY0PpZMTIW8Xi+7vv6KjDdeJ6Z5It0eeKjajetQuk3ESiDGNC6WTEy5CtK2kvbaZHJ//okmB/WgzeixNUokPlYiMaZxsmRi9rHj84VkvD6NqNhY2oweS8vjBtc4kdjU8cY0bpZMzD7iWrd2ZvcdMYrYpJp9+PvaSXzjRqx6y5jGyZKJ2UezXofTrNfhITmXr53Exo0Y07hZMolwXo8nJFVY5Y0bARs7YkykqHmLqmmQinbsYPOLE8l6560an6uicSNgY0eMiRRWMokwXo+HnV99QeasmXgLC9nvnL8EdXx5pRArfRhjwppMRKQHMBlIBrKAMar6a5l97gUuBorcn/Gq+lE442ysCrZsdrr7/qI06SG0HXMZ8e3aBXUO/7EiPlb6MMaEu2TyIjBRVaeKyCiclRxPLLPPYuBxVc0Rkd7A5yLSXlVzwxxro5P59hzyN26g7aWX0WLQ4ICnQSlvPRErhRhj/IUtmYhIG6AvcLK7aQbwnIikqmqGb78ypZAfcCabTAY2hivWxqrNiFEQBbEtA+vuW7Zbr60nYoypSDhLJp2BTapaDKCqxSKy2d2eUcExY4DfVdUSSQgEO2bEuvUaYwJVbxvgReR44AH2lmSCkpwcuUu7Zn37HU27dKZJhw4ApKZWPfnzvEVr+Xx56Zy9MSObAzq15KFrB9VGmGEXyH2IFHYv9rJ7ERrhTCYbgI4iEuOWSmKADu72UkRkIDAVOEdVtToXy8rKxuPx1ijghqZox3bSp00le/n3tDx+CG1HX0pqanMyMnZXetzCFZuYMs+5zf7TnXRKTaTvQSlVHt8QBHIfIoXdi73sXjiio6Nq/AU8bMlEVdNFZAUwAidRjACW+7eXAIhIf2AmcL6qLgtXfA2Z1+Nh5+cLyZwzC29RESnnXUCrk08N+Hhf4/qY08Sqsowx1RLuaq6rgckiMgHYjtMmgojMBSao6lLgeaAJ8JKI+I4braqrwhxrg+D1etn01OPk/LiGpoccSpvRY4lv0ybo80jnJEskxphqC2syUdWfgQHlbD/D7/f+4YypoYuKiiKxz5E0HzCQFsccW2l336qWyzXGmOqqtw3wJnBJQ8oO1SlfeQMOwQYdGmNqzpJJA+LJyyUqYb8qBxv6l0Di4mMoLCgGbMChMab2WDJpALxeL9nfLyF9xjRSLxpBi6OOBiqutvIfZOjPSiDGmNpiyaSeK9yWRfq019izcgUJXboS3659yXMVVVv5DzK0ro/GmHCwZFJPeT0edny2gMw5b4LXQ8oFF9Fq6ClExcSU2s+qrYwx9YElk3rKW1zMzs8+pcmBB9J21KXEpaaWPOer3rJeWMaY+sKSST0VHRdHpzvvJiax+T4N7v6JxNpAjDH1gSWTeiy2eYuS320aeGNMfWbL9tax4uxsMmbOoDi38uVa/JfGtRKJMaa+sZJJHfF6vexe/B0Zr0+nKDubuVvj2Jqyf4X7W2nEGFOfWTKpA4WZGaRNfY2c1T+Q0K078w8+k9U5TehSyTFWGjHG1GeWTMKscPt21v7jHgBSLx5J0olD2TVjBV0SsVKHMabBsmQSZnGtWpHyl/NJ7NOXuORkFq7YhG7Ysc9odWOMaUgsmdSBVkP3Lh7p66FlVVjGmIbMkkktKshIJz7VWVuksunfI3UtkdatmxETE74OhbY86152L/aKxHtRXOxh27Y9IT2nJZNaULx7N+lvzGD34u/oes99JHTubNO/lyMmJtrmDTOmDtRGArVkEkJer5fd3y5yx43ksLvPcTzz2RY8MRnWtdcY06hZMgmRwm1ZpE1+hZw1q9lv/wPodOllPLUwvSSJRHIJxBjT+FkyCZGo6GgKNm9mx+BhfBbdDfwSiZVGjDGNnU2nEiKxSa1Yf/FNvLi5FbpxJxDZ7SEN0fnnD+OPP34Lybm++upzJk58utJ9li1byuLF35Y8zszM4IYbrgrqOsuWLeWkk45l7NiRjB59IddffyXr1q2tTshh8d//vsiCBfNDes68vDwuv3w0uVVMSVSfbNuWxS23XMfFF5/LpZeOYM2a1ZXuv337doYNO4V77vlbybYXXniWsWNHlvyceOIxzJr1OgBz5szitddeqdXXUJaVTELA11PLt8LhmNMkIntnmb0GDTqeQYOOr3Sf5cu/Jzc3l6PclTNTUlJ59tmXgr5Wt2778/LLrwHw/PPP8MwzT/D4488EH3QFiouLiSmzjk51jRt3dUjO42/27Nc54YQTadKkSVDHFRUVERtbNx+BL774HL179+HJJyeycuUK7r//Hl5//a0Kl+R+/PGHGTjwWHJy9vbAuuaaG7jmmhsAJ9mcf/5ZnHjiUADOPvsvjBx5HueeewHNmoVnmQpLJkHas+oHdn27iHaXX0FUtFOw8/XU8l/h0FTf16u28NUPW2p0jkGHt+fYXu2r3jEAH374PjNmvEZUVBQdOnTib38bT6tWrSksLOSJJx5l+fLvadWqFQcd1INt27L4178eZe7c9/jmmy/5178eZf36tTz44D/Jy8vD4ynm9NOHMWDAQN55Zw4ej4elSxdz0kmnMHToKYwbN5oPPlgAwOrVPzBx4tPk5OQAcN11N5Uknor06XMk33zzVanY58yZRXFxMYmJidx++1106dKtytg/+WQ+rVol8eeff3L33fdSUFDIiy8+y549zofZuHFXc8wxg9i+fRv33XcP27dnAdCv31HceONtrFq1kieffBSPx0tRURGXXvpXTj75NB588D4OPvgQzjvvInJycnjqqf/w009rADj11DMYNWosANdffyWHHNKT1at/IDMzkxNPHFrywVnWu+++xTPPvFjy+LnnnmLFimUUFhaSlJTE3XdPoF279mzZsplx40Zz7rkXsnTpYk499XQGDTqBp556lLS0reTn5zN06KmMGfPXSs8TCp999gmzZr0HQO/eRxAfH8/PP//IIYf03Gff+fM/pHXr1hx88KF8882X5Z7vo48+oF+/o0hOTgEgNjaWo446mgULPubss/8SkpirYskkQEU7d5Ixcwa7F39LfPsOfLlIWbTWKVZb20jj9ccfv/Hii8/x8stTSUlJYdKkF3jyyf9w//0P8c47b5KWtpWpU9+guLiYG264ijZt2uxzjjlzZjNw4LGMHTsOgF27dtGiRQvOOedccnNzuf76mwHYsmVzyTG7du1k/Pg7ePDBR+nVqzfFxcUlH+QV8Xg8fPXV5wwdegoAK1cu59NPP2bixEnEx8ezaNHXPPTQ/bzwwv+qjH3VqhW8+uoMOnbsxO7du7nxxqv4z3+eISUlhczMTK64YgxTpsxk/vwPadeuHU8//XzJawOYNm0yF144ktNOOxOv10t2dvY+8b766n/xeDxMmTKTnJw9XHXVXznggIMYOPBYANLStjJx4iRycnK46KJzOOusc+jcufQMdmlpW8nLyyv1IT9q1NiSe/ree2/zwgvP8M9/PgTAzp076datO5df7lQn3nzztYwdO44jjuhLYWEhN910DYcccij9+x9d6Xn8ffjh+8ycOb3c/5PRo8dy0kmnlNq2c+cOvF4vSUl7Z71o27Yd6elp+ySTzMwMXn99Gs89938sXLig3GsAzJ37HuPGXVNqW8+evfj2268tmdQXXq+XXV9/RcYbr+MtyCf57OGs6tiHKR//DjjrrVvbSGgd2yt0pYqaWrZsKQMHHktKivON75xzzmXs2JHuc99z2mlnEBsbS2xsLEOHnsoPPyzf5xxHHNGHiROfprCwkL59+9G3b78qr7t69Sq6detOr169AYiJiaFFixbl7rt27R+MHTuSzMx0WrRoyYsv/g+Ar7/+gt9++5UrrxwLuF3Xd+8KKPZevY6gY8dObiwr2bJlM7fffmPJ81FRUWzatIGePXsxc+Z0Jk58miOO6MuAAQMB6Nu3H1OnvsrWrVvo3/9oevY8bJ+4ly5dzE033U5UVBTNmiUydOgpLF26uCSZDBlyEtHR0SQmJtK1a3c2bdq4TzLJyEinVavWpbZ9++3XzJkzi9zcHIqLi0s9Fx+fwIknOjNQ5Obmsnz59+zYsaPk+ZycPaxdu5b+/Y+u9Dz+Tj/9LE4//awKn6+JRx75F9deeyNNmzatcJ8ff1zN9u3bOeaYQaW2JyenkJ6eXitxlceSSRV2L/mOtFdfpslBPVjX/3TmbPGibiKxtpHGz+tln3ps30Ov1wuUX8ft74QTTuKwww5n8eJvmTr1VT744F0mTHigiut6A47R12ZSWFjIP/4xnscee5j7738IrxfOPPPsctspqoq9adMmfvvCAQccxMSJk8rd95VXprFkyXd89NFcpk59lRdeeJkLLxzJsccOZsmS73jqqUfp3/9orrzy2rJRULaJwP9ex8cnlPweHR1d7gd6QkICBQX5JY+3bt3Cs88+waRJU+jQoSOrVq3kn/+8p+T5Jk32K7mG1+shKiqK//53yj5tJ1Wdx1+wJZOWLZ0SyY4dO0pKJ2lpW2nTZt8vpKtXr+LPP533Sm5uDvn5+dx++4089tjeNrEPPniXU089Y5/XUFCQT0JCAuFivbmq0PzI/rS/8ho63XEXX2/xlrSNWCKJDEce2Z9Fi74mKysTcKo7+vU7CnC+fc+fP5eioiLy8/P59NOPyz3Hxo0baN06mTPOGMZll13Bjz86bQTNmjVjz559q38AevU6nLVr/2T16h8ApxHcV4VUkbi4OG6//S6+/fYbfv1VOfbY45g37wPS09NKzvHzzz8FFTvAYYcdzsaN61m2bGnJtp9+WoPX62Xz5k1uqeJUbrjhFlR/xuPxsH79Ojp27MTw4edxwQUjStpF/PXrN4D3338Hr9dLTs4eFiyYX3JvA9WlS1eysrIoKCgAYM+ePcTGxpGcnIzH4+Htt9+s8NimTZvRu3cfpk59tWRbWtpWsrIygzrP6aefxauvTi/3p2wi8RkyZChvvz0bgJUrV5Cfn4/IIfvs9+GHnzJ79nvMnv0e1113M0cffUypRJKfn8eCBfM588yz9zl27do/OfDAHhXGHWpWMqlCVEwMzY8aUPLY2kYat5tvvq5Uz6XJk1/nqquu45ZbrnMb4Dtyxx3jARg+/Dx+++0XRo++kDZt2iFyMHl5efuc89NPP2b+/HnExcUSFRXFTTfdBsDgwUP4+9/vYOzYkSUN8D4tWrTkwQcf5dlnnyQvL5eoqGiuu+4m+vcfsM/5/bVuncyIEaP43/8m8dBDj3Hllddy1123UlzsoaiokCFDhnLwwYcEHLsTSwsefvgJJk58mqeffpyiokI6dOjII488yfLl3/P661OJiYnF6/Vwxx13Ex0dzezZr7Ns2ffExcUSFxfPLbfcsc95x44dx5NPPsqYMRcBTgP80UcfU8X/UGkJCfvRt++RLF/+PQMGDOSAAw5kyJChjBp1EW3btqVPnyNZuXLfqkefCRMe4JlnniiJoWnTZtx994SgzxOsq6++nvvvn8C8eX8hISGBe++9n2i3Q8/DDz/AoEGDq+wNCPD555/RpUs3unffd2G9xYu/Lac0WHuigilONxDdgD+zsrLxeAJ/bZ68XPI3b6HJ/hWvdvjItGVAw1p3JDW1eb2d/6o+xxaonJw9NG3ajIKCAu6661aGDBnKsGHD6zqsgDTk2P2tWrWS6dOn8NBDj9d1KPXGunVr+c9//s1zz/1fuc+X/duLjo4iOTkRoDuwtjrXtJIJkL1yBenTpuAtLKL7I48RHR9f1yGZBuKmm66lsLCQgoJ8+vU7qtYaYmtDQ47dX69evTnmmOPIzc0NeqxJY5WevpXbbrsrrNeM6GRStHMH6TOmkb10CfEdOtL2qssqTCS2iJUpz6RJk+s6hGpryLGX1RBLVLWpf//KxyPVhohNJru/X0ra5P/hLSggefi5tD7tDKJiYytcd8Q3ut26ABtjzL4iNpn8kFEI8a1Z0WsI2TtbwUyn14wvaZQtgdjodmOMqVjEJpOvshJY3/EUujQtPW+NJY3wKS72ROQqd8bUteJiT8jPGdZkIiI9gMlAMpAFjFHVX8vsEwM8A5wGeIGHVfW/Nbmu1+stdwI16+Zbt0K9bGhlGkPPsVCxe7GX3YvQCXfJ5EVgoqpOFZFRwEvAiWX2uQQ4EDgIJ+ksF5FPVHVtsBcrzskhc85siIKfep5Uqi2kvCV0jTHGVE/YRsCLSBugLzDD3TQD6CsiqWV2vQiYpKoeVc0A3gYuCPZ6e1avZu2E8ez4/DO+/3UbUz78uaQ9BGytEWOMCaVwlkw6A5tUtRhAVYtFZLO7PcNvvy7AOr/H6919gpI+9VW2FcbyVsfT2bpfirWFGGNMLWqMDfAxABsPO54fm3WlW3Q0ww9K5ehDI7cUEh1d9WSEkcDuw152L/aye1HqHlR7FbRwJpMNQEcRiXFLJTFAB3e7v/VAV2CJ+7hsSaUq7QGG3zQCG8bkcKdJiHh2H/aye7GX3YtS2gO/V+fAsCUTVU0XkRXACGCq++9yt13E3yzgChGZg9MAPxwYHMSllgDHAVuAihchMMYY4xODk0iWVLVjRcJdzXU1MFlEJgDbgTEAIjIXmKCqS4HXgAGAr8vw/ar6RxDXyAe+qnIvY4wx/qpVIvFpjLMGG2OMCTNbHMsYY0yNWTIxxhhTY5ZMjDHG1JglE2OMMTVmycQYY0yNWTIxxhhTYw12OpW6ms6+PgrwXtwLXAwUuT/jVfWjcMda2wK5F377CrAceF5Vbw9flOER6L0QkQuBe4EonL+Toaq673KjDVSAfx9tgFdw5gGMBz4FblTVojCHW6tE5DHgPKAb0EtVV5ezT7U+NxtyycQ3nX0PYCLOdPZl+U9nPxC4T0S6hS3C8AnkXiwG+qtqb+CvwEwRaRLGGMMlkHvh+4N5CWdW6saqynshIv2A+4CTVfUwYBCwM5xBhkEg74nxwE+qejjQCzgSODd8IYbN2zgzilQ2RVW1PjcbZDIJ93T29Vmg90JVP1LVHPfhDzjfQpPDFmgYBPG+ALgLeB/4JUzhhVUQ9+IW4DFV3QqgqjtVNS98kdauIO6DF2guItFAAk7pZFPYAg0TVf1KVcvOh1hWtT43G2QyoZzp7AHfdPb+QjKdfT0X6L3wNwb4XVU3hiG+cAroXojI4cCpwJNhjzB8An1fHArsLyJfiMgyEblHRBrTNLqB3ocHgB44c/ptBT5S1a/DGWg9Uq3PzYaaTEw1icjxOH84I+o6lrogInHAJOBq3wdMhIsFDgdOBo4HTgdG12lEdeMCnBJ7e6AjMFhEzq/bkBqWhppMSqazh5L678qms/fpUs4+DV2g9wIRGYgzY/NwVdWwRhkegdyL9sABwFwRWQvcjDNL9f+FN9RaF+j7Yh0wW1XzVXU38A5wVFgjrV2B3ocbgGlu1c5OnPswJKyR1h/V+txskMlEVdMB33T2UPV09tFuHelw4M3wRVr7Ar0XItIfmAmcr6rLwhtleARyL1R1vaqmqGo3Ve0GPIVTP3xl2AOuRUH8jUwHThGRKLfUdhKwMnyR1q4g7sOfOL2XEJF4YCiwT0+nCFGtz80GmUxcVwM3iMgvON8qrgZnOnu3hwo409n/gTOd/bcEP519QxHIvXgeaAK8JCIr3J9edRNurQrkXkSKQO7F60A68CPOh+4a4OU6iLU2BXIfbgaOE5FVOPfhF5zq0EZFRJ4RkY1AJ+ATEVnjbq/x56ZNQW+MMabGGnLJxBhjTD1hycQYY0yNWTIxxhhTY5ZMjDHG1JglE2OMMTVmycQYY0yNWTIxjYKILBSRcXUdR2VE5BIRmV/J88eJSGOcmcBEABtnYuodd5qTtoD/3Fk9VHVzJccsBKaGcr0a95xH46z/kgd8AVynqltCdH4vcJCq/haK81VynfuAvwP5OK/lR+A2VV0U4PFhidM0bFYyMfXVMFVN9PupMJHUsutVNRFnRtkkGu5MwzPd15ECfIYzZYYxIdNgV1o0kUVEWuFM8zAA5337Nc7Mv/tMoy8iB+JMCXIEUAgsUNWL3OcOBp7FWfwoA7hXVd+o6vqquk1E3gSucc9zDPA0TpL5BbhJVb9xnxsLTABSgUzgHlWd5m4fp6qDROQL99Qr3W/+lwNpOKWrTiJyF9BPVUtmrhWRp4EoVb1RRFoCTwBnAB6cVQL/UdVMyKpaJCLTgPEikqqqGSJylPtaDgFyceZhulVVC8qLU1VnishZwL9wVuz7Eef/4oeq7qNpvKxkYhqKaJwPzK44s5jmAs9VsO8DwHygFc4cRM8CiEgz4GOcyQ3b4Ez697yI9Kzq4iKSgrPc6XIRaQ18gLO0aTLOh/oHIpLsXuMZ4HRVbQ4cgzPXUymqOtj9tbdb8ppZZpcZwBki0sK9fgxwoRs7OMvQFuGsiNcHOAWoss3IncRwDM7ytdvdzcU4i2Sl4KysdxJwbUVxikhf4H/AVe7rfwl4V0QSqrq+abysZGLqq7dFxLf+9kJVLTVzqYg8iFNdU55CnKTTwS25fOVuPwtYq6qvuI+XuaWN83EmOCzPM+662XuAhcCtwJnAr6r6mrvPDBG5ERiGU33kAQ4TkfVu+0rQbSyquk5EluHM2DoFOBHIUdVvRaQtzrojSaqaC+wRkSeBK6lgmWLgQrc00RzYAZznW99cVb/322+tiLyEs7bJUxWc6wrgJVX9zn08WUTG47QvfR7sazWNgyUTU18NV9VPfA9EpClOe8VpOCUOcJZZjSmnaudvOKWTxSKyHXhcVf+Hk2AGiMgOv31jcarPKnJj2UZ9EenAvmtorwM6quoeEbkIuB14WUS+xmns/jmA11zWdJzS0xRgJHtLJV2BOGCLiPj2jabyNSfeUNVRbgnrTZxqvoXu6+mBU7rqBzTFuSffV3Ae3/UvFZEb/LbF46wTYiKUJRPTUNwGCDBAVbeKyBHAcpy17Etx1zO/AkBEBuFMtf0Fzoft56p6cg1j2UzpxYPAqXqb517/I+AjEWmC064wCTiuGteZBTwuIp2Av+BUQYHzOvKBFF/pIlCqmikiVwFLRGS6v8WB1AAAAZNJREFUW3J6AedejlDV3SJyM05prSIbgAdV9cEgX49pxCyZmIaiOU47yQ63zeIfFe0oIhcAi9wqru2AF6dd4H3gYREZjbOOBziN9Nmq+lMQscwFnhWRkcAbOG0phwLvu1VQA4AFbrzZlO7i7C8N2B8ot8ut2zi+EKet6E9fjKq6xR2v8riI3OteozvQSVWrrGZS1Z9F5COcEtwtOPd2F5DtdlC4BqdzQkVxTgLeEpFPgMU4pZkTgC/c1RpNBLIGeNNQPIWzuFcmzoI98yrZtz/wnYhkA+/i9LT60/2gOwW4GKd0sRV4BAiq4VhVs3DaX27Dacj+G3CWqmbi/E3d5p5/G07bw7UVnOo+nPaGHSJyYQX7TMdZ9W96me1jcKqWfsRJmLNxliQO1H+AK0WkDU6V3EhgN06iKNsZoFScqroUp+T3nHvt34CxQVzbNEI2aNEYY0yNWcnEGGNMjVkyMcYYU2OWTIwxxtSYJRNjjDE1ZsnEGGNMjVkyMcYYU2OWTIwxxtSYJRNjjDE1ZsnEGGNMjf0/qcWhpH/pSEQAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "from sklearn.metrics import roc_auc_score\n",
    "from sklearn.metrics import roc_curve\n",
    "import matplotlib.pyplot as plt\n",
    "logit_roc_auc = roc_auc_score(test_labels, lr_clf.predict(test_features))\n",
    "fpr, tpr, thresholds = roc_curve(test_labels, lr_clf.predict_proba(test_features)[:,1])\n",
    "plt.figure()\n",
    "plt.plot(fpr, tpr, label='Logistic Regression (area = %0.2f)' % logit_roc_auc)\n",
    "plt.plot([0, 1], [0, 1],'r--')\n",
    "plt.xlim([0.0, 1.0])\n",
    "plt.ylim([0.0, 1.05])\n",
    "plt.xlabel('False Positive Rate')\n",
    "plt.ylabel('True Positive Rate')\n",
    "plt.title('Receiver operating characteristic')\n",
    "plt.legend(loc=\"lower right\")\n",
    "plt.savefig('Log_ROC')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "execution_count": null,
   "metadata": {
    "papermill": {
     "duration": 0.029153,
     "end_time": "2020-08-07T04:48:35.767128",
     "exception": false,
     "start_time": "2020-08-07T04:48:35.737975",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# TFIDF (optional)\n",
    "\n",
    "This part of kernel is optional. I wanted to explore how tf-idf would work on predicting the movement of DJIA."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-08-07T04:48:35.833410Z",
     "iopub.status.busy": "2020-08-07T04:48:35.832227Z",
     "iopub.status.idle": "2020-08-07T04:48:35.836039Z",
     "shell.execute_reply": "2020-08-07T04:48:35.835172Z"
    },
    "papermill": {
     "duration": 0.039524,
     "end_time": "2020-08-07T04:48:35.836197",
     "exception": false,
     "start_time": "2020-08-07T04:48:35.796673",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "from sklearn.feature_extraction.text import CountVectorizer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-08-07T04:48:35.906716Z",
     "iopub.status.busy": "2020-08-07T04:48:35.905313Z",
     "iopub.status.idle": "2020-08-07T04:48:36.006121Z",
     "shell.execute_reply": "2020-08-07T04:48:36.005214Z"
    },
    "papermill": {
     "duration": 0.140244,
     "end_time": "2020-08-07T04:48:36.006288",
     "exception": false,
     "start_time": "2020-08-07T04:48:35.866044",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "df = pd.read_csv('/kaggle/input/stocknews/Combined_News_DJIA.csv')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-08-07T04:48:36.080287Z",
     "iopub.status.busy": "2020-08-07T04:48:36.077544Z",
     "iopub.status.idle": "2020-08-07T04:48:36.223420Z",
     "shell.execute_reply": "2020-08-07T04:48:36.222547Z"
    },
    "papermill": {
     "duration": 0.186357,
     "end_time": "2020-08-07T04:48:36.223603",
     "exception": false,
     "start_time": "2020-08-07T04:48:36.037246",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "columns = ['Top1', 'Top2','Top3', 'Top4', 'Top5','Top6', 'Top7', 'Top8', 'Top9', 'Top10', 'Top11', 'Top12', 'Top13', 'Top14', 'Top15', 'Top16', 'Top17', 'Top18', 'Top19', 'Top20', 'Top21', 'Top22', 'Top23','Top24', 'Top25']\n",
    "df['combined_news'] = df[columns].apply(lambda row:'.'.join(row.values.astype(str)), axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-08-07T04:48:36.310950Z",
     "iopub.status.busy": "2020-08-07T04:48:36.292555Z",
     "iopub.status.idle": "2020-08-07T04:48:36.315040Z",
     "shell.execute_reply": "2020-08-07T04:48:36.314123Z"
    },
    "papermill": {
     "duration": 0.060837,
     "end_time": "2020-08-07T04:48:36.315205",
     "exception": false,
     "start_time": "2020-08-07T04:48:36.254368",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "df = df.drop(columns, axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-08-07T04:48:36.384678Z",
     "iopub.status.busy": "2020-08-07T04:48:36.383448Z",
     "iopub.status.idle": "2020-08-07T04:48:36.756508Z",
     "shell.execute_reply": "2020-08-07T04:48:36.755671Z"
    },
    "papermill": {
     "duration": 0.411078,
     "end_time": "2020-08-07T04:48:36.756665",
     "exception": false,
     "start_time": "2020-08-07T04:48:36.345587",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Date</th>\n",
       "      <th>Label</th>\n",
       "      <th>combined_news</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2008-08-08</td>\n",
       "      <td>0</td>\n",
       "      <td>Georgia 'downs two Russian warplanes' as count...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2008-08-11</td>\n",
       "      <td>1</td>\n",
       "      <td>Why wont America and Nato help us? If they won...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2008-08-12</td>\n",
       "      <td>0</td>\n",
       "      <td>Remember that adorable 9-year-old who sang at ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2008-08-13</td>\n",
       "      <td>0</td>\n",
       "      <td>U.S. refuses Israel weapons to attack Iran: r...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2008-08-14</td>\n",
       "      <td>1</td>\n",
       "      <td>All the experts admit that we should legalise ...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         Date  Label                                      combined_news\n",
       "0  2008-08-08      0  Georgia 'downs two Russian warplanes' as count...\n",
       "1  2008-08-11      1  Why wont America and Nato help us? If they won...\n",
       "2  2008-08-12      0  Remember that adorable 9-year-old who sang at ...\n",
       "3  2008-08-13      0   U.S. refuses Israel weapons to attack Iran: r...\n",
       "4  2008-08-14      1  All the experts admit that we should legalise ..."
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from bs4 import BeautifulSoup\n",
    "\n",
    "df= df.replace('b\\\"|b\\'|\\\\\\\\|\\\\\\\"', '', regex=True)\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-08-07T04:48:36.825583Z",
     "iopub.status.busy": "2020-08-07T04:48:36.824671Z",
     "iopub.status.idle": "2020-08-07T04:50:47.234193Z",
     "shell.execute_reply": "2020-08-07T04:50:47.233185Z"
    },
    "papermill": {
     "duration": 130.447601,
     "end_time": "2020-08-07T04:50:47.234393",
     "exception": false,
     "start_time": "2020-08-07T04:48:36.786792",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "bow_transformer = CountVectorizer(analyzer=punctuation_stopwords_removal).fit(df['combined_news'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-08-07T04:50:47.303011Z",
     "iopub.status.busy": "2020-08-07T04:50:47.302035Z",
     "iopub.status.idle": "2020-08-07T04:50:47.306781Z",
     "shell.execute_reply": "2020-08-07T04:50:47.307428Z"
    },
    "papermill": {
     "duration": 0.042453,
     "end_time": "2020-08-07T04:50:47.307669",
     "exception": false,
     "start_time": "2020-08-07T04:50:47.265216",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "79203"
      ]
     },
     "execution_count": 55,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(bow_transformer.vocabulary_)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {
    "collapsed": true,
    "execution": {
     "iopub.execute_input": "2020-08-07T04:50:47.379098Z",
     "iopub.status.busy": "2020-08-07T04:50:47.377836Z",
     "iopub.status.idle": "2020-08-07T04:50:47.440350Z",
     "shell.execute_reply": "2020-08-07T04:50:47.441121Z"
    },
    "papermill": {
     "duration": 0.103046,
     "end_time": "2020-08-07T04:50:47.441338",
     "exception": false,
     "start_time": "2020-08-07T04:50:47.338292",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Georgia 'downs two Russian warplanes' as countries move to brink of war.BREAKING: Musharraf to be impeached.'.Russia Today: Columns of troops roll into South Ossetia; footage from fighting (YouTube)'.Russian tanks are moving towards the capital of South Ossetia, which has reportedly been completely destroyed by Georgian artillery fire'.Afghan children raped with 'impunity,' U.N. official says - this is sick, a three year old was raped and they do nothing.150 Russian tanks have entered South Ossetia whilst Georgia shoots down two Russian jets.'.Breaking: Georgia invades South Ossetia, Russia warned it would intervene on SO's side.The 'enemy combatent' trials are nothing but a sham: Salim Haman has been sentenced to 5 1/2 years, but will be kept longer anyway just because they feel like it..Georgian troops retreat from S. Osettain capital, presumably leaving several hundred people killed. [VIDEO]'.Did the U.S. Prep Georgia for War with Russia?'.Rice Gives Green Light for Israel to Attack Iran: Says U.S. has no veto over Israeli military ops'.Announcing:Class Action Lawsuit on Behalf of American Public Against the FBI'.So---Russia and Georgia are at war and the NYT's top story is opening ceremonies of the Olympics?  What a fucking disgrace and yet further proof of the decline of journalism..China tells Bush to stay out of other countries' affairs.Did World War III start today?'.Georgia Invades South Ossetia - if Russia gets involved, will NATO absorb Georgia and unleash a full scale war?'.Al-Qaeda Faces Islamist Backlash'.Condoleezza Rice: The US would not act to prevent an Israeli strike on Iran. Israeli Defense Minister Ehud Barak: Israel is prepared for uncompromising victory in the case of military hostilities.'.This is a busy day:  The European Union has approved new sanctions against Iran in protest at its nuclear programme.'.Georgia will withdraw 1,000 soldiers from Iraq to help fight off Russian forces in Georgia's breakaway region of South Ossetia.Why the Pentagon Thinks Attacking Iran is a Bad Idea - US News &amp; World Report'.Caucasus in crisis: Georgia invades South Ossetia'.Indian shoe manufactory  - And again in a series of you do not like your work?'.Visitors Suffering from Mental Illnesses Banned from Olympics'.No Help for Mexico's Kidnapping Surge\n",
      "=====\n",
      "  (0, 53)\t1\n",
      "  (0, 280)\t1\n",
      "  (0, 2388)\t1\n",
      "  (0, 3454)\t1\n",
      "  (0, 3801)\t1\n",
      "  (0, 3825)\t1\n",
      "  (0, 4236)\t1\n",
      "  (0, 5755)\t1\n",
      "  (0, 5863)\t1\n",
      "  (0, 6543)\t1\n",
      "  (0, 6751)\t1\n",
      "  (0, 7285)\t1\n",
      "  (0, 7757)\t1\n",
      "  (0, 7835)\t1\n",
      "  (0, 8576)\t1\n",
      "  (0, 8623)\t1\n",
      "  (0, 9046)\t1\n",
      "  (0, 9094)\t1\n",
      "  (0, 9672)\t1\n",
      "  (0, 11628)\t1\n",
      "  (0, 11775)\t1\n",
      "  (0, 12311)\t1\n",
      "  (0, 12394)\t1\n",
      "  (0, 13050)\t2\n",
      "  (0, 13388)\t1\n",
      "  :\t:\n",
      "  (0, 71906)\t1\n",
      "  (0, 72044)\t2\n",
      "  (0, 72589)\t2\n",
      "  (0, 72970)\t1\n",
      "  (0, 73067)\t1\n",
      "  (0, 73370)\t1\n",
      "  (0, 73516)\t1\n",
      "  (0, 73978)\t4\n",
      "  (0, 74716)\t1\n",
      "  (0, 74816)\t1\n",
      "  (0, 74865)\t1\n",
      "  (0, 75697)\t3\n",
      "  (0, 75706)\t1\n",
      "  (0, 75719)\t1\n",
      "  (0, 75858)\t1\n",
      "  (0, 75958)\t1\n",
      "  (0, 76921)\t1\n",
      "  (0, 77351)\t1\n",
      "  (0, 77708)\t1\n",
      "  (0, 77714)\t2\n",
      "  (0, 77977)\t2\n",
      "  (0, 78299)\t1\n",
      "  (0, 78397)\t1\n",
      "  (0, 78740)\t1\n",
      "  (0, 78879)\t1\n"
     ]
    }
   ],
   "source": [
    "sample_djia_down = df['combined_news'][0]\n",
    "bow_sample_djia_down = bow_transformer.transform([sample_djia_down])\n",
    "print(sample_djia_down)\n",
    "print('=====')\n",
    "print(bow_sample_djia_down)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {
    "collapsed": true,
    "execution": {
     "iopub.execute_input": "2020-08-07T04:50:47.575095Z",
     "iopub.status.busy": "2020-08-07T04:50:47.570020Z",
     "iopub.status.idle": "2020-08-07T04:51:07.336905Z",
     "shell.execute_reply": "2020-08-07T04:51:07.336004Z"
    },
    "papermill": {
     "duration": 19.864941,
     "end_time": "2020-08-07T04:51:07.337075",
     "exception": false,
     "start_time": "2020-08-07T04:50:47.472134",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Printing bag-of-words for sample 1 (DJIA goes down) :\n",
      "1000\n",
      "12\n",
      "5\n",
      "absorb\n",
      "act\n",
      "action\n",
      "affairsdid\n",
      "american\n",
      "amp\n",
      "anyway\n",
      "approved\n",
      "artillery\n",
      "attack\n",
      "attacking\n",
      "backlashcondoleezza\n",
      "bad\n",
      "banned\n",
      "barak\n",
      "behalf\n",
      "breakaway\n",
      "brink\n",
      "bush\n",
      "busy\n",
      "capital\n",
      "case\n",
      "ceremonies\n",
      "children\n",
      "columns\n",
      "combatent\n",
      "completely\n",
      "countries\n",
      "crisis\n",
      "day\n",
      "decline\n",
      "defense\n",
      "destroyed\n",
      "disgrace\n",
      "downs\n",
      "ehud\n",
      "enemy\n",
      "entered\n",
      "european\n",
      "faces\n",
      "fbisorussia\n",
      "feel\n",
      "fight\n",
      "fighting\n",
      "fireafghan\n",
      "footage\n",
      "forces\n",
      "fucking\n",
      "full\n",
      "georgia\n",
      "georgian\n",
      "georgias\n",
      "gets\n",
      "gives\n",
      "green\n",
      "haman\n",
      "help\n",
      "hostilitiesthis\n",
      "hundred\n",
      "idea\n",
      "iii\n",
      "illnesses\n",
      "impeachedrussia\n",
      "impunity\n",
      "intervene\n",
      "invades\n",
      "involved\n",
      "iran\n",
      "iraq\n",
      "islamist\n",
      "israel\n",
      "israeli\n",
      "itgeorgian\n",
      "jetsbreaking\n",
      "journalismchina\n",
      "kept\n",
      "kidnapping\n",
      "killed\n",
      "lawsuit\n",
      "leaving\n",
      "light\n",
      "like\n",
      "longer\n",
      "manufactory\n",
      "mental\n",
      "mexicos\n",
      "military\n",
      "minister\n",
      "move\n",
      "moving\n",
      "musharraf\n",
      "nato\n",
      "new\n",
      "news\n",
      "nothing\n",
      "nothing150\n",
      "nuclear\n",
      "nyts\n",
      "official\n",
      "old\n",
      "olympics\n",
      "olympicsno\n",
      "opening\n",
      "opsannouncingclass\n",
      "osettain\n",
      "ossetia\n",
      "ossetiaindian\n",
      "ossetiawhy\n",
      "pentagon\n",
      "people\n",
      "prep\n",
      "prepared\n",
      "presumably\n",
      "prevent\n",
      "programmegeorgia\n",
      "proof\n",
      "protest\n",
      "public\n",
      "raped\n",
      "region\n",
      "reportcaucasus\n",
      "reportedly\n",
      "retreat\n",
      "rice\n",
      "roll\n",
      "russia\n",
      "russian\n",
      "russiarice\n",
      "salim\n",
      "sanctions\n",
      "says\n",
      "scale\n",
      "sentenced\n",
      "series\n",
      "several\n",
      "sham\n",
      "shoe\n",
      "shoots\n",
      "sick\n",
      "sidethe\n",
      "soldiers\n",
      "sos\n",
      "south\n",
      "start\n",
      "stay\n",
      "story\n",
      "strike\n",
      "suffering\n",
      "surge\n",
      "tanks\n",
      "tells\n",
      "thinks\n",
      "three\n",
      "today\n",
      "todaygeorgia\n",
      "top\n",
      "towards\n",
      "trials\n",
      "troops\n",
      "two\n",
      "un\n",
      "uncompromising\n",
      "union\n",
      "unleash\n",
      "us\n",
      "veto\n",
      "victory\n",
      "videodid\n",
      "war\n",
      "waralqaeda\n",
      "warbreaking\n",
      "warned\n",
      "warplanes\n",
      "whilst\n",
      "withdraw\n",
      "workvisitors\n",
      "world\n",
      "would\n",
      "year\n",
      "years\n",
      "yet\n",
      "youtuberussian\n"
     ]
    }
   ],
   "source": [
    "print('Printing bag-of-words for sample 1 (DJIA goes down) :')\n",
    "row, cols = bow_sample_djia_down.nonzero()\n",
    "for col in cols:\n",
    "    print(bow_transformer.get_feature_names()[col])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-08-07T04:51:07.437328Z",
     "iopub.status.busy": "2020-08-07T04:51:07.436008Z",
     "iopub.status.idle": "2020-08-07T04:51:07.440298Z",
     "shell.execute_reply": "2020-08-07T04:51:07.441195Z"
    },
    "papermill": {
     "duration": 0.06228,
     "end_time": "2020-08-07T04:51:07.441495",
     "exception": false,
     "start_time": "2020-08-07T04:51:07.379215",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(1, 79203)\n"
     ]
    }
   ],
   "source": [
    "print(np.shape(bow_sample_djia_down))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {
    "collapsed": true,
    "execution": {
     "iopub.execute_input": "2020-08-07T04:51:07.530623Z",
     "iopub.status.busy": "2020-08-07T04:51:07.529523Z",
     "iopub.status.idle": "2020-08-07T04:51:07.577275Z",
     "shell.execute_reply": "2020-08-07T04:51:07.576120Z"
    },
    "papermill": {
     "duration": 0.092055,
     "end_time": "2020-08-07T04:51:07.577455",
     "exception": false,
     "start_time": "2020-08-07T04:51:07.485400",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Printing bag-of-words for sample 2 (DJIA goes up/remains the same :)\n",
      "Why wont America and Nato help us? If they wont help us now, why did we help them in Iraq?'.Bush puts foot down on Georgian conflict'.Jewish Georgian minister: Thanks to Israeli training, we're fending off Russia .Georgian army flees in disarray as Russians advance - Gori abandoned to Russia without a shot fired'.Olympic opening ceremony fireworks 'faked'.What were the Mossad with fraudulent New Zealand Passports doing in Iraq?'.Russia angered by Israeli military sale to Georgia'.An American citizen living in S.Ossetia blames U.S. and Georgian leaders for the genocide of innocent people'.Welcome To World War IV! Now In High Definition!'.Georgia's move, a mistake of monumental proportions .Russia presses deeper into Georgia; U.S. says regime change is goal'.Abhinav Bindra wins first ever Individual Olympic Gold Medal for India'. U.S. ship heads for Arctic to define territory'.Drivers in a Jerusalem taxi station threaten to quit rather than work for their new boss - an Ara.The French Team is Stunned by Phelps and the 4x100m Relay Team'.Israel and the US behind the Georgian aggression?'.Do not believe TV, neither Russian nor Georgian. There are much more victims'.Riots are still going on in Montreal (Canada) because police murdered a boy on Saturday.'.China to overtake US as largest manufacturer'.War in South Ossetia [PICS]'.Israeli Physicians Group Condemns State Torture'. Russia has just beaten the United States over the head with Peak Oil'.Perhaps *the* question about the Georgia - Russia conflict '.Russia is so much better at war'.So this is what it's come to: trading sex for food.\n",
      "======\n",
      "  (0, 2382)\t1\n",
      "  (0, 3265)\t1\n",
      "  (0, 4146)\t1\n",
      "  (0, 4711)\t1\n",
      "  (0, 5741)\t1\n",
      "  (0, 5755)\t1\n",
      "  (0, 5995)\t1\n",
      "  (0, 6862)\t1\n",
      "  (0, 6911)\t1\n",
      "  (0, 7089)\t1\n",
      "  (0, 9523)\t1\n",
      "  (0, 9718)\t1\n",
      "  (0, 9822)\t1\n",
      "  (0, 10025)\t1\n",
      "  (0, 10291)\t1\n",
      "  (0, 10505)\t1\n",
      "  (0, 11338)\t1\n",
      "  (0, 11445)\t1\n",
      "  (0, 12811)\t1\n",
      "  (0, 13902)\t1\n",
      "  (0, 14020)\t1\n",
      "  (0, 15075)\t1\n",
      "  (0, 16194)\t1\n",
      "  (0, 16794)\t1\n",
      "  (0, 16950)\t1\n",
      "  :\t:\n",
      "  (0, 66458)\t1\n",
      "  (0, 66532)\t1\n",
      "  (0, 66733)\t1\n",
      "  (0, 67359)\t1\n",
      "  (0, 69174)\t1\n",
      "  (0, 69232)\t1\n",
      "  (0, 69242)\t1\n",
      "  (0, 69644)\t1\n",
      "  (0, 69895)\t1\n",
      "  (0, 70354)\t1\n",
      "  (0, 71204)\t1\n",
      "  (0, 71474)\t1\n",
      "  (0, 71555)\t1\n",
      "  (0, 72508)\t1\n",
      "  (0, 73430)\t1\n",
      "  (0, 73978)\t7\n",
      "  (0, 74798)\t1\n",
      "  (0, 75697)\t1\n",
      "  (0, 76026)\t1\n",
      "  (0, 77273)\t1\n",
      "  (0, 77376)\t1\n",
      "  (0, 77546)\t2\n",
      "  (0, 77590)\t1\n",
      "  (0, 77714)\t1\n",
      "  (0, 78979)\t1\n"
     ]
    }
   ],
   "source": [
    "print('Printing bag-of-words for sample 2 (DJIA goes up/remains the same :)')\n",
    "sample_djia_up = df['combined_news'][1]\n",
    "bow_sample_djia_up = bow_transformer.transform([sample_djia_up])\n",
    "print(sample_djia_up)\n",
    "print('======')\n",
    "print(bow_sample_djia_up)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {
    "collapsed": true,
    "execution": {
     "iopub.execute_input": "2020-08-07T04:51:07.661850Z",
     "iopub.status.busy": "2020-08-07T04:51:07.660895Z",
     "iopub.status.idle": "2020-08-07T04:53:17.615185Z",
     "shell.execute_reply": "2020-08-07T04:53:17.616291Z"
    },
    "papermill": {
     "duration": 130.001145,
     "end_time": "2020-08-07T04:53:17.617210",
     "exception": false,
     "start_time": "2020-08-07T04:51:07.616065",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  (0, 53)\t1\n",
      "  (0, 280)\t1\n",
      "  (0, 2388)\t1\n",
      "  (0, 3454)\t1\n",
      "  (0, 3801)\t1\n",
      "  (0, 3825)\t1\n",
      "  (0, 4236)\t1\n",
      "  (0, 5755)\t1\n",
      "  (0, 5863)\t1\n",
      "  (0, 6543)\t1\n",
      "  (0, 6751)\t1\n",
      "  (0, 7285)\t1\n",
      "  (0, 7757)\t1\n",
      "  (0, 7835)\t1\n",
      "  (0, 8576)\t1\n",
      "  (0, 8623)\t1\n",
      "  (0, 9046)\t1\n",
      "  (0, 9094)\t1\n",
      "  (0, 9672)\t1\n",
      "  (0, 11628)\t1\n",
      "  (0, 11775)\t1\n",
      "  (0, 12311)\t1\n",
      "  (0, 12394)\t1\n",
      "  (0, 13050)\t2\n",
      "  (0, 13388)\t1\n",
      "  :\t:\n",
      "  (0, 71906)\t1\n",
      "  (0, 72044)\t2\n",
      "  (0, 72589)\t2\n",
      "  (0, 72970)\t1\n",
      "  (0, 73067)\t1\n",
      "  (0, 73370)\t1\n",
      "  (0, 73516)\t1\n",
      "  (0, 73978)\t4\n",
      "  (0, 74716)\t1\n",
      "  (0, 74816)\t1\n",
      "  (0, 74865)\t1\n",
      "  (0, 75697)\t3\n",
      "  (0, 75706)\t1\n",
      "  (0, 75719)\t1\n",
      "  (0, 75858)\t1\n",
      "  (0, 75958)\t1\n",
      "  (0, 76921)\t1\n",
      "  (0, 77351)\t1\n",
      "  (0, 77708)\t1\n",
      "  (0, 77714)\t2\n",
      "  (0, 77977)\t2\n",
      "  (0, 78299)\t1\n",
      "  (0, 78397)\t1\n",
      "  (0, 78740)\t1\n",
      "  (0, 78879)\t1\n"
     ]
    }
   ],
   "source": [
    "from sklearn.feature_extraction.text import TfidfTransformer\n",
    "\n",
    "bow_data = bow_transformer.transform(df['combined_news'])\n",
    "print(bow_data[:1])\n",
    "tfidf_transformer = TfidfTransformer().fit(bow_data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {
    "collapsed": true,
    "execution": {
     "iopub.execute_input": "2020-08-07T04:53:17.701776Z",
     "iopub.status.busy": "2020-08-07T04:53:17.700765Z",
     "iopub.status.idle": "2020-08-07T04:53:17.762128Z",
     "shell.execute_reply": "2020-08-07T04:53:17.761058Z"
    },
    "papermill": {
     "duration": 0.105053,
     "end_time": "2020-08-07T04:53:17.762314",
     "exception": false,
     "start_time": "2020-08-07T04:53:17.657261",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  (0, 78879)\t0.08992988241047055\n",
      "  (0, 78740)\t0.0440219238356948\n",
      "  (0, 78397)\t0.0206391174376636\n",
      "  (0, 78299)\t0.02523380293229986\n",
      "  (0, 77977)\t0.04847120580388957\n",
      "  (0, 77714)\t0.038810753861782794\n",
      "  (0, 77708)\t0.08992988241047055\n",
      "  (0, 77351)\t0.053764968941441754\n",
      "  (0, 76921)\t0.0756739676993348\n",
      "  (0, 75958)\t0.0555587199071744\n",
      "  (0, 75858)\t0.040064265740091545\n",
      "  (0, 75719)\t0.08992988241047055\n",
      "  (0, 75706)\t0.08992988241047055\n",
      "  (0, 75697)\t0.06157076318476815\n",
      "  (0, 74865)\t0.08992988241047055\n",
      "  (0, 74816)\t0.047951988586972076\n",
      "  (0, 74716)\t0.06557683121864555\n",
      "  (0, 73978)\t0.05712795928743642\n",
      "  (0, 73516)\t0.07281411540057674\n",
      "  (0, 73370)\t0.03659664027396496\n",
      "  (0, 73067)\t0.08531586032896392\n",
      "  (0, 72970)\t0.025080281836451345\n",
      "  (0, 72589)\t0.046408460721043036\n",
      "  (0, 72044)\t0.05956080909074401\n",
      "  (0, 71906)\t0.05703866970681566\n",
      "  :\t:\n",
      "  (1988, 5636)\t0.08791748270130299\n",
      "  (1988, 4860)\t0.03916773041098513\n",
      "  (1988, 4460)\t0.03489117605344331\n",
      "  (1988, 3986)\t0.06074653580674998\n",
      "  (1988, 3880)\t0.036089635935201904\n",
      "  (1988, 3801)\t0.04003504732885746\n",
      "  (1988, 3725)\t0.02996229879134474\n",
      "  (1988, 3656)\t0.032027148993727085\n",
      "  (1988, 3612)\t0.05378559965480081\n",
      "  (1988, 3590)\t0.05576229082971022\n",
      "  (1988, 3558)\t0.06287201249651986\n",
      "  (1988, 3253)\t0.08791748270130299\n",
      "  (1988, 3244)\t0.0834067105790539\n",
      "  (1988, 2789)\t0.06287201249651986\n",
      "  (1988, 2157)\t0.0374329662056969\n",
      "  (1988, 1858)\t0.03447307091359384\n",
      "  (1988, 1612)\t0.04385370194783875\n",
      "  (1988, 1242)\t0.05256184707626112\n",
      "  (1988, 993)\t0.04284164701922489\n",
      "  (1988, 698)\t0.08791748270130299\n",
      "  (1988, 457)\t0.06124105868277477\n",
      "  (1988, 401)\t0.06347350670540843\n",
      "  (1988, 280)\t0.035624954019551486\n",
      "  (1988, 250)\t0.08791748270130299\n",
      "  (1988, 50)\t0.032361479316171435\n"
     ]
    }
   ],
   "source": [
    "final_tfidf = tfidf_transformer.transform(bow_data)\n",
    "print(final_tfidf)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-08-07T04:53:17.846432Z",
     "iopub.status.busy": "2020-08-07T04:53:17.844911Z",
     "iopub.status.idle": "2020-08-07T04:53:17.856229Z",
     "shell.execute_reply": "2020-08-07T04:53:17.855325Z"
    },
    "papermill": {
     "duration": 0.057646,
     "end_time": "2020-08-07T04:53:17.856389",
     "exception": false,
     "start_time": "2020-08-07T04:53:17.798743",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "features_train, features_test, labels_train, labels_test = train_test_split(final_tfidf, df['Label'], test_size=0.3, random_state=5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-08-07T04:53:17.936043Z",
     "iopub.status.busy": "2020-08-07T04:53:17.935057Z",
     "iopub.status.idle": "2020-08-07T04:53:19.310612Z",
     "shell.execute_reply": "2020-08-07T04:53:19.311357Z"
    },
    "papermill": {
     "duration": 1.419074,
     "end_time": "2020-08-07T04:53:19.311607",
     "exception": false,
     "start_time": "2020-08-07T04:53:17.892533",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "features_train = features_train.A\n",
    "features_test = features_test.A"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-08-07T04:53:19.394042Z",
     "iopub.status.busy": "2020-08-07T04:53:19.393142Z",
     "iopub.status.idle": "2020-08-07T04:53:19.933621Z",
     "shell.execute_reply": "2020-08-07T04:53:19.934624Z"
    },
    "papermill": {
     "duration": 0.587091,
     "end_time": "2020-08-07T04:53:19.934891",
     "exception": false,
     "start_time": "2020-08-07T04:53:19.347800",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.5293132328308208\n"
     ]
    }
   ],
   "source": [
    "from sklearn.naive_bayes import MultinomialNB\n",
    "from sklearn.metrics import classification_report, confusion_matrix, accuracy_score\n",
    "\n",
    "djia_movement_detect_model_MNB = MultinomialNB()\n",
    "djia_movement_detect_model_MNB.fit(features_train, np.asarray(labels_train, dtype=\"float64\"))\n",
    "pred_test_MNB = djia_movement_detect_model_MNB.predict(features_test)\n",
    "\n",
    "acc_MNB = accuracy_score(np.asarray(labels_test, dtype=\"float64\"), pred_test_MNB)\n",
    "print(acc_MNB)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-08-07T04:53:20.031027Z",
     "iopub.status.busy": "2020-08-07T04:53:20.029980Z",
     "iopub.status.idle": "2020-08-07T04:53:20.354153Z",
     "shell.execute_reply": "2020-08-07T04:53:20.353324Z"
    },
    "papermill": {
     "duration": 0.375322,
     "end_time": "2020-08-07T04:53:20.354309",
     "exception": false,
     "start_time": "2020-08-07T04:53:19.978987",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAVsAAAFaCAYAAACwk/5IAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO3dd5hU5fn/8ffuCjZQdFksSBGROxEswYIaayzB9tWvLRoNxkSNDYItGhKN0fCN+tPYCyFGiQpRsaABsQaNxi5YMN6CShGRpigYY4Td3x/PmWUYZ3fPzM6cnZn9vK6Li90zZ+bcZ2b2nmfu85SqhoYGRESkuKrbOgARkfZAyVZEJAFKtiIiCVCyFRFJgJKtiEgClGxFRBKgZFtGzGy6me3V1nGUCjMbYWZ/aqNj325mv2uLYxeamR1nZo/led+CvyfNrMHM+hbyMUtBlfrZ5sfMZgEbASuB5cBk4Ex3X96GYRWMma0JXAwcB9QBHwJ/BK5098TfNNEf9J3uvllCx6sChgKnAJsDnwLPA5e4+5tmdjvwobv/Ool4mmJmFwN93f34BI51OwU4ZzPrDXwAfBFtWgzc4u6XRbc3AFu6+8wWHufHwEnuvltr4kmKWratc4i7dwK2A74D/LKN48mZma3RxE33AvsABwKdgR8REs+1RYihysxK7b14LfBzYBiwIdAPeBA4qNAHauY1KLq2PDbQJfr7ORa4yMwGt2EsRaeWbZ6ilu1J7v5E9PsVQH93Pyj6fWfgD8BWwGzg5+4+JbptQ+Aq4PvA2sDT7n5YdNvBwO+A3sDbwKnu/kb6MaPt7wHd3f2T6LbvAI8Dm7j712b2E+A8YGPgJeAUd58d7dsAnAkMB9Zw980zzm0fYCKhdTE3bfsg4J+AuftMM5tCaO3tAxgwBTgxLabmnoMpwHPAXsBAYGtgd+AXwGbAIuBydx9lZusSWj9rAv+OwulHSP593f34tNbSj4FLgXWAq919ZHS8tYFbgP8BPgZuA4Zlaymb2ZbAO8Au7v5S5u3RPrcTWma9gT2i1+SH7v5edPu1wOHA+sAMYLi7/yO67WJgAPCfKJ6zgTcICf7bwJfAfcDZ7v7f6D79gWuA7YGvo31fAx4CqoCvgPfcfVszWz963g8E6qNz/Y27r4xagycT3hMnADcBM4laiFGL/g+EbzRrRq/bD4FdgRuBBuC/wN/d/ZD0vwMzqwHOB34KdAPeBQ5Lfw9F55J6rTq4+4po28vA3e5+ZXrLNjqX64EDCK/9aOD/CO+3qUCH6Pla4e5dsr1WpaLUWhNlycw2I7wZZka/dyckq98RWkXnAveZWV10lzsIyaA/4U15dXS/gcCfgZ8BtcAo4KHoK30jd/+IkOSOSNv8Q2B8lGgPA0YQ/tjrgH8A4zLCPgwYREiEmfYDXsz8I3H3FwnlhH3SNg8BfgJsCqwArov5HMCq1nJnwh/1QuBgYD3gROBqMxvo7l8Qnt+P3L1T9O+jLHED7Eb4Q9yH0Fr6drT9N4TE2Cc6v+a+du9D+LqcNdGmORb4LbAB4bUfmXbby4RvPBsCY4F7zWyttNsPBcYDXYC7COWos4CuwC5RDKcDmFln4AlCqWpToC/wpLtPJiSeu6PnZNvosccQXou+hG9c+xM+pFMGAe8T3nvpMRPtuwfhw6wL8ANgibv/MYrziuhYh2R5Ps6OnpMDCa/hT1j14ZhV9K3mu4S/halZdrme8IHVB9iT8H470d3/BZwKPB/FU9KJFqAtv0JUggejT+FOwFOEP2gIf8iT3H1S9PvjZvYKcGB0IeIAoNbdP41ufzr6/2RgVJTUAMaY2Qhg57R9UsYSEuzoqDVyDKE1AiFZ/z56Q2Jm/weMMLNeqdZtdPsnTZxXV2B+E7fNj25PucPd34qOcyEwzcxOaO45ICQDgNvdfXraY01M+/np6LnandCCi+u37v4l8LqZvQ5sC/wLOBo4LXrOPzWz6wg16Wxqafr8092fSshmdhehRQiAu9+Ztt9VZvZrwofA69G25939wejnL4FX0/afZWajCMnlGsIH0MfuflV0+3+AF8nCzDYivL+6RM/DF2Z2NeFDbVS020fufn308wozS3+Irwkfft8CXkq9h2I6CfiFu3v0++vN7Uz4ttJA+KZxgbs/mXEuNYRk/x13XwYsM7OrCB/St+YQV0lQsm2dw6KvT3sSkl9XYCnQCzjKzNI//TsAfwd6AJ+kJdp0vYATzGxo2raOhNZMpvHA9Wa2KbAl4U37j7THuTZ6Y6ZUAd0JLUiA1VqtGRZHj5nNJtHtKemPM5twnl1p/jnIdl/M7ADCB1Y/wreudYA3m4kzm4/Tfv434YMQwnOYfrzmzn8J4TzzPRZmdg4h+WxKeG3WY/UPqcxz70dI1jsQznsNViXgHoSyURy9CM/z/LQkWk3Mc3f3p8zsBkLJoKeZPQCc6+6fxzh2LnECdE2VEZq6nfD+n522bTbhfVx2lGwLwN2fjmp4VxK+ns8ltPhOztzXzDYBNjSzLu6+NOPmucDIVJ2xhWMujVp+RxPqfOPSegmkHueuZh6iuWL9E8BwM+uRUbPdifAH9VTavj3Sfu5JaBktppnnIFsMUankPsLXxAlROeRBwodES/HGMZ9QC347S9yZngRuNLMd3P2VXA9kZrsTapf7ANPdvd7MPmXVucA3z+dmwtfoY919mZkNB46MbptL+HqeTebjzCXUb5tLZM0+l+5+HXCdmXUD7iHU/i9s6X7RsbcA3mphv7gWE95PvVj1uvUE5kU/l9UFJyXbwrmG8PVvO+BO4GUz+z4hcXUglAJmuvuHZvYIcJOZnUHoNraLuz9DKP4/YGZPEC5grEO4gPRM9DUq01jCH3VPVq+j3gJcambT3H16dJFhf3e/N86JRK31Jwk11hMJF4t2JNSab3b3GWm7H29mfwFmAZcQ6sYrzazZ5yDLYTsSLsgsIny1PYBQP0z94S4Aas1sfXf/LM55ZLgH+GV0IWYdwgXCps5/hpndBIwzs5MJFwWrCR+kvVNdlJrRmVAzXQSsYWYXEFq2Ld3nc2C5mX0LOC26P8DfgD9ECfhmwnO1VVRuWgDsZ2bV7l7v7vOjD+GrorLOckLXtc3cPbMU9Q1mtmN0rq8RLgD+h1BPJjpWn2bu/ifC++5tQg17a2Ceuy9p6bjZRO+je4CRZjaEUP8+m9CoScWzmZl1TF1ILGW6QFYg7r4I+AtwYdQaPJRwkWoR4RP/PFY93z8ifGK/Q7goNDx6jFcIddsbCP06ZxKurjflIcLX/QXu3lgfc/cHgMuBv5rZ54SEdUCOp3QE4Sv/ZMIf7J2EOtnQjP3uAG4nfKVei9BVihjPwWqiD5NhhKT4KaEe/VDa7e8QLvK9b2ZLo/JJLi4hXNz7gJD8xxNagE0ZRngdbiSUht4D/hd4OMaxHgUeIVyNn01IWM2VLSBcQPwhsIzwoXt36oboudkPOITwPM8A9o5uTn2ALjGzVG17CCEhv014LscTrywC4UNhdHS/2YSSSiq53QpsFT3/D2a57x8Ir99jhA+OWwm9bVpjKCHpvw88S2hg/Dm67SlgOvCxmS3OfvfSkVjXLzO7kvAH3BvYOnVRJWOfGsLV7MGErwiXuXubjBCSlkXdt+4sx9fIzE4DjnH3Pds6FmkfkiwjPEjoG/iPZvY5jtBdZUvCFeGpZvaEu88qfnhSyaJaeR9Cl7ktgXMILVeRRCRWRnD3ZzP7bWbxA2B0VHtaREjQRxU/OmkHOhK6Pi0jfP2cQOjQL5KIUrtA1pPVu3nMofmrxtKG3H2vto4hrqh/8YC2jkPar1JLtoWwJuHK+XxWXUUVEWmtGsKFxpdp/uJqVqWWbOcQ+tS9HP2e2dKNY0earwuLiLTG7oSeETkptWR7L3Cymd1PuEB2GGGcdi7mA3z66RfU15dVn+dYams7sWRJRczimJXOr7xV6vm98PYCnp42jyuG7gHxhnJ/Q2LJNhqLfjhhFqonzGyJu/c3s0nARVEf0zsIk2SkOs1f4u7v53iolQD19Q0VmWyBij2vFJ1feavE8/v7qx/y6fLGykFe5cnEkq27DyPq8J6x/cC0n1cSRs6IiJSUPpu2NAiweRpBJiKSgFKr2YpUvA03XJeamspt59TVdW7rEFpt5cp6Pvnki5Z3zIGSrUjCamqqWbQo27xCUiqK8YFRuR+vIiIFMGXaPHxu5myouVOyFRFpwpRp8/jL5LDwxHZb1rWwd/NURhARyeLyu15rbNEOGWzsvNVGrXo8JVsRAeDzzz/nsMMGc+ihR/Dzn5/TuP3WW0fx5ZdfcuaZwxu33Xff3bzzzr/41a8uBmDOnNncfPP1zJw5g7XWWpO1116H44//MXvssVfWYzU0NDB8+OnMnPkuEyeuWnps/vyPuOqqy5g370Nqamo45pjjOPjgw1a7b319PRdeeD7vv/8eHTuuyQYbbMB5542ge/ewUPKZZ57CggULWHfddQE46qhjOOig/1ntMR555G+MHHkxl19+Nd/97u5ZY/S5S7EeXRjUfyP22q71K/Eo2YoIAI8//gj9+2/NE088yumnD6NDhw6x7rd48WLOPPMUTj99GL///ZWN215++YUm73PffXez8cabMHPmu43bGhoaGDHiXE488RT22GMvGhoaWLo021J9cMABB7PrrrtTXV3NfffdzRVXjOTaa29uvH348HObTKILFy5gwoT76d9/62bPa8hgK0iSTVHNVkQAmDjxIU444af06dOXZ59tcQWdRvfffw/f+c72DB58UOO2rl27csABB2fdf+7cOTz55GMcf/yPV9v+yisvss466za2hquqqthggw2/cf/q6mp2221PqqtD+urffxs+/jj+CNorrhjJ0KFnt/hhUshEC2rZipSU596cz7Nv5DX0HoDdttmE724ddwWcVWbMeJfPP/+c7bffkU8+WcLEiQ+x9977xrrvu+++w0477Rxr3/r6ei6//Hecffb5rLHG6unngw8+YL311ufXvz6fefPm0r17D4YOPYuNNtq42ce8//572G231adQuemmaxk16gb69u3HaacNpa6uGwAPPDCezTffgv79k59tUy1bEWHixAkMHnwQVVVV7Lnn3kyf/haLFi0EQgszm9T2XFbWGjfuDrbbbiBbbmnfuG3lypW89trLnHzyadx221gGDNiakSMvbvbxxo79C7NmfcDJJ5/euO3CCy/hrrvGc9ttY+nZsxcXXfRLAD76aB4PP/wAJ530s/gBF5BatiIl5Ltb59cybY2vv/6axx+fTMeOazJ58kQAVqxYwSOP/I0hQ35Cly4bfONr+tKlS+nSZQMAzL7F229Pj3Ws11+fysyZM5g8eSIrV65k2bJlHHnkIYwZM46NN94Ys2/Tq1dvAL7//QO59dZRTT7WfffdzeOPT+baa29mrbXWatyeagnX1NRw9NHHcttto6mvr+ett95k8eLFHHdcWPzlk0+WcNlll/Kzn53BwQcfGu/JagW1bEXauWeemULPnr154IFJjB//MOPHP8zVV9/ApElhIeGBA3fgxRefZ+HCBQB8/vlnPPXU442lg8MPP4pXX32Zxx6b3PiYixcv4qGHHvjGsa644hruv38i48c/zE03/YnOnTszfvzDrLtuJ3be+bssXLiAxYvDQrkvvPBP+vbdMmvMEybcz4QJ9/OHP9zIeuut37h9xYoVfPLJqpXTH3/8Ufr02YLq6mr2338wDz30aOM5brXVAC644MJEEi2oZSvS7k2a9DD777/6SvcDBmxDfX0906a9xnbbDWTYsLO54IJzqK+vp6GhgSOO+AE77LATAF271nHDDX/k5puvY/Tom1l77bWirl8n5BTH2muvzfDh53HuucNoaGhg/fXXZ8SIi4GQvM899+fcfvtY/v3vL7jyyt+z8cabcNZZZwDQoUMHRo8ew9dff8155w1nxYqvaWhooGvXbvz2t//X+iepABJbyjxBvYEPlixZXpHzatbVda7ocfXt4fyAij7HSpDtfVhdXUVtbSeAzYFZuT6myggiIglQshURSYCSrYhIApRsRUQSoGQrIpIAdf0SSdjKlfUVsXRMJVu5sr7gj6lkK5KwQq9tVUpKreteak7aQs/glQ8lWxGpGFOmzePF6Qsaf5+zcDnWo0ubJ1pQshWRCpBKsqmVFaxHFwB6duvEoP6tW2GhUJRsRaSspa8TVsiVFQpNyVZEylZ6oi2Fumxz1PVLRMpWqj5b6okWlGxFpMyVygWwlijZiogkQMlWRCQBSrYiIglQshWRsjRl2rzGfrXlQF2/RKSsZA5gKJVBCy1RshWRsvLi9AWNw3BLdQBDNkq2IlIWUi3aOQuX07NbJ84/bmBbh5QTJVsRKXnZhuSWGyVbESl55TRSrCnqjSAiZaFcRoo1RclWREpauXXxaoqSrYiUtFQJoRzrtOmUbEWk5JV7CQGUbEVEEqFkKyIlq1LqtaBkKyIlrFLqtaBkKyIlKtWqrYR6LSjZikiJqqRWLSjZikgJqrRWLSQ8XNfM+gFjgFpgCTDE3Wdk7NMNuA3oAXQEngKGufuKJGMVkbZTaa1aSL5lewtwo7v3A24ERmXZZwTwL3ffBtga2B44PLkQRaQUVFKrFhJMtlGLdSAwLto0DhhoZnUZuzYAnc2sGliT0Lqdl1ScIiLFkGTLtgcwz91XAkT/fxRtT3cp0A+YD3wMPOruzyUYp4i0oUrqW5uuFC+QHQW8AWwCdAf2MLMj2zYkEUlKJdZrIdkLZHOB7mZW4+4rzawG2DTanm4o8BN3rwc+M7MJwN7A+FwOVlvbqRAxl6S6us5tHUJR6fzKW2vOb/Lzs/C5SxmwRS1H7fetwgVVAhJLtu6+0MymAccCd0b/T3X3RRm7fgAMBl4ys47AvsD9uR5vyZLl1Nc3tDLq0lNX15lFi5a1dRhFo/Mrb605v/TVGAZu2bXknqfq6qpWNeKSLiOcCgw1s3cJLdhTAcxskpntEO0zHNjdzN4EpgHvAqMTjlNEEpSeaMt5NYbmJNrP1t3fAQZl2X5g2s/vAfslGZeItJ32kGihNC+QiUg7Ugnri8WhBR9FpOBSy47HMWfh8oobwJCNkq2ItFoquXboWMPX/13Z2E/WenRp8b49u3WquG5e2SjZikirpNdcB2xRC4QkO6j/RhXfWs2Fkq2I5CSzRJBqxQ4ZbBy137dKrstWqVCyFZGcvDh9AXMWLqdnt9DnVK3YeJRsRSS29Hlmzz9uYFuHU1bU9UtEYkmvzbaHC1qFppatiDQrVaNNr82qZJA7JVsR+Yb0i2Dp3bhUm82fkq2IfEP6RTAl2cJQshURYPXWbCrR6iJY4egCmYgAq1qz0H5GdSVJLVsRaaTWbPGoZSsikgAlWxGRBCjZiogkQMlWRCQBukAm0o5l6+4lxaGWrUg7pu5eyVHLVqSdU3evZKhlKyKSACVbkXYqNTetJENlBJF2JnPKRNVpk6FkK9LOpC6KaTavZCnZirQTqRatZvRqG0q2IhUus2yQatFKsmInWzPbDzgG6Obuh5jZDsB67v5U0aITkVZT2aA0xEq2ZjYU+DnwJ+DIaPOXwHXArsUJTURaQ2WD0hK369dwYF93vwyoj7a9A1hRohKRVkmthOtzl2pkWImIW0boDMyNfm6I/u8A/LfgEYlIq6QvOa6VcEtH3GT7DHABMDJt2zDg7wWPSETyoiXHS1vcZDsUeNjMTgY6m5kDnwOHFC0yEWlW+oxdoCXHS12sZOvu881sR2BHoBehpPCSu9c3f08RKZb0i1+gJFvq4vZGmODuhwIvRf9S2+9398OLFZyIfJN6GZSnuL0R9m5i+14FikNEYkpPtOplUD6abdma2SXRjx3Tfk7pA8wuSlQi0iy1aMtPS2WEHtH/1Wk/Q+j+NRe4uAgxiYhUnGaTrbufCGBm/3T30cmEJCJNSc1Baz26tHUokqO4vRFGA5hZZ6ArUJV22/vFCU1EMqW6eqlWW37i9kb4NjAW2JZQQqhi1UiymuKEJiKZfWlTE8qoe1f5idsb4WbCaLENCYMZNgBGAScUKS6Rdi99foMU9UAoX3FHkG0L7OfuX5tZlbt/ZmbnAW8BdxYvPJH2K9Wi1bDbyhA32f6HMPHM18BiM+sJfArUFiswkfYovWygkkFliVtG+AdwdPTzeOAR4GlAE4eLFFBqwAKoZFBp4vZGODrt1xHAdKATMKYYQYm0ZxqwUJnitmwbuXu9u98B3AqcWPiQREQqT4stWzPbB9gOmOnuE8xsDeB04HzgE+DGuAczs36E1nAtsAQY4u4zsux3NHAhq7qY7evuCzL3E6kkGrBQ2Zpt2ZrZ+cAEQr32LjO7AvgncBRwirtvnePxbgFudPd+hCQ9KssxdyAMA97P3QcAuwGf5XgckbKjAQuVraWW7c+APd39VTPbGXgOONfdr871QGbWDRgI7BdtGgfcYGZ17r4obdezgCvd/WMAd1eilYqX3qpV74PK1FLNtqu7vwrg7i8AXwHX5HmsHsA8d18ZPd5K4CNWn+AGYCugj5k9Y2avmdmvzawKkQqmVm3li1OzrSLUTqsI/W0xs8YkXYTVGtYAtiG0gDsCk4E5wF9yeZDa2k4FDqt01NV1busQiqo9nl+HjjUM2KKWo/b7VhtEVFiV/vrlq6Vk2wlYkfZ7VdrvqYtXcedGmAt0N7Mad19pZjXApqxatTdlNjDe3b8CvjKzCcBO5JhslyxZTn19Q8s7lpm6us4sWrSsrcMomvZ4flOmzeOt95ZgPbqU/blX8utXXV3VqkZcS8l287wfOYO7LzSzacCxhCG+xwJTM+q1ECa8OdDM7oji24cwkEKkomSuhqsSQmVraT7bQq/EcCowxswuIgz3HQJgZpOAi9z9FeCvwA7A20A98CihT69IRUmNFtNCje1DVUNDxX3V7g18oDJCeWoP53fv4+9U7IKNlfz6pZURNgdm5Xz/QgckIs3Tgo3tU9xZv0SkACY/P6uxP20ltWilZTm1bM2sRzS4QUTy8PTUDwFdDGuP4i6L05Mw4ms7QnevTmZ2JDDY3U8qYnwiFUejxNqnuC3bUcBEoDNhAnGAx1k19FZEWpDqTyvtU9xkuxNwWTRarAEa5yxYv1iBiVSS1HpioBJCexU32S4A+qZvMLOtCMNoRaQFqbkPzjhyW5UQ2qm4yfZK4G9mdiKwhpkdC9wNXF60yEQqjPXowuBderd1GNJGYiVbd/8z8AvCPLZzCSO/LnT3u4oYm4hIxYjbG6HG3R8EHixyPCIiFSluGeFjM7vJzL5b1GhERCpU3BFk+xNm6RpnZvWEPrdj3f3NokUmIlJB4tZsp7r7L9y9J3ACsAHwpJm9UdToRCpAaskbad/ymYjGgX8RLpT1Lmg0IhVIS94IxL9A1gU4AvghsDPwGKHb10PFC02kPKUmBU9JzVmr/rXtW9ya7UeEJczHAodrxVuRpqVPoQhoKkUB4ifbLdx9flEjESlj6a3ZSpwUXFqvyWRrZnu4+zPRr982s29n28/dnypKZCJlJL01q5asZNNcy/YmYED0c1NrgDUAfQoakUiZUmtWmtNksnX3AWk/F2yVXZFKk+raZT26tHUoUsJidf0yswlNbL+/sOGIlBdNnShxxb1AtncT2/cqUBwiZSV1QSw1WGHIYFPXLmlWs8nWzC6JfuyY9nNKH2B2UaISKXGpC2LWowuD+m+kRCstaqll2yP6vzrtZwgXxuYCFxchJpGSlWrRqnuX5KrZZOvuJwKY2T/dfXQyIYmUpvT6bKpFKxJXc/1se7v7rOjXJ80saxcvd3+/GIGJlJrUoAXVZyUfzbVs3ySspgswk1A6qMrYpwGoKUJcIiVJcxxIvprrZ9s57ed8ZgcTEZFIXknUzPqYWa9CByNSqjQnrbRW3EEN48xs1+jnE4HpwNtm9tNiBidSKjQnrbRW3JbtPsAr0c9nA/sCOwEXFCMokVKkeq20Rtxk29Hd/2tm3YEN3f05d58O6GNeKp5KCFIIcYfrTjOzXwK9gIkAUeL9vFiBiZQKlRCkEOK2bH8KbA2sDVwYbdsFuKsYQYmUivQZvVRCkNaI1bJ19/cI64+lbxsPjC9GUCKlQDN6SSHFLSOkeiH8COgOzAPucPfbihWYSFvTiDEppLir6/4KGAJcRZjpqxfwCzPb1N1HFjE+kcSlTzaj8oEUStyW7UnAXu7eOKWimT0KPAMo2UpFSZ/VS+UDKZS4yXZdYFHGtiWEC2YiFUfTJ0qhxU22k4G7zOwCYA6hjDASeLRYgYkkKdtS5CKFFLfr15nAMuB1YDkwDfgCGFqkuEQSlSodACofSFG02LI1sy6EJXDOAH4MdAUWu3t9cUMTSZZKB1JMLa1BdhBwD6E2uww4zN3/nkRgIsWm0oEkqaWW7aXA+cCfgZMJddpdix2USDFlroxrPbqodCBF11Ky7ePuNwCY2Y3Ar4ofkkhxZEuyWhlXktJSsm28gObuK8ws9ogzkVKSbbFGJVlJUkvJcx0zeybt984Zv+PuexQ+LJHCSU+0GnorbaWlZJu5EsOtrTmYmfUDxgC1hEERQ9x9RhP7GjAVuMndz23NcaV90xwHUgqaTbbuPqbAx7sFuNHd7zSz44FRwPcydzKzmui2Bwt8fGmnNMeBtLXEVs01s27AQGBctGkcMNDM6rLsfgHwN+DdhMKTCqVVFqRUJLlEeQ9gnruvBIj+/yja3sjMtgG+D1ydYGxSgTQfrZSSkupdYGYdgNHAie6+MpRt81NbW7kd1OvqOrd1CEWV7/lNfn4WT0/9sPH3t95bAsAZR27L4F16FyCywtDr1z4lmWznAt3NrCZKpDXAptH2lE2ALYBJUaLtAlSZ2XrufkouB1uyZDn19Q0FCr101NV1ZtGiZW0dRtG05vyeeHH2aiPBUl28tu9bWzLPmV6/8lVdXdWqRlzcycPXBC4CjgVq3X19M9sf6Jca9NASd19oZtOix7gz+n+quy9K22cOYe6F1HEvBjqpN4K0JH2tMM1vIKUobs32amAAcByQai5OB07L8XinAkPN7F3CjGGnApjZJDPbIcfHEmmkFXCl1MUtI/wv0NfdvzCzegB3nxctZx6bu78DDMqy/cAm9r84l8eX9kdL2L3EIDQAABBOSURBVEi5iJts/5u5b9Rla0nBIxKJKdsQXJFSFTfZ3guMMbOzAMxsE+Aa4K/FCkykJRoZJuUkbrIdAVwBvAmsA8wgdNH6bZHiEskqcw5alQ6kXMRKtu7+X2A4MDwqHyx298rrVyUlL33lW81BK+UkbtevPhmbOqcGHLj7+4UOSiRT+oUwLV8j5ShuGWEmoctXVdq2VMu2pqARiaRpasJvkXITt4ywWn9cM9sY+A3wj2IEJZKS3q1LE35LOctruK67f2xmwwmzco0tbEjS3mVbiFFlAyl3rZn1ywg9E0QKKtWaBXQRTCpG3Atk/2BVjRZCku0PXFKMoKT9mvz8LM1xIBUpbhnhTxm/fwG83tSSNiL50PyzUslaTLbRVIjfA05x96+KH5K0N5k9DjQiTCpRizXbaEWF/YH64ocj7VF6j4MzjtxWiVYqUi5TLP42WklBpGBS89CmehyU0ooKIoXUbBnBzI5193GEuWc3Bs42s0WkXSxz957FDVEqmeahlfaipZrtKMIquMcnEIu0A+l9aEGTyUj70VKyrQJw96cTiEUqXOb8s6B+tNJ+tJRsa8xsb1afE2E17v5UYUOSSpHZilVvA2nPWkq2awK30nSybQAyZwQTydqK1fwG0p61lGy/cHclU8mZVlEQWV1eE9GIZKNVFESa1lI/2yZrtSKZNIGMSNOabdm6e+ekApHylhqcoAlkRLJrzRSLIo00OEGkearZSt5UoxWJTy1byZtqtCLxqWUreVGNViQ3atlKXlSjFcmNkq3kTTVakfiUbEVEEqBkKzlL1WtFJD5dIJPYMtcKU71WJD4lW4ktfa0wzd4lkhslW2lRqkU7Z+HyxrXCRCQ3SrbSrMx5aVU6EMmPkq1klVmf1by0Iq2jZCtA00vYqD4rUhhKtgKwWk0WlGRFCk3Jth3LnLVLF79EikeDGtqp1IWvVLlAs3aJFJdatu1Qeg8DXfgSSYZatu2QVr4VSZ6SbTuTPg+tEq1IcpRs2xnNQyvSNpRs2yG1akWSp2QrIpKARHsjmFk/YAxQCywBhrj7jIx9LgSOAVZE/0a4+6NJxikiUmhJt2xvAW50937AjcCoLPu8BOzo7tsCPwHuNrO1E4xRRKTgEmvZmlk3YCCwX7RpHHCDmdW5+6LUfhmt2DeAKkJL+MOkYq0kmXMepA/JFZHkJNmy7QHMc/eVANH/H0XbmzIEeM/dlWjzlJrzIEUjxUTaRsmOIDOzPYFLWdUSlhyl96nVnAcibSvJZDsX6G5mNe6+0sxqgE2j7asxs12AO4FD3d3zOVhtbeV+Va6r6xxrv9dmLAZg30G9Yt+nFJRTrPnQ+bVPiSVbd19oZtOAYwmJ9Fhganq9FsDMdgTuBo5099fyPd6SJcupr29oTcglqa6uM4sWLWtxvynT5vHWe0uwHl3Yvm9trPuUgrjnV650fuWrurqqVY24pMsIpwJjzOwi4FNCTRYzmwRc5O6vADcBawOjzCx1vx+5+5sJx1qWtAKuSGlKNNm6+zvAoCzbD0z7ecckYyp3WmFBpDyU7AUyiUcrLIiUByXbMqbeBiLlQ3MjlDHN4CVSPpRsy5xm8BIpD0q2ZSpVQhCR8qBkW4bS1xBTCUGkPCjZliGtISZSfpRsy8zk52dpDTGRMqSuXyWuqUELKh+IlBcl2xKWXpu1Hl0AGLBFLQO37KpWrUiZUbItIU21YtNrs5U80YdIJVOyLQGZk8ekWrEaeitSOZRsS0BqfgMlV5HKpWTbhlIt2tREMprfQKRyKdm2kcyLX+pdIFLZlGzbiAYmiLQvGtTQhjQwQaT9UMs2YZl1WhFpH5RsE6Q6rUj7pWSbgMx+tKrTirQ/SrYJUD9aEVGyLZL0obfqRysi6o1QJKnWLEDPbp1UnxVp59SyLSK1ZkUkRcm2wNS1S0SyURmhwNITrUoHIpKilm0RqHwgIpnUshURSYCSbQFNmTavceCCiEg6JdsCSvWrVa1WRDIp2RZIqlWrmbxEJBtdIGulzHkP1KoVkWyUbFsh2yxeatWKSDZKtq2g1RZEJC4l2zykjxJTjVZE4lCyzZEmABeRfCjZ5iA90ap0ICK5UNevHKhGKyL5Usu2GekTgAOq0YpI3tSybUKqZJA+/FYzeYlIvtSyzUK1WREpNLVsMyjRikgxqGUb0XLjIlJMSrYRLTcuIsWkZMvqM3ZphQURKYZ2nWw1Y5eIJKVik+0Lby/g769+2Ow+qSSr0oGIFFuiydbM+gFjgFpgCTDE3Wdk7FMDXAcMBhqAy9z9T7kea9qMRS0uJ64kKyJJSbplewtwo7vfaWbHA6OA72XscxzQF9iSkJSnmtkT7j4rlwN9tPjfWuVWREpGYv1szawbMBAYF20aBww0s7qMXX8AjHb3endfBDwIHJXr8Tbtuo5qsCJSMpIc1NADmOfuKwGi/z+KtqfrCcxO+31Oln1adOqhA1QeEJGSUYkXyGoAqqur2jqOoqnkcwOdX7mr1PNLO6+afO6fZLKdC3Q3sxp3XxldCNs02p5uDtALeDn6PbOl25JNADbYYN1Whlu6amubvuhXCXR+5a3Sz4+QY97L9U6JJVt3X2hm04BjgTuj/6dGddl09wInm9n9hAtkhwF75HCol4HdgfnAylYHLiIS1BAS7cst7ZhN0mWEU4ExZnYR8CkwBMDMJgEXufsrwB3AICDVJewSd38/h2N8BTxbuJBFRBrl3KJNqWpoaChkICIikoWmWBQRSYCSrYhIApRsRUQSoGQrIpIAJVsRkQQo2YqIJKBsh+smOV1j0mKe24XAMcCK6N8Id3806VjzEef80vY1YCpwk7ufm1yU+Yt7fmZ2NHAhUEV4f+7r7guSjDUfMd+f3YDbCPOadASeAoa5+4qEw82JmV0JHAH0BrZ297ey7JNXXinnlm1qusZ+wI2E6RozpU/XuAtwsZn1TizC/MU5t5eAHd19W+AnwN1mtnaCMbZGnPNLvalHEWZ+Kyctnp+Z7QBcDOzn7gOA3YDPkgyyFeK8fiOAf7n7NsDWwPbA4cmFmLcHCSNWm5siIK+8UpbJNunpGpMU99zc/VF3/3f06xuE1lFtYoHmKYfXDuAC4G/AuwmF12o5nN9ZwJXu/jGAu3/m7v9JLtL85HB+DUBnM6sG1iS0buclFmie3P1Zd8+cryVTXnmlLJMtCU/XmLC455ZuCPCeuze/DlBpiHV+ZrYN8H3g6sQjbJ24r99WQB8ze8bMXjOzX5tZOUyXFff8LgX6EeYo+Rh41N2fSzLQIsorr5RrspWIme1JeGMf29axFIqZdQBGA6em/qgr0BrANsB+wJ7AAcCP2jSiwjqK8I1rE6A7sIeZHdm2IbWtck22jdM1QmNtr7npGlN6Ztmn1MQ9N8xsF8IMaoe5uycaZf7inN8mwBbAJDObBQwnzAT3x2RDzUvc1282MN7dv3L3ZcAEYKdEI81P3PMbCtwVfdX+jHB+eycaafHklVfKMtm6+0IgNV0jtDxdY3VUUzoMuC+5SHMX99zMbEfgbuBId38t2SjzF+f83H2Ou3d1997u3hu4hlAjOyXxgHOUw3tzLLC/mVVFLfl9gNeTizQ/OZzfB4Sr9ZhZR2Bf4BtX9stUXnmlLJNt5FRgqJm9S/gUPRXCdI3RlV4I0zW+T5iu8QVyn66xrcQ5t5uAtYFRZjYt+rd124SbszjnV87inN9fgYXA24TkNR24tQ1izUec8xsO7G5mbxLO711Caaikmdl1ZvYhsBnwhJlNj7a3Oq9oikURkQSUc8tWRKRsKNmKiCRAyVZEJAFKtiIiCVCyFRFJgJKtiEgClGylKMxsipmd1NZxNMfMjjOzx5q5fXczK5eReVLi1M9WWhQNmd0ISJ+noJ+7f9TMfaYAdxZy/uDoMXcmzN/7H+AZ4Ax3n1+gx28AtnT3mYV4vGaOczHwK+Arwrm8DZzj7s/HvH8icUphqWUrcR3i7p3S/jWZaIvsTHfvRJhRqgvlNytYyt3ReXQF/k4YAioVrGxXapC2ZWYbEIYtDiK8j54jzNL1jWkezawvYSjqdsDXwJPu/oPotm8B1xMml14EXOju97R0fHf/xMzuA06LHmdX4FpCEn4X+Lm7/zO67cfARUAdsBj4tbvfFW0/yd13M7Nnood+PWo5/hRYQGidb2ZmFwA7uHvjzFVmdi1Q5e7DzGx94A/AgUA9YZWC37Q0a5m7rzCzu4ARZlbn7ovMbKfoXL4NfEkYd3+2u/83W5zufreZHQz8jrDCwNuE1+KNlp5HSY5atpKvakJC6UWY9ehL4IYm9r0UeAzYgDDm/HoAM1sXeJwwKUs3wqQmN5lZ/5YObmZdCcuXTDWzDYGJhKVKaglJb6KZ1UbHuA44wN07A7sSxuqvxt33iH7cNmq5352xyzjgQDNbLzp+DXB0FDuEZWJWEGbw/w6wP9BizTqapGUIYXmZT6PNKwmTi3clrASwD3B6U3Ga2UDgz8DPovMfBTxkZmu2dHxJjlq2EteDZpZaP2qKu68205GZjSR8Hc7ma0JS3jRq+T4bbT8YmOXut0W/vxa1Vo8kTMySzXXROlFfAFOAs4GDgBnufke0zzgzGwYcQvh6Xg8MMLM5UX035xqvu882s9cIMzz9Bfge8G93f8HMNiLMR9vF3b8EvjCzq4FTaGLJH+DoqDXaGVgKHJFan8vdX03bb5aZjSLMeXtNE491MjDK3V+Mfh9jZiMI9e2ncz1XKQ4lW4nrMHd/IvWLma1DqJcOJrRYISyDUpPlq/MvCK3bl8zsU+Aqd/8zIQEPMrOlafuuQShPNGVY5kU3M9uUb64ZNRvo7u5fmNkPgHOBW83sOcLFqHdinHOmsYTW91+AH7KqVdsL6ADMN7PUvtU0P8fpPe5+fNRCv49QRpkSnU8/Qut8B2AdwnPyahOPkzr+CWY2NG1bR8I8s1IilGwlX+cABgxy94/NbDvCKrjfWNolWmfrZAAz240wdd0zhGT0tLvv18pYPmL1yZwhlDYmR8d/FHjUwoKYvyNM9bd7Hse5F7jKzDYD/pfwFR/CeXwFdM119Vh3X2xmPwNeNrOxUcv7ZsJzeay7LzOz4YTWflPmAiPdfWSO5yMJUrKVfHUm1GmXRjXT3zS1o5kdBTwflRA+JSwGuJKwmONlZvYjwvyuEC6iLXf3f+UQyyTgejP7IXAPoZa7FfC36Cv+IODJKN7lrN6FLd0CoA+QtUtVdPFqCqFW/UEqRnefH/XXvcrCEvPLgc2Bzdy9xa/x7v6OmT1K+AZwFuG5/RxYHl1API1w8bCpOEcDD5jZE4RVl9cB9gKeiVaBkBKgC2SSr2sIk5cvJkygPLmZfXcEXjSz5cBDhJ4CH0SJYH/gGELr9GPgcsJqrLG5+xJC/fccwoWmXwAHu/tiwnv8nOjxPyHUPk9v4qEuJtQ7l5rZ0U3sM5aw6sDYjO1DCF/d3yZ8oIwnLO8T1/8DTrGweu25hDLFMkIizbxYt1qc7v4K4ZvDDdGxZwI/zuHYkgANahARSYBatiIiCVCyFRFJgJKtiEgClGxFRBKgZCsikgAlWxGRBCjZiogkQMlWRCQBSrYiIgn4/5SuZMhADOenAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 360x360 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "from sklearn.metrics import roc_curve, auc\n",
    "\n",
    "fpr, tpr, thr = roc_curve(np.asarray(labels_test, dtype=\"float64\"), djia_movement_detect_model_MNB.predict_proba(features_test)[:,1])\n",
    "plt.figure(figsize=(5, 5))\n",
    "plt.plot(fpr, tpr)\n",
    "plt.xlabel('False Positive Rate')\n",
    "plt.ylabel('True Positive Rate')\n",
    "plt.title('Receiver Operating Characteristic Plot')\n",
    "auc_knn4 = auc(fpr, tpr) * 100\n",
    "plt.legend([\"AUC {0:.3f}\".format(auc_knn4)]);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "papermill": {
     "duration": 0.036611,
     "end_time": "2020-08-07T04:53:20.427762",
     "exception": false,
     "start_time": "2020-08-07T04:53:20.391151",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  },
  "papermill": {
   "duration": 516.692689,
   "end_time": "2020-08-07T04:53:20.678868",
   "environment_variables": {},
   "exception": null,
   "input_path": "__notebook__.ipynb",
   "output_path": "__notebook__.ipynb",
   "parameters": {},
   "start_time": "2020-08-07T04:44:43.986179",
   "version": "2.1.0"
  },
  "widgets": {
   "application/vnd.jupyter.widget-state+json": {
    "state": {
     "08c712ccba324989960094911ff3b69b": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "ProgressStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "ProgressStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "bar_color": null,
       "description_width": "initial"
      }
     },
     "14e18380daec4b9581fe37f7d915101a": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "1531f3e738f44ceb8e2824b9b1499dfd": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "173f25b1ba434795ba6a530ca9488fcb": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "197cec13dd234875bcffcb032b372bd2": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "207c78f0351b48afa46af1910d85a401": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "248c2d0ba7fc43938ecc92e010238710": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "FloatProgressModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "FloatProgressModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "ProgressView",
       "bar_style": "success",
       "description": "Downloading: 100%",
       "description_tooltip": null,
       "layout": "IPY_MODEL_29e9f077056b486ba2d1fdd02f1caa22",
       "max": 442.0,
       "min": 0.0,
       "orientation": "horizontal",
       "style": "IPY_MODEL_e567afe5512d497c9be1c366246c9381",
       "value": 442.0
      }
     },
     "29e9f077056b486ba2d1fdd02f1caa22": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "2c2277a9e0f844878664f1f7b1be63cb": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "2e65558064cf4ad48e85a62ddfc53fff": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_97289cb6e43940ff90da96711b324386",
       "placeholder": "​",
       "style": "IPY_MODEL_bc2c117ded184e149b4469425c298ca6",
       "value": " 232k/232k [00:00&lt;00:00, 656kB/s]"
      }
     },
     "48e855ab70914f32a6d9c704ca75af3b": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "61c70158d930473696c9bf26689f0efd": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "FloatProgressModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "FloatProgressModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "ProgressView",
       "bar_style": "success",
       "description": "Downloading: 100%",
       "description_tooltip": null,
       "layout": "IPY_MODEL_173f25b1ba434795ba6a530ca9488fcb",
       "max": 267967963.0,
       "min": 0.0,
       "orientation": "horizontal",
       "style": "IPY_MODEL_08c712ccba324989960094911ff3b69b",
       "value": 267967963.0
      }
     },
     "70607a78c4c744c8bf4be0ac7c0c4265": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_197cec13dd234875bcffcb032b372bd2",
       "placeholder": "​",
       "style": "IPY_MODEL_207c78f0351b48afa46af1910d85a401",
       "value": " 268M/268M [00:18&lt;00:00, 14.8MB/s]"
      }
     },
     "77c99bff83854cfdbe56e1a57591556f": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "FloatProgressModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "FloatProgressModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "ProgressView",
       "bar_style": "success",
       "description": "Downloading: 100%",
       "description_tooltip": null,
       "layout": "IPY_MODEL_1531f3e738f44ceb8e2824b9b1499dfd",
       "max": 231508.0,
       "min": 0.0,
       "orientation": "horizontal",
       "style": "IPY_MODEL_788c05d8a1924c899673be9520f3fab8",
       "value": 231508.0
      }
     },
     "788c05d8a1924c899673be9520f3fab8": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "ProgressStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "ProgressStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "bar_color": null,
       "description_width": "initial"
      }
     },
     "96cc8b0e9bac48a3b0566b290bc352a9": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HBoxModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HBoxModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HBoxView",
       "box_style": "",
       "children": [
        "IPY_MODEL_77c99bff83854cfdbe56e1a57591556f",
        "IPY_MODEL_2e65558064cf4ad48e85a62ddfc53fff"
       ],
       "layout": "IPY_MODEL_48e855ab70914f32a6d9c704ca75af3b"
      }
     },
     "97289cb6e43940ff90da96711b324386": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "bc0eeb798a024015822bc0c5ed0e3e76": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "bc2c117ded184e149b4469425c298ca6": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "bea62a5d76f9468f8a6e639ef00fb2ca": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HBoxModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HBoxModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HBoxView",
       "box_style": "",
       "children": [
        "IPY_MODEL_248c2d0ba7fc43938ecc92e010238710",
        "IPY_MODEL_f103d1bc647f48b5943560c053dfdfaf"
       ],
       "layout": "IPY_MODEL_14e18380daec4b9581fe37f7d915101a"
      }
     },
     "db1d2a9ba497421cb2267b317b7b8cfd": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HBoxModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HBoxModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HBoxView",
       "box_style": "",
       "children": [
        "IPY_MODEL_61c70158d930473696c9bf26689f0efd",
        "IPY_MODEL_70607a78c4c744c8bf4be0ac7c0c4265"
       ],
       "layout": "IPY_MODEL_e7cb3ee34f6c4117b6ccec7544e3a6b4"
      }
     },
     "e567afe5512d497c9be1c366246c9381": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "ProgressStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "ProgressStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "bar_color": null,
       "description_width": "initial"
      }
     },
     "e7cb3ee34f6c4117b6ccec7544e3a6b4": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "f103d1bc647f48b5943560c053dfdfaf": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_bc0eeb798a024015822bc0c5ed0e3e76",
       "placeholder": "​",
       "style": "IPY_MODEL_2c2277a9e0f844878664f1f7b1be63cb",
       "value": " 442/442 [00:00&lt;00:00, 615B/s]"
      }
     }
    },
    "version_major": 2,
    "version_minor": 0
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}