fine_tuneing.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "775c0235-e509-483e-b231-585a9b343ab3",
   "metadata": {},
   "source": [
    "# Fine-tuning a Model"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "53d709e5-aea7-4272-ae63-fb16bb543614",
   "metadata": {},
   "source": [
    "The code in this notebook works through an example of fine-tuning a DistilBERT model using Hugging Face. The code accompanies my lessons as part of the Introduction to Hugging Face course available through 365DataScience."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ab6f6294",
   "metadata": {},
   "source": [
    "## Choose model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "ceae2e63",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_name = 'distilbert-base-cased'"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d3f259f4",
   "metadata": {},
   "source": [
    "## Load dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "fe353370",
   "metadata": {},
   "outputs": [],
   "source": [
    "from datasets import load_dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "37b9c16f",
   "metadata": {},
   "outputs": [],
   "source": [
    "dataset = load_dataset(\"jonathanli/law-stack-exchange\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "f8d12409",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "DatasetDict({\n",
       "    train: Dataset({\n",
       "        features: ['Id', 'PostTypeId', 'CreationDate', 'Score', 'ViewCount', 'LastActivityDate', 'AnswerCount', 'CommentCount', 'ContentLicense', 'body', 'text_label', 'title'],\n",
       "        num_rows: 638\n",
       "    })\n",
       "    validation: Dataset({\n",
       "        features: ['Id', 'PostTypeId', 'CreationDate', 'Score', 'ViewCount', 'LastActivityDate', 'AnswerCount', 'CommentCount', 'ContentLicense', 'body', 'text_label', 'title'],\n",
       "        num_rows: 319\n",
       "    })\n",
       "    test: Dataset({\n",
       "        features: ['Id', 'PostTypeId', 'CreationDate', 'Score', 'ViewCount', 'LastActivityDate', 'AnswerCount', 'CommentCount', 'ContentLicense', 'body', 'text_label', 'title'],\n",
       "        num_rows: 1596\n",
       "    })\n",
       "})"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "03558635",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'Id': 71340,\n",
       " 'PostTypeId': 1,\n",
       " 'CreationDate': '2021-09-01T02:39:58.830',\n",
       " 'Score': 0,\n",
       " 'ViewCount': 128.0,\n",
       " 'LastActivityDate': '2021-09-01T06:43:55.503',\n",
       " 'AnswerCount': 1.0,\n",
       " 'CommentCount': 9,\n",
       " 'ContentLicense': 'CC BY-SA 4.0',\n",
       " 'body': \"I'm trying to comfortably use my laptop at night and now could not find an answer by web-search to the following: Why TVs / laptops etc. do not warn about too bright screen whereas many smartphones warn about too loud sound amplification? I'm asking on law site cause I suspect there were a number of successful lawsuits for impaired hearing from headphones but none for impaired vision from TVs. Am I right here? Digging down - has it happened because there are studies proving impairment for one and none for the other?\\n\\n\\nAdded: I actually doubt my second guess cause there are a lot of enhancements in screen brightness - auto-brightness, night mode.\\n\\n\\n\",\n",
       " 'text_label': 'civil-law',\n",
       " 'title': 'Why TVs / laptops etc. do not warn about too brirght screen whereas many smartphones warn about too loud sound amplification?'}"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dataset['train'][0]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "275c053e",
   "metadata": {},
   "source": [
    "## Prepare data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "a68e8e10",
   "metadata": {},
   "outputs": [],
   "source": [
    "dataset = dataset.remove_columns([\"Id\", \"PostTypeId\", \"CreationDate\", \n",
    "                                  \"Score\", \"ViewCount\", \"LastActivityDate\",\n",
    "                                  \"AnswerCount\", \"CommentCount\", \"ContentLicense\", \n",
    "                                  \"title\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "fc7def9a",
   "metadata": {},
   "outputs": [],
   "source": [
    "dataset = dataset.rename_column(\"text_label\", \"label\")\n",
    "dataset = dataset.rename_column(\"body\", \"text\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "0ab4687b",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "DatasetDict({\n",
       "    train: Dataset({\n",
       "        features: ['text', 'label'],\n",
       "        num_rows: 638\n",
       "    })\n",
       "    validation: Dataset({\n",
       "        features: ['text', 'label'],\n",
       "        num_rows: 319\n",
       "    })\n",
       "    test: Dataset({\n",
       "        features: ['text', 'label'],\n",
       "        num_rows: 1596\n",
       "    })\n",
       "})"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "7c1df637",
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "519e25ab",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<Axes: ylabel='0'>"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAr4AAAGdCAYAAAD0V3BwAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjAsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvlHJYcgAAAAlwSFlzAAAPYQAAD2EBqD+naQAAbQZJREFUeJzt3QV4U+f7PvCnRQrF3WW4FJchw4cW3YaNHz5kuNvGsKFfYMgYNgYMh+E+3F2GDi/OcBguzf+6n/9OriQ0bQotbXLuz3WdtUlOjqSsvfPked/jZbFYLEJERERE5OG8I/oAiIiIiIg+BgZfIiIiIjIFBl8iIiIiMgUGXyIiIiIyBQZfIiIiIjIFBl8iIiIiMgUGXyIiIiIyBQZfIiIiIjKFqBF9AESRRWBgoNy4cUPixIkjXl5eEX04RERE5AJci+3ff/+VlClTird38DVdBl+i/yD0pkmTJqIPg4iIiN7D1atXJXXq1MGuw+BL9B9Ueo3/ceLGjRvRh0NEREQuePz4sRaujL/jwWHwJfqP0d6A0MvgS0RE5F5caVPk4DYiIiIiMgUGXyIiIiIyBbY6EDnw67devH18I/QYAob5R+j+iYiIPBErviZy7949SZo0qQQEBMjWrVu1F+bhw4f62IwZMyR+/Pih2l769OllzJgxwa6DfSxbtky/x35x++jRo3rb8RjCyqlTp3RU59OnT8N0u0REROTeGHxNZPDgwVKjRg0NrI7q1q0rZ8+eDfN93rx5UypXrhzkY8WKFdPH48WL997hOyg5cuSQIkWKyOjRoz94W0REROQ5GHxN4tmzZzJt2jRp3rx5kI/HjBlTq8FhLXny5OLj4xPkY9GjR9fHw+NiEU2bNpWJEyfKmzdvwnzbRERE5J4YfE1izZo1GkBRCQ2KY7X1woULWh1OliyZxI4dWwoVKiQbN25853m4Ukr9+vUlVqxYkipVKpkwYYLTVgdHtq0O+B5h9dGjR3oflv79+8vAgQPFz8/vnefmzZtX+vbt6/R8y5cvL/fv35dt27YF+7oQERGReTD4msSOHTukQIECLq//5MkTqVKlimzatEmOHDkilSpVkmrVqsmVK1fs1vvf//4nefLk0XV69eolHTt2lA0bNoT6+ND2gH5hzJ+L9gcs3bp1k2bNmsnp06flwIED1nWxr2PHjmlQdgbVZIRjnLczL1++1EmvbRciIiLyXAy+JnH58mW9hrWrEGZbtWql1dbMmTPLoEGDJGPGjLJixQq79YoXL66BN0uWLNK+fXv56quv5Keffgr18SGootcXlV60P2BBpRmD1CpWrCjTp0+3rovvS5UqJRkyZAh2mzhfnLczQ4cO1X0aCy9XTERE5NkYfE3i+fPnEiNGjFBVfFFxzZ49u7ZAIISi8upY8S1atOg7t7FeWGrRooXMmzdPXrx4Ia9evZK5c+dqJTgk6FtGb7MzvXv31tYKY8GliomIiMhzcR5fk0icOLE8ePDA5fURetGyMHLkSMmUKZOGSFRzETw/NrRYoD956dKlWhl+/fq1HktI0OOLKrUz2KazgXdERETkeRh8TSJfvnwye/Zsl9fftWuXNGnSRGrVqmWtAGMeXkd79+595zaqxO8Dofbt27fv3B81alRp3LixtjhgnXr16mkQD8mJEydcCshERERkDgy+JoE+WXy0j6pvggQJQlwffb1LlizRaiv6bjGDQmBgYJABecSIEVKzZk2tEC9atEhWr179XseI+YURsDGgDj3Gvr6+usA333xjDdTYZ0gQ0q9fvy6ff/75ex0LEREReR72+JpErly5JH/+/LJw4UKX1sfFHxCQMdsCwi+CM57vqGvXrnLw4EGtKP/444/6PKz7PrCv1q1b68U0kiRJooHaNojj8WzZssmnn34a5LRothVp9ARXqFBB0qVL917HQkRERJ7Hy2KxWCL6IOjjQCW2e/fu2gLg7e1e73nwzxTht02bNtKlSxe7x9ACMWTIEL1UcbRo0bQPGetiEBxmnXAVpjPD7A4Y6IZp1YiIiCjyC83fb7Y6mIi/v7+cO3dOWwDcaequO3fuyPz58+XWrVtBzt2Li3Mg+CL0Amae6NOnT6hCLxEREXk+Vnwp0kMbA2alGDt2rHz99dfhth9WfImIiNwPK77kUfjejIiIiMKCezV6EhERERG9JwZfIiIiIjIFBl8iIiIiMgUGXyIiIiIyBQZfIiIiIjIFBl8iIiIiMgUGXyIiIiIyBQZfIiIiIjIFXsDCA927d0+yZ88u+/fvl/Tp0we7bpMmTeThw4eybNkyl7bdv39/Xffo0aMuPx/H0KlTJ10+hl69esnTp09l/Pjx7/V8v37rxdvHVyKzgGH+EX0IREREbofB1wMNHjxYatSoEWLoBVwGODRXRuvWrZu0b99eIjMcY4YMGaRz5876lYiIiAjY6uBhnj17JtOmTZPmzZu7tD6ubR0/fnyXtx87dmxJlCiRRGaJEyeWihUrysSJEyP6UIiIiCgSYfD1MGvWrBEfHx8pUqSI9b6TJ09K1apVJW7cuBInThwpUaKEXLhwwdqqULNmTf1+ypQpkjJlSgkMDLTbJqrHzZo1s7Y65M2b94OOcfTo0ZIrVy6JFSuWpEmTRtq0aSNPnjzRx1B9TpIkifzxxx/W9bG/FClSWG/v3LlTzxEh35lq1arJ/PnzP+g4iYiIyLMw+HqYHTt2SIECBay3r1+/LiVLltSguHnzZjl06JCG2Ddv3rzz3Nq1a2t/8JYtW6z33b9/X9atWycNGjQIs2P09vaWcePGaSCfOXOmHlePHj30MS8vLz3erVu36u0HDx7I6dOn5fnz5/L333/rfdu2bZNChQqJr6/zPtzChQvLtWvXJCAgwOk6L1++lMePH9stRERE5LkYfD3M5cuXtWprmDBhgrYzoPpZsGBByZIlizRt2lSyZs36znMTJEgglStXlrlz51rvQ+UVrQNlypQJs2PEIDdsDz3IZcuWlR9//FEWLlxofbx06dLW4Lt9+3bJly+f3X34WqpUqWD3YbwGeD2cGTp0qL42xoLqMxEREXkuBl8Pg8pojBgxrLcx+wJaG6JFi+bS81HZXbx4sVZDYc6cOVKvXj2t0oYE66IH2FhQfQ7Kxo0bpVy5cpIqVSptvWjYsKFWmo3WBYTaU6dOyZ07d7S6i9BrBN/Xr1/L7t279XZwYsaMqV+Da4fo3bu3PHr0yLpcvXo1xHMkIiIi98Xg62FQnUV7gGMAdBV6Y9Fnu3r1ag2CCK+utjlUr15dg7axoMLsCK0H6DfOnTu3Bmy0XqAqDa9evdKv6P9NmDChhl7b4IvvDxw4oOG3WLFiwR4LWjQA/cLOoP0Dfc+2CxEREXkuTmfmYdAWMHv2bOttBEz00SIsulL1RbX4iy++0Ort+fPntSUif/78Lu0b1VsswUHQxeC5UaNGWavItm0ORp8vqtTLly/XPuDPPvtM+3lRhZ48ebIGagyMC86JEyf0fHPmzOnSsRMREZHnY8XXw2AaL4RFo+rbrl07HbSFdoWDBw/KuXPnZNasWXLmzBmn20CFFxXf3377LUwHtUGmTJk0hOPiEhcvXtRjmTRp0jvrocI7b948ndEBbRMIyRj0hkAeUn8voFKN8BzaijcRERF5LgZfD4M2AVRojSoq5tzFrAmYLgyBETM+TJ06NdjqLwacodUA4fjrr78O0+PLkyePTmc2fPhw8fPz0yCLQWaOcKxv37616+XF9473GVOyOd6HwXwtWrQI02MnIiIi9+ZlCc1lu8gtoFrbvXt3/bjflUFp7g4hGbNEYI5hWLt2rXTt2lWOHTsmUaO63s2Dyjhmd8BAN/b7EhERuYfQ/P1mj68H8vf315YGzOHr6VN04R85LsaBsG94+vSpTJ8+PVShl4iIiDwfK75E/2HFl4iIyLP/fnv+5+BERERERAy+RERERGQWDL5EREREZAoMvkRERERkCgy+RERERGQKDL5EREREZAoMvkRERERkCgy+RERERGQKDL5EREREZAq8pquHuHfvnmTPnl32798vAQEBUqZMGXnw4IHEjx8/XPbXv39/WbZsmRw9elQio3r16kmhQoWka9euoX6uX7/14u3jK+4oYJh/RB8CERFRpMWKr4cYPHiw1KhRQ9KnT/9R9tetWzfZtGmTRFbff/+9via4fCERERERMPh6gGfPnsm0adOkefPmH22fsWPHlkSJEklk5efnJxkzZpTZs2dH9KEQERFRJMHg6wHWrFkjPj4+UqRIEbv7d+3aJblz55YYMWLoYydOnLBrVcibN6/d+mPGjLGrGG/dulUKFy4ssWLF0paJ4sWLy+XLl4N8fpMmTaRmzZoycuRISZEihYbitm3byuvXr63rvHz5UivFqVKl0m1++umnug8Dtl2tWjVJkCCBPp4zZ049N0DbRoMGDSRJkiQSM2ZMyZw5s0yfPj3Y1wXbmj9//nu8okREROSJ2OPrAXbs2CEFChR45/7u3bvL2LFjJXny5NKnTx8NgmfPnpVo0aKFuM03b95okG3RooXMmzdPXr16pf3DXl5eTp+zZcsWDb34ev78ealbt66GY2wD2rVrJ6dOndIwmjJlSlm6dKlUqlRJjh8/rkEWQRn72b59uwZfrIvKMvTt21dvr127VhInTqzbf/78ebDngNCOdgcEbrwxcIT7sRgeP34c4utCRERE7ovB1wOgUoog6ahfv35Svnx5/X7mzJmSOnVqDZt16tQJcZsIgeiPrVq1qrYMAAbPBQeV2p9//lmiRIki2bJlE39/f+0DRvC9cuWKVmjx1ThWVH/XrVun9w8ZMkQf+/LLLyVXrlz6eIYMGazbxmP58uWTggUL6m1XepmxHwTpW7duSbp06d55fOjQoTJgwIAQt0NERESega0OHgCVT7QzOCpatKj1+4QJE0rWrFnl9OnTLm0T66N9oWLFilopRuX45s2bwT4HrQkIvQZUf2/fvq3fo6r79u1byZIli1ZxjWXbtm1y4cIFXadDhw7y448/aksFQvuxY8es2/r222+1UowKco8ePWT37t0hngNaIowe6KD07t1bw72xXL161aXXhoiIiNwTg68HwEf/6IENDW9vb7FYLHb32fbjAiqxe/bskWLFismCBQs0tO7du9fpNh1bKNAWERgYqN8/efJEQ/GhQ4d0CjRjQRBHqIZvvvlGLl68KA0bNtSgjOru+PHj9bHKlStrZbtz585y48YNKVeunFaMg3P//n39ir7goKD9IW7cuHYLEREReS4GXw+AFgD0vzqyDakIxujvNdoVEAbRAmAbfoOakxfbRmUUFVbMlDB37tz3PkZUfFEBzpQpk92CHmRDmjRppHXr1rJkyRKdg3fq1KnWx3DMjRs31pkaMBBvypQpwe4Tg/nQ3oE3BkREREQMvh4A7QgnT558p+o7cOBA7bFFAETbAgIgBqxB6dKl5c6dOzJixAhtNZgwYYIOHDNcunRJAy8qvqi0/vnnn3Lu3LkQ+3ydQbUYszI0atRIQy22j8Fy6LNdvXq1rtOpUydZv369Pnb48GEdJGfs74cffpDly5froDac66pVq0I8Fgz6q1ChwnsdLxEREXkeBl8PgMFg+fPnl4ULF9rdP2zYMOnYsaPO+IDq7sqVKyV69Oj6GELjL7/8ooE3T548GkJtWwd8fX3l77//1sFmCK0tW7bUWRdatWr13seJ1gkEX1Ry0W+MEH7gwAFJmzatPo6KMPaBY8NsD9gvjhFw3AjimJ6tZMmS2jZhO1UZgjzCveHFixd6ZTljRgkiIiIiL4tjoye5JVRNMX0Zqrvo3zUbzNqAGRqM8Dtx4kSdwQKValdhJot48eLpQDf2+xIREbmH0Pz95nRmHgJTh6EV4fr169onayZofcA/eFSTbQfaGQPjiIiIiIAVX6L/sOJLRETk2X+/zfeZOBERERGZEoMvEREREZkCgy8RERERmQKDLxERERGZAoMvEREREZkCgy8RERERmQKDLxERERGZAoMvEREREZkCgy8RERERmQIvWewh7t27J9mzZ5f9+/dL+vTpP3h7/fv3l4kTJ8rt27dl6dKlUrNmTXEXr169kixZssgff/whBQsWDPXz/fqtF28fX3FHAcP8I/oQiIiIIi1WfD3E4MGDpUaNGmESek+fPi0DBgyQyZMny82bN6Vy5cq63TFjxog7iB49unTr1k169uwZ0YdCREREkQiDrwd49uyZTJs2TZo3bx4m27tw4YJ+RZBOnjy5+Pj4SER4+/atBAYGvtdzGzRoIDt37pSTJ0+G+XERERGRe2Lw9QBr1qzRcFqkSBG9/eDBAw1+SZIkkZgxY0rmzJll+vTp1vWPHz8uZcuW1ccSJUokLVu2lCdPnlhbHKpVq6bfe3t7i5eXl5QuXVouX74snTt31ttYLBaLbh/tBIa8efNKihQprLcRPHFcCOYwevRoyZUrl8SKFUvSpEkjbdq0se4XZsyYIfHjx5cVK1ZIjhw59LlXrlyRly9fagU3VapU+txPP/1Utm7dGuxrkiBBAilevLjMnz8/zF5nIiIicm8Mvh5gx44dUqBAAevtvn37yqlTp2Tt2rXatoBe3cSJE+tjT58+lYoVK2owPHDggCxatEg2btwo7dq108cRMI2QjDYHLEuWLJHUqVPLwIEDrfch/JYsWdIaQBG2sa/nz5/L33//rfdt27ZNChUqJL6+vtYgPW7cOK3Czpw5UzZv3iw9evSwOxeE5OHDh8uvv/6q6yVNmlSPbc+ePRpijx07JrVr15ZKlSrJuXPngn1dChcurK8NEREREXBwmwdANTZlypTW26iS5suXzzqwy7bvd+7cufLixQv5/ffftXoKP//8s1Z5ETiTJUumVVdAm4MhSpQoEidOHLv7UAlGHzBs375d94nHEYazZcumX0uVKmVdv1OnTtbvcUw//vijtG7dWn755Rfr/a9fv9bbefLksZ4Lgji+GueIcL5u3Tq9f8iQIU5fF6yP18YZVJKxGB4/fhzMq0xERETujhVfD4Aqa4wYMay3v/32W62OovUAFdXdu3dbH0NVFqHSCL2AlgD00p45cyZU+0WoRWX5zp07Wt1FEMaCwIsAi/3itgGV5XLlymnLAkJ0w4YNdTYKoxXCGJiWO3duu7YM9PpilobYsWNbF+zP6EV2Bq0cttt2NHToUIkXL551QfsFEREReS4GXw+ANga0GhgwC4PRk3vjxg0Nm6iShjX06yZMmFBDqG3wxfdoo0D4LVasmK4bEBAgVatW1VC7ePFiOXTokEyYMME6/ZhtWEUbhQE9wKg2Y/2jR49aFwT4sWPHBnt89+/f1z5kZ3r37i2PHj2yLlevXg2DV4WIiIgiK7Y6eAC0GMyePdvuPgS+xo0b61KiRAnp3r27jBw5Uuf6xSAy9PoaVd9du3Zp/23WrFmd7gOVWFRebSGgYtvLly/XftzPPvtM+3nRPoAWCLRaGPtAcEVVedSoUbovWLhwoUvnhv1iPmHsKzROnDihz3cGg+ciasYKIiIi+vhY8fUAGKyG4GlUfX/44QcNo+fPn9f7V61apYEXMNsD2iIQiBEMt2zZIu3bt9e2A/T3OoOeXPTxXr9+Xe7evWu9HxXeefPmaVsFWhAQajHobc6cOXb9vZkyZdIK8Pjx4+XixYsya9YsmTRpUojnhhYHHHOjRo10kN2lS5f0Ih1oU1i9enWwz8XAtgoVKrj0GhIREZHnY/D1AGg5yJ8/v7WCiuosPsZHWwFCKFoFjGm9UJFdv369tgFgxoWvvvpKWyEwwC04mNEB7QoZM2a0ax9AuEVF1raXF9873oe+YkxnhgF0fn5+GowRXl2BQWwIvl27dtWqNK4ih1aKtGnT2lWfUck2YBYItC/g/IiIiIjAy4IJWcntofqJdgZUcY1WArNAFRiVYQy0w5zFULduXQ3bffr0cXk7mNUBg9wQmOPGjRuOR0xERERhJTR/v9nj6yH8/f11Xlu0IphtdgJcwAMX4TBCLwbLoQqOwX1EREREBlZ8if7Dii8REZFn//0212fiRERERGRaDL5EREREZAoMvkRERERkCgy+RERERGQKDL5EREREZAoMvkRERERkCgy+RERERGQKDL5EREREZAoMvkRERERkCrxksYe5d++eZM+eXfbv3y/p06cXM8Ili7NkySJ//PGHFCxYMNTP9+u3Xrx9fMXTBQzzj+hDICIi+qhY8fUwgwcPlho1any00DtjxgyJHz++fCw4rzFjxgS7TvTo0aVbt27Ss2fPj3ZcREREFPkx+HqQZ8+eybRp06R58+YSGauwH1ODBg1k586dcvLkyY+6XyIiIoq8GHw9yJo1a8THx0eKFClivQ/Br2rVqhI3blyJEyeOlChRQi5cuKCPBQYGysCBAyV16tT6vLx588q6deuszw0ICBAvLy9ZsmSJlClTRnx9fSVPnjyyZ88efXzr1q3StGlTefToka6HpX///tbK7KBBg6RRo0a675YtW+r9qMKiDQHbypAhg/Tt21dev35tdx4rV66UQoUKSYwYMSRx4sRSq1Ytvb906dJy+fJl6dy5s3V/ziRIkECKFy8u8+fPD9PXmIiIiNwXg68H2bFjhxQoUMB6+/r161KyZEkNtZs3b5ZDhw5Js2bN5M2bN/r42LFjZdSoUTJy5Eg5duyYVKxYUapXry7nzp2z2+53332nrQNHjx7V0Fq/fn3dRrFixbTtAMH25s2bumA9A7aLoHzkyBENuIDwjfaIU6dO6f6nTp0qP/30k/U5q1ev1qBbpUoVfd6mTZukcOHC+hgCOEI6wrqxv+DgeXhNiIiIiICD2zwIqqEpU6a03p4wYYLEixdPq57RokXT+xBcbYMpKrD16tXT28OHD5ctW7ZomMVzDQiz/v7/fyDUgAEDJGfOnHL+/HnJli2bbh+V1+TJk79zPGXLlpWuXbva3ff9999bv0dVGNvG8fXo0cPao4zjwX4MCM+QMGFCiRIliobnoPbnCK8FXhNnXr58qYvh8ePHIW6TiIiI3Bcrvh7k+fPn2h5gQIUWrQ1G6LWFkHfjxg1tB7CF26dPn7a7L3fu3NbvU6RIoV9v374d4vEENaPCggULdB8IrrFjx9YgfOXKFbtjLleunISFmDFjat+zM0OHDtXgbixp0qQJk/0SERFR5MTg60HQD/vgwQO74BcWbIOz0VeL/uCQxIoVy+42eoMx6AxtDKtWrdJWBrRR2A58C6tjhvv370uSJEmcPt67d2/tTzaWq1evhtm+iYiIKPJh8PUg+fLl095Z20otelwdB48B+nLRCrBr1y67+3E7R44cLu8TU4e9ffvWpXV3794t6dKl07CLanDmzJnfaUXAMaOvNyz2d+LECX1NnEHvM14H24WIiIg8F4OvB8HgNMziYFR927Vrpy0N6Jk9ePCgDlqbNWuWnDlzRh/v3r279vWi/QD39erVS1sNOnbs6PI+0af75MkTDat3794NtrUAQRdtDejpxcwS48aNk6VLl9qt069fP5k3b55+RcvF8ePH9Rht97d9+3YduIf9BQehv0KFCi6fCxEREXk2Bl8PkitXLsmfP78sXLhQbydKlEhnc0AwLVWqlM74gFkUjNaFDh06SJcuXXQAGp6LqcxWrFihAdVVmNmhdevWUrduXW0rGDFihNN1MWMEpiJDIMfUaagAG7M9GDBl2aJFi/Q4sA4GyOEqdAbM6IBp1jJmzGjXxoAWDMwWYdtWgfaFr776yuVzISIiIs/mZbFYLBF9EBR2MB0YKrn4mN/b2xzvay5duqSzVaDNwwjtCOKYDaJPnz4ubwfVcQxyQ2Bm2wMREZF7CM3fb05n5mEw7RhaGtAKYJZZCnDhDlwgwwi9GCyHCjaqy0REREQGVnyJ/sOKLxERkWf//TbHZ+FEREREZHoMvkRERERkCgy+RERERGQKDL5EREREZAoMvkRERERkCgy+RERERGQKDL5EREREZAoMvkRERERkCgy+RERERGQKvGQxkQO/fuvF28dXzCRgmH9EHwIREVG485iK77179yRp0qQSEBAgkVXp0qWlU6dO4bJtnLeXl5ccPXpUPqYZM2ZI/Pjxg12nSZMmUrNmzY92TOvWrZO8efNKYGDgR9snERERRX4eE3wHDx4sNWrUkPTp00f0ocjWrVs1hD58+NDu/iVLlsigQYOst3GsY8aMCfX2gwqSadKkkZs3b4qfn5+YXaVKlSRatGgyZ86ciD4UIiIiikQ8Ivg+e/ZMpk2bJs2bN5fILGHChBInTpxw2XaUKFEkefLkEjUqu1eMNwfjxo2L6MMgIiKiSMQjgu+aNWvEx8dHihQpYr3v5MmTUrVqVYkbN66GzRIlSsiFCxf0MXwEPnDgQEmdOrU+Dx+L4+Nxx7YBVGjLlCkjvr6+kidPHtmzZ491ncuXL0u1atUkQYIEEitWLMmZM6ceB56L5wAew3YQwhxbHfA9ttG5c2ddBwv0799fj8cWqsJGJRuPz5w5U5YvX259HirMQbU6bNu2TQoXLqznmCJFCunVq5e8efPG+jiOoUOHDtKjRw8N5QjO2L6t0aNHS65cufQcUVVu06aNPHny5IN+XnitP/vsM22RSJQokf6cjJ8NfPXVV9KuXTvrbbxmOLe///5bb7969UqPZ+PGjU73gZ/NwYMH7bZLRERE5uYRwXfHjh1SoEAB6+3r169LyZIlNfBt3rxZDh06JM2aNbOGvrFjx8qoUaNk5MiRcuzYMalYsaJUr15dzp07Z7fd7777Trp166ZhMkuWLFK/fn3rNtq2bSsvX76U7du3y/Hjx2X48OESO3ZsDYeLFy/Wdc6cOaPtB9ifI4RqBG8EcKyDxRU4njp16ujH+cbzihUr9s56eA2qVKkihQoVkr/++ksmTpyoVfEff/zRbj2EaITIffv2yYgRI/R4NmzYYH3c29tbK6d4I4F18XoiKH+Ip0+fSpcuXTSYbtq0SfdRq1Yta09uqVKlNMzbBvjEiRNb7ztw4IC8fv06yPM2pE2bVpIlS6b/NpzBz+/x48d2CxEREXkuj/hcHJXTlClTWm9PmDBB4sWLJ/Pnz9deT0BwNSDw9uzZU+rVq6e3EVq3bNmilVU81zZk+vv//9HuAwYM0Kru+fPnJVu2bHLlyhX58ssvtRoKGTJksD4P1VPAYDtnA7+wDtoTUI1GpdVVCNcxY8bU0Bbc83755RcN4T///LNWS3HMN27c0PP+4YcfNGxC7ty5pV+/fvp95syZdX2E0fLly+t9toPxUHVGcG7durVu/33hdbP122+/SZIkSeTUqVPao4xKdMeOHeXOnTvauoH7+/btq8EX+8ZXBHpU4oODfxP4t+HM0KFD9edKRERE5uARFd/nz59LjBgxrLdRoUVrgxF6baGqhwBYvHhxu/tx+/Tp03b3IRQa0CoAt2/f1q9oEUAIxPMQHFE5jkxwLkWLFrW2UACOFW0K165dC/IcjfM0zhHQTlCuXDlJlSqVhvSGDRvqDBroq3aENwMI5sYyZMiQII8NlXVUz/FmAa0oRhsHng8Iv3hjgEovKrb58uXTdgjcBnxFOA4J3iAEdZyG3r17y6NHj6zL1atXQ9wmERERuS+PCL74GPzBgwd2gScs2AZnI0AaH8d/8803cvHiRQ2CaHUoWLCgjB8//oP3iUqsxWKxuw8f64cXxzcHOE/jHNE3jMCJcIz2DbSMGBVx9NkGVWHFmw5jQXXWWf/t/fv3ZerUqdpigcV2mzgGtKqgsmuEXBwDqtwnTpyQ3bt3aztESLAPVJKdQSsMgrftQkRERJ7LI4IvKoL4ONyAkIRKYVCBEeEGAW3Xrl129+N2jhw5QrVftBIg3KFft2vXrhrkIHr06Pr17du3wT4f6zmug6B269Ytu/DrODdvUM9zlD17dh2MZ7sdnCOqtugtdgWCLkIw+qExcBDtIqiWO4O2hEyZMlkXo+XDFqrF6H3+/vvvtZKM47R902Iw+nyxIPjiDQHC8P/+9z8NwI4Ve0cvXrzQgW34t0FERETkMcEXg9Mw+MoIUJgRAC0N6OHFACp8tD5r1iwNXNC9e3ft612wYIHeh9kOEC7RV+oq9L6uX79eLl26JIcPH9YeYYQ4SJcunVYtV61apX2qzmZBwEf8GByHgWh3797V+xDy8BwMNENwQ4V17dq17zwPrRU4djwvqICP2Rfw0X379u11NgTMAoGWDAwqM/p7Q4Lwim2jko3qNl7DSZMmyYfATBeYyWHKlCnaL43BcjgmR3gd8GYGP1fMAGHch7l5UV3HgLzg7N27Vyu6aPcgIiIi8pjgiwFm+fPnl4ULF+ptBCsEKgROVA4x4wOqscbH+ujPRdhClRbPxfRaK1as0MFdrkLFFTM7IOxihgVUQ40BX+iHxaApBGrMLGA7NZctzKCAdoKMGTNaP5LH9rAdBF5MobZ//34dZGerRYsWkjVrVg2AeJ5j9do4BkyvhudjO6hMY55jVFpdhedhOjO8SUDfLUInBoR9CIRuDDpENRnbxHRuqOI6ws8FAwMxtRv6hY3gi9fdsb8XV4+z7WWGefPmSYMGDUIcAEdERETm4WVxbCh1U6tXr9ZKLnpAXa1okmdAJRu9wMZ0Z6iC440Bqv2ffPKJy9vBpwSYDQQD3djvS0RE5B5C8/fbI6YzA0w7hpYGtA2g95bMA60gmIbNgCo6quahCb1ERETk+Tym4kv0oVjxJSIi8uy/3+wJICIiIiJTYPAlIiIiIlNg8CUiIiIiU2DwJSIiIiJTYPAlIiIiIlNg8CUiIiIiU2DwJSIiIiJTYPAlIiIiIlPwmCu3EYUVv37rxdvHV8wkYJh/RB8CERFRuGPF183du3dPkiZNqpfp3bp1q3h5ecnDhw/1sRkzZkj8+PElsujfv7/kzZs33PfTq1cvad++fbjvh4iIiNwLg6+bGzx4sNSoUUPSp0//zmN169aVs2fPSmTRrVs32bRp00fZz8yZM+XixYvhvi8iIiJyHwy+buzZs2cybdo0ad68eZCPx4wZU6vBkUXs2LElUaJE4b6fxIkTS8WKFWXixInhvi8iIiJyHwy+bmzNmjXi4+MjRYoUCfLxoFodVq5cKYUKFZIYMWJoQKxVq5b1sZcvX2q1NFWqVBIrViz59NNPtX3CcXvr16+X7Nmza5CtVKmS3Lx507oO1i9cuLA+H+sWL15cLl++HGSrQ5MmTaRmzZoycuRISZEihYbitm3byuvXr63rYNv+/v4a4j/55BOZO3euVrfHjBkT7GtTrVo1mT9/fqheTyIiIvJsDL5ubMeOHVKgQAGX11+9erUG3SpVqsiRI0e07QAh1dCuXTvZs2ePBsZjx45J7dq1NdieO3fOrsqMoDpr1izZvn27XLlyRcMyvHnzRoNsqVKl9PnYVsuWLbXv2JktW7bIhQsX9CvaExCusRgaNWokN27c0EC9ePFimTJlity+fTvEc8V5Xbt2TXufnUHQf/z4sd1CREREnouzOrgxVFJTpkwZqn7gevXqyYABA6z35cmTR78iwE6fPl2/GttEoF23bp3eP2TIEL0P1dhJkyZJxowZrWF54MCB+j2C46NHj6Rq1arWx1EZDk6CBAnk559/lihRoki2bNm0uotA3qJFC/n7779l48aNcuDAASlYsKCu/+uvv0rmzJlDPFfjHPAaBdX/DEOHDrV7LYiIiMizseLrxp4/f64tC646evSolCtXLsjHjh8/Lm/fvpUsWbJoC4OxbNu2TSuyBl9fX2uoBbQoGBXYhAkTavsC+mvRajB27Fi7Noig5MyZU0NvUNs7c+aMRI0aVfLnz299PFOmTBqWQ4LWCKNC7Uzv3r01qBvL1atXQ9wuERERuS9WfN0YenQfPHjg8vpGGAzKkydPNIAeOnTILogCArAhWrRodo+hjcFisVhvozrcoUMHrRQvWLBAvv/+e9mwYYPTPuSgthcYGCgf6v79+/o1SZIkTtdBfzQWIiIiMgdWfN1Yvnz55NSpUy6vnzt3bqfTiWFbqPii2oqqqu2SPHnyUB8Xqqm7d+8WPz8/HZD2PrJmzap9w+hHNpw/f96lsH/ixAkN1agoExEREQGDrxtDS8HJkyddrvr269dP5s2bp19Pnz6t7Q3Dhw/Xx9Di0KBBAx1MtmTJErl06ZLs379f+2AxKM4VeA4CLwa1obf2zz//1IFxIfX5OoOe388//1wHyOFYEIDxPSrXwQ2YMwb+lShRItgqNxEREZkLg68by5Url/a/Lly40KX1S5cuLYsWLZIVK1botGJly5bVQGnbpoDg27VrV622YoYGDCxLmzatS9tH/y8GpH355ZcapBFSMT1Zq1at3vscf//9d0mWLJmULFlSZ6TAoLc4ceLY9TajrxjnZgszU2BdIiIiIoOXxbZBk9wOqrHdu3fXj/a9vT3/fQymKEuTJo3O9mAM1MP0aWXKlNF5gmHt2rUa3jGlGgbHuQqzUsSLF08HusWNGzfczoGIiIjCTmj+fnNwm5vD9F9oJ7h+/boGQk+zefNmHXiH6jZmiOjRo4dOT4YKMOAfOWadsG3HePr0qVavQxN6iYiIyPOx4kuRGq4Sh+rtxYsXtcWhWLFietW2dOnShfm+WPElIiJyP6H5+83gS/QfBl8iIiLP/vvt+U2hREREREQMvkRERERkFgy+RERERGQKDL5EREREZAoMvkRERERkCgy+RERERGQKDL5EREREZAq8tBWRA79+68Xbx1fMLmCYf0QfAhERkWdVfO/duydJkyaVgIAA2bp1q3h5ecnDhw/Dfb9NmjSRmjVrWm+XLl1aOnXqJO4Al+zF1csoaK9evdLX6ODBgxF9KERERBSJRHjwHTx4sNSoUUODCi5He/PmTb36xvsG2MjCnYJ0ZIE3P3jjc/To0Q/aTvTo0aVbt27Ss2fPMDs2IiIicn8RGnyfPXsm06ZNk+bNm1sDS/LkyTX80MeFK1e/efMmQqu0YalBgwayc+dOOXnyZJhul4iIiNxXhAbfNWvWiI+PjxQpUkRvO7Y6zJgxQ+LHjy/r16+X7NmzS+zYsaVSpUpaFYb+/fvLzJkzZfny5fo8LNgGXL16VerUqaPPT5gwoVaVUVF01cuXL7VqmCpVKokVK5Z8+umn1m0bdu3apZVdX19fSZAggVSsWFEePHigVeht27bJ2LFjrceFfRvnY2vZsmV2Qf/ChQt6rMmSJdPzLVSokGzcuPG9Kqfz58/XKnqMGDHEz89Pj8lgvNZr166VAgUK6M8BQRHn3aFDB20/wfM+++wzOXDgwDvPW716teTOnVvXwc/vxIkTdseAbZUoUUJixowpadKk0W0+ffrU+jgq/IMGDZJGjRrpdbVbtmwpn3zyiT6WL18+3Qde2+3bt0u0aNHk1q1bdttHNR3bdwY/j+LFi+trQERERBThwXfHjh0aukKqCo8cOVJmzZqlIejKlSsaSAFfEW6NMIwFQe/169caQuPEiaP7QEA1QrOrlcV27drJnj17NDgdO3ZMateurc8/d+6cPo6P48uVKyc5cuTQ9RD0qlWrJm/fvtXAW7RoUWnRooX1uBD+XPHkyROpUqWKbNq0SY4cOaL7xHZx3qHVvXt36dq1q24Hx4PtoKfaVq9evWTYsGFy+vRpDbI9evSQxYsX6xuKw4cPS6ZMmfS1vH///jvbHjVqlIbiJEmS6LbxuhvhHcf95Zdf6mu3YMECfX3wmtrCzzVPnjx6fH379pX9+/fr/Qj6eM2WLFkiJUuWlAwZMujP34D9zJkzR5o1axbs+RcuXFh//s4g5D9+/NhuISIiIs8VocH38uXLkjJlymDXQciZNGmSFCxYUPLnz6/hCaEQEGZRUUS1Ei0SWNAugaAVGBgov/76q+TKlUurxdOnT9fw6Fi1DQrWw/qLFi3SqmLGjBk1ZKP6ifthxIgReky//PKLhrecOXPqsSVOnFh7lHEcqAQbxxUlShSXXhNsq1WrVlqhzZw5s1ZFsf8VK1ZIaOF4ED5x/hMnTtTjQmuJrYEDB0r58uV1H3gdsd7//vc/qVy5sob6qVOn6mvs+Lx+/frp8/D6IiT/888/snTpUn1s6NCh2mqAqizOAW9Gxo0bJ7///ru8ePHCuo2yZctqMMe+sSBAQ6JEifQ1Q6Ue0ApjvO6wcuVK3Q7e9AQH/7bwb8wZHCdeE2Nx9c0JERERuacIDb7Pnz/Xj8qDg/CIUGRIkSKF3L59O9jn/PXXX3L+/Hmt+CIcY0GIQlhCNTIkx48f18ptlixZrM/HglYB4/lGxTesoeKLkI2wirYI7BfVWGcV39atW9sdoy1UeQ1Ro0bVoI5t2cJ9Bpwb3migRcCANgNUTh2fZ7ttvLZZs2a1roPXH20dtseFqjHejFy6dCnIfQcHrSP4ee7du1dvY9sIvWhBCQ4COz4xcKZ3797y6NEj64L2GCIiIvJcETqPL6qj6IkNDoKXLfR+YiBWSOERLRT4ONyRUVUM6fmo0B46dOidSq0RLhGqQsvb2/udYzfaAwwIvRs2bNA2ALQZYD9fffWV0xYNVGyN1o/3EVJ4fB94/VC1Rl+vo7Rp04Z63+g3RisFqr7oA0ZfsiuVe7RnBPfzRoUbCxEREZlDhAZfDGKaPXv2B20DLQWoztpCSwTaHRCYMHDqfY4L20Rl2dkAKvTDouViwIABLh8XQti///6rg7yM0Oc4dRf6kVHhrFWrljVEBjcoD+eIJSiokKJHFjBjA4K8Y5+tLVTWcdw4hnTp0lmDOfp4Hadmw7aNEIs3L2fPntUqtfH6nzp1SoN7aGDf4Pi6wTfffCP169eX1KlT63HaVqWdwYA7/CyJiIiIIrzVAR9/Y7qpkKq+wcHsABhAdebMGbl7964GNfSXopqM2REwuAkfr6NCiArktWvXQtwmWhywDcw4gAFWeD4GXqEnFLMZGB+TIxC2adNG9//3339rfyyOwTiuffv2aWjFffiYHzNDoHWjT58+2lYwd+5c/djeFnpisU8EYrQMfP311/rc9zFhwgTtu8WxtW3bVl/n4AaEIYx/++23OnBt3bp1Gl4xQA/tAsaUc7aVZgR/hEsEdbzexnzKmD939+7dGrJxHhgQiJk3ggvdgACPCjf2jZ5htB/Y/lvBm5gff/xRmjZt6tL542dfoUIFl9YlIiIizxehwRcDo1AdXLhw4XtvA8EM/aXoF0VFFdVKhEvMAIGK5BdffKGVSAQ39Pi6WgHGx+oIvhh8he0j1CHoGlVOhOM///xTwyl6YNHzinCHXlpA+wHaJDBADMeFHl30wqLCjWnccO7z5s3TKdlsjR49WqfiwoAwfLyPwIfX6H1gtgYsGDCHWRUwQA4BNaTnYEBcw4YNdb/orcV0cjgmx/U6duyoLSWYagwDzoyKLarh6IdGFRgVc1Rdf/jhhxAHMuK1wyC4yZMn67p442LbJoKAjWowfi6O0AJj+yYCM20gOKNNhIiIiAi8LCE1zIYzVFBRYUTlEOGGPhyqzOiFxTRhefPmDdNto3JepkwZrR47zkkc3vDm5c6dO+/McIGKPN6IoEKNijnUrVtXAz+q667CdGaY3QGB+X1aZIiIiOjjC83f7wjt8QV/f3/9KPz69eucToqChH/ImGkDrSFBTeuGCjougGGEXgwEREW9c+fOEXC0REREFFlFePAFx4FTRLbQ8oAea0zdhrmDHaF/2RZaLr7//vuPeIRERETkka0OGKj122+/aQ+lcRlZXGwAPanowXRlujCiyIitDkRERJ799ztUTbUY3IVeSgxAwg4wVRYWfI/7smXLJgcPHvzQ4yciIiIiitiKb5EiRXTAEC4hjFH0trAZfBSNqb1QDSZyN6z4EhERuZ9wG9xmXIrWMfQC7sNgIl4wgIiIiIgio1C1OqCXF4OMnMFjyZIlC4vjIiIiIiIKU6Gq+OKiDJg2Cpe+LVeunDXk4ipbuIrX1KlTZeTIkWF7hEREREREHzv4YtooXPnrp59+kl9++UWvogW4Qhmu4IU2iDp16oTFcRERERERRY4rt71+/VqnNgOE4WjRooXtkRF9ZBzcRkRE5H4+ypXbEHRTpEjxvk8nirT8+q0Xbx/fiD4MjxAwzD+iD4GIiOj9BreRe7t3754kTZpUAgICPvq+MevHsmXLPtr+6tWrJ6NGjfpo+yMiIqLIj8HXRAYPHqyX/02fPr1s3bpVw+jDhw/FE+GSxThffOxBREREBAy+JvHs2TOZNm2aNG/eXMzAz89PMmbMKLNnz47oQyEiIqJIgsHXJNasWSM+Pj569T20OpQpU0bvT5AggVZ+mzRporfXrVsnn332mcSPH18SJUokVatWlQsXLli38/vvv0vs2LHl3Llz1vvatGmjl6tGuHZVz5499fLXvr6+kiFDBunbt68OmARUaTFTiHH568DAQEmYMKEeuwGBNk2aNMHuo1q1ajJ//nyXj4mIiIg8G4OvSezYsUOnnAMExsWLF+v3Z86ckZs3b8rYsWP19tOnT6VLly4aOjE3s7e3t9SqVUvDJzRq1EiqVKkiDRo0kDdv3sjq1avl119/lTlz5miIdVWcOHF0+rtTp07pvjEHNKbJA4zMzJs3r7ZjwPHjxzWcHzlyRJ48eaL3bdu2TUqVKhXsPgoXLqwXVXn58mWQj+N+jAS1XYiIiMhzMfiaxOXLlyVlypT6PaqpqKACBrvhinwIm/Dll1/KF198IZkyZdLw+dtvv2nwREA1TJ48WcNyhw4dtHWif//+1lAdmh7cYsWKab8xKrO4OMrChQutj5cuXdoafPG1fPnykj17dtm5c6f1vpCCL8731atXcuvWrSAfHzp0qJ63sYRUQSYiIiL3xuBrEs+fP5cYMWKEuB5aGOrXr6/tB5gLD8EUrly5Yl0H7RHoF544caL20fbq1cv62JAhQ7QVwlhsn2drwYIFUrx4cQ3dWA9B2HZdhFqEXFwkBdVdBGEjDN+4cUPOnz+vt4MTM2ZM/eqsBaN3797aVmEsV69eDfH1ISIiIvfF4GsSuMjIgwcPQlwP1df79+9r68G+fft0AVRObW3fvl0rx6j8oj3C0Lp1azl69Kh1MarMtvbs2aOtEmiZWLVqlbYwfPfdd3b7KFmypPz7779y+PBh3Zdt8EUQxnYzZ84c7LngPCBJkiRBPo6eZ4R724WIiIg8F4OvSeTLl8+uXSF69Oj61bjstDHPL3p+UX0tV66cthYEFZZ3794tw4cPl5UrV2q1tl27dtbH0EKBNgljiRo1apDPT5cunYbdggULaoBFK4YtDK7LnTu3/Pzzz3qxFAyeQxhGSEZYDqnNAU6cOCGpU6fW0E9ERETE4GsSFStWlJMnT1qDLIInBowhRN65c0cHjaGFATM5TJkyRVsJNm/erAPdbKEK27BhQ+3vrVy5sg5qQ9vCH3/84fKxIOiirQEzLmDGiHHjxsnSpUvfWQ8VXmzfCLkI1Qjj2J8rwRcD+ipUqODycREREZFnY/A1iVy5ckn+/PmtA8hSpUolAwYM0P7cZMmSadUWMzggjB46dEjnwe3cubP873//s9tOx44dJVasWNrLa2wX37dq1UquX7/u0rFUr15dt419YgAdKsCYzswRwi0q0ra9vPje8T7jfmNKNnjx4oVeKa5FixahfKWIiIjIU3lZLBZLRB8EfRyYeqx79+7aAoCQ60lQwUaQN8IvBt6hivznn3+6vA1MZ4bZHTDQjf2+RERE7iE0f7/fbcAkj+Xv76+zNqAy60lTd6GFA//gMcewAX3B48ePj9DjIiIiosiFFV+i/7DiS0RE5Nl/vz3r824iIiIiIicYfImIiIjIFBh8iYiIiMgUGHyJiIiIyBQYfImIiIjIFBh8iYiIiMgUGHyJiIiIyBQYfImIiIjIFHjlNiIHfv3Wi7ePb0QfhscJGOYf0YdAREQmx4qvSdy7d0+SJk0qAQEBH7SdJk2aSM2aNSUy69Wrl7Rv3z6iD4OIiIgiGV6y2CS6dOki//77r0ydOvWDtoPLAeKfTPz48SWyunv3rmTIkEGOHj2qX0N7ycM0nRay4hsOWPElIqLwwEsWk51nz57JtGnTpHnz5u+9jbdv30pgYKD+w4rMoRcSJ04sFStWlIkTJ0b0oRAREVEkwuBrAmvWrBEfHx8pUqSI3t66dat4eXnJ6tWrJXfu3BIjRgx97MSJE9bnzJgxQwPuihUrJEeOHPr8K1eu2LU6TJkyRVKmTKmB2FaNGjWkWbNm+v2FCxf0drJkySR27NhSqFAh2bhxo936L1++lJ49e0qaNGl0P5kyZdKgjsoyvh85cqTd+qjk4vjPnz/v9JyrVasm8+fPD4NXj4iIiDwFg68J7NixQwoUKPDO/d27d5dRo0bJgQMHJEmSJBoWX79+bVcpHj58uPz6669y8uRJ7RG2Vbt2be0d3rJli/W++/fvy7p166RBgwZ6+8mTJ1KlShXZtGmTHDlyRCpVqqT7QYg2NGrUSObNmyfjxo2T06dPy+TJkzUkI9wiQE+fPt1uv7hdsmRJDcXOFC5cWK5duxZsTzMCNz4esV2IiIjIczH4msDly5e1MuuoX79+Ur58ecmVK5fMnDlT/vnnH1m6dKn1cYTgX375RYoVKyZZs2YVX1/7vtcECRJI5cqVZe7cudb7/vjjD201KFOmjN7OkyePtGrVSvz8/CRz5swyaNAgyZgxo1aS4ezZs7Jw4UL57bffpFatWtqTW65cOalbt64+jgrzmTNnZP/+/dZjwv6MirIzxvni3J0ZOnSotm4YCyrORERE5LkYfE3g+fPn2s7gqGjRotbvEyZMqOEWFVdD9OjRtRUiOKjsLl68WKunMGfOHKlXr554e3tbK77dunWT7Nmza+sEKrnYh1HxRdtClChRpFSpUk4DrL+/vwZjWLlype4L1ebgxIwZ01q1dqZ3797aCG8sV69eDXabRERE5N4YfE0AFdgHDx6E+nkIj2g3CA7aFtCLi35hBEe0VRhtDoDQiyrykCFD9DEEXVSYX716Zd1HSL755hvt10WAR5sDqsGO1WdHaLkAtHA4g35ijP60XYiIiMhzMfiaQL58+eTUqVPv3L93717r9wjGaDtAZTY0UEn+4osvtNKLPl1UjfPnz299fNeuXdqugDYGBN7kyZPb9d3iPgyO27Ztm9N9oEc4VqxYOksD+odDanMADNSLFi2a5MyZM1TnQ0RERJ6LwdcEMLUXBqc5Vn0HDhyog84QEhFOURl+n4tToMKLii/aEWyrvYC+3iVLlmil96+//pKvv/7abhaI9OnTS+PGjTXMLlu2TC5duqSzTqDv14BWCBwfWhOwPdsWDWdQXS5RooRLFWUiIiIyBwZfE0BVFVVY2zAJw4YNk44dO+qMD7du3dL+WfT1hlbZsmW1RxiD0BBsbY0ePVoHwWGAHNoiEMJtK8KASu5XX30lbdq0kWzZskmLFi3k6dOndutgDmK0RzRt2vSd/SMUly5d2u4+tEZgO0REREQGXrnNJFCRxfRlqO5u375dZ11ABTiyX4zCtoKL2R7QR4w5gW1hYBzOp3///np77dq10rVrVzl27JhEjRo1XK78QkRERJFDaP5+u54KyK1hZoRz587J9evXxZ1gBoc7d+5oqMVMDo6hF//IcZEMBHsDqsUYBBea0EtERESej60OJtKpUye3m6sWA+bSpUsnDx8+lBEjRrzzON7h4UIVmCbNgLaJTz/99CMfKREREUV2bHUg+g9bHYiIiDz77zcrvkRERERkCgy+RERERGQKDL5EREREZAoMvkRERERkCgy+RERERGQKDL5EREREZAoMvkRERERkCry0FZEDv37rxdvHN6IPw6MFDPOP6EMgIiITYsXXg927d0+SJk0qAQEBsnXrVvHy8tIroHm6V69eSfr06eXgwYMRfShEREQUiTD4erDBgwdLjRo1NAQWK1ZMbt68qVc2cVWTJk2kZs2aEpngXMaMGRPsOtGjR5du3bpJz549P9pxERERUeTH4Ouhnj17JtOmTZPmzZtbw2Dy5Mm16hsRFdiPrUGDBrJz5045efLkR983ERERRU4Mvh5qzZo14uPjI0WKFNHbjq0OM2bMkPjx48v69esle/bsEjt2bKlUqZJWhaF///4yc+ZMWb58uT4PC7YBV69elTp16ujzEyZMqFVltFM4VopRcU6ZMqVkzZpVH8c2lixZImXKlBFfX1/JkyeP7Nmzx+64EVZLlCghMWPGlDRp0kiHDh3k6dOn+ljp0qXl8uXL0rlzZ+sxOZMgQQIpXry4zJ8/PxxeXSIiInJHDL4easeOHVKgQIEQq8IjR46UWbNmyfbt2+XKlSvaIgD4inBrhGEsaJd4/fq1VKxYUeLEiaP72LVrlzU021Z2N23aJGfOnJENGzbIqlWrrPd/9913uu2jR49KlixZpH79+vLmzRt97MKFC7qdL7/8Uo4dOyYLFizQINyuXTt9HKE5derUMnDgQOsxBadw4cJ6jM68fPlSHj9+bLcQERGR5+KsDh4KlVFUW4ODEDtp0iTJmDGj3kbARKgEhFlUXREO0SJhmD17tgQGBsqvv/5qrbhOnz5dq7+oCFeoUEHvixUrlq6DFgswKsIIvf7+/39E/4ABAyRnzpxy/vx5yZYtmwwdOlRbFDp16qSPZ86cWcaNGyelSpWSiRMnanU5SpQoGrptj8kZnD9eB2ewPxwDERERmQMrvh7q+fPnEiNGjGDXQbuBEXohRYoUcvv27WCf89dff2lQRfhEOMaCQPrixQut2Bpy5cplDb22cufObbc/MPaJbaMFw9guFlSXEbQvXbokoYXgjqq2M71795ZHjx5ZF7RwEBERkedixddDJU6cWB48eBDsOtGiRbO7jQquxWIJ9jlPnjzRFoo5c+a881iSJEms36PiG9I+jYoxgq2x7VatWmlfr6O0adNKaN2/f9/umByhBxoLERERmQODr4fKly+ftiV8CFRs3759a3df/vz5tfcW8wPHjRtXwhK2ferUKcmUKVOojsmZEydO6OtAREREBGx18FBoEcBUXiFVfUOaMxeDzDBI7e7du9oTjB5cVJMxkwMGjqEFAb29qNJeu3btg44Z8+7u3r1be40x+O3cuXM6q4QxuM04JgzEu379uh5TcHB8Rs8xEREREYOvh0KPLSqoCxcufO9ttGjRQqciK1iwoLYMYAYH9AUjeKL14IsvvtCp0DBXMHp8P7QCjP7fbdu2ydmzZ3VKM1Rrf/jhB7tBehh8h4Fy6E22bWNA2wT6gw2YJg19u1999dUHHRMRERF5Di9LSE2d5LZWr14t3bt314/8vb099z0Oqs6YGg1tEpgJAurWravzBPfp08fl7WA6M1zZDoE5rNs4iIiIKHyE5u83e3w9GKYNQ7sA2gJwMQhPvlhHy5YtraEX8wmj4o0LXRAREREZWPEl+g8rvkRERJ7999tzP/8mIiIiIrLB4EtEREREpsDgS0RERESmwOBLRERERKbA4EtEREREpsDgS0RERESmwOBLRERERKbA4EtEREREpsDgS0RERESmwEsWh5N79+5J9uzZZf/+/ZI+ffoIO44ZM2ZIp06d5OHDh2IWd+/elRw5csjhw4clderUoX6+X7/14u3jGy7HRq4JGOYf0YdAREQeiBXfcDJ48GCpUaNGhIZeT+Tl5SXLli0Ldp3EiRNLo0aNpF+/fh/tuIiIiCjyY/ANB8+ePZNp06ZJ8+bNI/pQTKtp06YyZ84cuX//fkQfChEREUUSDL7hYM2aNeLj4yNFihSx3nfixAmpXLmyxI4dW5IlSyYNGzbUj+QNpUuXlvbt22tbQoIECXSdqVOnytOnTzXExYkTRzJlyiRr1661Pmfr1q1aAV29erXkzp1bYsSIofvEvoIzceJEyZgxo0SPHl2yZs0qs2bNsj7WrFkzqVq1qt36r1+/lqRJk2qYf99jdfU16NChg/To0UMSJkwoyZMnl/79+1sfN6rntWrV0vMOrpqeM2dOSZkypSxdujTY14KIiIjMg8E3HOzYsUMKFChgvY3+2rJly0q+fPnk4MGDsm7dOvnnn3+kTp06ds+bOXOmfkyPvmAEy2+//VZq164txYoV037VChUqaFhERdlW9+7dZdSoUXLgwAFJkiSJVKtWTcNqUBAEO3bsKF27dtUg2qpVKw2rW7Zs0ce/+eYbPb6bN29an7Nq1SrdZ926dd/7WEPzGsSKFUv27dsnI0aMkIEDB8qGDRv0MZwfTJ8+XY/PuO1M4cKF9WfhzMuXL+Xx48d2CxEREXkuL4vFYonog/A0NWvWlESJElkrpD/++KMGsPXr11vXuXbtmqRJk0bOnDkjWbJk0Wrn27dvrUEN38eLF0+++OIL+f333/W+W7duSYoUKWTPnj1a2UXFt0yZMjJ//nxrKMVH+xjQhUFtCJWOg9uKFy+u1dApU6ZYjwXroVqLyjHg8caNG2vlFapXr67ng8AJ73Os7/MaGOEVgXnYsGF6G5VehHe8xiHp0qWLHDlyxBrqHaGaPGDAgHfuT9NpIQe3RTAObiMiIlehcIUc8ujRI4kbN26w67LiGw6eP3+ubQeGv/76S8MXPuI3lmzZsuljFy5csK6HdgVDlChRNGzmypXLeh/aA+D27dt2+ytatKj1e7QIoH3h9OnTQR4b7kf4tYXbtuuj6muEXFRl0bKAFghboT3W93kNAOHZ8XxdFTNmzHeq47Z69+6t/5MYy9WrV99rP0REROQeOJ1ZOEALwIMHD6y3nzx5ou0Hw4cPf2ddBDtDtGjR7B5DddP2PtyGwMBACU+YEaFXr15ard29e7d88sknUqJECbt1QnusH/IavO/5ovqN1g9n0IeNhYiIiMyBwTccoI919uzZ1tv58+eXxYsX62CsqFHD/iXfu3evpE2bVr9H4D579qzOIRwU3L9r1y5tZTDgNua9NaB6i1YCVH0RftED/KHC6jVAMEY7hCvQw4z2CSIiIiJgq0M4qFixopw8edJa9W3btq1WH+vXr68DsvDRPnpdEShdDXHBwQCwTZs2adBr0qSJVpyd9cBiIBz6fjGzw7lz52T06NGyZMkS6datm916aHfAQDO0QNiG5PcVVq8BgjPOFT3EtlV1R2hxOHTokA6yIyIiIgIG33CAXldUOBcuXKi3Ma0WqqoIeAhieBwDzuLHjy/e3h/+I8DAL8zUgJkkEAhXrlypU5UFBYF47NixMnLkSB3ENnnyZK3sOlZGP//8c21BQIjH8X+osHoNMHsFZnnAoDhU1iEgIEBbIjDYz7B8+XKtgju2aBAREZF5cVaHcIIZElBdRRU2LMJtUIxZHVD5RIAMS+jJTZUqlYZizNYQmWHQHI7x4sWLOq8wYCYJzAn89ddfh8uoUCIiIoocQvP3mz2+4cTf319bCa5fv67VSXeBgWS4qAQqqwjTmMrMHS4Y0qdPH2voxfEjCKOtgoiIiMjAiq8bC4+KL9oGMIuDMRdwuXLlxCxY8SUiInI/rPiaBPpyw/p9CwaP8b0QEREReSIObiMiIiIiU2DwJSIiIiJTYPAlIiIiIlNg8CUiIiIiU2DwJSIiIiJTYPAlIiIiIlNg8CUiIiIiU2DwJSIiIiJT4AUsIpl79+5J9uzZZf/+/XoxifCGq7N16tRJHj58KJGRcSW5I0eOSN68eV16Tq9eveTp06cyfvz499qnX7/14u3j+17PpbAXMMw/og+BiIg8BCu+kczgwYOlRo0aGnpxSWIvL69IG0ojq27dusnMmTPl4sWLEX0oREREFIkw+EYiz549k2nTpknz5s1D9bxXr16JJ3rf80qcOLFUrFhRJk6cGObHRERERO6LwTcSWbNmjfj4+EiRIkX0I/4yZcro/QkSJNDKb5MmTfR26dKlpV27dtqiYIQ8GD16tOTKlUtixYoladKkkTZt2siTJ0/eaW1Imzat+Pr6Sq1atbS1wtHy5cslf/78EiNGDMmQIYMMGDBA3rx5Y30cxzJ58mSpWrWqbgetGXv27JHz58/rsWH/xYoVkwsXLlifg+9RyU6WLJnEjh1bChUqJBs3brTbL6rcgwYNkkaNGkncuHGlZcuW7xzb27dvpVmzZpItWza5cuWK09eyWrVqMn/+/FC8+kREROTpGHwjkR07dkiBAgX0ewTXxYsX6/dnzpyRmzdvytixY63r4qP86NGjy65du2TSpEl6n7e3t4wbN05Onjypj2/evFl69Ohhfc6+ffu0mozQfPToUQ3WP/744zvHgODZsWNHOXXqlAZchGW0YNgyAiq2gxD69ddfS6tWraR3795y8OBBsVgsuh8DAniVKlVk06ZN2q9bqVIlDaeO4XXkyJGSJ08eXadv3752j718+VJq166t+8RxIsA7U7hwYbl27Zq+gXAG23v8+LHdQkRERJ7Ly4KEQpFCzZo1JVGiRNruAOjxRTh98OCBxI8f37oeqqoIaYcPHw52e3/88Ye0bt1a7t69q7cRTh89eiSrV6+2rlOvXj1Zt26dtY/4888/l3LlymmANcyePVsD9I0bN6wV3++//17DL+zdu1eKFi2qx41qLKDa2rRpU3n+/LnT4/Pz89PjMwIyKr758uWTpUuXvjO4DUG3f//+GlZXrVol8eLFC/bc8fpgHbyGpUqVCnIdbA/VbEdpOi3k4LZIhIPbiIjIlb/5yDj4xDg4rPhGIgiJaC9whVEZtoXWAYTWVKlSSZw4caRhw4bayoDeYTh9+rR8+umnds9BYLX1119/ycCBA7UdwVhatGihFWdjO5A7d27r92hfALRZ2N734sULaxUVFV8MOkNbBEI8tovjcaz4FixYMMjzrV+/vs7U8Oeff4YYeiFmzJj61faYHSHc438SY7l69WqI2yUiIiL3xeAbiaBfF9VdV6CP1hYqo+i5RSBFi8ShQ4dkwoQJoR4khoCKKijaCYzl+PHjcu7cObtQHi1aNOv3qAA7uy8wMFC/IvSikjtkyBCt3mK7CMqOx+Z4Xga0SRw7dkx7iV1x//59/ZokSRKn66CfGu8MbRciIiLyXJzHNxLBx/xoKzCgh9cY0BUSBF2EzFGjRmmvLyxcuNBuHVRb0edrC20KtjCoDT3FmTJlkrCEXmQMzsOAOiNgB9d/6+jbb7/V1ojq1atrq4az9gXDiRMnNIjnzJnzg4+diIiIPAMrvpEIZmfAwDSj6psuXTqtnKKn9c6dO+/M0GALQfX169d60QbMXztr1izroDdDhw4dtJ8XA8hQwf3555/1tq0ffvhBfv/9d6364ljQjoB+XfT0fojMmTPLkiVLtNKLdgr0GxvVYFe1b99eB+Ohsr1z585g10VVuUSJEtaWByIiIiIG30gEH/2j4mpUatGriwCKK5GhZ9Z2lgRHmAkB05kNHz5cK6Nz5syRoUOH2q2DadKmTp2qs0NgffTLOgZahG8EbTyGKcfwnJ9++klD+IfAsWFaNkxzhtkcsB+ca2hhCje8Jmh92L17t96HSjIG/NlCWEdvMhEREZGBszpEMvgYv3v37vpRvdGyQMFD2wNmv8AsDbB27Vrp2rWr9gRHjRo1XEaFEhERUeQQmr/f7PGNZPz9/bUN4fr16zqXLwUP/8hxcQzbKdow+8P06dNDFXqJiIjI87HiS/QfVnyJiIjcD+fxJSIiIiJywOBLRERERKbA4EtEREREpsDgS0RERESmwOBLRERERKbA4EtEREREpsDgS0RERESmwOBLRERERKbA4EtEREREpsBruoaje/fuSfbs2WX//v2SPn16Mav+/fvLsmXL5OjRox9lf6dOnZIKFSrImTNnJFasWKF+vl+/9eLt4xsux0ZkNgHD/CP6EIiIrFjxDUeDBw+WGjVqfLTQO2PGDIkfP758LDivMWPGSGSTI0cOKVKkiIwePTqiD4WIiIgiEQbfcPLs2TOZNm2aNG/eXCKbV69eiadr2rSpTJw4Ud68eRPRh0JERESRBINvOFmzZo34+Pho5dFw8uRJqVq1qsSNG1fixIkjJUqUkAsXLuhjgYGBMnDgQEmdOrU+L2/evLJu3TrrcwMCAsTLy0uWLFkiZcqUEV9fX8mTJ4/s2bNHH9+6dauGvUePHul6WNBiYFRmBw0aJI0aNdJ9t2zZUu/v2bOnZMmSRbeVIUMG6du3r7x+/druPFauXCmFChWSGDFiSOLEiaVWrVp6f+nSpeXy5cvSuXNn6/5cdeDAASlfvrxuL168eFKqVCk5fPiw9fFu3brp62RAVRnbt309MmXKJL/++qvTfWD79+/fl23btrl8XEREROTZGHzDyY4dO6RAgQLW29evX5eSJUtqqN28ebMcOnRImjVrZq1Ijh07VkaNGiUjR46UY8eOScWKFaV69epy7tw5u+1+9913GgzRL4vQWr9+fd1GsWLFNCAi2N68eVMXrGfAdhGUjxw5ogEXEL7RHoGeWOx/6tSp8tNPP1mfs3r1ag26VapU0edt2rRJChcurI8hgCOkI6wb+3PVv//+K40bN5adO3fK3r17JXPmzLoP3A8Iwnjs7du3ehvhFSEZ4d54LfGGAeHbmejRo+ubB/wciIiIiICD28IJqqEpU6a03p4wYYJWN+fPny/RokXT+xBcbYMpKrD16tXT28OHD5ctW7ZomMVzDQiz/v7/f7DIgAEDJGfOnHL+/HnJli2bbh+V0eTJk79zPGXLlpWuXbva3ff9999bv0dVGNvG8fXo0cPao4zjwX4MCM+QMGFCiRIliobnoPYXHByLrSlTpmhvMgIuKr2ohCMEI2zjzcP27dule/fuOkAOEIBTpUqlVd/g4PXHz8GZly9f6mJ4/PhxqM6DiIiI3AsrvuHk+fPn2h5gQIUWgc4IvbYQuG7cuCHFixe3ux+3T58+bXdf7ty5rd+nSJFCv96+fTvE4ylYsOA79y1YsED3geAaO3ZsDcJXrlyxO+Zy5cpJaGA7xtK6desg1/nnn3+kRYsWWulFWEeV+smTJ9Z9IwQjYCPgHj9+XKu3aM9AEMZ6CMioCockZsyY2mvtzNChQ3X/xpImTZpQnSsRERG5F1Z8wwk+mn/w4IFdCAsLtsHZ6KtFf3BIHKf1Qm9wgwYNtJqLtgqjGo12iw85ZtspyxBog4I2B0z1hvaKdOnSaftH0aJF7QbdoY0BwRePIeSiwoyp4dACgeDrWL0OCnp8M2bM6PTx3r17S5cuXezegDD8EhEReS5WfMNJvnz5tHfWtlKLflPHwWNGQMTH8rt27bK7H7cxNZerUBk1+mJDsnv3bg2d6BlGNRjVV8e2ABwz+npDsz+0HxhL0qRJg3wezqtDhw7a14tWDYTbu3fv2q1j9Pli/0YvL77OmzdPzp49G2x/r+HEiRP6c3AG+8Vrb7sQERGR52LwDSeoomIWB6Pq265dO60oomf24MGDOmht1qxZepEFQA8r+nrRfoD7evXqpdXTjh07urxP9OmiFQBhEUEyuI/5EXTRWoAqLwaKjRs3TpYuXWq3Tr9+/TRo4itaLtB2gGO03R/6bzHYzDG4Bgf7xrljm/v27dPKs2N1GQMB0ee7atUqu+A7Z84cbfGw7Y8OCmbBwHF9/vnnLh8XEREReTYG33CSK1cuyZ8/vyxcuFBvJ0qUSGdzQDBFNRODtjCLgtG6gAooPnbHR/h4LqbuWrFihYZEV2FmB/TV1q1bV5IkSSIjRoxwui5mjMBUZAjkmP0AFWBjtgcDguaiRYv0OLAOBqXhKnQGzOiAgIl2AuzPVZjfGG8I8Po0bNhQz92xOpwgQQJ9HbBdDNwzwjDaOhz7e9ESgbYPHIsBgR1Xb0NVm4iIiAi8LBaLhS9F+MB0YKjk4iN3b2++xwgv06dPlyFDhmhrCd5IoFcYbxjmzp37zoDB4KAij15nzIXMtgciIiL3EJq/3xzcFo4w7RhaGvCROwdNhe/FQhB8jeo5Wjj69OkTqtBLREREno8VX6L/sOJLRETk2X+/+fk7EREREZkCgy8RERERmQKDLxERERGZAoMvEREREZkCgy8RERERmQKDLxERERGZAoMvEREREZkCgy8RERERmQKDLxERERGZAi9Z/BHcu3dPsmfPLvv375f06dO/1zbwvE6dOunyvvr37y/Lli2To0ePSljZunWrlClTRh48eCDx48f/aPsNzqlTp6RChQpy5swZiRUrVqif79dvvXj7+IbLsRGRvYBh/hF9CERkIqz4fgSDBw+WGjVqvHfohQMHDkjLli0/6Di6desmmzZtEk+XI0cOKVKkiIwePTqiD4WIiIgiEQbfcPbs2TOZNm2aNG/e/L2e/+rVK/2aJEkS8fX9sCpk7NixJVGiRGIGTZs2lYkTJ8qbN28i+lCIiIgokmDwDWdr1qwRHx8frUAaTp48KVWrVpW4ceNKnDhxpESJEnLhwgV9rEmTJlKzZk2tEqdMmVKyZs2q96NaPGbMGOs2vLy8ZPLkybodBGK0UuzZs0fOnz8vpUuX1o/4ixUrZt2u0XKQN29e621jXyNHjpQUKVJoKG7btq28fv3aus6sWbOkYMGCepzJkyeXr7/+Wm7fvv1Brwmq1+XLl5fEiRNLvHjxpFSpUnL48GG7yjTOy4DzxvmuW7fOel+mTJnk119/dboPbP/+/fuybdu2DzpWIiIi8hwMvuFsx44dUqBAAevt69evS8mSJTUMb968WQ4dOiTNmjWzq0yiHQH9qRs2bJBVq1Y53fagQYOkUaNG2jubLVs2DaWtWrWS3r17y8GDB8VisUi7du2CPb4tW7ZoOMbXmTNnyowZM3QxIARjP3/99Zf26QYEBGhg/hD//vuvNG7cWHbu3Cl79+6VzJkzS5UqVfR+QBDGY2/fvtXbCK8IyegnNl5DHDMCvjPRo0fXkI/Xn4iIiAg4uC2cXb58WSu3hgkTJmiVc/78+RItWjS9L0uWLHbPQbUW1UyEt5A+zq9Tp45+37NnTylatKj07dtXKlasqPd17NhR1wlOggQJ5Oeff5YoUaJoePb399fg3aJFC30codyQIUMGGTdunBQqVEiePHmirRPvo2zZsna3p0yZogPjEHBR6UUFHCH4yJEj+qZh+/bt0r17dw3egACcKlUqrfoGB687Xn9nXr58qYvh8ePH73U+RERE5B5Y8Q1nz58/lxgxYlhvozqLYGeE3qDkypUrxNALuXPntn6fLFky63Nt73vx4kWwgS5nzpwaeg1oebBtZUBFulq1apI2bVptd0A1Fq5cuRLk9hCGjaV169ZBrvPPP/9osEalF28C0PKBIG1sEyE4T548GnCPHz+urwUG9iEIYz0EZOM4ghMzZkztsXZm6NChun9jSZMmTYjbJCIiIvfFim84w0f0mOrLNoyFxNUpuGzDM3pgnd0XGBjo0jaM5xjrP336VKvHWObMmaMD7BBOcdsYdOfIdsoyBNqgoM0BU7yNHTtW0qVLp20fqFbbbhNtDAi+eAwhN2HChNrHjBYIBN+uXbuG+PqgxzdjxoxOH0dLSJcuXay38QaB4ZeIiMhzMfiGs3z58sns2bPtqrTopUXvbHBV38jg77//1oA6bNgwayBE73BwQmo/gF27dskvv/yifb1w9epVuXv3rt06CLu//fabRI0aVSpVqmQNw/PmzZOzZ88G299rOHHihHz11VdOH0eoxkJERETmwFaHcIbqKGZxMKq+GGyGymK9evU0RJ47d05nTsBgtsgG7Q1oMxg/frxcvHhRVqxYoQPdPhRaHHDOp0+fln379kmDBg3eqYRjACD6fDG4zwi5+IrKM9oxHPuiHWEQHgbBff755x98vEREROQZGHzDGXpu8+fPLwsXLtTbmDIMszmgVxVVTQzemjp1aqSs/qK1ATM8LFq0SC8Kgcovpj77UJjXGG8E8Lo0bNhQOnToIEmTJn1n0B1eOxwDBt0ZYRhtGI79vWiJQIsGwq4BlWFcvQ2tFERERETgZcGcVxSuVq9erbMS4KN3b2++1whr06dPlyFDhuilivEGAr3CqCrPnTtXihcv7vJ2UInHILdHjx457U8mIiKiyCU0f7/Z4/sRYIowtDTgo3cOngqfi4Qg+BpVcwzA69OnT6hCLxEREXk+VnyJ/sOKLxERkWf//ebn7kRERERkCgy+RERERGQKDL5EREREZAoMvkRERERkCgy+RERERGQKDL5EREREZAoMvkRERERkCgy+RERERGQKDL5EREREZAq8ZDGRA79+68XbxzeiD4OIiMijBAzzj+hDYMXX3dy7d0+SJk0qAQEBEplt3bpVvLy85OHDhy4/p3///pI3b94w2X+RIkVk8eLFYbItIiIi8gwMvm5m8ODBUqNGDUmfPr1EZsWKFZObN2/qtbPDUunSpaVTp04hrvf9999Lr169JDAwMEz3T0RERO6LwdeNPHv2TKZNmybNmzeXyOz169cSPXp0SZ48uVZ9I0LlypXl33//lbVr10bI/omIiCjyYfB1I2vWrBEfHx/9GN9w8uRJqVq1qsSNG1fixIkjJUqUkAsXLuhjqHYOHDhQUqdOrc9DG8G6deusz0W7BILp/PnztUIbI0YM8fPzk23btunjFotFMmXKJCNHjrQ7jqNHj+rzzp8/r7fx/cSJE6V69eoSK1YsrUoH1eowdepUSZMmjfj6+kqtWrVk9OjREj9+/HfOc9asWVrRRrW4Xr16GmChSZMmemxjx47VbWNx1vIRJUoUqVKlip4bERERETD4upEdO3ZIgQIFrLevX78uJUuW1FC7efNmOXTokDRr1kzevHmjjyMgjho1SoPrsWPHpGLFihpOz507Z7fd7t27S9euXeXIkSNStGhRqVatmvYSI1hie9OnT7dbH7exX4Ri2/5chNnjx4/rcxzt2rVLWrduLR07dtTgXL58eQ3IjhDaly1bJqtWrdIFQXfYsGHW88HxtWjRQtsosCBIO1O4cGF9zZx5+fKlPH782G4hIiIiz8Xg60YuX74sKVOmtN6eMGGCVkVR1SxYsKBkyZJFmjZtKlmzZtXHEXh79uypVVPcN3z4cK36jhkzxm677dq1ky+//FKyZ8+ulVtsEy0VRpX1zJkzsn//fmsbw9y5c98Jt19//bXuO0OGDJI2bdp3jn38+PHaftCtWzc9zjZt2uhtR6hSz5gxQyvPqF43bNhQNm3apI/huNBCgYox2iiwoLLrDF6rq1evOu3zHTp0qG7TWIIL0UREROT+GHzdyPPnz7UdwYDKKcJhtGjR3lkX1csbN25I8eLF7e7H7dOnT9vdhyqqIWrUqBqijXUQHv39/eW3337T2ytXrtRKae3ate22gecEB+EZFVhbjrcBLQ5o2TCkSJFCbt++Le8jZsyYGnpxvEHp3bu3PHr0yLogJBMREZHnYvB1I4kTJ5YHDx7YBbuP4ZtvvtGqMoI32hzq1q2rVVdb6O0NC44hHu0W7zszw/379/W4nL1OaBFBb7TtQkRERJ6LwdeN5MuXT06dOmW9nTt3bu1hRfuBI4Q4VGvRW2sLt3PkyGF33969e63foz8YvcJoezBgkBgCJNogMDguqB7ekKDV4sCBA3b3Od52BVod3r5969K6J06c0NeMiIiICBh83QgGp2EWB6Pqi95ctDSgh/fgwYM6aA0zIqCtwBi0hr7eBQsW6H2Y1xbtERhgZgu9wkuXLpW///5b2rZtq9u3Dbfoo0WvL1oDMmfObNca4ar27dvrrBSYyQHHOXnyZJ1qLLTTnaEVYt++fTqbw927d4OtBuNNQYUKFUJ9rEREROSZGHzdSK5cuSR//vyycOFCvZ0oUSKdzeHJkydSqlQpnfEBU4YZ7QIdOnSQLl266IwNeC6qtStWrNDwaguzJmDJkyeP7Ny5U9dBW4UtzB386tUrHcD2PtBbPGnSJA2+2A+OpXPnznY9y67A4DgEcVStkyRJIleuXLEGYswsYTvjxe7du9/7eImIiMjzeFkwWSu5jdWrV2slFx/je3t/2PsWVE0/+eQTncYspEsFo3parlw5HQCWLFkyCQuYlgxV5uCmHHP1wh54E4AKMq7sBpjNApXrKVOmuLwdVM8xuwMGurHfl4iIyD2E5u931I92VBQmMMMCWgVQ0fwY029hRoQ7d+5oNRUzOXxI6MX0api/F/3CCKkzZ86UX3755YOPccuWLVK2bFlr6IWkSZNqtZuIiIjIwIqviblS8cWcumhzwONogUiVKtV7769OnTp6RTdciQ3z/aLvFxe1iCxY8SUiInI/ofn7zeBL9B8GXyIiIvfDVgei92C8B+Sli4mIiNyH8XfblVougy/Rf+7du6dfeeliIiIi94NWSlR+g8PgS/SfhAkT6ldMkRbS/zju+o4YoR4zc3hiKwfPz73x/Nwbz8+9PXbz80OlF6EXF+4KCYMv0X+M6eEQet3xf3xXefrlmXl+7o3n5954fu4trhufn6sFK17AgoiIiIhMgcGXiIiIiEyBwZfoPz4+PtKvXz/96ol4fu6N5+feeH7ujefnOTiPLxERERGZAiu+RERERGQKDL5EREREZAoMvkRERERkCgy+RERERGQKDL5E/5kwYYKkT59eYsSIIZ9++qns379f3NHQoUOlUKFCEidOHEmaNKnUrFlTzpw5Y7fOixcvpG3btpIoUSKJHTu2fPnll/LPP/+Iuxk2bJh4eXlJp06dPObcrl+/Lv/3f/+nxx8zZkzJlSuXHDx40Po4xiP/8MMPkiJFCn38888/l3Pnzok7ePv2rfTt21c++eQTPfaMGTPKoEGD9Jzc8fy2b98u1apV06tF4d/hsmXL7B535Vzu378vDRo00IsGxI8fX5o3by5PnjyRyH5+r1+/lp49e+q/z1ixYuk6jRo1khs3bnjE+Tlq3bq1rjNmzBiPOr/Tp09L9erV9eIP+DnibweuXuopv0+DwuBLJCILFiyQLl266HQuhw8fljx58kjFihXl9u3b4m62bdumv6j27t0rGzZs0D9QFSpUkKdPn1rX6dy5s6xcuVIWLVqk6+OP1RdffCHu5MCBAzJ58mTJnTu33f3ufG4PHjyQ4sWLS7Ro0WTt2rVy6tQpGTVqlCRIkMC6zogRI2TcuHEyadIk2bdvn/6xwr9V/IGK7IYPHy4TJ06Un3/+Wf/g4jbOZ/z48W55fvh/Cr8r8KY5KK6cC0LTyZMn9f/VVatWaVhp2bKlRPbze/bsmf6uxBsZfF2yZIm+wUaIsuWu52dr6dKl+vs0qMvhuvP5XbhwQT777DPJli2bbN26VY4dO6Y/TxR/POH3qVOYzozI7AoXLmxp27at9fbbt28tKVOmtAwdOtTi7m7fvo1ymmXbtm16++HDh5Zo0aJZFi1aZF3n9OnTus6ePXss7uDff/+1ZM6c2bJhwwZLqVKlLB07dvSIc+vZs6fls88+c/p4YGCgJXny5Jb//e9/1vtwzj4+PpZ58+ZZIjt/f39Ls2bN7O774osvLA0aNHD788O/saVLl1pvu3Iup06d0ucdOHDAus7atWstXl5eluvXr1si8/kFZf/+/bre5cuXPeb8rl27ZkmVKpXlxIkTlnTp0ll++ukn62Pufn5169a1/N///Z/T57j771NnWPEl03v16pUcOnRIP4Y0eHt76+09e/aIu3v06JF+TZgwoX7FuaIKbHu+eMefNm1atzlfVLT9/f3tzsETzm3FihVSsGBBqV27trap5MuXT6ZOnWp9/NKlS3Lr1i2788NHlGjNcYfzK1asmGzatEnOnj2rt//66y/ZuXOnVK5c2SPOz5Yr54Kv+HgcP3MD1sfvH1SI3fF3DT5Sxzl5wvkFBgZKw4YNpXv37pIzZ853Hnfn8wsMDJTVq1dLlixZ9FMI/L7Bv03bdgh3/33qDIMvmd7du3e19zBZsmR29+M2/nC5M/xyQ/8rPj738/PT+3BO0aNHt/5xcrfznT9/vn60il5mR+5+bhcvXtRWgMyZM8v69evl22+/lQ4dOsjMmTP1ceMc3PXfaq9evaRevXr6xxPtHAj2+PeJj4s94fxsuXIu+IrAYStq1Kj6JtXdzhftG+j5rV+/vva7esL5oRUHx4v/B4Pizud3+/Zt7UXGOIlKlSrJn3/+KbVq1dI2BrQ0eMLvU2eiRvQBEFH4VkZPnDihVTVPcPXqVenYsaP209n2oXkKvFFB9WjIkCF6G8EQPz/0iDZu3Fjc3cKFC2XOnDkyd+5craAdPXpUgy96Jz3h/MwKVcE6deroYD68cfMEqHaOHTtW32Sjiu2Jv2ugRo0a2scLefPmld27d+vvm1KlSomnYsWXTC9x4sQSJUqUd0aq4nby5MnFXbVr104HW2zZskVSp05tvR/nhPaOhw8fut354o8RKhX58+fXygoWVCcwgAjfoxLhrucGGP2fI0cOu/uyZ89uHWVtnIO7/lvFR8ZG1RezAeBjZPzRNar37n5+tlw5F3x1HED75s0bnSnAXc7XCL2XL1/WN6RGtdfdz2/Hjh167PhY3/hdg3Ps2rWrzv7j7ueXOHFiPaeQft+48+9TZxh8yfTwUU6BAgW099D23TBuFy1aVNwNqi4IvRiJvHnzZp06yhbOFR8z254vRmPjl11kP99y5crJ8ePHtVJoLKiQ4qNy43t3PTdAS4rj1HPoh02XLp1+j58l/uDYnt/jx4+1n9Adzg8zAaD/0RbedBrVJ3c/P1uunAu+IlTgDZ0B/8/i9UC/pbuEXkzRtnHjRp3yypY7nx/elGGWA9vfNfhkAm/e0Ibk7ucXPXp0nbosuN837vy3IlgRPbqOKDKYP3++jraeMWOGjtRt2bKlJX78+JZbt25Z3M23335riRcvnmXr1q2WmzdvWpdnz55Z12ndurUlbdq0ls2bN1sOHjxoKVq0qC7uyHZWB3c/N4yKjxo1qmXw4MGWc+fOWebMmWPx9fW1zJ4927rOsGHD9N/m8uXLLceOHbPUqFHD8sknn1ieP39uiewaN26sI+RXrVpluXTpkmXJkiWWxIkTW3r06OGW54fZRY4cOaIL/pyOHj1avzdmNXDlXCpVqmTJly+fZd++fZadO3fqbCX169e3RPbze/XqlaV69eqW1KlTW44ePWr3u+bly5duf35BcZzVwd3Pb8mSJTprw5QpU/T3zfjx4y1RokSx7NixwyN+nzrD4Ev0H/xPj//Bo0ePrtOb7d271+KO8AsuqGX69OnWdfCHt02bNpYECRJosKpVq5b+wfKE4Ovu57Zy5UqLn5+fvhHLli2b/lGyhWmy+vbta0mWLJmuU65cOcuZM2cs7uDx48f6s8L/ZzFixLBkyJDB8t1339kFJXc6vy1btgT5/xoCvqvncu/ePQ1KsWPHtsSNG9fStGlTDSyR/fzwxsXZ7xo8z93Pz9Xg6+7nN23aNEumTJn0/8c8efJYli1bZrcNd/99GhQv/Ceiq85EREREROGNPb5EREREZAoMvkRERERkCgy+RERERGQKDL5EREREZAoMvkRERERkCgy+RERERGQKDL5EREREZAoMvkRERERkCgy+RERERGQKDL5EREREZAoMvkRERERkCgy+RERERCRm8P8ADwW9pv2jwUYAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "train_labels = pd.DataFrame(dataset['train']['label']).value_counts()\n",
    "train_labels.plot(kind='barh')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "b68d2392",
   "metadata": {},
   "outputs": [],
   "source": [
    "# balance the dataset and take just 4 classes\n",
    "from datasets import concatenate_datasets\n",
    "\n",
    "NUM_TRAINING_SAMPLES = 30\n",
    "NUM_VAL_SAMPLES = 7\n",
    "NUM_TESTING_SAMPLES = 5\n",
    "\n",
    "# criminal-law \n",
    "label_6_train_data = dataset['train'].filter(lambda example: example[\"label\"] == 'criminal-law').select(range(NUM_TRAINING_SAMPLES))\n",
    "label_6_val_data = dataset['validation'].filter(lambda example: example[\"label\"] == 'criminal-law').select(range(NUM_VAL_SAMPLES))\n",
    "label_6_test_data = dataset['test'].filter(lambda example: example[\"label\"] == 'criminal-law').select(range(NUM_TESTING_SAMPLES))\n",
    "# employment\n",
    "label_7_train_data = dataset['train'].filter(lambda example: example[\"label\"] == 'employment').select(range(NUM_TRAINING_SAMPLES))\n",
    "label_7_val_data = dataset['validation'].filter(lambda example: example[\"label\"] == 'employment').select(range(NUM_VAL_SAMPLES))\n",
    "label_7_test_data = dataset['test'].filter(lambda example: example[\"label\"] == 'employment').select(range(NUM_TESTING_SAMPLES))\n",
    "# tax-law \n",
    "label_14_train_data = dataset['train'].filter(lambda example: example[\"label\"] == 'tax-law').select(range(NUM_TRAINING_SAMPLES))\n",
    "label_14_val_data = dataset['validation'].filter(lambda example: example[\"label\"] == 'tax-law').select(range(NUM_VAL_SAMPLES))\n",
    "label_14_test_data = dataset['test'].filter(lambda example: example[\"label\"] == 'tax-law').select(range(NUM_TESTING_SAMPLES))\n",
    "# trademark\n",
    "label_15_train_data = dataset['train'].filter(lambda example: example[\"label\"] == 'trademark').select(range(NUM_TRAINING_SAMPLES))\n",
    "label_15_val_data = dataset['validation'].filter(lambda example: example[\"label\"] == 'trademark').select(range(NUM_VAL_SAMPLES))\n",
    "label_15_test_data = dataset['test'].filter(lambda example: example[\"label\"] == 'trademark').select(range(NUM_TESTING_SAMPLES))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "64e6a9d6",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Dataset({\n",
       "    features: ['text', 'label'],\n",
       "    num_rows: 120\n",
       "})"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_data = concatenate_datasets([label_6_train_data, label_7_train_data, label_14_train_data, label_15_train_data])\n",
    "train_data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "5a245615",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Dataset({\n",
       "    features: ['text', 'label'],\n",
       "    num_rows: 28\n",
       "})"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "val_data = concatenate_datasets([label_6_val_data, label_7_val_data, label_14_val_data, label_15_val_data])\n",
    "val_data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "0d88821c",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Dataset({\n",
       "    features: ['text', 'label'],\n",
       "    num_rows: 20\n",
       "})"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test_data = concatenate_datasets([label_6_test_data, label_7_test_data, label_14_test_data, label_15_test_data])\n",
    "test_data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "bed0639f",
   "metadata": {},
   "outputs": [],
   "source": [
    "NUM_LABELS = 4"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "6a62938f",
   "metadata": {},
   "outputs": [],
   "source": [
    "# need to properly encode the labels \n",
    "from datasets import Dataset "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "fb4de03d",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'text': Value(dtype='string', id=None),\n",
       " 'label': Value(dtype='string', id=None)}"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_data.features"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "5f2cd85a",
   "metadata": {},
   "outputs": [],
   "source": [
    "def encode_labels(dataset):\n",
    "    encoded_data = pd.DataFrame(dataset)\n",
    "    encoded_data = Dataset.from_pandas(encoded_data)\n",
    "    encoded_data = encoded_data.class_encode_column(\"label\")\n",
    "    return encoded_data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "17625ec6",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "d7c3485b7afc48f6bfc03dc42a1d560b",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Casting to class labels:   0%|          | 0/120 [00:00<?, ? examples/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "f95041015136458dac039f73dc7e424d",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Casting to class labels:   0%|          | 0/28 [00:00<?, ? examples/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "15653d04132a4499a6800cf353a8da7d",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Casting to class labels:   0%|          | 0/20 [00:00<?, ? examples/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "train_data = encode_labels(train_data)\n",
    "val_data = encode_labels(val_data)\n",
    "test_data = encode_labels(test_data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "1d4e9021",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'text': Value(dtype='string', id=None),\n",
       " 'label': ClassLabel(names=['criminal-law', 'employment', 'tax-law', 'trademark'], id=None)}"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_data.features"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "8d6a17c2",
   "metadata": {},
   "outputs": [],
   "source": [
    "# create id2label and label2id\n",
    "labels = train_data.features['label'].names\n",
    "ids = range(NUM_LABELS)\n",
    "id2label = dict(zip(ids, labels))   \n",
    "label2id = dict(zip(labels, ids)) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "86e0f069",
   "metadata": {},
   "outputs": [],
   "source": [
    "# tokenize\n",
    "from transformers import AutoTokenizer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "00b688a9",
   "metadata": {},
   "outputs": [],
   "source": [
    "tokenizer = AutoTokenizer.from_pretrained(model_name)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "97fb3daf",
   "metadata": {},
   "outputs": [],
   "source": [
    "def tokenize_function(examples):\n",
    "    return tokenizer(examples[\"text\"], padding=\"max_length\", truncation=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "e15dcdb3",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "f92525ab425e42c1b9df4b70899fc726",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Map:   0%|          | 0/120 [00:00<?, ? examples/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "ca61c7b4b5d54fae90f3b12605c4c1ff",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Map:   0%|          | 0/28 [00:00<?, ? examples/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "2b6c9f2815bc4fc4974ed254708edad0",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Map:   0%|          | 0/20 [00:00<?, ? examples/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "tokenized_train = train_data.map(tokenize_function, batched=True)\n",
    "tokenized_val = val_data.map(tokenize_function, batched=True)\n",
    "tokenized_test = test_data.map(tokenize_function, batched=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "a9cea1fc",
   "metadata": {},
   "outputs": [],
   "source": [
    "tokenized_train = tokenized_train.remove_columns([\"text\"])\n",
    "tokenized_val = tokenized_val.remove_columns([\"text\"])\n",
    "tokenized_test = tokenized_test.remove_columns([\"text\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "017b06d8",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Dataset({\n",
       "    features: ['label', 'input_ids', 'attention_mask'],\n",
       "    num_rows: 120\n",
       "})"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tokenized_train"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3b586dc9",
   "metadata": {},
   "source": [
    "## Train model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "60dcaac7",
   "metadata": {},
   "outputs": [],
   "source": [
    "from transformers import AutoModelForSequenceClassification"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "b07d544e",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Some weights of DistilBertForSequenceClassification were not initialized from the model checkpoint at distilbert-base-cased and are newly initialized: ['classifier.bias', 'classifier.weight', 'pre_classifier.bias', 'pre_classifier.weight']\n",
      "You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.\n"
     ]
    }
   ],
   "source": [
    "model = AutoModelForSequenceClassification.from_pretrained(model_name, num_labels=NUM_LABELS, label2id=label2id, id2label=id2label)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "id": "74d23bcc",
   "metadata": {},
   "outputs": [],
   "source": [
    "from transformers import TrainingArguments"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "id": "90f79a91",
   "metadata": {},
   "outputs": [],
   "source": [
    "training_args = TrainingArguments(output_dir=\"my_test_model\", eval_strategy=\"epoch\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "id": "3ef305fe",
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import evaluate"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "id": "445a0f2f",
   "metadata": {},
   "outputs": [],
   "source": [
    "metric = evaluate.load(\"accuracy\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "id": "42fdc6ee",
   "metadata": {},
   "outputs": [],
   "source": [
    "def compute_metrics(eval_pred):\n",
    "    logits, labels = eval_pred\n",
    "    predictions = np.argmax(logits, axis=-1)\n",
    "    return metric.compute(predictions=predictions, references=labels)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "id": "3dcac915",
   "metadata": {},
   "outputs": [],
   "source": [
    "from transformers import Trainer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "id": "3371234b",
   "metadata": {},
   "outputs": [],
   "source": [
    "trainer = Trainer(\n",
    "    model=model,\n",
    "    args=training_args,\n",
    "    train_dataset=tokenized_train,\n",
    "    eval_dataset=tokenized_val,\n",
    "    compute_metrics=compute_metrics,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "id": "3c853c60",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "    <div>\n",
       "      \n",
       "      <progress value='45' max='45' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
       "      [45/45 15:54, Epoch 3/3]\n",
       "    </div>\n",
       "    <table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       " <tr style=\"text-align: left;\">\n",
       "      <th>Epoch</th>\n",
       "      <th>Training Loss</th>\n",
       "      <th>Validation Loss</th>\n",
       "      <th>Accuracy</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>No log</td>\n",
       "      <td>1.089008</td>\n",
       "      <td>0.857143</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>No log</td>\n",
       "      <td>0.570478</td>\n",
       "      <td>0.892857</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>No log</td>\n",
       "      <td>0.421259</td>\n",
       "      <td>0.892857</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table><p>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "TrainOutput(global_step=45, training_loss=0.8214864095052083, metrics={'train_runtime': 977.2892, 'train_samples_per_second': 0.368, 'train_steps_per_second': 0.046, 'total_flos': 47689964421120.0, 'train_loss': 0.8214864095052083, 'epoch': 3.0})"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "trainer.train()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "295d691b",
   "metadata": {},
   "source": [
    "## Evaluate model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "id": "ef9c330e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "    <div>\n",
       "      \n",
       "      <progress value='3' max='3' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
       "      [3/3 00:08]\n",
       "    </div>\n",
       "    "
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "{'eval_loss': 0.3957909047603607,\n",
       " 'eval_accuracy': 0.9,\n",
       " 'eval_runtime': 14.7954,\n",
       " 'eval_samples_per_second': 1.352,\n",
       " 'eval_steps_per_second': 0.203,\n",
       " 'epoch': 3.0}"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "trainer.evaluate(tokenized_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4b7b85d3",
   "metadata": {},
   "source": [
    "## Hyperparameter tuning"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "id": "fbc2cd70",
   "metadata": {},
   "outputs": [],
   "source": [
    "import optuna"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "id": "956a02fc",
   "metadata": {},
   "outputs": [],
   "source": [
    "# initialise a new model for each tuning run\n",
    "def model_init():\n",
    "    return AutoModelForSequenceClassification.from_pretrained(model_name, num_labels=NUM_LABELS, label2id=label2id, id2label=id2label)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "id": "c6f5b64b",
   "metadata": {},
   "outputs": [],
   "source": [
    "# function to control the search of hyperparams to tune\n",
    "def hp_space(trial):\n",
    "    return {\n",
    "        \"learning_rate\": trial.suggest_float(\"learning_rate\", 1e-5, 5e-4, log=True),\n",
    "        \"per_device_train_batch_size\": trial.suggest_categorical(\"per_device_train_batch_size\", [8, 16, 32]),\n",
    "        \"num_train_epochs\": trial.suggest_categorical(\"num_train_epochs\", [2, 3, 5])\n",
    "    }"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "id": "3516dfed",
   "metadata": {},
   "outputs": [],
   "source": [
    "# create the training args for each tuning run\n",
    "tuning_training_args = TrainingArguments(\n",
    "    output_dir=\"model_tuning_output\",\n",
    "    eval_strategy=\"epoch\",\n",
    "    learning_rate=0.1,  # Placeholder; will be overridden by the hyperparameter search\n",
    "    per_device_train_batch_size=16,  # Placeholder; will be overridden by the hyperparameter search\n",
    "    num_train_epochs=3,  # Placeholder; will be overridden by the hyperparameter search\n",
    "    logging_dir=\"./logs\",\n",
    "    #push_to_hub=True\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "id": "9929cd35",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Some weights of DistilBertForSequenceClassification were not initialized from the model checkpoint at distilbert-base-cased and are newly initialized: ['classifier.bias', 'classifier.weight', 'pre_classifier.bias', 'pre_classifier.weight']\n",
      "You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.\n"
     ]
    }
   ],
   "source": [
    "tuning_trainer = Trainer(\n",
    "    model_init=model_init,\n",
    "    args=tuning_training_args,\n",
    "    train_dataset=tokenized_train,\n",
    "    eval_dataset=tokenized_val,\n",
    "    processing_class=tokenizer,\n",
    "    compute_metrics=compute_metrics\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "id": "9fc9e05d",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2025-02-01 16:08:23,814] A new study created in memory with name: no-name-2be3205c-c33e-4b02-8b56-2ae2b7ccd655\n",
      "Some weights of DistilBertForSequenceClassification were not initialized from the model checkpoint at distilbert-base-cased and are newly initialized: ['classifier.bias', 'classifier.weight', 'pre_classifier.bias', 'pre_classifier.weight']\n",
      "You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "    <div>\n",
       "      \n",
       "      <progress value='16' max='16' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
       "      [16/16 04:50, Epoch 2/2]\n",
       "    </div>\n",
       "    <table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       " <tr style=\"text-align: left;\">\n",
       "      <th>Epoch</th>\n",
       "      <th>Training Loss</th>\n",
       "      <th>Validation Loss</th>\n",
       "      <th>Accuracy</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>No log</td>\n",
       "      <td>0.522721</td>\n",
       "      <td>0.892857</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>No log</td>\n",
       "      <td>0.209599</td>\n",
       "      <td>0.964286</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table><p>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2025-02-01 16:13:34,507] Trial 0 finished with value: 0.9642857142857143 and parameters: {'learning_rate': 0.0002728032165901902, 'per_device_train_batch_size': 16, 'num_train_epochs': 2}. Best is trial 0 with value: 0.9642857142857143.\n",
      "Some weights of DistilBertForSequenceClassification were not initialized from the model checkpoint at distilbert-base-cased and are newly initialized: ['classifier.bias', 'classifier.weight', 'pre_classifier.bias', 'pre_classifier.weight']\n",
      "You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "    <div>\n",
       "      \n",
       "      <progress value='30' max='30' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
       "      [30/30 05:05, Epoch 2/2]\n",
       "    </div>\n",
       "    <table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       " <tr style=\"text-align: left;\">\n",
       "      <th>Epoch</th>\n",
       "      <th>Training Loss</th>\n",
       "      <th>Validation Loss</th>\n",
       "      <th>Accuracy</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>No log</td>\n",
       "      <td>0.373767</td>\n",
       "      <td>0.857143</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>No log</td>\n",
       "      <td>0.431960</td>\n",
       "      <td>0.892857</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table><p>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2025-02-01 16:18:50,407] Trial 1 finished with value: 0.8928571428571429 and parameters: {'learning_rate': 0.0004841251983791936, 'per_device_train_batch_size': 8, 'num_train_epochs': 2}. Best is trial 0 with value: 0.9642857142857143.\n",
      "Some weights of DistilBertForSequenceClassification were not initialized from the model checkpoint at distilbert-base-cased and are newly initialized: ['classifier.bias', 'classifier.weight', 'pre_classifier.bias', 'pre_classifier.weight']\n",
      "You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "    <div>\n",
       "      \n",
       "      <progress value='24' max='24' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
       "      [24/24 07:29, Epoch 3/3]\n",
       "    </div>\n",
       "    <table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       " <tr style=\"text-align: left;\">\n",
       "      <th>Epoch</th>\n",
       "      <th>Training Loss</th>\n",
       "      <th>Validation Loss</th>\n",
       "      <th>Accuracy</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>No log</td>\n",
       "      <td>0.410274</td>\n",
       "      <td>0.928571</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>No log</td>\n",
       "      <td>0.218974</td>\n",
       "      <td>0.892857</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>No log</td>\n",
       "      <td>0.379487</td>\n",
       "      <td>0.928571</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table><p>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2025-02-01 16:26:39,501] Trial 2 finished with value: 0.9285714285714286 and parameters: {'learning_rate': 0.0003169465104468718, 'per_device_train_batch_size': 16, 'num_train_epochs': 3}. Best is trial 0 with value: 0.9642857142857143.\n",
      "Some weights of DistilBertForSequenceClassification were not initialized from the model checkpoint at distilbert-base-cased and are newly initialized: ['classifier.bias', 'classifier.weight', 'pre_classifier.bias', 'pre_classifier.weight']\n",
      "You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "    <div>\n",
       "      \n",
       "      <progress value='20' max='20' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
       "      [20/20 14:08, Epoch 5/5]\n",
       "    </div>\n",
       "    <table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       " <tr style=\"text-align: left;\">\n",
       "      <th>Epoch</th>\n",
       "      <th>Training Loss</th>\n",
       "      <th>Validation Loss</th>\n",
       "      <th>Accuracy</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>No log</td>\n",
       "      <td>0.951409</td>\n",
       "      <td>0.892857</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>No log</td>\n",
       "      <td>0.300698</td>\n",
       "      <td>0.964286</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>No log</td>\n",
       "      <td>0.218479</td>\n",
       "      <td>0.964286</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>4</td>\n",
       "      <td>No log</td>\n",
       "      <td>0.174622</td>\n",
       "      <td>0.964286</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>5</td>\n",
       "      <td>No log</td>\n",
       "      <td>0.169968</td>\n",
       "      <td>0.964286</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table><p>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2025-02-01 16:41:47,202] Trial 3 finished with value: 0.9642857142857143 and parameters: {'learning_rate': 0.00021073447095197996, 'per_device_train_batch_size': 32, 'num_train_epochs': 5}. Best is trial 0 with value: 0.9642857142857143.\n",
      "Some weights of DistilBertForSequenceClassification were not initialized from the model checkpoint at distilbert-base-cased and are newly initialized: ['classifier.bias', 'classifier.weight', 'pre_classifier.bias', 'pre_classifier.weight']\n",
      "You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "    <div>\n",
       "      \n",
       "      <progress value='8' max='8' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
       "      [8/8 05:17, Epoch 2/2]\n",
       "    </div>\n",
       "    <table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       " <tr style=\"text-align: left;\">\n",
       "      <th>Epoch</th>\n",
       "      <th>Training Loss</th>\n",
       "      <th>Validation Loss</th>\n",
       "      <th>Accuracy</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>No log</td>\n",
       "      <td>1.348463</td>\n",
       "      <td>0.428571</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>No log</td>\n",
       "      <td>1.309576</td>\n",
       "      <td>0.821429</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table><p>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[I 2025-02-01 16:47:52,344] Trial 4 finished with value: 0.8214285714285714 and parameters: {'learning_rate': 3.9687757840299386e-05, 'per_device_train_batch_size': 32, 'num_train_epochs': 2}. Best is trial 0 with value: 0.9642857142857143.\n"
     ]
    }
   ],
   "source": [
    "best_trial = tuning_trainer.hyperparameter_search(\n",
    "    direction=\"maximize\",\n",
    "    backend=\"optuna\",\n",
    "    hp_space=hp_space,\n",
    "    n_trials=5\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "id": "8c3dbf0e",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Best Hyperparameters: {'learning_rate': 0.0002728032165901902, 'per_device_train_batch_size': 16, 'num_train_epochs': 2}\n"
     ]
    }
   ],
   "source": [
    "print(\"Best Hyperparameters:\", best_trial.hyperparameters)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "id": "8a9e4d6f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "('tokenizer\\\\tokenizer_config.json',\n",
       " 'tokenizer\\\\special_tokens_map.json',\n",
       " 'tokenizer\\\\vocab.txt',\n",
       " 'tokenizer\\\\added_tokens.json',\n",
       " 'tokenizer\\\\tokenizer.json')"
      ]
     },
     "execution_count": 54,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tuning_trainer.model.save_pretrained(\"my_tuned_model\") # not used when pushing model to hub\n",
    "#tuning_trainer.push_to_hub(\"my_tuned_model\") # to be used when push_to_hub = True\n",
    "tuning_trainer.processing_class.save_pretrained(\"tokenizer\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fe71d85b",
   "metadata": {},
   "source": [
    "## Inference"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "id": "cd4d85fe",
   "metadata": {},
   "outputs": [],
   "source": [
    "from transformers import pipeline\n",
    "import random "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "id": "60b16e31",
   "metadata": {},
   "outputs": [],
   "source": [
    "my_model = AutoModelForSequenceClassification.from_pretrained(\"my_tuned_model\")\n",
    "#my_model = AutoModelForSequenceClassification.from_pretrained(\"lnewbould/my_tuned_model\") # to be used when push_to_hub = True\n",
    "my_tokenizer = AutoTokenizer.from_pretrained(\"tokenizer\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "id": "05ebbad2",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Device set to use cpu\n"
     ]
    }
   ],
   "source": [
    "my_text_classifier = pipeline(\"text-classification\", model=my_model, tokenizer=my_tokenizer)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "id": "7733289c",
   "metadata": {},
   "outputs": [],
   "source": [
    "random_sample = random.randint(0, len(test_data))\n",
    "sample = test_data[random_sample]\n",
    "sample_text = sample['text']\n",
    "sample_label = sample['label']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "id": "5eefb5de",
   "metadata": {},
   "outputs": [],
   "source": [
    "my_model_result = my_text_classifier(sample_text)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "id": "7b7c7344",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Text to classify:\n",
      " About 15 years ago while my father was working 55 hour weeks being paid $8/hr, he completed his taxi license and started working more hours a week as a taxi driver. On the highway, one of his passengers struck him with a hammer on the back of his head landing him in the hospital for a few months for no apparent reason.\n",
      "\n",
      "\n",
      "My father, being an immigrant, did not understand the law or his rights so did not pursue the issue further, despite police catching the individual in question.\n",
      "\n",
      "\n",
      "I, his son, would like to look at this case and pursue it, despite my father's unwillingness to do so. Is there a way that I, independently, can figure out the identity of the criminal. The city in question is Calgary, Alberta, Canada.\n",
      "\n",
      "\n",
      " \n",
      "\n",
      "The correct label from the dataset: 0 \n",
      "Our model classification: criminal-law with a score of 0.2640526294708252\n"
     ]
    }
   ],
   "source": [
    "print(\"Text to classify:\\n\", sample_text, \n",
    "      \"\\n\\nThe correct label from the dataset:\", sample_label, \n",
    "      \"\\nOur model classification:\", my_model_result[0]['label'], \"with a score of\", my_model_result[0]['score'])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1fde2f31",
   "metadata": {},
   "source": [
    "## Pushing model to hub"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0c0d7348",
   "metadata": {},
   "source": [
    "Will need to move this section up in finished file so hyperparam tuning can be uploaded to hub. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "id": "a0cc2a21",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "ae78bcb52a3f4542a6d84a9e53f9d552",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "VBox(children=(HTML(value='<center> <img\\nsrc=https://huggingface.co/front/assets/huggingface_logo-noborder.sv…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from huggingface_hub import notebook_login\n",
    "notebook_login()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c8a55a5f",
   "metadata": {},
   "outputs": [],
   "source": [
    "my_model.push_to_hub(\"my_final_model\")\n",
    "my_tokenizer.push_to_hub(\"my_final_model\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9cb0e58e",
   "metadata": {},
   "source": [
    "## Gradio"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "id": "302329fa",
   "metadata": {},
   "outputs": [],
   "source": [
    "import gradio as gr"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "id": "8802ec4b",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "* Running on local URL:  http://127.0.0.1:7860\n",
      "* Running on public URL: https://2118699b11ea8ef5b2.gradio.live\n",
      "\n",
      "This share link expires in 72 hours. For free permanent hosting and GPU upgrades, run `gradio deploy` from the terminal in the working directory to deploy to Hugging Face Spaces (https://huggingface.co/spaces)\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div><iframe src=\"https://2118699b11ea8ef5b2.gradio.live\" width=\"100%\" height=\"500\" allow=\"autoplay; camera; microphone; clipboard-read; clipboard-write;\" frameborder=\"0\" allowfullscreen></iframe></div>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": []
     },
     "execution_count": 59,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "demo = gr.Interface.from_pipeline(my_text_classifier)\n",
    "demo.launch(share=True)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "fresh_env2",
   "language": "python",
   "name": "fresh_env2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}