nn_fmnist.ipynb

{
 "metadata": {
  "kernelspec": {
   "language": "python",
   "display_name": "Python 3",
   "name": "python3"
  },
  "language_info": {
   "pygments_lexer": "ipython3",
   "nbconvert_exporter": "python",
   "version": "3.6.4",
   "file_extension": ".py",
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "name": "python",
   "mimetype": "text/x-python"
  }
 },
 "nbformat_minor": 4,
 "nbformat": 4,
 "cells": [
  {
   "cell_type": "markdown",
   "source": [
    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/xiptos/is_notes/blob/main/nn_fmnist.ipynb)"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "# Introduction\n",
    "\n",
    "This notebook presents a simple guide to creating an artificial neural network with PyTorch. It will predict the outcome of the fashion images from the [Zalando's article images](https://github.com/zalandoresearch/fashion-mnist) (Fashion-MNIST).\n",
    "\n",
    "The guide contains the most elementary PyTorch elements to create and evaluate a network.\n",
    " "
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "source": [
    "import torch\n",
    "from torch import nn\n",
    "from torch.nn import functional as F\n",
    "\n",
    "from torch.utils.data import DataLoader # loads data in batches\n",
    "from torchvision import datasets # load Fasion-MNIST\n",
    "import torchvision.transforms as T # transformers for computer vision "
   ],
   "metadata": {
    "_uuid": "8f2839f25d086af736a60e9eeb907d3b93b6e0e5",
    "_cell_guid": "b1076dfc-b9ad-4769-8c92-a6c4dae69d19",
    "execution": {
     "iopub.status.busy": "2022-11-20T11:59:55.135283Z",
     "iopub.execute_input": "2022-11-20T11:59:55.135779Z",
     "iopub.status.idle": "2022-11-20T11:59:57.228738Z",
     "shell.execute_reply.started": "2022-11-20T11:59:55.135682Z",
     "shell.execute_reply": "2022-11-20T11:59:57.227699Z"
    },
    "trusted": true
   },
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "code",
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from tqdm import tqdm # progress bar"
   ],
   "metadata": {
    "execution": {
     "iopub.status.busy": "2022-11-20T12:00:09.84164Z",
     "iopub.execute_input": "2022-11-20T12:00:09.842235Z",
     "iopub.status.idle": "2022-11-20T12:00:09.84754Z",
     "shell.execute_reply.started": "2022-11-20T12:00:09.842194Z",
     "shell.execute_reply": "2022-11-20T12:00:09.846605Z"
    },
    "trusted": true
   },
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "source": [
    "# Torchvision datasets\n",
    "\n",
    "We obtain the [Fashion-MNIST dataset](http://pytorch.org/vision/main/generated/torchvision.datasets.FashionMNIST.html) via torchvision. The dataset contains a training set of 60,000 examples and a test set of 10,000 examples. Each example is a 28x28 grayscale image, associated with a label from 10 classes. Note that `datasets` is an object imported from torchvision, not to confuse with from the Dataset object (used in torch.utils.data import Dataset)\n",
    "\n",
    "```from torchvision import datasets```\n",
    "\n",
    "When called for the first time, the datasets will be downloaded to the path specified in the `root` argument. After that, Torchvision will look first for a local copy before attempting another download."
   ],
   "metadata": {
    "execution": {
     "iopub.status.busy": "2022-08-05T23:37:48.188818Z",
     "iopub.execute_input": "2022-08-05T23:37:48.18922Z",
     "iopub.status.idle": "2022-08-05T23:37:48.193684Z",
     "shell.execute_reply.started": "2022-08-05T23:37:48.18919Z",
     "shell.execute_reply": "2022-08-05T23:37:48.192628Z"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "> **torchvision.transforms**. A transformer operates on the data. Using the ' transform' argument, we can apply multiple transformations (reshape, convert to tensor, normalize, etc.) to the data obtained."
   ],
   "metadata": {
    "execution": {
     "iopub.status.busy": "2022-08-06T22:12:57.413031Z",
     "iopub.execute_input": "2022-08-06T22:12:57.413408Z",
     "iopub.status.idle": "2022-08-06T22:12:57.419602Z",
     "shell.execute_reply.started": "2022-08-06T22:12:57.413377Z",
     "shell.execute_reply": "2022-08-06T22:12:57.418347Z"
    }
   }
  },
  {
   "cell_type": "code",
   "source": [
    "mytransform = T.ToTensor() # image (3D array) to Tensor\n",
    "\n",
    "train_data = datasets.FashionMNIST(root = './', download=True, train = True, transform = mytransform)\n",
    "test_data = datasets.FashionMNIST(root = './', download=True, train = False, transform = mytransform)"
   ],
   "metadata": {
    "execution": {
     "iopub.status.busy": "2022-11-20T12:00:16.383148Z",
     "iopub.execute_input": "2022-11-20T12:00:16.3836Z",
     "iopub.status.idle": "2022-11-20T12:00:19.65425Z",
     "shell.execute_reply.started": "2022-11-20T12:00:16.383563Z",
     "shell.execute_reply": "2022-11-20T12:00:19.653126Z"
    },
    "trusted": true
   },
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "source": [
    "Note that the first image in the dataset is a 3D tensor (C, H, W) for the number of channels (C), Height (H), and Width (W)."
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "source": [
    "img, label = train_data[0]\n",
    "img.shape # returns a Tensor of Size 1,28,28"
   ],
   "metadata": {
    "execution": {
     "iopub.status.busy": "2022-11-20T12:00:24.385689Z",
     "iopub.execute_input": "2022-11-20T12:00:24.386099Z",
     "iopub.status.idle": "2022-11-20T12:00:24.39587Z",
     "shell.execute_reply.started": "2022-11-20T12:00:24.386067Z",
     "shell.execute_reply": "2022-11-20T12:00:24.394778Z"
    },
    "trusted": true
   },
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "source": [
    "We plot the first image if we reshape the image into a 2D array (HxW)."
   ],
   "metadata": {
    "execution": {
     "iopub.status.busy": "2022-08-06T22:28:48.870011Z",
     "iopub.execute_input": "2022-08-06T22:28:48.870692Z",
     "iopub.status.idle": "2022-08-06T22:28:48.877179Z",
     "shell.execute_reply.started": "2022-08-06T22:28:48.870647Z",
     "shell.execute_reply": "2022-08-06T22:28:48.875971Z"
    }
   }
  },
  {
   "cell_type": "code",
   "source": [
    "# We could simply plot the tensor\n",
    "plt.imshow(img.reshape(28,28), cmap = 'gist_yarg'); # gist_yarg plots inverse of W&B\n",
    "plt.axis('off');"
   ],
   "metadata": {
    "execution": {
     "iopub.status.busy": "2022-11-20T12:00:26.922833Z",
     "iopub.execute_input": "2022-11-20T12:00:26.923289Z",
     "iopub.status.idle": "2022-11-20T12:00:27.049719Z",
     "shell.execute_reply.started": "2022-11-20T12:00:26.923244Z",
     "shell.execute_reply": "2022-11-20T12:00:27.048123Z"
    },
    "trusted": true
   },
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "source": [
    "# DataLoader\n",
    "\n",
    "The PyTorch DataLoader object allows the preparation of the dataset in batches of different sizes and shuffles them if necessary when exposing them to the training. \n",
    "\n",
    "```from torch.utils.data import DataLoader```\n",
    "\n",
    "> Note that the DataLoader object shuffles the data by default."
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "source": [
    "torch.manual_seed(101)\n",
    "\n",
    "train_loader = DataLoader(train_data, batch_size = 100, shuffle=True)\n",
    "# the test loader can be bigger and doesn't need to be shuffled\n",
    "test_loader =  DataLoader(test_data,  batch_size = 500, shuffle=False) "
   ],
   "metadata": {
    "execution": {
     "iopub.status.busy": "2022-11-20T12:00:30.888542Z",
     "iopub.execute_input": "2022-11-20T12:00:30.889634Z",
     "iopub.status.idle": "2022-11-20T12:00:30.898479Z",
     "shell.execute_reply.started": "2022-11-20T12:00:30.88958Z",
     "shell.execute_reply": "2022-11-20T12:00:30.89723Z"
    },
    "trusted": true
   },
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "source": [
    "If we run one iteration now, we will have one batch of the training dataset (100 images and labels)."
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "source": [
    "# Plot 10 images\n",
    "for img, label in train_loader:\n",
    "    break # we run only one iteration , after that we break\n",
    "img.shape # bz, ch, W H"
   ],
   "metadata": {
    "execution": {
     "iopub.status.busy": "2022-11-20T12:00:51.706437Z",
     "iopub.execute_input": "2022-11-20T12:00:51.70682Z",
     "iopub.status.idle": "2022-11-20T12:00:51.736293Z",
     "shell.execute_reply.started": "2022-11-20T12:00:51.706789Z",
     "shell.execute_reply": "2022-11-20T12:00:51.735106Z"
    },
    "trusted": true
   },
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "source": [
    "Let's select the 50 first images of the batch to plot them."
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "source": [
    "myimages = img[:50].numpy() # we now obtain NumPy arrays\n",
    "myimages.shape"
   ],
   "metadata": {
    "execution": {
     "iopub.status.busy": "2022-11-20T12:00:55.244671Z",
     "iopub.execute_input": "2022-11-20T12:00:55.245108Z",
     "iopub.status.idle": "2022-11-20T12:00:55.253224Z",
     "shell.execute_reply.started": "2022-11-20T12:00:55.245063Z",
     "shell.execute_reply": "2022-11-20T12:00:55.252062Z"
    },
    "trusted": true
   },
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "source": [
    "We will need to transpose the NumPy array to plot it with matplotlib (accepts height x width matrices)."
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "source": [
    "myimages[0].shape # channel, height, width"
   ],
   "metadata": {
    "execution": {
     "iopub.status.busy": "2022-11-20T12:00:56.314685Z",
     "iopub.execute_input": "2022-11-20T12:00:56.315147Z",
     "iopub.status.idle": "2022-11-20T12:00:56.323107Z",
     "shell.execute_reply.started": "2022-11-20T12:00:56.315106Z",
     "shell.execute_reply": "2022-11-20T12:00:56.321927Z"
    },
    "trusted": true
   },
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "code",
   "source": [
    "myimages[0].transpose(1,2,0).shape # height, width, channel"
   ],
   "metadata": {
    "execution": {
     "iopub.status.busy": "2022-11-20T12:00:58.23949Z",
     "iopub.execute_input": "2022-11-20T12:00:58.239877Z",
     "iopub.status.idle": "2022-11-20T12:00:58.24684Z",
     "shell.execute_reply.started": "2022-11-20T12:00:58.239846Z",
     "shell.execute_reply": "2022-11-20T12:00:58.245747Z"
    },
    "trusted": true
   },
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "code",
   "source": [
    "fig, ax = plt.subplots(nrows = 5, ncols = 10, figsize=(8,4), subplot_kw={'xticks': [], 'yticks': []})\n",
    "for row in range(0,5):\n",
    "    for col in range(0,10):\n",
    "        myid = (10*row) + col # (ncols*rows) + cols\n",
    "        \n",
    "        ax[row,col].imshow( myimages[myid].transpose(1,2,0), cmap = 'gist_yarg' ) # W,H,C\n",
    "        ax[row,col].axis('off')"
   ],
   "metadata": {
    "execution": {
     "iopub.status.busy": "2022-11-20T12:01:00.409224Z",
     "iopub.execute_input": "2022-11-20T12:01:00.409613Z",
     "iopub.status.idle": "2022-11-20T12:01:01.80208Z",
     "shell.execute_reply.started": "2022-11-20T12:01:00.409583Z",
     "shell.execute_reply": "2022-11-20T12:01:01.800892Z"
    },
    "trusted": true
   },
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "source": [
    "# Create the network\n",
    "\n",
    "The training set contains 60,000 records with 784 incoming features. The first layer is 784 neurons. After that, we create two fully connected layers of 120 and 84 neurons, respectively. The activation function we use is a Rectified Linear Unit (ReLU) function. This piecewise function that makes negative inputs to the neuron will be zero and scales up to positive ones.\n",
    "\n",
    "Finally, the output layer contains ten layers. Every layer is the probability of having the label given (from 1 to 10), with the condition that the sum of the probabilities is one (Log softmax)."
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "source": [
    "class MultilayerPerceptron(nn.Module):\n"
   ],
   "metadata": {
    "execution": {
     "iopub.status.busy": "2022-11-20T12:01:06.042304Z",
     "iopub.execute_input": "2022-11-20T12:01:06.042691Z",
     "iopub.status.idle": "2022-11-20T12:01:06.05135Z",
     "shell.execute_reply.started": "2022-11-20T12:01:06.042661Z",
     "shell.execute_reply": "2022-11-20T12:01:06.050039Z"
    },
    "trusted": true
   },
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "code",
   "source": [
    "torch.manual_seed(101)\n",
    "\n",
    "mymodel = MultilayerPerceptron() # default params are in_features = 784, out_features=10\n",
    "mymodel # check topology"
   ],
   "metadata": {
    "execution": {
     "iopub.status.busy": "2022-11-20T12:01:07.534523Z",
     "iopub.execute_input": "2022-11-20T12:01:07.535632Z",
     "iopub.status.idle": "2022-11-20T12:01:07.54395Z",
     "shell.execute_reply.started": "2022-11-20T12:01:07.535591Z",
     "shell.execute_reply": "2022-11-20T12:01:07.542867Z"
    },
    "trusted": true
   },
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "source": [
    "* We select the [cross-entroy](https://pytorch.org/docs/stable/generated/torch.nn.functional.cross_entropy.html?highlight=entropy) as the cost function. The cross-entropy is similar to the quadratic formula, but it predicts the probability distribution of each class.\n",
    "\n",
    "* We define the optimization method. The simplest one is the [Adaptative Stochastic Gradient Descent method](https://pytorch.org/docs/stable/generated/torch.optim.Adam.html). "
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "source": [
    "learning_rate = 1e-3\n",
    "criterion = \n",
    "\n",
    "optimizer = "
   ],
   "metadata": {
    "execution": {
     "iopub.status.busy": "2022-11-20T12:01:10.616172Z",
     "iopub.execute_input": "2022-11-20T12:01:10.617368Z",
     "iopub.status.idle": "2022-11-20T12:01:10.622927Z",
     "shell.execute_reply.started": "2022-11-20T12:01:10.617323Z",
     "shell.execute_reply": "2022-11-20T12:01:10.621805Z"
    },
    "trusted": true
   },
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "source": [
    "How many parameters do we need to evaluate?\n",
    "\n",
    "* Number of weights = (784 x 120) + (120 x 84) + (84 x 10) = 105,000\n",
    "* Number of biases = 120 + 84 + 10 = 214\n",
    "\n",
    "Total = 105,214"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "source": [
    "params = [p.numel() for p in mymodel.parameters() if p.requires_grad]\n",
    "np.sum(params)"
   ],
   "metadata": {
    "execution": {
     "iopub.status.busy": "2022-11-20T12:01:13.523456Z",
     "iopub.execute_input": "2022-11-20T12:01:13.523874Z",
     "iopub.status.idle": "2022-11-20T12:01:13.531517Z",
     "shell.execute_reply.started": "2022-11-20T12:01:13.523838Z",
     "shell.execute_reply": "2022-11-20T12:01:13.530481Z"
    },
    "trusted": true
   },
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "source": [
    "# Training and evaluation \n",
    "\n",
    "1. Before starting, we must consider that the DataLoader returns a tensor of size [100,1,28,28], but our model accepts 1D vectors of 784 pixels (28x28). Therefore, we must flatten the tensor to accommodate the model's input."
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "source": [
    "# Plot 10 images\n",
    "myiter = iter(myloader.train)\n",
    "img, label = myiter.__next__() # only one iteration\n",
    "img.shape # batch_size, channel, Height, Width"
   ],
   "metadata": {
    "execution": {
     "iopub.status.busy": "2022-11-20T12:01:15.761224Z",
     "iopub.execute_input": "2022-11-20T12:01:15.761628Z",
     "iopub.status.idle": "2022-11-20T12:01:15.786276Z",
     "shell.execute_reply.started": "2022-11-20T12:01:15.761596Z",
     "shell.execute_reply": "2022-11-20T12:01:15.785453Z"
    },
    "trusted": true
   },
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "source": [
    "2. We will flatten the dimensions of the batch (1,28,28) that correspond to channel, height, and width. That's a common preprocessing step when using images to allocate a 1D vector to the entry of the network (in our case, a 28 x 28 = 784 vector)."
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "source": [
    "img.view(100,-1).shape # 100 batches of 784 pixels"
   ],
   "metadata": {
    "execution": {
     "iopub.status.busy": "2022-11-20T12:01:17.609688Z",
     "iopub.execute_input": "2022-11-20T12:01:17.610583Z",
     "iopub.status.idle": "2022-11-20T12:01:17.617167Z",
     "shell.execute_reply.started": "2022-11-20T12:01:17.610539Z",
     "shell.execute_reply": "2022-11-20T12:01:17.615907Z"
    },
    "trusted": true
   },
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "source": [
    "Let's evaluate that batch without training the model. The prediction returns a 100 x 10 tensor. It means that we obtain ten probabilities for every batch of 100 images (shape is [100,10])."
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "source": [
    "y_pred = mymodel( img.view(100,-1) )\n",
    "y_pred.shape # 100 x 10, meaning for every batch (100) we obtain  (10 probabilities) predictions "
   ],
   "metadata": {
    "execution": {
     "iopub.status.busy": "2022-11-20T12:01:19.582222Z",
     "iopub.execute_input": "2022-11-20T12:01:19.582677Z",
     "iopub.status.idle": "2022-11-20T12:01:19.60963Z",
     "shell.execute_reply.started": "2022-11-20T12:01:19.582639Z",
     "shell.execute_reply": "2022-11-20T12:01:19.608831Z"
    },
    "trusted": true
   },
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "source": [
    "If we calculate the index with the highest probability for every label outcome, we obtain:"
   ],
   "metadata": {
    "execution": {
     "iopub.status.busy": "2022-08-07T00:53:24.531953Z",
     "iopub.execute_input": "2022-08-07T00:53:24.532474Z",
     "iopub.status.idle": "2022-08-07T00:53:24.540818Z",
     "shell.execute_reply.started": "2022-08-07T00:53:24.532436Z",
     "shell.execute_reply": "2022-08-07T00:53:24.53935Z"
    }
   }
  },
  {
   "cell_type": "code",
   "source": [
    "val, idx = torch.max(y_pred, dim=1) # dim 1 is for the output\n",
    "idx # indices == predictions"
   ],
   "metadata": {
    "execution": {
     "iopub.status.busy": "2022-11-20T12:01:20.433201Z",
     "iopub.execute_input": "2022-11-20T12:01:20.433897Z",
     "iopub.status.idle": "2022-11-20T12:01:20.443568Z",
     "shell.execute_reply.started": "2022-11-20T12:01:20.433859Z",
     "shell.execute_reply": "2022-11-20T12:01:20.442461Z"
    },
    "trusted": true
   },
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "source": [
    "3. We will calculate the model's accuracy in every epoch (number of correct projections in/batch size) for both the train and the test dataset."
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "source": [
    "# tracking variables\n",
    "\n",
    "class Loss:\n",
    "    \"\"\" Class to monitor train and test lost\"\"\"\n",
    "    train: list = []\n",
    "    test: list = []\n",
    "    \n",
    "\n",
    "class Accuracy:\n",
    "    \"\"\" Class to monitor train and test accuracy\"\"\"\n",
    "    train: list = []\n",
    "    test: list = []"
   ],
   "metadata": {
    "execution": {
     "iopub.status.busy": "2022-11-20T12:03:23.607963Z",
     "iopub.execute_input": "2022-11-20T12:03:23.608414Z",
     "iopub.status.idle": "2022-11-20T12:03:23.616421Z",
     "shell.execute_reply.started": "2022-11-20T12:03:23.608381Z",
     "shell.execute_reply": "2022-11-20T12:03:23.615034Z"
    },
    "trusted": true
   },
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "code",
   "source": [
    "%%time\n",
    "# Train for 10 epocs\n"
   ],
   "metadata": {
    "execution": {
     "iopub.status.busy": "2022-11-20T12:03:24.595312Z",
     "iopub.execute_input": "2022-11-20T12:03:24.596362Z",
     "iopub.status.idle": "2022-11-20T12:04:51.579405Z",
     "shell.execute_reply.started": "2022-11-20T12:03:24.596303Z",
     "shell.execute_reply": "2022-11-20T12:04:51.578093Z"
    },
    "trusted": true
   },
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "source": [
    "# Visualization"
   ],
   "metadata": {
    "execution": {
     "iopub.status.busy": "2022-08-06T02:24:07.967843Z",
     "iopub.execute_input": "2022-08-06T02:24:07.968441Z",
     "iopub.status.idle": "2022-08-06T02:24:07.972846Z",
     "shell.execute_reply.started": "2022-08-06T02:24:07.968408Z",
     "shell.execute_reply": "2022-08-06T02:24:07.971569Z"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "We'll see train and test losses, together with its accuracies per epoch. Note that the training data have more minor losses and reach an accuracy of almost 100%. On the other hand, the test data reach almost a plateau of > 95% accuracy, and we could think of using more than two epochs because this is where the training data crosses the accuracy of the testing data."
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "source": [
    "fig, ax = plt.subplots(nrows=1, ncols=2, figsize=(12,4))\n",
    "ax[0].plot(Loss.train, label = 'Training')\n",
    "ax[0].plot(Loss.test, label='test/validation')\n",
    "ax[0].set_ylabel('Loss', fontsize=16)\n",
    "\n",
    "\n",
    "ax[1].plot(Accuracy.train, label = 'Training')\n",
    "ax[1].plot(Accuracy.test, label='test/validation')\n",
    "ax[1].set_yticks(range(85,110,5))\n",
    "ax[1].axvline(x=2, color='gray', linestyle=':')\n",
    "ax[1].axhline(y=100, color='gray', linestyle=':')\n",
    "ax[1].set_ylabel('Accuracy (%)', fontsize=16)\n",
    "\n",
    "for myax in ax:\n",
    "    myax.set_xlabel('Epoch', fontsize=16)\n",
    "    myax.set_xticks(range(epochs))\n",
    "    myax.legend(frameon=False)\n",
    "\n"
   ],
   "metadata": {
    "execution": {
     "iopub.status.busy": "2022-11-20T12:19:02.347292Z",
     "iopub.execute_input": "2022-11-20T12:19:02.347668Z",
     "iopub.status.idle": "2022-11-20T12:19:02.694434Z",
     "shell.execute_reply.started": "2022-11-20T12:19:02.347638Z",
     "shell.execute_reply": "2022-11-20T12:19:02.693078Z"
    },
    "trusted": true
   },
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "source": [
    "Finally, we evaluate all the test data at once and visualize the accuracy of every outcome for every prediction (i.e., confusion matrix)."
   ],
   "metadata": {
    "execution": {
     "iopub.status.busy": "2022-08-08T00:59:08.129764Z",
     "iopub.execute_input": "2022-08-08T00:59:08.130182Z",
     "iopub.status.idle": "2022-08-08T00:59:08.137731Z",
     "shell.execute_reply.started": "2022-08-08T00:59:08.130148Z",
     "shell.execute_reply": "2022-08-08T00:59:08.136336Z"
    }
   }
  },
  {
   "cell_type": "code",
   "source": [
    "test_loader =  DataLoader(test_data,  batch_size = 10_000, shuffle=False) # the whole test is 10,000 images\n",
    "myiter = iter(test_loader)\n",
    "img, label = myiter.__next__()\n",
    "img.shape"
   ],
   "metadata": {
    "execution": {
     "iopub.status.busy": "2022-11-20T12:19:10.729477Z",
     "iopub.execute_input": "2022-11-20T12:19:10.729899Z",
     "iopub.status.idle": "2022-11-20T12:19:11.901358Z",
     "shell.execute_reply.started": "2022-11-20T12:19:10.729859Z",
     "shell.execute_reply": "2022-11-20T12:19:11.900128Z"
    },
    "trusted": true
   },
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "code",
   "source": [
    "with torch.no_grad():\n",
    "    correct = 0\n",
    "    \n",
    "    for X, y_label in test_loader:\n",
    "            y_val = mymodel( X.view(X.shape[0],-1) ) # flatten\n",
    "            _, predicted = torch.max( y_val, dim = 1)\n",
    "            correct += (predicted == y_label).sum()\n",
    "\n",
    "print(f'Test accuracy: = {correct.item()*100/(len(test_data)):2.4f} %')"
   ],
   "metadata": {
    "execution": {
     "iopub.status.busy": "2022-11-20T12:19:15.641037Z",
     "iopub.execute_input": "2022-11-20T12:19:15.641444Z",
     "iopub.status.idle": "2022-11-20T12:19:16.637063Z",
     "shell.execute_reply.started": "2022-11-20T12:19:15.641413Z",
     "shell.execute_reply": "2022-11-20T12:19:16.635805Z"
    },
    "trusted": true
   },
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "code",
   "source": [
    "# Show the confusion matrix"
   ],
   "metadata": {
    "execution": {
     "iopub.status.busy": "2022-11-20T12:19:19.35135Z",
     "iopub.execute_input": "2022-11-20T12:19:19.351999Z",
     "iopub.status.idle": "2022-11-20T12:19:19.802896Z",
     "shell.execute_reply.started": "2022-11-20T12:19:19.351962Z",
     "shell.execute_reply": "2022-11-20T12:19:19.801705Z"
    },
    "trusted": true
   },
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "code",
   "source": [
    "# Show the heatmap corresponding to the confusion matrix"
   ],
   "metadata": {
    "execution": {
     "iopub.status.busy": "2022-11-20T12:19:22.262835Z",
     "iopub.execute_input": "2022-11-20T12:19:22.263252Z",
     "iopub.status.idle": "2022-11-20T12:19:23.045818Z",
     "shell.execute_reply.started": "2022-11-20T12:19:22.263218Z",
     "shell.execute_reply": "2022-11-20T12:19:23.04464Z"
    },
    "trusted": true
   },
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "code",
   "source": [],
   "metadata": {},
   "execution_count": null,
   "outputs": []
  }
 ]
}