MNIST_Handwritten_Digits.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Introduction\n",
    "\n",
    "In this project, you will build a neural network of your own design to evaluate the MNIST dataset.\n",
    "\n",
    "Some of the benchmark results on MNIST include can be found [on Yann LeCun's page](https://webcache.googleusercontent.com/search?q=cache:stAVPik6onEJ:yann.lecun.com/exdb/mnist) and include:\n",
    "\n",
    "88% [Lecun et al., 1998](https://hal.science/hal-03926082/document)\n",
    "\n",
    "95.3% [Lecun et al., 1998](https://hal.science/hal-03926082v1/document)\n",
    "\n",
    "99.65% [Ciresan et al., 2011](http://people.idsia.ch/~juergen/ijcai2011.pdf)\n",
    "\n",
    "\n",
    "MNIST is a great dataset for sanity checking your models, since the accuracy levels achieved by large convolutional neural networks and small linear models are both quite high. This makes it important to be familiar with the data.\n",
    "\n",
    "## Installation"
   ]
  },
  {
   "cell_type": "code",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-09-25T19:34:01.346552Z",
     "start_time": "2024-09-25T19:34:01.333662Z"
    }
   },
   "source": [
    "# Update the PATH to include the user installation directory. \n",
    "import os\n",
    "os.environ['PATH'] = f\"{os.environ['PATH']}:/root/.local/bin\"\n",
    "\n",
    "# Restart the Kernel before you move on to the next step."
   ],
   "outputs": [],
   "execution_count": 2
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Important: Restart the Kernel before you move on to the next step."
   ]
  },
  {
   "cell_type": "code",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-09-25T19:38:43.698130Z",
     "start_time": "2024-09-25T19:34:05.341942Z"
    }
   },
   "source": [
    "# Install requirements\n",
    "!python -m pip install -r requirements.txt"
   ],
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting opencv-python-headless==4.5.3.56 (from -r requirements.txt (line 1))\n",
      "  Using cached opencv-python-headless-4.5.3.56.tar.gz (89.2 MB)\n",
      "  Installing build dependencies: started\n",
      "  Installing build dependencies: still running...\n",
      "  Installing build dependencies: still running...\n",
      "  Installing build dependencies: finished with status 'error'\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING: Ignoring invalid distribution ~atplotlib (C:\\Users\\anils\\AppData\\Roaming\\Python\\Python311\\site-packages)\n",
      "WARNING: Ignoring invalid distribution ~atplotlib (C:\\Users\\anils\\AppData\\Roaming\\Python\\Python311\\site-packages)\n",
      "  error: subprocess-exited-with-error\n",
      "  \n",
      "  pip subprocess to install build dependencies did not run successfully.\n",
      "  exit code: 1\n",
      "  \n",
      "  [330 lines of output]\n",
      "  WARNING: Ignoring invalid distribution ~atplotlib (C:\\Users\\anils\\AppData\\Roaming\\Python\\Python311\\site-packages)\n",
      "  Ignoring numpy: markers 'python_version == \"3.6\" and platform_machine != \"aarch64\" and platform_machine != \"arm64\"' don't match your environment\n",
      "  Ignoring numpy: markers 'python_version >= \"3.6\" and sys_platform == \"linux\" and platform_machine == \"aarch64\"' don't match your environment\n",
      "  Ignoring numpy: markers 'python_version >= \"3.6\" and sys_platform == \"darwin\" and platform_machine == \"arm64\"' don't match your environment\n",
      "  Ignoring numpy: markers 'python_version == \"3.7\" and platform_machine != \"aarch64\" and platform_machine != \"arm64\"' don't match your environment\n",
      "  Ignoring numpy: markers 'python_version == \"3.8\" and platform_machine != \"aarch64\" and platform_machine != \"arm64\"' don't match your environment\n",
      "  Collecting setuptools\n",
      "    Using cached setuptools-75.1.0-py3-none-any.whl.metadata (6.9 kB)\n",
      "  Collecting wheel\n",
      "    Using cached wheel-0.44.0-py3-none-any.whl.metadata (2.3 kB)\n",
      "  Collecting scikit-build\n",
      "    Using cached scikit_build-0.18.1-py3-none-any.whl.metadata (18 kB)\n",
      "  Collecting cmake\n",
      "    Using cached cmake-3.30.3-py3-none-win_amd64.whl.metadata (6.4 kB)\n",
      "  Collecting pip\n",
      "    Using cached pip-24.2-py3-none-any.whl.metadata (3.6 kB)\n",
      "  Collecting numpy==1.19.3\n",
      "    Using cached numpy-1.19.3.zip (7.3 MB)\n",
      "    Installing build dependencies: started\n",
      "    Installing build dependencies: finished with status 'done'\n",
      "    Getting requirements to build wheel: started\n",
      "    Getting requirements to build wheel: finished with status 'done'\n",
      "    Preparing metadata (pyproject.toml): started\n",
      "    Preparing metadata (pyproject.toml): still running...\n",
      "    Preparing metadata (pyproject.toml): finished with status 'error'\n",
      "    error: subprocess-exited-with-error\n",
      "  \n",
      "    Preparing metadata (pyproject.toml) did not run successfully.\n",
      "    exit code: 1\n",
      "  \n",
      "    [282 lines of output]\n",
      "    setup.py:67: RuntimeWarning: NumPy 1.19.3 may not yet support Python 3.11.\n",
      "      warnings.warn(\n",
      "    Running from numpy source directory.\n",
      "    setup.py:480: UserWarning: Unrecognized setuptools command, proceeding with generating Cython sources and expanding templates\n",
      "      run_build = parse_setuppy_commands()\n",
      "    Processing numpy/random\\_bounded_integers.pxd.in\n",
      "    Processing numpy/random\\bit_generator.pyx\n",
      "    Processing numpy/random\\mtrand.pyx\n",
      "    Processing numpy/random\\_bounded_integers.pyx.in\n",
      "    Processing numpy/random\\_common.pyx\n",
      "    Processing numpy/random\\_generator.pyx\n",
      "    Processing numpy/random\\_mt19937.pyx\n",
      "    Processing numpy/random\\_pcg64.pyx\n",
      "    Processing numpy/random\\_philox.pyx\n",
      "    Processing numpy/random\\_sfc64.pyx\n",
      "    Cythonizing sources\n",
      "    blas_opt_info:\n",
      "    blas_mkl_info:\n",
      "    No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils\n",
      "    customize MSVCCompiler\n",
      "      libraries mkl_rt not found in ['C:\\\\Users\\\\anils\\\\anaconda3\\\\lib', 'C:\\\\', 'C:\\\\Users\\\\anils\\\\anaconda3\\\\libs']\n",
      "      NOT AVAILABLE\n",
      "  \n",
      "    blis_info:\n",
      "      libraries blis not found in ['C:\\\\Users\\\\anils\\\\anaconda3\\\\lib', 'C:\\\\', 'C:\\\\Users\\\\anils\\\\anaconda3\\\\libs']\n",
      "      NOT AVAILABLE\n",
      "  \n",
      "    openblas_info:\n",
      "      libraries openblas not found in ['C:\\\\Users\\\\anils\\\\anaconda3\\\\lib', 'C:\\\\', 'C:\\\\Users\\\\anils\\\\anaconda3\\\\libs']\n",
      "    get_default_fcompiler: matching types: '['gnu', 'intelv', 'absoft', 'compaqv', 'intelev', 'gnu95', 'g95', 'intelvem', 'intelem', 'flang']'\n",
      "    customize GnuFCompiler\n",
      "    Could not locate executable g77\n",
      "    Could not locate executable f77\n",
      "    customize IntelVisualFCompiler\n",
      "    Could not locate executable ifort\n",
      "    Could not locate executable ifl\n",
      "    customize AbsoftFCompiler\n",
      "    Could not locate executable f90\n",
      "    customize CompaqVisualFCompiler\n",
      "    Could not locate executable DF\n",
      "    customize IntelItaniumVisualFCompiler\n",
      "    Could not locate executable efl\n",
      "    customize Gnu95FCompiler\n",
      "    Could not locate executable gfortran\n",
      "    Could not locate executable f95\n",
      "    customize G95FCompiler\n",
      "    Could not locate executable g95\n",
      "    customize IntelEM64VisualFCompiler\n",
      "    customize IntelEM64TFCompiler\n",
      "    Could not locate executable efort\n",
      "    Could not locate executable efc\n",
      "    customize PGroupFlangCompiler\n",
      "    Could not locate executable flang\n",
      "    don't know how to compile Fortran code on platform 'nt'\n",
      "      NOT AVAILABLE\n",
      "  \n",
      "    atlas_3_10_blas_threads_info:\n",
      "    Setting PTATLAS=ATLAS\n",
      "      libraries tatlas not found in ['C:\\\\Users\\\\anils\\\\anaconda3\\\\lib', 'C:\\\\', 'C:\\\\Users\\\\anils\\\\anaconda3\\\\libs']\n",
      "      NOT AVAILABLE\n",
      "  \n",
      "    atlas_3_10_blas_info:\n",
      "      libraries satlas not found in ['C:\\\\Users\\\\anils\\\\anaconda3\\\\lib', 'C:\\\\', 'C:\\\\Users\\\\anils\\\\anaconda3\\\\libs']\n",
      "      NOT AVAILABLE\n",
      "  \n",
      "    atlas_blas_threads_info:\n",
      "    Setting PTATLAS=ATLAS\n",
      "      libraries ptf77blas,ptcblas,atlas not found in ['C:\\\\Users\\\\anils\\\\anaconda3\\\\lib', 'C:\\\\', 'C:\\\\Users\\\\anils\\\\anaconda3\\\\libs']\n",
      "      NOT AVAILABLE\n",
      "  \n",
      "    atlas_blas_info:\n",
      "      libraries f77blas,cblas,atlas not found in ['C:\\\\Users\\\\anils\\\\anaconda3\\\\lib', 'C:\\\\', 'C:\\\\Users\\\\anils\\\\anaconda3\\\\libs']\n",
      "      NOT AVAILABLE\n",
      "  \n",
      "    accelerate_info:\n",
      "      NOT AVAILABLE\n",
      "  \n",
      "    C:\\Users\\anils\\AppData\\Local\\Temp\\pip-install-357unvi0\\numpy_b70e9bfa1fee4034b563f15b605e89df\\numpy\\distutils\\system_info.py:1914: UserWarning:\n",
      "        Optimized (vendor) Blas libraries are not found.\n",
      "        Falls back to netlib Blas library which has worse performance.\n",
      "        A better performance should be easily gained by switching\n",
      "        Blas library.\n",
      "      if self._calc_info(blas):\n",
      "    blas_info:\n",
      "      libraries blas not found in ['C:\\\\Users\\\\anils\\\\anaconda3\\\\lib', 'C:\\\\', 'C:\\\\Users\\\\anils\\\\anaconda3\\\\libs']\n",
      "      NOT AVAILABLE\n",
      "  \n",
      "    C:\\Users\\anils\\AppData\\Local\\Temp\\pip-install-357unvi0\\numpy_b70e9bfa1fee4034b563f15b605e89df\\numpy\\distutils\\system_info.py:1914: UserWarning:\n",
      "        Blas (http://www.netlib.org/blas/) libraries not found.\n",
      "        Directories to search for the libraries can be specified in the\n",
      "        numpy/distutils/site.cfg file (section [blas]) or by setting\n",
      "        the BLAS environment variable.\n",
      "      if self._calc_info(blas):\n",
      "    blas_src_info:\n",
      "      NOT AVAILABLE\n",
      "  \n",
      "    C:\\Users\\anils\\AppData\\Local\\Temp\\pip-install-357unvi0\\numpy_b70e9bfa1fee4034b563f15b605e89df\\numpy\\distutils\\system_info.py:1914: UserWarning:\n",
      "        Blas (http://www.netlib.org/blas/) sources not found.\n",
      "        Directories to search for the sources can be specified in the\n",
      "        numpy/distutils/site.cfg file (section [blas_src]) or by setting\n",
      "        the BLAS_SRC environment variable.\n",
      "      if self._calc_info(blas):\n",
      "      NOT AVAILABLE\n",
      "  \n",
      "    non-existing path in 'numpy\\\\distutils': 'site.cfg'\n",
      "    lapack_opt_info:\n",
      "    lapack_mkl_info:\n",
      "      libraries mkl_rt not found in ['C:\\\\Users\\\\anils\\\\anaconda3\\\\lib', 'C:\\\\', 'C:\\\\Users\\\\anils\\\\anaconda3\\\\libs']\n",
      "      NOT AVAILABLE\n",
      "  \n",
      "    openblas_lapack_info:\n",
      "      libraries openblas not found in ['C:\\\\Users\\\\anils\\\\anaconda3\\\\lib', 'C:\\\\', 'C:\\\\Users\\\\anils\\\\anaconda3\\\\libs']\n",
      "      NOT AVAILABLE\n",
      "  \n",
      "    openblas_clapack_info:\n",
      "      libraries openblas,lapack not found in ['C:\\\\Users\\\\anils\\\\anaconda3\\\\lib', 'C:\\\\', 'C:\\\\Users\\\\anils\\\\anaconda3\\\\libs']\n",
      "      NOT AVAILABLE\n",
      "  \n",
      "    flame_info:\n",
      "      libraries flame not found in ['C:\\\\Users\\\\anils\\\\anaconda3\\\\lib', 'C:\\\\', 'C:\\\\Users\\\\anils\\\\anaconda3\\\\libs']\n",
      "      NOT AVAILABLE\n",
      "  \n",
      "    atlas_3_10_threads_info:\n",
      "    Setting PTATLAS=ATLAS\n",
      "      libraries lapack_atlas not found in C:\\Users\\anils\\anaconda3\\lib\n",
      "      libraries tatlas,tatlas not found in C:\\Users\\anils\\anaconda3\\lib\n",
      "      libraries lapack_atlas not found in C:\\\n",
      "      libraries tatlas,tatlas not found in C:\\\n",
      "      libraries lapack_atlas not found in C:\\Users\\anils\\anaconda3\\libs\n",
      "      libraries tatlas,tatlas not found in C:\\Users\\anils\\anaconda3\\libs\n",
      "    <class 'numpy.distutils.system_info.atlas_3_10_threads_info'>\n",
      "      NOT AVAILABLE\n",
      "  \n",
      "    atlas_3_10_info:\n",
      "      libraries lapack_atlas not found in C:\\Users\\anils\\anaconda3\\lib\n",
      "      libraries satlas,satlas not found in C:\\Users\\anils\\anaconda3\\lib\n",
      "      libraries lapack_atlas not found in C:\\\n",
      "      libraries satlas,satlas not found in C:\\\n",
      "      libraries lapack_atlas not found in C:\\Users\\anils\\anaconda3\\libs\n",
      "      libraries satlas,satlas not found in C:\\Users\\anils\\anaconda3\\libs\n",
      "    <class 'numpy.distutils.system_info.atlas_3_10_info'>\n",
      "      NOT AVAILABLE\n",
      "  \n",
      "    atlas_threads_info:\n",
      "    Setting PTATLAS=ATLAS\n",
      "      libraries lapack_atlas not found in C:\\Users\\anils\\anaconda3\\lib\n",
      "      libraries ptf77blas,ptcblas,atlas not found in C:\\Users\\anils\\anaconda3\\lib\n",
      "      libraries lapack_atlas not found in C:\\\n",
      "      libraries ptf77blas,ptcblas,atlas not found in C:\\\n",
      "      libraries lapack_atlas not found in C:\\Users\\anils\\anaconda3\\libs\n",
      "      libraries ptf77blas,ptcblas,atlas not found in C:\\Users\\anils\\anaconda3\\libs\n",
      "    <class 'numpy.distutils.system_info.atlas_threads_info'>\n",
      "      NOT AVAILABLE\n",
      "  \n",
      "    atlas_info:\n",
      "      libraries lapack_atlas not found in C:\\Users\\anils\\anaconda3\\lib\n",
      "      libraries f77blas,cblas,atlas not found in C:\\Users\\anils\\anaconda3\\lib\n",
      "      libraries lapack_atlas not found in C:\\\n",
      "      libraries f77blas,cblas,atlas not found in C:\\\n",
      "      libraries lapack_atlas not found in C:\\Users\\anils\\anaconda3\\libs\n",
      "      libraries f77blas,cblas,atlas not found in C:\\Users\\anils\\anaconda3\\libs\n",
      "    <class 'numpy.distutils.system_info.atlas_info'>\n",
      "      NOT AVAILABLE\n",
      "  \n",
      "    lapack_info:\n",
      "      libraries lapack not found in ['C:\\\\Users\\\\anils\\\\anaconda3\\\\lib', 'C:\\\\', 'C:\\\\Users\\\\anils\\\\anaconda3\\\\libs']\n",
      "      NOT AVAILABLE\n",
      "  \n",
      "    C:\\Users\\anils\\AppData\\Local\\Temp\\pip-install-357unvi0\\numpy_b70e9bfa1fee4034b563f15b605e89df\\numpy\\distutils\\system_info.py:1748: UserWarning:\n",
      "        Lapack (http://www.netlib.org/lapack/) libraries not found.\n",
      "        Directories to search for the libraries can be specified in the\n",
      "        numpy/distutils/site.cfg file (section [lapack]) or by setting\n",
      "        the LAPACK environment variable.\n",
      "      return getattr(self, '_calc_info_{}'.format(name))()\n",
      "    lapack_src_info:\n",
      "      NOT AVAILABLE\n",
      "  \n",
      "    C:\\Users\\anils\\AppData\\Local\\Temp\\pip-install-357unvi0\\numpy_b70e9bfa1fee4034b563f15b605e89df\\numpy\\distutils\\system_info.py:1748: UserWarning:\n",
      "        Lapack (http://www.netlib.org/lapack/) sources not found.\n",
      "        Directories to search for the sources can be specified in the\n",
      "        numpy/distutils/site.cfg file (section [lapack_src]) or by setting\n",
      "        the LAPACK_SRC environment variable.\n",
      "      return getattr(self, '_calc_info_{}'.format(name))()\n",
      "      NOT AVAILABLE\n",
      "  \n",
      "    numpy_linalg_lapack_lite:\n",
      "      FOUND:\n",
      "        language = c\n",
      "        define_macros = [('HAVE_BLAS_ILP64', None), ('BLAS_SYMBOL_SUFFIX', '64_')]\n",
      "  \n",
      "    C:\\Users\\anils\\AppData\\Local\\Temp\\pip-build-env-3weomz5m\\overlay\\Lib\\site-packages\\setuptools\\_distutils\\dist.py:275: UserWarning: Unknown distribution option: 'define_macros'\n",
      "      warnings.warn(msg)\n",
      "    running dist_info\n",
      "    running build_src\n",
      "    build_src\n",
      "    building py_modules sources\n",
      "    creating build\n",
      "    creating build\\src.win-amd64-3.11\n",
      "    creating build\\src.win-amd64-3.11\\numpy\n",
      "    creating build\\src.win-amd64-3.11\\numpy\\distutils\n",
      "    building library \"npymath\" sources\n",
      "    Traceback (most recent call last):\n",
      "      File \"C:\\Users\\anils\\anaconda3\\Lib\\site-packages\\pip\\_vendor\\pyproject_hooks\\_in_process\\_in_process.py\", line 353, in <module>\n",
      "        main()\n",
      "      File \"C:\\Users\\anils\\anaconda3\\Lib\\site-packages\\pip\\_vendor\\pyproject_hooks\\_in_process\\_in_process.py\", line 335, in main\n",
      "        json_out['return_val'] = hook(**hook_input['kwargs'])\n",
      "                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n",
      "      File \"C:\\Users\\anils\\anaconda3\\Lib\\site-packages\\pip\\_vendor\\pyproject_hooks\\_in_process\\_in_process.py\", line 149, in prepare_metadata_for_build_wheel\n",
      "        return hook(metadata_directory, config_settings)\n",
      "               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n",
      "      File \"C:\\Users\\anils\\AppData\\Local\\Temp\\pip-build-env-3weomz5m\\overlay\\Lib\\site-packages\\setuptools\\build_meta.py\", line 157, in prepare_metadata_for_build_wheel\n",
      "        self.run_setup()\n",
      "      File \"C:\\Users\\anils\\AppData\\Local\\Temp\\pip-build-env-3weomz5m\\overlay\\Lib\\site-packages\\setuptools\\build_meta.py\", line 249, in run_setup\n",
      "        self).run_setup(setup_script=setup_script)\n",
      "              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n",
      "      File \"C:\\Users\\anils\\AppData\\Local\\Temp\\pip-build-env-3weomz5m\\overlay\\Lib\\site-packages\\setuptools\\build_meta.py\", line 142, in run_setup\n",
      "        exec(compile(code, __file__, 'exec'), locals())\n",
      "      File \"setup.py\", line 508, in <module>\n",
      "        setup_package()\n",
      "      File \"setup.py\", line 500, in setup_package\n",
      "        setup(**metadata)\n",
      "      File \"C:\\Users\\anils\\AppData\\Local\\Temp\\pip-install-357unvi0\\numpy_b70e9bfa1fee4034b563f15b605e89df\\numpy\\distutils\\core.py\", line 169, in setup\n",
      "        return old_setup(**new_attr)\n",
      "               ^^^^^^^^^^^^^^^^^^^^^\n",
      "      File \"C:\\Users\\anils\\AppData\\Local\\Temp\\pip-build-env-3weomz5m\\overlay\\Lib\\site-packages\\setuptools\\__init__.py\", line 165, in setup\n",
      "        return distutils.core.setup(**attrs)\n",
      "               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n",
      "      File \"C:\\Users\\anils\\AppData\\Local\\Temp\\pip-build-env-3weomz5m\\overlay\\Lib\\site-packages\\setuptools\\_distutils\\core.py\", line 148, in setup\n",
      "        dist.run_commands()\n",
      "      File \"C:\\Users\\anils\\AppData\\Local\\Temp\\pip-build-env-3weomz5m\\overlay\\Lib\\site-packages\\setuptools\\_distutils\\dist.py\", line 967, in run_commands\n",
      "        self.run_command(cmd)\n",
      "      File \"C:\\Users\\anils\\AppData\\Local\\Temp\\pip-build-env-3weomz5m\\overlay\\Lib\\site-packages\\setuptools\\_distutils\\dist.py\", line 986, in run_command\n",
      "        cmd_obj.run()\n",
      "      File \"C:\\Users\\anils\\AppData\\Local\\Temp\\pip-build-env-3weomz5m\\overlay\\Lib\\site-packages\\setuptools\\command\\dist_info.py\", line 31, in run\n",
      "        egg_info.run()\n",
      "      File \"C:\\Users\\anils\\AppData\\Local\\Temp\\pip-install-357unvi0\\numpy_b70e9bfa1fee4034b563f15b605e89df\\numpy\\distutils\\command\\egg_info.py\", line 24, in run\n",
      "        self.run_command(\"build_src\")\n",
      "      File \"C:\\Users\\anils\\AppData\\Local\\Temp\\pip-build-env-3weomz5m\\overlay\\Lib\\site-packages\\setuptools\\_distutils\\cmd.py\", line 313, in run_command\n",
      "        self.distribution.run_command(command)\n",
      "      File \"C:\\Users\\anils\\AppData\\Local\\Temp\\pip-build-env-3weomz5m\\overlay\\Lib\\site-packages\\setuptools\\_distutils\\dist.py\", line 986, in run_command\n",
      "        cmd_obj.run()\n",
      "      File \"C:\\Users\\anils\\AppData\\Local\\Temp\\pip-install-357unvi0\\numpy_b70e9bfa1fee4034b563f15b605e89df\\numpy\\distutils\\command\\build_src.py\", line 144, in run\n",
      "        self.build_sources()\n",
      "      File \"C:\\Users\\anils\\AppData\\Local\\Temp\\pip-install-357unvi0\\numpy_b70e9bfa1fee4034b563f15b605e89df\\numpy\\distutils\\command\\build_src.py\", line 155, in build_sources\n",
      "        self.build_library_sources(*libname_info)\n",
      "      File \"C:\\Users\\anils\\AppData\\Local\\Temp\\pip-install-357unvi0\\numpy_b70e9bfa1fee4034b563f15b605e89df\\numpy\\distutils\\command\\build_src.py\", line 288, in build_library_sources\n",
      "        sources = self.generate_sources(sources, (lib_name, build_info))\n",
      "                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n",
      "      File \"C:\\Users\\anils\\AppData\\Local\\Temp\\pip-install-357unvi0\\numpy_b70e9bfa1fee4034b563f15b605e89df\\numpy\\distutils\\command\\build_src.py\", line 378, in generate_sources\n",
      "        source = func(extension, build_dir)\n",
      "                 ^^^^^^^^^^^^^^^^^^^^^^^^^^\n",
      "      File \"numpy\\core\\setup.py\", line 658, in get_mathlib_info\n",
      "        st = config_cmd.try_link('int main(void) { return 0;}')\n",
      "             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n",
      "      File \"C:\\Users\\anils\\AppData\\Local\\Temp\\pip-build-env-3weomz5m\\overlay\\Lib\\site-packages\\setuptools\\_distutils\\command\\config.py\", line 243, in try_link\n",
      "        self._link(body, headers, include_dirs,\n",
      "      File \"C:\\Users\\anils\\AppData\\Local\\Temp\\pip-install-357unvi0\\numpy_b70e9bfa1fee4034b563f15b605e89df\\numpy\\distutils\\command\\config.py\", line 162, in _link\n",
      "        return self._wrap_method(old_config._link, lang,\n",
      "               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n",
      "      File \"C:\\Users\\anils\\AppData\\Local\\Temp\\pip-install-357unvi0\\numpy_b70e9bfa1fee4034b563f15b605e89df\\numpy\\distutils\\command\\config.py\", line 96, in _wrap_method\n",
      "        ret = mth(*((self,)+args))\n",
      "              ^^^^^^^^^^^^^^^^^^^^\n",
      "      File \"C:\\Users\\anils\\AppData\\Local\\Temp\\pip-build-env-3weomz5m\\overlay\\Lib\\site-packages\\setuptools\\_distutils\\command\\config.py\", line 137, in _link\n",
      "        (src, obj) = self._compile(body, headers, include_dirs, lang)\n",
      "                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n",
      "      File \"C:\\Users\\anils\\AppData\\Local\\Temp\\pip-install-357unvi0\\numpy_b70e9bfa1fee4034b563f15b605e89df\\numpy\\distutils\\command\\config.py\", line 105, in _compile\n",
      "        src, obj = self._wrap_method(old_config._compile, lang,\n",
      "                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n",
      "      File \"C:\\Users\\anils\\AppData\\Local\\Temp\\pip-install-357unvi0\\numpy_b70e9bfa1fee4034b563f15b605e89df\\numpy\\distutils\\command\\config.py\", line 96, in _wrap_method\n",
      "        ret = mth(*((self,)+args))\n",
      "              ^^^^^^^^^^^^^^^^^^^^\n",
      "      File \"C:\\Users\\anils\\AppData\\Local\\Temp\\pip-build-env-3weomz5m\\overlay\\Lib\\site-packages\\setuptools\\_distutils\\command\\config.py\", line 132, in _compile\n",
      "        self.compiler.compile([src], include_dirs=include_dirs)\n",
      "      File \"C:\\Users\\anils\\AppData\\Local\\Temp\\pip-build-env-3weomz5m\\overlay\\Lib\\site-packages\\setuptools\\_distutils\\_msvccompiler.py\", line 401, in compile\n",
      "        self.spawn(args)\n",
      "      File \"C:\\Users\\anils\\AppData\\Local\\Temp\\pip-build-env-3weomz5m\\overlay\\Lib\\site-packages\\setuptools\\_distutils\\_msvccompiler.py\", line 505, in spawn\n",
      "        return super().spawn(cmd, env=env)\n",
      "               ^^^^^^^^^^^^^^^^^^^^^^^^^^^\n",
      "      File \"C:\\Users\\anils\\AppData\\Local\\Temp\\pip-install-357unvi0\\numpy_b70e9bfa1fee4034b563f15b605e89df\\numpy\\distutils\\ccompiler.py\", line 90, in <lambda>\n",
      "        m = lambda self, *args, **kw: func(self, *args, **kw)\n",
      "                                      ^^^^^^^^^^^^^^^^^^^^^^^\n",
      "    TypeError: CCompiler_spawn() got an unexpected keyword argument 'env'\n",
      "    [end of output]\n",
      "  \n",
      "    note: This error originates from a subprocess, and is likely not a problem with pip.\n",
      "  error: metadata-generation-failed\n",
      "  \n",
      "  Encountered error while generating package metadata.\n",
      "  \n",
      "  See above for output.\n",
      "  \n",
      "  note: This is an issue with the package mentioned above, not pip.\n",
      "  hint: See above for details.\n",
      "  WARNING: Ignoring invalid distribution ~atplotlib (C:\\Users\\anils\\AppData\\Roaming\\Python\\Python311\\site-packages)\n",
      "  WARNING: Ignoring invalid distribution ~atplotlib (C:\\Users\\anils\\AppData\\Roaming\\Python\\Python311\\site-packages)\n",
      "  WARNING: Ignoring invalid distribution ~atplotlib (C:\\Users\\anils\\AppData\\Roaming\\Python\\Python311\\site-packages)\n",
      "  \n",
      "  [notice] A new release of pip is available: 24.0 -> 24.2\n",
      "  [notice] To update, run: python.exe -m pip install --upgrade pip\n",
      "  [end of output]\n",
      "  \n",
      "  note: This error originates from a subprocess, and is likely not a problem with pip.\n",
      "error: subprocess-exited-with-error\n",
      "\n",
      "pip subprocess to install build dependencies did not run successfully.\n",
      "exit code: 1\n",
      "\n",
      "See above for output.\n",
      "\n",
      "note: This error originates from a subprocess, and is likely not a problem with pip.\n",
      "WARNING: Ignoring invalid distribution ~atplotlib (C:\\Users\\anils\\AppData\\Roaming\\Python\\Python311\\site-packages)\n",
      "WARNING: Ignoring invalid distribution ~atplotlib (C:\\Users\\anils\\AppData\\Roaming\\Python\\Python311\\site-packages)\n",
      "WARNING: Ignoring invalid distribution ~atplotlib (C:\\Users\\anils\\AppData\\Roaming\\Python\\Python311\\site-packages)\n",
      "\n",
      "[notice] A new release of pip is available: 24.0 -> 24.2\n",
      "[notice] To update, run: python.exe -m pip install --upgrade pip\n"
     ]
    }
   ],
   "execution_count": 3
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Imports"
   ]
  },
  {
   "cell_type": "code",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-09-27T15:27:12.203612Z",
     "start_time": "2024-09-27T15:27:10.464255Z"
    }
   },
   "source": [
    "## This cell contains the essential imports you will need – DO NOT CHANGE THE CONTENTS! ##\n",
    "import torch\n",
    "import torch.nn as nn\n",
    "import torch.nn.functional as F\n",
    "import torch.optim as optim\n",
    "import torchvision\n",
    "import torchvision.transforms as transforms\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np"
   ],
   "outputs": [
    {
     "ename": "ModuleNotFoundError",
     "evalue": "No module named 'torch'",
     "output_type": "error",
     "traceback": [
      "\u001B[1;31m---------------------------------------------------------------------------\u001B[0m",
      "\u001B[1;31mModuleNotFoundError\u001B[0m                       Traceback (most recent call last)",
      "Cell \u001B[1;32mIn[1], line 2\u001B[0m\n\u001B[0;32m      1\u001B[0m \u001B[38;5;66;03m## This cell contains the essential imports you will need – DO NOT CHANGE THE CONTENTS! ##\u001B[39;00m\n\u001B[1;32m----> 2\u001B[0m \u001B[38;5;28;01mimport\u001B[39;00m \u001B[38;5;21;01mtorch\u001B[39;00m\n\u001B[0;32m      3\u001B[0m \u001B[38;5;28;01mimport\u001B[39;00m \u001B[38;5;21;01mtorch\u001B[39;00m\u001B[38;5;21;01m.\u001B[39;00m\u001B[38;5;21;01mnn\u001B[39;00m \u001B[38;5;28;01mas\u001B[39;00m \u001B[38;5;21;01mnn\u001B[39;00m\n\u001B[0;32m      4\u001B[0m \u001B[38;5;28;01mimport\u001B[39;00m \u001B[38;5;21;01mtorch\u001B[39;00m\u001B[38;5;21;01m.\u001B[39;00m\u001B[38;5;21;01mnn\u001B[39;00m\u001B[38;5;21;01m.\u001B[39;00m\u001B[38;5;21;01mfunctional\u001B[39;00m \u001B[38;5;28;01mas\u001B[39;00m \u001B[38;5;21;01mF\u001B[39;00m\n",
      "\u001B[1;31mModuleNotFoundError\u001B[0m: No module named 'torch'"
     ]
    }
   ],
   "execution_count": 1
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Load the Dataset\n",
    "\n",
    "Specify your transforms as a list if you intend to .\n",
    "The transforms module is already loaded as `transforms`.\n",
    "\n",
    "MNIST is fortunately included in the torchvision module.\n",
    "Then, you can create your dataset using the `MNIST` object from `torchvision.datasets` ([the documentation is available here](https://pytorch.org/vision/stable/datasets.html#mnist)).\n",
    "Make sure to specify `download=True`! \n",
    "\n",
    "Once your dataset is created, you'll also need to define a `DataLoader` from the `torch.utils.data` module for both the train and the test set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Define transforms\n",
    "transform = transforms.Compose([\n",
    "    transforms.ToTensor(),\n",
    "    transforms.Normalize((0.1307,), (0.3081,))\n",
    "])\n",
    "\n",
    "# Create training set and define training dataloader\n",
    "train_set = torchvision.datasets.MNIST(root='./data', train=True, download=True, transform=transform)\n",
    "train_loader = torch.utils.data.DataLoader(train_set, batch_size=64, shuffle=True)\n",
    "\n",
    "# Create test set and define test dataloader\n",
    "test_set = torchvision.datasets.MNIST(root='./data', train=False, download=True, transform=transform)\n",
    "test_loader = torch.utils.data.DataLoader(test_set, batch_size=1000, shuffle=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Justify your preprocessing\n",
    "\n",
    "In your own words, why did you choose the transforms you chose? If you didn't use any preprocessing steps, why not?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The chosen transforms are:  \n",
    "transforms.ToTensor(): This transform converts the images from the dataset into PyTorch tensors. This is necessary because PyTorch models require input data to be in tensor format.  \n",
    "transforms.Normalize((0.1307,), (0.3081,)): This normalizes the tensor images with a mean of 0.1307 and a standard deviation of 0.3081. These values are specific to the MNIST dataset and help in stabilizing and speeding up the training process by ensuring that the input data has a consistent distribution."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Explore the Dataset\n",
    "Using matplotlib, numpy, and torch, explore the dimensions of your data.\n",
    "\n",
    "You can view images using the `show5` function defined below – it takes a data loader as an argument.\n",
    "Remember that normalized images will look really weird to you! You may want to try changing your transforms to view images.\n",
    "Typically using no transforms other than `toTensor()` works well for viewing – but not as well for training your network.\n",
    "If `show5` doesn't work, go back and check your code for creating your data loaders and your training/test sets."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "## This cell contains a function for showing 5 images from a dataloader – DO NOT CHANGE THE CONTENTS! ##\n",
    "def show(img_loader):\n",
    "    dataiter = iter(img_loader)\n",
    "    \n",
    "    batch = next(dataiter)\n",
    "    labels = batch[1][0:5]\n",
    "    images = batch[0][0:5]\n",
    "    for i in range(5):\n",
    "        print(int(labels[i].detach()))\n",
    "    \n",
    "        image = images[i].numpy()\n",
    "        plt.imshow(image.T.squeeze().T)\n",
    "        plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Explore data\n",
    "show(train_loader)\n",
    "show(test_loader)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Build your Neural Network\n",
    "Using the layers in `torch.nn` (which has been imported as `nn`) and the `torch.nn.functional` module (imported as `F`), construct a neural network based on the parameters of the dataset.\n",
    "Use any architecture you like. \n",
    "\n",
    "*Note*: If you did not flatten your tensors in your transforms or as part of your preprocessing and you are using only `Linear` layers, make sure to use the `Flatten` layer in your network!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch.nn as nn\n",
    "import torch.nn.functional as F\n",
    "\n",
    "class Net(nn.Module):\n",
    "    def __init__(self):\n",
    "        super(Net, self).__init__()\n",
    "        self.fc1 = nn.Linear(28 * 28, 512)\n",
    "        self.fc2 = nn.Linear(512, 256)\n",
    "        self.fc3 = nn.Linear(256, 10)\n",
    "\n",
    "    def forward(self, x):\n",
    "        x = x.view(-1, 28 * 28)\n",
    "        x = F.relu(self.fc1(x))\n",
    "        x = F.relu(self.fc2(x))\n",
    "        x = self.fc3(x)\n",
    "        return F.log_softmax(x, dim=1)\n",
    "\n",
    "model = Net()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Specify a loss function and an optimizer, and instantiate the model.\n",
    "\n",
    "If you use a less common loss function, please note why you chose that loss function in a comment."
   ]
  },
  {
   "metadata": {},
   "cell_type": "code",
   "outputs": [],
   "execution_count": null,
   "source": [
    "import torch.optim as optim\n",
    "\n",
    "criterion = nn.CrossEntropyLoss()\n",
    "optimizer = optim.SGD(model.parameters(), lr=0.01, momentum=0.9)"
   ]
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": [
    "## Running your Neural Network\n",
    "Use whatever method you like to train your neural network, and ensure you record the average loss at each epoch. \n",
    "Don't forget to use `torch.device()` and the `.to()` method for both your model and your data if you are using GPU!\n",
    "\n",
    "If you want to print your loss **during** each epoch, you can use the `enumerate` function and print the loss after a set number of batches. 250 batches works well for most people!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n",
    "model.to(device)\n",
    "\n",
    "num_epochs = 10\n",
    "train_losses = []\n",
    "\n",
    "for epoch in range(num_epochs):\n",
    "    model.train()\n",
    "    running_loss = 0.0\n",
    "    for i, data in enumerate(train_loader, 0):\n",
    "        inputs, labels = data\n",
    "        inputs, labels = inputs.to(device), labels.to(device)\n",
    "\n",
    "        optimizer.zero_grad()\n",
    "        outputs = model(inputs)\n",
    "        loss = criterion(outputs, labels)\n",
    "        loss.backward()\n",
    "        optimizer.step()\n",
    "\n",
    "        running_loss += loss.item()\n",
    "        if i % 250 == 249:\n",
    "            print(f\"[{epoch + 1}, {i + 1}] loss: {running_loss / 250:.3f}\")\n",
    "            running_loss = 0.0\n",
    "\n",
    "    train_losses.append(running_loss / len(train_loader))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Plot the training loss (and validation loss/accuracy, if recorded)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plt.plot(train_losses, label='Training loss')\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Testing your model\n",
    "Using the previously created `DataLoader` for the test set, compute the percentage of correct predictions using the highest probability prediction. \n",
    "\n",
    "If your accuracy is over 90%, great work, but see if you can push a bit further! \n",
    "If your accuracy is under 90%, you'll need to make improvements.\n",
    "Go back and check your model architecture, loss function, and optimizer to make sure they're appropriate for an image classification task."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.eval()\n",
    "correct = 0\n",
    "total = 0\n",
    "with torch.no_grad():\n",
    "    for data in test_loader:\n",
    "        images, labels = data\n",
    "        images, labels = images.to(device), labels.to(device)\n",
    "        outputs = model(images)\n",
    "        _, predicted = torch.max(outputs.data, 1)\n",
    "        total += labels.size(0)\n",
    "        correct += (predicted == labels).sum().item()\n",
    "\n",
    "print(f\"Accuracy of the network on the 10000 test images: {100 * correct / total:.2f}%\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Improving your model\n",
    "\n",
    "Once your model is done training, try tweaking your hyperparameters and training again below to improve your accuracy on the test set!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# possible improvements:\n",
    "## Adjust hyperparameters\n",
    "learning_rate = 0.001\n",
    "batch_size = 128\n",
    "num_epochs = 20\n",
    "\n",
    "## Update dataloaders with new batch size\n",
    "train_loader = torch.utils.data.DataLoader(train_set, batch_size=batch_size, shuffle=True)\n",
    "test_loader = torch.utils.data.DataLoader(test_set, batch_size=batch_size, shuffle=False)\n",
    "\n",
    "## Update optimizer with new learning rate\n",
    "optimizer = optim.SGD(model.parameters(), lr=learning_rate, momentum=0.9)\n",
    "\n",
    "\"\"\"\n",
    "# Retrain the model with new hyperparameters\n",
    "train_losses = []\n",
    "\n",
    "for epoch in range(num_epochs):\n",
    "    model.train()\n",
    "    running_loss = 0.0\n",
    "    for i, data in enumerate(train_loader, 0):\n",
    "        inputs, labels = data\n",
    "        inputs, labels = inputs.to(device), labels.to(device)\n",
    "\n",
    "        optimizer.zero_grad()\n",
    "        outputs = model(inputs)\n",
    "        loss = criterion(outputs, labels)\n",
    "        loss.backward()\n",
    "        optimizer.step()\n",
    "\n",
    "        running_loss += loss.item()\n",
    "        if i % 250 == 249:\n",
    "            print(f\"[{epoch + 1}, {i + 1}] loss: {running_loss / 250:.3f}\")\n",
    "            running_loss = 0.0\n",
    "\n",
    "    train_losses.append(running_loss / len(train_loader))\n",
    "\n",
    "# Plot the training loss\n",
    "plt.plot(train_losses, label='Training loss')\n",
    "plt.legend()\n",
    "plt.show()\n",
    "\n",
    "# Test the model\n",
    "model.eval()\n",
    "correct = 0\n",
    "total = 0\n",
    "with torch.no_grad():\n",
    "    for data in test_loader:\n",
    "        images, labels = data\n",
    "        images, labels = images.to(device), labels.to(device)\n",
    "        outputs = model(images)\n",
    "        _, predicted = torch.max(outputs.data, 1)\n",
    "        total += labels.size(0)\n",
    "        correct += (predicted == labels).sum().item()\n",
    "\n",
    "print(f\"Accuracy of the network on the 10000 test images: {100 * correct / total:.2f}%\")\n",
    "\"\"\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Saving your model\n",
    "Using `torch.save`, save your model for future loading."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": "torch.save(model.state_dict(), 'mnist_model.pth')"
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}