daal/pca-example.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Principal Component Analysis with Intel® Data Analytics Acceleration Library in Amazon SageMaker\n",
    "\n",
    "## Introduction\n",
    "\n",
    "Intel® Data Analytics Acceleration Library (Intel® DAAL) is the library of Intel® architecture optimized building blocks covering all stages of data analytics: data acquisition from a data source, preprocessing, transformation, data mining, modeling, validation, and decision making. One of its algorithms is PCA.\n",
    "\n",
    "Principal Component Analysis (PCA) is a method for exploratory data analysis. PCA transforms a set of observations of possibly correlated variables to a new set of uncorrelated variables, called principal components. Principal components are the directions of the largest variance, that is, the directions where the data is mostly spread out.\n",
    "\n",
    "Because all principal components are orthogonal to each other, there is no redundant information. This is a way of replacing a group of variables with a smaller set of new variables. PCA is one of powerful techniques for dimension reduction.\n",
    "\n",
    "Intel® DAAL developer guide: https://software.intel.com/en-us/daal-programming-guide\n",
    "\n",
    "Intel® DAAL documentation for PCA: https://software.intel.com/en-us/daal-programming-guide-principal-component-analysis"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## PCA Usage with SageMaker Estimator\n",
    "Firstly, you need to import SageMaker package, get execution role and create session."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "import sagemaker\n",
    "\n",
    "role = sagemaker.get_execution_role()\n",
    "sess = sagemaker.Session()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Secondly, you can specify parameters of PCA.\n",
    "#### Hyperparameters\n",
    "All hyperparameters of PCA algorithm are optional.\n",
    "<table style=\"border: 1px solid black;\">\n",
    "    <tr>\n",
    "        <td><strong>Parameter name</strong></td>\n",
    "        <td><strong>Type</strong></td>\n",
    "        <td><strong>Default value</strong></td>\n",
    "        <td><strong>Description</strong></td>\n",
    "    </tr>\n",
    "    <tr>\n",
    "        <td>fptype</td>\n",
    "        <td>str</td>\n",
    "        <td>\"double\"</td>\n",
    "        <td>The floating-point type that the algorithm uses for intermediate computations. Can be \"float\" or \"double\"</td>\n",
    "    </tr>\n",
    "    <tr>\n",
    "        <td>method</td>\n",
    "        <td>str</td>\n",
    "        <td>\"correlationDense\"</td>\n",
    "        <td>Available methods for PCA computation: \"correlationDense\" (\"defaultDense\") or \"svdDense\"</td>\n",
    "    </tr>\n",
    "    <tr>\n",
    "        <td>resultsToCompute</td>\n",
    "        <td>str</td>\n",
    "        <td>\"none\"</td>\n",
    "        <td>Provide one of the following values to request a single characteristic or use bitwise OR to request a combination of the characteristics: mean, variance, eigenvalue. For example, combination of all is \"mean|variance|eigenvalue\"</td>\n",
    "    </tr>\n",
    "    <tr>\n",
    "        <td>nComponents</td>\n",
    "        <td>int</td>\n",
    "        <td>0</td>\n",
    "        <td>Number of principal components.<br/> If it is zero, the algorithm will compute the result for number of principal components = number of features.<br/>Remember that number of components must be equal or less than number of features for PCA algorithm</td>\n",
    "    </tr>\n",
    "    <tr>\n",
    "        <td>isDeterministic</td>\n",
    "        <td>bool</td>\n",
    "        <td>False</td>\n",
    "        <td>If True, the algorithm applies the \"sign flip\" technique to the results</td>\n",
    "    </tr>\n",
    "    <tr>\n",
    "        <td>transformOnTrain</td>\n",
    "        <td>bool</td>\n",
    "        <td>False</td>\n",
    "        <td>If True, training data will be transformed and saved in model package on training stage</td>\n",
    "    </tr>\n",
    "</table>\n",
    "\n",
    "Example of hyperparameters dictionary:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "pca_params = {\n",
    "    \"fptype\": \"float\",\n",
    "    \"method\": \"svdDense\",\n",
    "    \"resultsToCompute\": \"mean|eigenvalue\",\n",
    "    \"nComponents\": 4,\n",
    "    \"isDeterministic\": True\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Then, you need to create SageMaker Estimator instance with following parameters:\n",
    "<table style=\"border: 1px solid black;\">\n",
    "    <tr>\n",
    "        <td><strong>Parameter name</strong></td>\n",
    "        <td><strong>Description</strong></td>\n",
    "    </tr>\n",
    "    <tr>\n",
    "        <td>image_name</td>\n",
    "        <td>The container image to use for training</td>\n",
    "    </tr>\n",
    "    <tr>\n",
    "        <td>role</td>\n",
    "        <td>An AWS IAM role. The SageMaker training jobs and APIs that create SageMaker endpoints use this role to access training data and models</td>\n",
    "    </tr>\n",
    "    <tr>\n",
    "        <td>train_instance_count</td>\n",
    "        <td>Number of Amazon EC2 instances to use for training. Should be 1, because it is not distributed version of algorithm</td>\n",
    "    </tr>\n",
    "    <tr>\n",
    "        <td>train_instance_type</td>\n",
    "        <td>Type of EC2 instance to use for training. See available types on Amazon Marketplace page of algorithm</td>\n",
    "    </tr>\n",
    "    <tr>\n",
    "        <td>input_mode</td>\n",
    "        <td>The input mode that the algorithm supports. May be \"File\" or \"Pipe\"</td>\n",
    "    </tr>\n",
    "    <tr>\n",
    "        <td>output_path</td>\n",
    "        <td>S3 location for saving the trainig result (model artifacts and output files)</td>\n",
    "    </tr>\n",
    "    <tr>\n",
    "        <td>sagemaker_session</td>\n",
    "        <td>Session object which manages interactions with Amazon SageMaker APIs and any other AWS services needed</td>\n",
    "    </tr>\n",
    "    <tr>\n",
    "        <td>hyperparameters</td>\n",
    "        <td>Dictionary containing the hyperparameters to initialize this estimator with</td>\n",
    "    </tr>\n",
    "</table>\n",
    "Full SageMaker Estimator documentation: https://sagemaker.readthedocs.io/en/latest/estimators.html"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "daal_pca_arn = \"<algorithm-arn>\" # you can find it on algorithm page in your subscriptions\n",
    "\n",
    "daal_pca = sagemaker.algorithm.AlgorithmEstimator(\n",
    "    algorithm_arn=daal_pca_arn,\n",
    "    role=role,\n",
    "    base_job_name=\"<base-job-name>\",\n",
    "    train_instance_count=1,\n",
    "    train_instance_type='ml.m4.xlarge',\n",
    "    input_mode=\"File\",\n",
    "    output_path=\"s3://<bucket-name>/<output-path>\",\n",
    "    sagemaker_session=sess,\n",
    "    hyperparameters=pca_params\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Training stage\n",
    "On training stage, PCA algorithm consume input data from S3 location and computes eigen vectors and other results (if they are specified in \"resultsToCompute\" parameter).\n",
    "This container supports only .csv (\"comma-separated values\") files."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:sagemaker:Creating training-job with name: daal-pca-alg-test-2018-11-30-06-50-06-484\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2018-11-30 06:50:06 Starting - Starting the training job...\n",
      "2018-11-30 06:50:07 Starting - Launching requested ML instances......\n",
      "2018-11-30 06:51:15 Starting - Preparing the instances for training...\n",
      "2018-11-30 06:52:07 Downloading - Downloading input data...\n",
      "2018-11-30 06:52:30 Training - Downloading the training image...\n",
      "2018-11-30 06:53:01 Uploading - Uploading generated training model\n",
      "2018-11-30 06:53:01 Completed - Training job completed\n",
      "\n",
      "\u001b[31m2018-11-30 06:52:49 INFO     Training stage started\u001b[0m\n",
      "\u001b[31m2018-11-30 06:52:49 INFO     Default Paramaters:\u001b[0m\n",
      "\u001b[31m2018-11-30 06:52:49 INFO     {'fptype': 'double', 'method': 'correlationDense', 'resultsToCompute': '', 'nComponents': '0', 'isDeterministic': 'False', 'transformOnTrain': 'False'}\u001b[0m\n",
      "\u001b[31m2018-11-30 06:52:49 INFO     Updated with user hyperparameters, uncorrect parameters changed or deleted\u001b[0m\n",
      "\u001b[31m2018-11-30 06:52:49 INFO     Final Hyperparameters:\u001b[0m\n",
      "\u001b[31m2018-11-30 06:52:49 INFO     {'fptype': 'float', 'method': 'defaultDense', 'resultsToCompute': 'mean|eigenvalue', 'nComponents': 3, 'isDeterministic': True, 'transformOnTrain': False}\u001b[0m\n",
      "\u001b[31m2018-11-30 06:52:54 INFO     Train data shape: (1600000, 10)\u001b[0m\n",
      "\u001b[31m2018-11-30 06:52:54 INFO     Files loading time: 4.69575047492981\u001b[0m\n",
      "\u001b[31m2018-11-30 06:52:54 INFO     Starting DAAL PCA algorithm...\u001b[0m\n",
      "\u001b[31m2018-11-30 06:52:54 INFO     Training time, sec: 0.10286879539489746\u001b[0m\n",
      "\u001b[31m2018-11-30 06:52:54 INFO     Means table saved at /opt/ml/model/mean.csv\u001b[0m\n",
      "\u001b[31m2018-11-30 06:52:54 INFO     Eigenvalues table saved at /opt/ml/model/eigenvalue.csv\u001b[0m\n",
      "\u001b[31m2018-11-30 06:52:54 INFO     Eigen vectors saved at /opt/ml/model/vectors.csv\u001b[0m\n",
      "\u001b[31m2018-11-30 06:52:54 INFO     Parameters saved at /opt/ml/model/parameters.json\u001b[0m\n",
      "Billable seconds: 54\n"
     ]
    }
   ],
   "source": [
    "daal_pca.fit({\"training\": \"s3://<bucket-name>/<training-data-path>\"})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Real-time prediction\n",
    "On prediction stage, PCA algorithm transforms input data using previously computed results.\n",
    "Firstly, you need to deploy SageMaker endpoint that consumes data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:sagemaker:Creating model package with name: intel-daal-pca1542385402-d0d25e75ca6ef4-2018-11-29-15-37-47-871\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      ".........."
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:sagemaker:Creating model with name: intel-daal-pca1542385402-d0d25e75ca6ef4-2018-11-29-15-38-33-468\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:sagemaker:Creating endpoint with name intel-daal-pca1542385402-d0d25e75ca6ef4-2018-11-29-15-31-41-921\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "-------------------------------------------------------------!"
     ]
    }
   ],
   "source": [
    "predictor = daal_pca.deploy(1, \"ml.m4.xlarge\", serializer=sagemaker.predictor.csv_serializer)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Secondly, you should pass data as numpy array to predictor instance and get transformed data as space-separated values.\n",
    "\n",
    "In this example we are passing random data, but you can use any numpy 2D array with one specific condition for PCA: training data and data to transform must have same numbers of features."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1.185592651367187500e+00 2.620933353900909424e-01 3.085311949253082275e-01\n",
      "6.714667081832885742e-01 8.762556910514831543e-01 -8.037568628787994385e-02\n",
      "1.111641526222229004e+00 3.375906124711036682e-02 -1.244278624653816223e-01\n",
      "1.294844508171081543e+00 3.067855909466743469e-02 -9.089314937591552734e-02\n",
      "8.781186342239379883e-01 2.732140570878982544e-02 -5.232766270637512207e-01\n",
      "5.412490963935852051e-01 3.470270335674285889e-01 -8.399704098701477051e-02\n",
      "1.048457860946655273e+00 -1.020107120275497437e-01 6.852779537439346313e-02\n",
      "8.287011384963989258e-01 2.592234015464782715e-01 -1.914716511964797974e-01\n",
      "1.426655769348144531e+00 1.917291432619094849e-02 -1.039648428559303284e-01\n",
      "1.183746933937072754e+00 -8.203709125518798828e-02 3.905816972255706787e-01\n",
      "\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "\n",
    "predict_data = np.random.random(size=(10,10))\n",
    "print(predictor.predict(predict_data).decode(\"utf-8\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Don't forget to delete endpoint if you don't need it anymore."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:sagemaker:Deleting endpoint with name: intel-daal-pca1542385402-d0d25e75ca6ef4-2018-11-29-15-31-41-921\n"
     ]
    }
   ],
   "source": [
    "sess.delete_endpoint(predictor.endpoint)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Batch transform job\n",
    "If you don't need real-time prediction, you can use transform job. It uses saved model, compute transformed data one time and saves it in specified or auto-generated output path.\n",
    "\n",
    "More about transform jobs: https://docs.aws.amazon.com/sagemaker/latest/dg/how-it-works-batch.html\n",
    "\n",
    "Transformer API: https://sagemaker.readthedocs.io/en/latest/transformer.html"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:sagemaker:Creating model package with name: intel-daal-pca1542385402-d0d25e75ca6ef4-2018-11-30-07-42-48-772\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      ".........."
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:sagemaker:Creating model with name: intel-daal-pca1542385402-d0d25e75ca6ef4-2018-11-30-07-43-34-517\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:sagemaker:Creating transform job with name: daal-pca-alg-test-2018-11-30-07-43-35-199\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "......................................!\n",
      "s3://<output-path>\n"
     ]
    }
   ],
   "source": [
    "transformer = daal_pca.transformer(1, 'ml.m4.xlarge')\n",
    "transformer.transform(\"s3://<bucket-name>/<prediction-data-path>\", content_type='text/csv')\n",
    "transformer.wait()\n",
    "print(transformer.output_path)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}