DataCamp Data Science Toolbox Python.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "e7d47923-8d5d-4452-87aa-d54111eba404",
   "metadata": {},
   "source": [
    "## Python Data Science Toolbox (Part 1)\n",
    "&\n",
    "## Python Data Science Toolbox (Part 2)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0e8f21b2-7ade-4721-9df1-e32cfa001d17",
   "metadata": {},
   "source": [
    "# Course Description\n",
    "\n",
    "It's time to push forward and develop your Python chops even further. There are tons of fantastic functions in Python and its library ecosystem. However, as a data scientist, you'll constantly need to write your own functions to solve problems that are dictated by your data. You will learn the art of function writing in this first Python Data Science Toolbox course. You'll come out of this course being able to write your very own custom functions, complete with multiple parameters and multiple return values, along with default arguments and variable-length arguments. You'll gain insight into scoping in Python and be able to write lambda functions and handle errors in your function writing practice. And you'll wrap up each chapter by using your new skills to write functions that analyze Twitter DataFrames."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "720d5338-f97a-4f00-84e8-b84da1e1e3af",
   "metadata": {},
   "source": [
    "##  Writing your own functions\n",
    "Free\n",
    "7%\n",
    "\n",
    "In this chapter, you'll learn how to write simple functions, as well as functions that accept multiple arguments and return multiple values. You'll also have the opportunity to apply these new skills to questions commonly encountered by data scientists.\n",
    "\n",
    "    User-defined functions    50 xp\n",
    "    Strings in Python    50 xp\n",
    "    Recapping built-in functions    50 xp\n",
    "    Write a simple function    100 xp\n",
    "    Single-parameter functions    100 xp\n",
    "    Functions that return single values    100 xp\n",
    "    Multiple parameters and return values    50 xp\n",
    "    Functions with multiple parameters    100 xp\n",
    "    A brief introduction to tuples    100 xp\n",
    "    Functions that return multiple values    100 xp\n",
    "    Bringing it all together    50 xp\n",
    "    Bringing it all together (1)    100 xp\n",
    "    Bringing it all together (2)    100 xp\n",
    "    Congratulations!    50 xp \n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "be8938f2-a1f9-45c3-9df2-fb79485596e1",
   "metadata": {},
   "source": [
    "##  Default arguments, variable-length arguments and scope\n",
    "0%\n",
    "\n",
    "In this chapter, you'll learn to write functions with default arguments so that the user doesn't always need to specify them, and variable-length arguments so they can pass an arbitrary number of arguments on to your functions. You'll also learn about the essential concept of scope.\n",
    "\n",
    "    Scope and user-defined functions    50 xp\n",
    "    Pop quiz on understanding scope    50 xp\n",
    "    The keyword global    100 xp\n",
    "    Python's built-in scope    50 xp\n",
    "    Nested functions    50 xp\n",
    "    Nested Functions I    100 xp\n",
    "    Nested Functions II    100 xp\n",
    "    The keyword nonlocal and nested functions    100 xp\n",
    "    Default and flexible arguments    50 xp\n",
    "    Functions with one default argument    100 xp\n",
    "    Functions with multiple default arguments    100 xp\n",
    "    Functions with variable-length arguments (*args)    100 xp\n",
    "    Functions with variable-length keyword arguments (**kwargs)    100 xp\n",
    "    Bringing it all together    50 xp\n",
    "    Bringing it all together (1)    100 xp\n",
    "    Bringing it all together (2)    100 xp \n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1d385243-06b4-405e-abe6-124e8a1af4de",
   "metadata": {},
   "source": [
    "##  Lambda functions and error-handling\n",
    "0%\n",
    "\n",
    "Learn about lambda functions, which allow you to write functions quickly and on the fly. You'll also practice handling errors in your functions, which is an essential skill. Then, apply your new skills to answer data science questions.\n",
    "\n",
    "    Lambda functions    50 xp\n",
    "    Pop quiz on lambda functions    50 xp\n",
    "    Writing a lambda function you already know    100 xp\n",
    "    Map() and lambda functions    100 xp\n",
    "    Filter() and lambda functions    100 xp\n",
    "    Reduce() and lambda functions    100 xp\n",
    "    Introduction to error handling    50 xp\n",
    "    Pop quiz about errors    50 xp\n",
    "    Error handling with try-except    100 xp\n",
    "    Error handling by raising an error    100 xp\n",
    "    Bringing it all together    50 xp\n",
    "    Bringing it all together (1)    100 xp\n",
    "    Bringing it all together (2)    100 xp\n",
    "    Bringing it all together (3)    100 xp\n",
    "    Bringing it all together: testing your error handling skills    50 xp\n",
    "    Congratulations!    50 xp\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fa8770cd-17b8-4922-94fd-09d8203d64b1",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "4064b253-c4f0-42a1-b9a8-35b4d067a8b4",
   "metadata": {},
   "source": [
    "## Course Description\n",
    "\n",
    "In this second Python Data Science Toolbox course, you'll continue to build your Python data science skills. First, you'll learn about iterators, objects you have already encountered in the context of for loops. You'll then learn about list comprehensions, which are extremely handy tools for all data scientists working in Python. You'll end the course by working through a case study in which you'll apply all the techniques you learned in both parts of this course."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5eb59e77-1f04-49d2-a8b8-6a5aa9954c76",
   "metadata": {},
   "source": [
    "##  Using iterators in PythonLand\n",
    "Free\n",
    "0%\n",
    "\n",
    "You'll learn all about iterators and iterables, which you have already worked with when writing for loops. You'll learn some handy functions that will allow you to effectively work with iterators. And you’ll finish the chapter with a use case that is pertinent to the world of data science and dealing with large amounts of data—in this case, data from Twitter that you will load in chunks using iterators.\n",
    "\n",
    "    Introduction to iterators    50 xp\n",
    "    Iterators vs Iterables    50 xp\n",
    "    Iterating over iterables (1)    100 xp\n",
    "    Iterating over iterables (2)    100 xp\n",
    "    Iterators as function arguments    100 xp\n",
    "    Playing with iterators    50 xp\n",
    "    Using enumerate    100 xp\n",
    "    Using zip    100 xp\n",
    "    Using * and zip to 'unzip'    100 xp\n",
    "    Using iterators to load large files into memory    50 xp\n",
    "    Processing large amounts of Twitter data    100 xp\n",
    "    Extracting information for large amounts of Twitter data    100 xp\n",
    "    Congratulations!    50 xp \n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "31232714-cd5f-4f8f-af20-b553970b2ca1",
   "metadata": {},
   "source": [
    "##  List comprehensions and generators\n",
    "0%\n",
    "\n",
    "In this chapter, you'll build on your knowledge of iterators and be introduced to list comprehensions, which allow you to create complicated lists—and lists of lists—in one line of code! List comprehensions can dramatically simplify your code and make it more efficient, and will become a vital part of your Python data science toolbox. You'll then learn about generators, which are extremely helpful when working with large sequences of data that you may not want to store in memory, but instead generate on the fly.\n",
    "\n",
    "    List comprehensions    50 xp\n",
    "    Write a basic list comprehension    50 xp\n",
    "    List comprehension over iterables    50 xp\n",
    "    Writing list comprehensions    100 xp\n",
    "    Nested list comprehensions    100 xp\n",
    "    Advanced comprehensions    50 xp\n",
    "    Using conditionals in comprehensions (1)    100 xp\n",
    "    Using conditionals in comprehensions (2)    100 xp\n",
    "    Dict comprehensions    100 xp\n",
    "    Introduction to generator expressions    50 xp\n",
    "    List comprehensions vs generators    50 xp\n",
    "    Write your own generator expressions    100 xp\n",
    "    Changing the output in generator expressions    100 xp\n",
    "    Build a generator    100 xp\n",
    "    Wrapping up comprehensions and generators.    50 xp\n",
    "    List comprehensions for time-stamped data    100 xp\n",
    "    Conditional list comprehensions for time-stamped data    100 xp \n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b741d9d5-27f4-4925-87d1-4966d9035680",
   "metadata": {},
   "source": [
    "##  Bringing it all together!\n",
    "0%\n",
    "\n",
    "This chapter will allow you to apply your newly acquired skills toward wrangling and extracting meaningful information from a real-world dataset—the World Bank's World Development Indicators. You'll have the chance to write your own functions and list comprehensions as you work with iterators and generators to solidify your Python data science chops.\n",
    "\n",
    "    Welcome to the case study!    50 xp\n",
    "    Dictionaries for data science    100 xp\n",
    "    Writing a function to help you    100 xp\n",
    "    Using a list comprehension    100 xp\n",
    "    Turning this all into a DataFrame    100 xp\n",
    "    Using Python generators for streaming data    50 xp\n",
    "    Processing data in chunks (1)    100 xp\n",
    "    Writing a generator to load data in chunks (2)    100 xp\n",
    "    Writing a generator to load data in chunks (3)    100 xp\n",
    "    Using pandas' read_csv iterator for streaming data    50 xp\n",
    "    Writing an iterator to load data in chunks (1)    100 xp\n",
    "    Writing an iterator to load data in chunks (2)    100 xp\n",
    "    Writing an iterator to load data in chunks (3)    100 xp\n",
    "    Writing an iterator to load data in chunks (4)    100 xp\n",
    "    Writing an iterator to load data in chunks (5)    100 xp\n",
    "    Final thoughts    50 xp\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fe10c7db-7757-42d2-9d01-cfd815215f60",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "e922c2ed-5a34-48a0-9f9b-ea24b7be1f84",
   "metadata": {},
   "source": [
    "## In this course, the first of the Python Data Science Toolbox courses, you'll learn how to write your very own functions and you'll have the opportunity to apply these newfound skills to questions that commonly arise in Data Science contexts.  \n",
    "\n",
    "Specifically,in this video and in the interactive exercises that follows it, you'll learn to do the following: Define a function without parameters, Define a function with single parameters, and Define a function that return a single value.  \n",
    "\n",
    "In the next section, you'll learn how to pass multiple arguments to functions, as well as return multiple values from them.  "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7a54b58e-1e33-4f5a-9ee3-974078abdf1d",
   "metadata": {},
   "source": [
    "Lets check out Python's build-in function str(), which accepts an object such as number and returns a string object.  \n",
    "\n",
    "As a data scientist, you'll need functions that have functionality specific to your needs.  (like I defined ff_sum() function in Efficient Python course last question.)  \n",
    "\n",
    "\n",
    "We'll now see how to define functions via an example, a function that squares a number.  \n",
    "  To define a function, we begin with the keyword def, followed by the function name (square in this example); This is then followed by a set of parentheses and a colon.  This piece of code is called function header.  \n",
    "  To complete the function definition, lets write the function body by squaring a value, say 4, and printing the output.  \n",
    "  Right now, our function does not have any parameters within the parentheses, we'll add them later.  \n",
    "  Now whenever this function is called, the code in the function body is run. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "8a4e88c0-94d4-4322-89c7-a620d0237c8e",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "16\n"
     ]
    }
   ],
   "source": [
    "def square():    # <- Function Header\n",
    "    new_value = 4**2\n",
    "    print(new_value)\n",
    "    \n",
    "    \n",
    "square()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ceccffcb-51ab-4f7b-be71-6d3d6994882c",
   "metadata": {},
   "source": [
    "We can add a parameter to the function definition in between the parentheses, here you see that we've added a parameter value and in the new function body, the new_value variable takes the square of value, which is then printed out.  We can now square any number that we pass to the function n_square as an argument.  \n",
    "\n",
    "# ***************************************************************************************************\n",
    "# ***************************************************************************************************\n",
    "# A quick word on parameters and arguments: when you define a function, you write parameters in the function header.  When you call the function, you pass arguments into the function.  \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "8dfacf21-3bc7-4348-86fd-6dab649e5eb4",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "144\n"
     ]
    }
   ],
   "source": [
    "def n_square(value):\n",
    "    new_value = value**2\n",
    "    print(new_value)\n",
    "    \n",
    "    \n",
    "n_square(12)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0e0b0974-921d-4d17-aa1e-224dfed40151",
   "metadata": {},
   "source": [
    "you can have your function return the new value by adding the return keyword, followed by the value to return.  Now we can assign to a variable ss the result of the function call.  \n",
    "\n",
    "\n",
    "There is another essqntial aspectof writing functions in Python: docstrings.  It's used to describe what your function does, such as the computations it performs or its return values.  These descriptions serve as documentation for your function so that anyone who read your function's docstring understands what your function does, without having to trace through all the code in the function definition.  \n",
    "\n",
    "Function docstrings are placed in the immediate line after the function header and are placed in between triple quotation marks.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "988d4b3b-526e-4a4f-b453-fe7105953646",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "144\n"
     ]
    }
   ],
   "source": [
    "def n_square(value):\n",
    "    '''Return the square of a value\n",
    "    '''\n",
    "    new_value = value**2\n",
    "    return new_value\n",
    "    \n",
    "    \n",
    "ss = n_square(12)\n",
    "print(ss)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2770067e-432e-4d56-bcf9-a9088f9fdcea",
   "metadata": {},
   "source": [
    "## Strings in Python\n",
    "\n",
    "In the video, you learned of another standard Python datatype, strings. Recall that these represent textual data. To assign the string 'DataCamp' to a variable company, you execute:\n",
    "\n",
    "company = 'DataCamp'\n",
    "\n",
    "You've also learned to use the operations + and * with strings. Unlike with numeric types such as ints and floats, the + operator concatenates strings together, while the * concatenates multiple copies of a string together. In this exercise, you will use the + and * operations on strings to answer the question below. Execute the following code in the shell:\n",
    "\n",
    "object1 = \"data\" + \"analysis\" + \"visualization\"\n",
    "object2 = 1 * 3\n",
    "object3 = \"1\" * 3\n",
    "\n",
    "What are the values in object1, object2, and object3, respectively?\n",
    "Instructions\n",
    "50 XP\n",
    "Possible Answers\n",
    "\n",
    "    object1 contains \"data + analysis + visualization\", object2 contains \"1*3\", object3 contains 13.\n",
    "    object1 contains \"data+analysis+visualization\", object2 contains 3, object3 contains \"13\".\n",
    "#    object1 contains \"dataanalysisvisualization\", object2 contains 3, object3 contains \"111\".\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "00bf9094-7f89-42c2-9c52-14ba3d8866c2",
   "metadata": {},
   "source": [
    "## Recapping built-in functions\n",
    "\n",
    "In the video, Hugo briefly examined the return behavior of the built-in functions print() and str(). Here, you will use both functions and examine their return values. A variable x has been preloaded for this exercise. Run the code below in the console. Pay close attention to the results to answer the question that follows.\n",
    "\n",
    "    Assign str(x) to a variable y1: y1 = str(x)\n",
    "    Assign print(x) to a variable y2: y2 = print(x)\n",
    "    Check the types of the variables x, y1, and y2.\n",
    "\n",
    "What are the types of x, y1, and y2?\n",
    "Instructions\n",
    "50 XP\n",
    "Possible Answers\n",
    "\n",
    "    They are all str types.\n",
    "    x is a float, y1 is an float, and y2 is a str.\n",
    "#    x is a float, y1 is a str, and y2 is a NoneType.\n",
    "    They are all NoneType types.\n",
    "    \n",
    "In [1]:\n",
    "x\n",
    "Out[1]:\n",
    "4.89\n",
    "In [3]:\n",
    "y1 = str(x)\n",
    "In [4]:\n",
    "y2 = print(x)\n",
    "4.89\n",
    "In [5]:\n",
    "type(y2)\n",
    "Out[5]:\n",
    "NoneType\n",
    "# *****************************************************************************************************"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "60828f94-bdda-44f5-abf6-6efee0ff42bc",
   "metadata": {},
   "source": [
    "## Write a simple function\n",
    "\n",
    "In the last video, Hugo described the basics of how to define a function. You will now write your own function!\n",
    "\n",
    "Define a function, shout(), which simply prints out a string with three exclamation marks '!!!' at the end. The code for the square() function that we wrote earlier is found below. You can use it as a pattern to define shout().\n",
    "\n",
    "def square():\n",
    "    new_value = 4 ** 2\n",
    "    return new_value\n",
    "\n",
    "Note that the function body is indented 4 spaces already for you. Function bodies need to be indented by a consistent number of spaces and the choice of 4 is common.\n",
    "\n",
    "This course touches on a lot of concepts you may have forgotten, so if you ever need a quick refresher, download the Python for data science Cheat Sheet and keep it handy!\n",
    "Instructions\n",
    "100 XP\n",
    "\n",
    "    Complete the function header by adding the appropriate function name, shout.\n",
    "    In the function body, concatenate the string, 'congratulations' with another string, '!!!'. Assign the result to shout_word.\n",
    "    Print the value of shout_word.\n",
    "    Call the shout function.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "01dd6dc9-d8b2-4497-8eea-fd79222f693e",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "congratulations!!!\n"
     ]
    }
   ],
   "source": [
    "# Define the function shout\n",
    "def shout():\n",
    "    \"\"\"Print a string with three exclamation marks\"\"\"\n",
    "    # Concatenate the strings: shout_word\n",
    "    shout_word = 'congratulations' + '!!!'\n",
    "\n",
    "    # Print shout_word\n",
    "    print(shout_word)\n",
    "\n",
    "# Call shout\n",
    "shout()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "90c4c27f-554c-462f-81d1-d9b8162e3156",
   "metadata": {},
   "source": [
    "## Single-parameter functions\n",
    "\n",
    "Congratulations! You have successfully defined and called your own function! That's pretty cool.\n",
    "\n",
    "In the previous exercise, you defined and called the function shout(), which printed out a string concatenated with '!!!'. You will now update shout() by adding a parameter so that it can accept and process any string argument passed to it. \n",
    "# Also note that shout(word), the part of the header that specifies the function name and parameter(s), is known as the signature of the function. You may encounter this term in the wild!\n",
    "Instructions\n",
    "100 XP\n",
    "\n",
    "    Complete the function header by adding the parameter name, word.\n",
    "    Assign the result of concatenating word with '!!!' to shout_word.\n",
    "    Print the value of shout_word.\n",
    "    Call the shout() function, passing to it the string, 'congratulations'.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "3d15225a-3a13-4e28-bfc5-239f8efacb50",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "congratulations!!!\n"
     ]
    }
   ],
   "source": [
    "# Define shout with the parameter, word\n",
    "# *************************************************************************************************** #\n",
    "def shout(word):   # <- shout(word) is called the signature of the function\n",
    "    \"\"\"Print a string with three exclamation marks\"\"\"\n",
    "    # Concatenate the strings: shout_word\n",
    "    shout_word = word + '!!!'\n",
    "\n",
    "    # Print shout_word\n",
    "    print(shout_word)\n",
    "\n",
    "# Call shout with the string 'congratulations'\n",
    "shout('congratulations')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "749ff04c-cd84-4b8b-b724-1f65c94ffebc",
   "metadata": {},
   "source": [
    "## Functions that return single values\n",
    "\n",
    "You're getting very good at this! Try your hand at another modification to the shout() function so that it now returns a single value instead of printing within the function. Recall that the return keyword lets you return values from functions. Parts of the function shout(), which you wrote earlier, are shown. \n",
    "# Returning values is generally more desirable than printing them out because, as you saw earlier, a print() call assigned to a variable has type NoneType.\n",
    "Instructions\n",
    "100 XP\n",
    "\n",
    "    In the function body, concatenate the string in word with '!!!' and assign to shout_word.\n",
    "    Replace the print() statement with the appropriate return statement.\n",
    "    Call the shout() function, passing to it the string, 'congratulations', and assigning the call to the variable, yell.\n",
    "    To check if yell contains the value returned by shout(), print the value of yell.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "76a5da70-7804-4899-a487-be6e2fdf2588",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "congratulations!!!\n",
      "<class 'str'>\n"
     ]
    }
   ],
   "source": [
    "# Define shout with the parameter, word\n",
    "def shout(word):\n",
    "    \"\"\"Return a string with three exclamation marks\"\"\"\n",
    "    # Concatenate the strings: shout_word\n",
    "    shout_word = word + '!!!'\n",
    "\n",
    "    # Replace print with return\n",
    "    return shout_word\n",
    "\n",
    "# Pass 'congratulations' to shout: yell\n",
    "yell = shout('congratulations')\n",
    "\n",
    "# Print yell\n",
    "print(yell)\n",
    "\n",
    "print(type(yell))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "50f38e18-69eb-4d31-8545-40e879a2e621",
   "metadata": {},
   "source": [
    "## Multiple parameters and return values\n",
    "\n",
    "\n",
    "\n",
    "# **At this point, you already know how to define your own functions and even return values from them.  What you'll learn next is how to pass multiple arguments to functions, as well as return not just one, but multiple values from them.  \n",
    "\n",
    "Let's tweak the square function we've been working on a little bit more.  Suppose that, instead of simply squaring a value, we'd like to raise a value to the power of another value that's also passed to the function.  We can do this by having our function accept two parameters instead of just one.  \n",
    "\n",
    "You should also changeyour function name and docstring to reflect this new behavior.  you can call the function by passing in two arguments because the function has two parameters, as declared in the function header.  The order in which the arguments are passed correspond to the order of the parameters in the function header.  \n",
    "This means that when we call raise_to_power(2, 3), when the function is executed, 2 would be assigned to value1 and 3 to value2.  Looking at the function body, this meansthat the computation value1 to the power of value2 translates to 2 to the power of 3.  This function call then returns the value 8.  \n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "fd7e4ce3-e8a2-44c4-8653-1eaa9f24ddd5",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "81"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def raise_to_power(value1, value2):\n",
    "    '''Raise value1 to the power of value2\n",
    "    '''\n",
    "    new_value = value1**value2\n",
    "    return new_value\n",
    "\n",
    "\n",
    "raise_to_power(3, 4)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "276c42f5-3afd-4fec-8822-d938bca913cd",
   "metadata": {},
   "source": [
    "# You can also make your function return multiple values, you can do that by constructing object known as tuples in your functions.  A tuple is like a list, in that it can contain multiple values.  There are some difference however. \n",
    "\n",
    "Firstly, unlike a list, a tuple is immutable, that is you cannot modify the values in a tuple once it has been constructed.  \n",
    "Secondly, while lists are defined using square brackets, tuples are constructed using a set of parentheses.  \n",
    "\n",
    "\n",
    "# Additionally, you can also access individual tuple elements like you do with lists.  \n",
    "Recall that with lists, you can use zero-indexing to access list elements. You can do the same thing with tuple. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "53e37aeb-dbc2-4341-a52e-5f818336b2e8",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1\n",
      "4\n",
      "[1, 2, 3, 4, 6, 8]\n",
      "4\n"
     ]
    }
   ],
   "source": [
    "my_tuple = (1, 2, 3, 4, 6, 8)\n",
    "[*aa] = my_tuple\n",
    "\n",
    "a,b,c,d,e,f = my_tuple\n",
    "\n",
    "print(a)\n",
    "print(d)\n",
    "print(aa)\n",
    "\n",
    "\n",
    "print(my_tuple[3])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3fd7d87f-4d13-49f9-86e2-b33da6b7c16c",
   "metadata": {},
   "source": [
    "Lets now modify the behaviorof our raise function, insted of returning just the value1 raise to the power of value2, lets also return the value2 to the power of value1.  You thus need to make raise return two values insted of one.  We can now use what we know of tuples to do this.  \n",
    "\n",
    "\n",
    "We first change the name of our function and the docstring to reflect the new behavior of our function. \n",
    "We then in the function body, construct a tuple consisting of values we want the function to return, and also in the function body, we return the tuple.  \n",
    "\n",
    "Calling the function constructeddemonstrates that it does exactly what we want.  \n",
    "\n",
    "\n",
    "Now its your turn to play with writing functionthat accept multiple arguments and return multiple values.   "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "6a8cf097-cbe5-4ffa-8549-0c4bfa0a3666",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(8, 9)"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def raise_bose(value1, value2):\n",
    "    '''Raise value1 to the power of value2, and vice versa\n",
    "    '''\n",
    "    new_value1 = value1**value2\n",
    "    new_value2 = value2**value1\n",
    "    \n",
    "    new_tuple = (new_value1, new_value2)\n",
    "    \n",
    "    return new_tuple\n",
    "\n",
    "\n",
    "raise_bose(2, 3)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dc1fcfbe-2275-4102-8c83-4c97d24a5268",
   "metadata": {},
   "source": [
    "## Functions with multiple parameters\n",
    "\n",
    "Hugo discussed the use of multiple parameters in defining functions in the last lecture. You are now going to use what you've learned to modify the shout() function further. Here, you will modify shout() to accept two arguments. Parts of the function shout(), which you wrote earlier, are shown.\n",
    "Instructions\n",
    "100 XP\n",
    "\n",
    "    Modify the function header such that it accepts two parameters, word1 and word2, in that order.\n",
    "    Concatenate each of word1 and word2 with '!!!' and assign to shout1 and shout2, respectively.\n",
    "    Concatenate shout1 and shout2 together, in that order, and assign to new_shout.\n",
    "    Pass the strings 'congratulations' and 'you', in that order, to a call to shout(). Assign the return value to yell.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "14524cad-ca27-4a28-a0f5-5e1e7b60e7ef",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "('congratulations!!!', 'you!!!')\n"
     ]
    }
   ],
   "source": [
    "# Define shout with parameters word1 and word2\n",
    "def shout(word1, word2):\n",
    "    \"\"\"Concatenate strings with three exclamation marks\"\"\"\n",
    "    # Concatenate word1 with '!!!': shout1\n",
    "    shout1 = word1 + '!!!'\n",
    "    \n",
    "    # Concatenate word2 with '!!!': shout2\n",
    "    shout2 = word2 + '!!!'\n",
    "    \n",
    "    # Concatenate shout1 with shout2: new_shout\n",
    "    new_shout = (shout1, shout2)\n",
    "\n",
    "    # Return new_shout\n",
    "    return new_shout\n",
    "\n",
    "# Pass 'congratulations' and 'you' to shout(): yell\n",
    "yell = shout('congratulations', 'you')\n",
    "\n",
    "# Print yell\n",
    "print(yell)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "245491df-df58-402b-9206-00bddfda556f",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "('congratulations!!!', 'you!!!')\n"
     ]
    }
   ],
   "source": [
    "# Define shout with parameters word1 and word2\n",
    "def shout(word1, word2):\n",
    "    \"\"\"Concatenate strings with three exclamation marks\"\"\"\n",
    "    # Concatenate word1 with '!!!': shout1\n",
    "    shout1 = word1 + '!!!'\n",
    "    \n",
    "    # Concatenate word2 with '!!!': shout2\n",
    "    shout2 = word2 + '!!!'\n",
    "    \n",
    "    # Concatenate shout1 with shout2: new_shout\n",
    "    new_shout = shout1, shout2     ### cause , is the key to construct a tuple, not the paretheses\n",
    "\n",
    "    # Return new_shout\n",
    "    return new_shout\n",
    "\n",
    "# Pass 'congratulations' and 'you' to shout(): yell\n",
    "yell = shout('congratulations', 'you')\n",
    "\n",
    "# Print yell\n",
    "print(yell)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "885805ee-7fe4-4bb5-bb7a-e5b5a4fcd248",
   "metadata": {},
   "source": [
    "## A brief introduction to tuples\n",
    "\n",
    "Alongside learning about functions, you've also learned about tuples! Here, you will practice what you've learned about tuples: how to construct, unpack, and access tuple elements. Recall how Hugo unpacked the tuple even_nums in the video:\n",
    "\n",
    "a, b, c = even_nums\n",
    "\n",
    "A three-element tuple named nums has been preloaded for this exercise. Before completing the script, perform the following:\n",
    "\n",
    "    Print out the value of nums in the IPython shell. Note the elements in the tuple.\n",
    "    In the IPython shell, try to change the first element of nums to the value 2 by doing an assignment: nums[0] = 2. What happens?\n",
    "\n",
    "Instructions\n",
    "100 XP\n",
    "\n",
    "    Unpack nums to the variables num1, num2, and num3.\n",
    "    Construct a new tuple, even_nums composed of the same elements in nums, but with the 1st element replaced with the value, 2.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "id": "7250d96c-e4e9-48fc-8327-8ce633d48e0f",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(2, 4, 6)\n"
     ]
    }
   ],
   "source": [
    "nums = (3, 4, 6)\n",
    "\n",
    "\n",
    "num1, num2, num3 = nums\n",
    "\n",
    "\n",
    "\n",
    "# Construct even_nums\n",
    "even_nums = 2, num2, num3\n",
    "\n",
    "\n",
    "print(even_nums)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bb6486a5-4933-4f06-a1c0-f6aadecc8d83",
   "metadata": {},
   "source": [
    "## Functions that return multiple values\n",
    "\n",
    "In the previous exercise, you constructed tuples, assigned tuples to variables, and unpacked tuples. Here you will return multiple values from a function using tuples. Let's now update our shout() function to return multiple values. Instead of returning just one string, we will return two strings with the string !!! concatenated to each.\n",
    "\n",
    "# ****************************************************************************************************\n",
    "# Note that the return statement return x, y has the same result as return (x, y): the former actually packs x and y into a tuple under the hood!\n",
    "Instructions\n",
    "100 XP\n",
    "\n",
    "    Modify the function header such that the function name is now shout_all, and it accepts two parameters, word1 and word2, in that order.\n",
    "    Concatenate the string '!!!' to each of word1 and word2 and assign to shout1 and shout2, respectively.\n",
    "    Construct a tuple shout_words, composed of shout1 and shout2.\n",
    "    Call shout_all() with the strings 'congratulations' and 'you' and assign the result to yell1 and yell2 (remember, shout_all() returns 2 variables!).\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "0c7992bd-13fa-4342-8bf7-349a1c5f4b23",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "congratulations!!!\n",
      "you!!!\n"
     ]
    }
   ],
   "source": [
    "# Define shout_all with parameters word1 and word2\n",
    "def shout_all(word1, word2):\n",
    "    '''Return a tuple of strings'''\n",
    "    # Concatenate word1 with '!!!': shout1\n",
    "    shout1 = word1 + '!!!'\n",
    "    \n",
    "    # Concatenate word2 with '!!!': shout2\n",
    "    shout2 = word2 + '!!!'\n",
    "    \n",
    "    # Construct a tuple with shout1 and shout2: shout_words\n",
    "    shout_words = shout1, shout2\n",
    "\n",
    "    # Return shout_words\n",
    "    return shout_words\n",
    "\n",
    "# Pass 'congratulations' and 'you' to shout_all(): yell1, yell2\n",
    "yell1, yell2 = shout_all('congratulations', 'you')\n",
    "\n",
    "# Print yell1 and yell2\n",
    "print(yell1)\n",
    "print(yell2)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5e96b6d1-d40e-4688-b74d-70f7302a6474",
   "metadata": {},
   "source": [
    "## Bringing it all together\n",
    "\n",
    "\n",
    "\n",
    "**So now you're able to write functions that accept multiple parameters and return multiple values, it is time to see how these learnt skills can be valuable in a Data Science context.  In the following exercrises, you'll write a function that analyzes a DataFrame of Twitter data.  \n",
    "\n",
    "The function that you'll write will return a dictionary containing data of how many times each language was used across all the tweets in the DataFrame.  We'll see later in this course that we can generalize such a function to count occurances of any items in a DataFrame column.  \n",
    "\n",
    "\n",
    "\n",
    "Lets now recap the basic ingredients of a function.  \n",
    "We have a function header which begins with the keyword def, this is followedby the function name, parameters in parentheses and a colon.  \n",
    "We than have the function body, which contains docstrings enclosed in tiple quotation marks; docstrings describe what the function does; the rest of function body performs the computation that the function does; the function body then closes with the keyword return, followed by the value or values returned by the function.  \n",
    "\n",
    "\n",
    "Happy function writing\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "55b72197-5bbe-400b-a162-9afed747c363",
   "metadata": {},
   "source": [
    "## Bringing it all together (1)\n",
    "\n",
    "You've got your first taste of writing your own functions in the previous exercises. You've learned how to add parameters to your own function definitions, return a value or multiple values with tuples, and how to call the functions you've defined.\n",
    "\n",
    "In this and the following exercise, you will bring together all these concepts and apply them to a simple data science problem. You will load a dataset and develop functionalities to extract simple insights from the data.\n",
    "\n",
    "For this exercise, your goal is to recall how to load a dataset into a DataFrame. The dataset contains Twitter data and you will iterate over entries in a column to build a dictionary in which the keys are the names of languages and the values are the number of tweets in the given language. The file tweets.csv is available in your current directory.\n",
    "\n",
    "Be aware that this is real data from Twitter and as such there is always a risk that it may contain profanity or other offensive content (in this exercise, and any following exercises that also use real Twitter data).\n",
    "Instructions\n",
    "100 XP\n",
    "\n",
    "    Import the pandas package with the alias pd.\n",
    "    Import the file 'tweets.csv' using the pandas function read_csv(). Assign the resulting DataFrame to df.\n",
    "    Complete the for loop by iterating over col, the 'lang' column in the DataFrame df.\n",
    "    Complete the bodies of the if-else statements in the for loop: if the key is in the dictionary langs_count, add 1 to the value corresponding to this key in the dictionary, else add the key to langs_count and set the corresponding value to 1. Use the loop variable entry in your code.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7994d451-8b81-49c2-a30e-0ba141f042c1",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Import pandas\n",
    "import pandas as pd\n",
    "\n",
    "# Import Twitter data as DataFrame: df\n",
    "df = pd.read_csv('tweets.csv')\n",
    "\n",
    "# Initialize an empty dictionary: langs_count\n",
    "langs_count = {}\n",
    "\n",
    "# Extract column from DataFrame: col\n",
    "col = df['lang']\n",
    "\n",
    "# Iterate over lang column in DataFrame\n",
    "for entry in col:\n",
    "\n",
    "    # If the language is in langs_count, add 1 \n",
    "    if entry in langs_count.keys():\n",
    "        langs_count[entry] += 1\n",
    "    # Else add the language to langs_count, set the value to 1\n",
    "    else:\n",
    "        langs_count[entry] = 1\n",
    "\n",
    "# Print the populated dictionary\n",
    "print(langs_count)\n",
    "\n",
    "\n",
    "# Return\n",
    "# {'en': 97, 'et': 1, 'und': 2}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1747372e-a485-4eae-935c-78b04dce9e5e",
   "metadata": {},
   "outputs": [],
   "source": [
    "# df.columns\n",
    "\n",
    "# Return:\n",
    "#Index(['contributors', 'coordinates', 'created_at', 'entities', 'extended_entities', 'favorite_count', 'favorited', 'filter_level', 'geo', 'id', 'id_str', 'in_reply_to_screen_name',\n",
    "#       'in_reply_to_status_id', 'in_reply_to_status_id_str', 'in_reply_to_user_id', 'in_reply_to_user_id_str', 'is_quote_status', 'lang', 'place', 'possibly_sensitive', 'quoted_status',\n",
    "#       'quoted_status_id', 'quoted_status_id_str', 'retweet_count', 'retweeted', 'retweeted_status', 'source', 'text', 'timestamp_ms', 'truncated', 'user'],\n",
    "#      dtype='object')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "688874bb-5c9e-48bb-9a29-09c1135ea9fa",
   "metadata": {},
   "source": [
    "## Bringing it all together (2)\n",
    "\n",
    "Great job! You've now defined the functionality for iterating over entries in a column and building a dictionary with keys the names of languages and values the number of tweets in the given language.\n",
    "\n",
    "In this exercise, you will define a function with the functionality you developed in the previous exercise, return the resulting dictionary from within the function, and call the function with the appropriate arguments.\n",
    "\n",
    "For your convenience, the pandas package has been imported as pd and the 'tweets.csv' file has been imported into the tweets_df variable.\n",
    "Instructions\n",
    "100 XP\n",
    "\n",
    "    Define the function count_entries(), which has two parameters. The first parameter is df for the DataFrame and the second is col_name for the column name.\n",
    "    Complete the bodies of the if-else statements in the for loop: if the key is in the dictionary langs_count, add 1 to its current value, else add the key to langs_count and set its value to 1. Use the loop variable entry in your code.\n",
    "    Return the langs_count dictionary from inside the count_entries() function.\n",
    "    Call the count_entries() function by passing to it tweets_df and the name of the column, 'lang'. Assign the result of the call to the variable result.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2a908459-923a-46a4-b793-624ea309fa8b",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Define count_entries()\n",
    "# How can we take a DataFrame and a DF column name as input argument,  Think, Think, Think, \n",
    "# *************************************************************************************************** #\n",
    "def count_entries(df, col_name):\n",
    "    \"\"\"Return a dictionary with counts of \n",
    "    occurrences as value for each key.\"\"\"\n",
    "\n",
    "    # Initialize an empty dictionary: langs_count\n",
    "    langs_count = {}\n",
    "    \n",
    "    # Extract column from DataFrame: col\n",
    "    col = df[col_name]\n",
    "    \n",
    "    # Iterate over lang column in DataFrame\n",
    "    for entry in col:\n",
    "\n",
    "        # If the language is in langs_count, add 1\n",
    "        if entry in langs_count.keys():\n",
    "            langs_count[entry] += 1\n",
    "        # Else add the language to langs_count, set the value to 1\n",
    "        else:\n",
    "            langs_count[entry] = 1\n",
    "\n",
    "    # Return the langs_count dictionary\n",
    "    return langs_count\n",
    "\n",
    "# Call count_entries(): result\n",
    "result = count_entries(tweets_df, 'lang')\n",
    "\n",
    "# Print the result\n",
    "print(result)\n",
    "\n",
    "\n",
    "# Return:\n",
    "# {'en': 97, 'et': 1, 'und': 2}"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "944fcdc9-15cf-42fd-afee-5afebda4cb50",
   "metadata": {},
   "source": [
    "## Congratulations!\n",
    "\n",
    "\n",
    "\n",
    "Congratulations, you're now a bonafide Python function writer.  On top of that, you have just written you very first DataScience function.  At this point, although you can write basic functions, you've really touched the surface of function writing capabilities.  \n",
    "\n",
    "\n",
    "\n",
    "# ****************************************************************************************************\n",
    "# In the following chapter, you'll learn how to write functions that have default arguments so that when you call them, you dont always have to specify all the parameters, you'll learn how write functions that can accept an arbitrary number of parameters and how to nest functions within one another, on top of this, you'll learn how to handle errors when writing functions, which will make your functions as robust as they need to be.  \n",
    "\n",
    "Moreover, you'll see the importance of such techniques in DataScience by writing functions that are pertinent to the DataScience sphere like Twitter DataFrame analysis that you just performed.  \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "05edd4bb-9eda-4dc2-b66e-f6df9d325ea8",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "8435ae31-94d4-4ff9-9745-8f6684d65819",
   "metadata": {},
   "source": [
    "## Scope and user-defined functions\n",
    "\n",
    "\n",
    "\n",
    "At this point, you know how to define your own functions - but not only that, you know how to write functions with multiple parameyters and can return multiple values using tuples.  \n",
    "\n",
    "\n",
    "# We'll now talk about the idea of scope in the context of user-defined functions. \n",
    "You've been defining variables in your programs and so far, you've been using these variables without any problems.  However, one of the thing you should know is that not all objects that you defined are always accessable everywhere in a program.  \n",
    "\n",
    "**Enter the idea of scope, which tells you which part of a program an object or a name may be accessed.  \n",
    "Names refer to the variables or more generally, objects such as functions that are defined in your program, for example a variable x has a name, as does the function sum.  \n",
    "\n",
    "\n",
    "\n",
    "\n",
    "# There are three types of scopes that you should know.  \n",
    "The first one is the idea of the global scope.  A name that is in the global scope means that it is defined in the main body of a script or a Python program.  \n",
    "\n",
    "The second one, is the local scope.  A name that is in a local scope means that it is defined within a function.  Once the execution of a function is done, any name inside the local scope ceases to exist, which means you cannot access these names outside the function definition.  \n",
    "\n",
    "The third is something called the built-in scope; this consists of names in the pre-defined build-ins module Python provides, such as print and sum. \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "id": "bb266ba0-39bc-4297-a583-b2687ccefcf8",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "144"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def square(value):\n",
    "    '''Return the square of value'''\n",
    "    new_value = value**2\n",
    "    return new_value\n",
    "\n",
    "\n",
    "square(12)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "02bfd7f7-75e1-4e37-bb93-677341e521ac",
   "metadata": {},
   "source": [
    "\n",
    "You'll play around with the built-ins module in the interactive exercise, lets look at a couple of examples to clarify these definitions.  Lets check out our example function square from earlier.  \n",
    "\n",
    "\n",
    "# ****************************************************************************************************\n",
    "# We define the function and then call it, if we then try to access the variable name new_value after function execution, the name is not accessable.  This is because it was defined only within the local scope of the function.  The name new_value was not defined globally.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "id": "6004a6e1-76fb-48d4-8f6c-43c803fa3488",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "144"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "new_value = 118\n",
    "\n",
    "\n",
    "def square(value):\n",
    "    '''Return the square of value'''\n",
    "    new_value = value**2   # global csope name new_value updated\n",
    "    return new_value\n",
    "\n",
    "\n",
    "square(12)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "id": "c0f962d3-22e0-4921-9680-4603ef57d1c2",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "118"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "new_value"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8bd95306-b194-48d9-aa43-0285a01ea5e8",
   "metadata": {},
   "source": [
    "Now what if we define the name new_value globally before defining and calling the function?  In short, any time we call the name in the global scope, it will access the name in the global, such as you see here.  \n",
    "\n",
    "Anytime we call the name in the local scope of the function, it will look first in the local scope, that' why calling square(12) results in 144 and not 118.  \n",
    "\n",
    "If Python can not find the name in the local scope, it will then and only then looking in the global scope.  \n",
    "\n",
    "Here for example, we access new_value defined globally with the function square.  Note that the global value accessed is the value at the time the function is called, not the value when the function is defined.  Thus if we reassign new_value and call the square function, we see that the new value of new_value is accessed.  \n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "id": "fc748ed8-1f74-4c02-bada-99abfbf54351",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "100"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "new_value = 10\n",
    "\n",
    "def square(value):\n",
    "    '''Returns the square of value'''\n",
    "    new_value1 = new_value**2    # Just as an example, if global scope value was used in a function\n",
    "    return new_value1\n",
    "\n",
    "\n",
    "square(12)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "id": "cbf7691e-9cda-4f07-8f3c-b9f29dfb5c06",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "144"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "new_value = 12\n",
    "\n",
    "square(7)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "02d08447-1d91-441d-b47f-5c28eec7d04c",
   "metadata": {},
   "source": [
    "\n",
    "# To recap, when we reference a name, first the local scope is searched, then the global, If the nameis in neither then the built-in scope is searched.  \n",
    "\n",
    "Now what if we want to alter the value of a global name within a function call?  This is where the keyword global comes in play.  \n",
    "\n",
    "\n",
    "Within the function definition, we use keyword global followed by the name of the global variable that we wish to access and alter.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "id": "008a56f7-0803-4d34-a234-e8d141e35b58",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "9"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "new_value = 10\n",
    "\n",
    "def square(value):\n",
    "    '''Return the square of a value'''\n",
    "    \n",
    "    global new_value\n",
    "    new_value2 = new_value**2\n",
    "    return new_value2\n",
    "\n",
    "\n",
    "square(12)\n",
    "\n",
    "new_value = 3\n",
    "\n",
    "square(18)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "09f4b7ca-1b06-485a-9412-0bf2e4bace46",
   "metadata": {},
   "source": [
    "## Pop quiz on understanding scope\n",
    "\n",
    "In this exercise, you will practice what you've learned about scope in functions. The variable num has been predefined as 5, alongside the following function definitions:\n",
    "\n",
    "def func1():\n",
    "    num = 3\n",
    "    print(num)\n",
    "\n",
    "def func2():\n",
    "    global num\n",
    "    double_num = num * 2\n",
    "    num = 6\n",
    "    print(double_num)\n",
    "\n",
    "Try calling func1() and func2() in the shell, then answer the following questions:\n",
    "\n",
    "    What are the values printed out when you call func1() and func2()?    3 and 10\n",
    "    What is the value of num in the global scope after calling func1() and func2()?    6\n",
    "\n",
    "Instructions\n",
    "50 XP\n",
    "Possible Answers\n",
    "\n",
    "    func1() prints out 3, func2() prints out 6, and the value of num in the global scope is 3.\n",
    "    func1() prints out 3, func2() prints out 3, and the value of num in the global scope is 3.\n",
    "    func1() prints out 3, func2() prints out 10, and the value of num in the global scope is 10.\n",
    "#    func1() prints out 3, func2() prints out 10, and the value of num in the global scope is 6.\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "id": "f7d500d6-9041-406a-95ae-2b1b6b077096",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "3\n",
      "10\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "6"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "num = 5\n",
    "\n",
    "\n",
    "def func1():\n",
    "    num = 3\n",
    "    print(num)\n",
    "\n",
    "def func2():\n",
    "    global num\n",
    "    double_num = num * 2\n",
    "    num = 6\n",
    "    print(double_num)\n",
    "\n",
    "    \n",
    "    \n",
    "func1()\n",
    "func2()\n",
    "num"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "02df706d-5366-4ceb-b33c-f35129c5cb5f",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Define count_entries()\n",
    "def count_entries(df, col_name):\n",
    "    \"\"\"Return a dictionary with counts of \n",
    "    occurrences as value for each key.\"\"\"\n",
    "\n",
    "    # Initialize an empty dictionary: langs_count\n",
    "    langs_count = {}\n",
    "    \n",
    "    # Extract column from DataFrame: col\n",
    "    col = df[col_name]\n",
    "    \n",
    "    # Iterate over lang column in DataFrame\n",
    "    for entry in col:\n",
    "\n",
    "        # If the language is in langs_count, add 1\n",
    "        if entry in langs_count.keys():\n",
    "            langs_count[entry] += 1\n",
    "        # Else add the language to langs_count, set the value to 1\n",
    "        else:\n",
    "            langs_count[entry] = 1\n",
    "\n",
    "    # Return the langs_count dictionary\n",
    "    return langs_count\n",
    "\n",
    "# Call count_entries(): result\n",
    "\n",
    "import time\n",
    "start = time.time()\n",
    "result = count_entries(tweets_df, 'lang')\n",
    "t_spend = time.time() - start\n",
    "print(t_spend)\n",
    "\n",
    "import line_profiler\n",
    "\n",
    "%load_ext line_profiler\n",
    "%lprun -f count_entries count_entries(tweets_df, 'lang')\n",
    "\n",
    "# Print the result\n",
    "print(result)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "952149be-247b-4bdb-b49e-08481c5501a4",
   "metadata": {},
   "source": [
    "## The keyword global\n",
    "\n",
    "Let's work more on your mastery of scope. In this exercise, you will use the keyword global within a function to alter the value of a variable defined in the global scope.\n",
    "Instructions\n",
    "100 XP\n",
    "\n",
    "    Use the keyword global to alter the object team in the global scope.\n",
    "    Change the value of team in the global scope to the string \"justice league\". Assign the result to team.\n",
    "    Hit the Submit button to see how executing your newly defined function change_team() changes the value of the name team!\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "id": "ab9460a1-c10c-4667-bc51-dc98a007096a",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "teen titans\n",
      "justice league\n"
     ]
    }
   ],
   "source": [
    "# Create a string: team\n",
    "team = \"teen titans\"\n",
    "\n",
    "# Define change_team()\n",
    "def change_team():\n",
    "    \"\"\"Change the value of the global variable team.\"\"\"\n",
    "\n",
    "    # Use team in global scope\n",
    "    global team\n",
    "\n",
    "    # Change the value of team in global: team\n",
    "    team = 'justice league'\n",
    "# Print team\n",
    "print(team)\n",
    "\n",
    "# Call change_team()\n",
    "change_team()\n",
    "\n",
    "# Print team\n",
    "print(team)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "56d845e7-1479-44b4-81da-5795e082326d",
   "metadata": {},
   "source": [
    "## Python's built-in scope\n",
    "\n",
    "Here you're going to check out Python's built-in scope, which is really just a built-in module called builtins. However, to query builtins, you'll need to import builtins 'because the name builtins is not itself built in…No, I’m serious!' (Learning Python, 5th edition, Mark Lutz). After executing import builtins in the IPython Shell, execute dir(builtins) to print a list of all the names in the module builtins. Have a look and you'll see a bunch of names that you'll recognize! Which of the following names is NOT in the module builtins?\n",
    "Instructions\n",
    "50 XP\n",
    "Possible Answers\n",
    "\n",
    "    'sum'\n",
    "    'range'\n",
    "#    'array'\n",
    "    'tuple'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "id": "2630172f-c0b1-4ba9-819f-edebd4d4026f",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['ArithmeticError', 'AssertionError', 'AttributeError', 'BaseException', 'BlockingIOError', 'BrokenPipeError', 'BufferError', 'BytesWarning', 'ChildProcessError', 'ConnectionAbortedError', 'ConnectionError', 'ConnectionRefusedError', 'ConnectionResetError', 'DeprecationWarning', 'EOFError', 'Ellipsis', 'EnvironmentError', 'Exception', 'False', 'FileExistsError', 'FileNotFoundError', 'FloatingPointError', 'FutureWarning', 'GeneratorExit', 'IOError', 'ImportError', 'ImportWarning', 'IndentationError', 'IndexError', 'InterruptedError', 'IsADirectoryError', 'KeyError', 'KeyboardInterrupt', 'LookupError', 'MemoryError', 'ModuleNotFoundError', 'NameError', 'None', 'NotADirectoryError', 'NotImplemented', 'NotImplementedError', 'OSError', 'OverflowError', 'PendingDeprecationWarning', 'PermissionError', 'ProcessLookupError', 'RecursionError', 'ReferenceError', 'ResourceWarning', 'RuntimeError', 'RuntimeWarning', 'StopAsyncIteration', 'StopIteration', 'SyntaxError', 'SyntaxWarning', 'SystemError', 'SystemExit', 'TabError', 'TimeoutError', 'True', 'TypeError', 'UnboundLocalError', 'UnicodeDecodeError', 'UnicodeEncodeError', 'UnicodeError', 'UnicodeTranslateError', 'UnicodeWarning', 'UserWarning', 'ValueError', 'Warning', 'ZeroDivisionError', '__IPYTHON__', '__build_class__', '__debug__', '__doc__', '__import__', '__loader__', '__name__', '__package__', '__spec__', 'abs', 'all', 'any', 'ascii', 'bin', 'bool', 'breakpoint', 'bytearray', 'bytes', 'callable', 'chr', 'classmethod', 'compile', 'complex', 'copyright', 'credits', 'delattr', 'dict', 'dir', 'display', 'divmod', 'enumerate', 'eval', 'exec', 'execfile', 'filter', 'float', 'format', 'frozenset', 'get_ipython', 'getattr', 'globals', 'hasattr', 'hash', 'help', 'hex', 'id', 'input', 'int', 'isinstance', 'issubclass', 'iter', 'len', 'license', 'list', 'locals', 'map', 'max', 'memoryview', 'min', 'next', 'object', 'oct', 'open', 'ord', 'pow', 'print', 'property', 'range', 'repr', 'reversed', 'round', 'runfile', 'set', 'setattr', 'slice', 'sorted', 'staticmethod', 'str', 'sum', 'super', 'tuple', 'type', 'vars', 'zip']\n"
     ]
    }
   ],
   "source": [
    "import builtins\n",
    "\n",
    "print(dir(builtins))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cbd4dea9-8197-44be-9939-6efcfce17bd3",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "a598d07a-20df-4922-9cd4-42cf0f55525a",
   "metadata": {},
   "source": [
    "## Nested functions\n",
    "# ****************************************************************************************************\n",
    "\n",
    "\n",
    "**Now that you've come to grips with scope, both local and global, its time to dive a bit deeper.  \n",
    "\n",
    "What if we have a function inner() defined within another function outer() and we reference a name x in the inner function?  \n",
    "\n",
    "\n",
    "**The answer is intuitive, Python searchesthe local scope of the function inner(), then if it doesnt find x, it searches the scope of the function outer, which is called an exclosing function, because it encloses the function inner().  If Python cant find x in the scope of the enclosing function, it only then searches the global scope and then the built-in scope.  \n",
    "\n",
    "\n",
    "\n",
    "# Why are we even nesting functions?  There are number of good reasons to do so.  \n",
    "\n",
    "**Lets say that we want to use a process a number of times within a function.**\n",
    "For example, we want a function that takes 3 numbers as parameters and performs the same function on each of them.  One way would be write out the computation 3 times, but this definitely does not scale if you need to perform the computation many times.  What we can do instead is to define a inner function within our function definition, such as we did below, and call it when necessary.  This is called nested function.  \n",
    "The syntax for the inner function is exactly the same as for the any other function.  \n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c0dbe4c0-866d-4345-a589-eae610e66d2f",
   "metadata": {},
   "outputs": [],
   "source": [
    "x = 1\n",
    "\n",
    "def outer():\n",
    "    '''Outer function'''\n",
    "    x = 123\n",
    "    \n",
    "    def inner():\n",
    "        '''Inner function'''\n",
    "        y = x**2\n",
    "        return y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "id": "47e4f15d-d57c-4352-9d6c-297619b61e70",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(6, 5, 6)"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def mod2plus5(x1, x2, x3):\n",
    "    new_x1 = x1%2 + 5\n",
    "    new_x2 = x2%2 + 5\n",
    "    new_x3 = x3%2 + 5\n",
    "    \n",
    "    return new_x1, new_x2, new_x3\n",
    "\n",
    "\n",
    "mod2plus5(7, 8, 9)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "id": "2b667e21-0bf5-45b3-9fd1-b8359b852a39",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(5, 6, 5)"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def mod2plus5(x1, x2, x3):\n",
    "    '''Returns the remainder plus 5 of three values'''\n",
    "    \n",
    "    def inner(x):\n",
    "        '''Returns the remainder plus 5'''\n",
    "        return x%2 +5\n",
    "    \n",
    "    return inner(x1), inner(x2), inner(x3)\n",
    "\n",
    "\n",
    "mod2plus5(8, 9, 10)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "350ad309-beeb-4aa4-a0be-598cea984def",
   "metadata": {},
   "source": [
    "\n",
    "# Lets now look at another important use case of nested functions. \n",
    "\n",
    "In this example, we define a function raise_val, which contains an inner function called inner.  Now look at what raise_val returns: it returns the inner function inner.  The raise_val takes an argument n and create a function inner that returns the nth power of any number.  \n",
    "That's a bit complicated and will be clearer when we use the function raise_val.  Passing the number 2 to raise_val create a function that squares any number, similarly, passing the number 3 to raise_val creates a function that cubes any number.  \n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "id": "1b2316e6-bf8e-4d02-a80a-f36619bc516e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "8"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def raise_val(n):\n",
    "    '''Return the inner func'''\n",
    "    \n",
    "    def inner(x):\n",
    "        '''Raise x to the power of n'''\n",
    "        raised = x**n\n",
    "        return raised\n",
    "    return inner\n",
    "\n",
    "\n",
    "ff = raise_val(3)\n",
    "ff(2)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "105347c5-7196-40a3-b362-320f8db20d7a",
   "metadata": {},
   "source": [
    "\n",
    "One interesting detail: when we call the function ff, it remembers the value n=3, although the enclosing scope defined by raise_val and to which n=3 is local, has finished execution.  This is a subtlety referred to as a closure in Computer Science circles and shouldnt concern you too much.  It is worth mentioning, however, as you may encounter it out there in the wild.  \n",
    "\n",
    "Recall from our discussion of scope that you can use the keyword global in function definitions, to create and change the global names; similarly, in a nested function, you can use the keyword nonlocal to create and changes names in an enclosing scope.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "id": "aac5ef12-14f5-4565-ba88-68f5b0f82fde",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2\n",
      "2\n"
     ]
    }
   ],
   "source": [
    "def outer():\n",
    "    '''Prints the value of n'''\n",
    "    n = 1\n",
    "    \n",
    "    def inner():\n",
    "        nonlocal n\n",
    "        n = 2\n",
    "        print(n)\n",
    "    \n",
    "    inner()\n",
    "    print(n)\n",
    "    \n",
    "    \n",
    "outer()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "18b748aa-0919-4a39-8c4c-8fc112ecf768",
   "metadata": {},
   "source": [
    "**To summarize: name reference search at most 4 scopes, the local scope, then those of enclosing function, if there arent any, then global, then built-in.  This is know as LEGB rule.  Also remember that assigning names will only create or change local names. unless they are declared in global or nonlocal statements using keyword global or the keyword nonlocal respectively.  \n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "eb7f9779-01ac-47c4-950d-2ca4076cc7f8",
   "metadata": {},
   "source": [
    "## Nested Functions I\n",
    "\n",
    "You've learned in the last video about nesting functions within functions. \n",
    "# One reason why you'd like to do this is to avoid writing out the same computations within functions repeatedly. \n",
    "There's nothing new about defining nested functions: you simply define it as you would a regular function with def and embed it inside another function!\n",
    "\n",
    "In this exercise, inside a function three_shouts(), you will define a nested function inner() that concatenates a string object with !!!. three_shouts() then returns a tuple of three elements, each a string concatenated with !!! using inner(). Go for it!\n",
    "Instructions\n",
    "100 XP\n",
    "\n",
    "    Complete the function header of the nested function with the function name inner() and \n",
    "# a single parameter word.\n",
    "    Complete the return value: each element of the tuple should be a call to inner(), passing in the parameters from three_shouts() as arguments to each call.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "id": "1d2a5fa5-272f-4093-8fed-5bdd77809e74",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "('a!!!', 'b!!!', 'c!!!')\n"
     ]
    }
   ],
   "source": [
    "# Define three_shouts\n",
    "def three_shouts(word1, word2, word3):\n",
    "    \"\"\"Returns a tuple of strings\n",
    "    concatenated with '!!!'.\"\"\"\n",
    "\n",
    "    # Define inner\n",
    "    def inner(word):\n",
    "        \"\"\"Returns a string concatenated with '!!!'.\"\"\"\n",
    "        return word + '!!!'\n",
    "\n",
    "    # Return a tuple of strings\n",
    "    return (inner(word1), inner(word2), inner(word3))\n",
    "\n",
    "# Call three_shouts() and print\n",
    "print(three_shouts('a', 'b', 'c'))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "26b12759-dce5-4cb3-b468-e57f27eeb094",
   "metadata": {},
   "source": [
    "## Nested Functions II\n",
    "\n",
    "Great job, you've just nested a function within another function. One other pretty cool reason for nesting functions is the idea of a closure. This means that the nested or inner function remembers the state of its enclosing scope when called. Thus, anything defined locally in the enclosing scope is available to the inner function even when the outer function has finished execution.\n",
    "\n",
    "Let's move forward then! In this exercise, you will complete the definition of the inner function inner_echo() and then call echo() a couple of times, each with a different argument. Complete the exercise and see what the output will be!\n",
    "Instructions\n",
    "100 XP\n",
    "\n",
    "    Complete the function header of the inner function with the function name inner_echo() and a single parameter word1.\n",
    "    Complete the function echo() so that it returns inner_echo.\n",
    "    We have called echo(), passing 2 as an argument, and assigned the resulting function to twice. Your job is to call echo(), passing 3 as an argument. Assign the resulting function to thrice.\n",
    "    Hit Submit to call twice() and thrice() and print the results.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "id": "6daf3525-0ef6-40d3-8561-152dcaf73814",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "hellohello hellohellohello\n"
     ]
    }
   ],
   "source": [
    "# Define echo\n",
    "def echo(n):\n",
    "    \"\"\"Return the inner_echo function.\"\"\"\n",
    "\n",
    "    # Define inner_echo\n",
    "    def inner_echo(word1):\n",
    "        \"\"\"Concatenate n copies of word1.\"\"\"\n",
    "        echo_word = word1 * n\n",
    "        return echo_word\n",
    "\n",
    "    # Return inner_echo\n",
    "    return inner_echo\n",
    "\n",
    "# Call echo: twice\n",
    "twice = echo(2)\n",
    "\n",
    "# Call echo: thrice\n",
    "thrice = echo(3)\n",
    "\n",
    "\n",
    "\n",
    "# Call twice() and thrice() then print\n",
    "print(twice('hello'), thrice('hello'))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "46203978-8226-444a-bdc2-6246a81a4379",
   "metadata": {},
   "source": [
    "## The keyword nonlocal and nested functions\n",
    "\n",
    "Let's once again work further on your mastery of scope! In this exercise, you will use the keyword nonlocal within a nested function to alter the value of a variable defined in the enclosing scope.\n",
    "Instructions\n",
    "100 XP\n",
    "\n",
    "    Assign to echo_word the string word, concatenated with itself.\n",
    "    Use the keyword nonlocal to alter the value of echo_word in the enclosing scope.\n",
    "    Alter echo_word to echo_word concatenated with '!!!'.\n",
    "    Call the function echo_shout(), passing it a single argument 'hello'.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "id": "36dce30f-e6eb-432c-a140-a235cf763f41",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "hello\n",
      "hello!!!\n"
     ]
    }
   ],
   "source": [
    "# Define echo_shout()\n",
    "def echo_shout(word):\n",
    "    \"\"\"Change the value of a nonlocal variable\"\"\"\n",
    "    \n",
    "    # Concatenate word with itself: echo_word\n",
    "    echo_word = word\n",
    "    \n",
    "    # Print echo_word\n",
    "    print(echo_word)\n",
    "    \n",
    "    # Define inner function shout()\n",
    "    def shout():\n",
    "        \"\"\"Alter a variable in the enclosing scope\"\"\"    \n",
    "        # Use echo_word in nonlocal scope\n",
    "        nonlocal echo_word\n",
    "        \n",
    "        # Change echo_word to echo_word concatenated with '!!!'\n",
    "        echo_word = echo_word + '!!!'\n",
    "    \n",
    "    # Call function shout()\n",
    "    shout()\n",
    "    \n",
    "    # Print echo_word\n",
    "    print(echo_word)\n",
    "    \n",
    "    \n",
    "    \n",
    "# Call function echo_shout() with argument 'hello'\n",
    "echo_shout('hello')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2983b2d8-fda2-4770-ac59-4068df641c78",
   "metadata": {},
   "source": [
    "## Default and flexible arguments\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "**Lets say you're writing a function that takes nultiple parameters and that there is often a common value for some of these parameters.  In this case, you would like to be able to call the function without explicitly specifying every parameters.  In other words yo would like some parameters to have default arguments that are used when it is not specified otherwise.  \n",
    "\n",
    "In this video, you'll learn how to write function with default arguments, along with using flexible arguments, which allows you to pass any number of arguments to a function, as we'll soon see.  \n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a3f8192f-7286-47d0-85c1-8f979414ddb2",
   "metadata": {},
   "source": [
    "First up, to define a function with a default argument value, in the function header, we follow the parameter of interest with an equal sign and the default argument value.  Notice that below function raise the first argument to the power of second of argument, and the default second argument value is 1."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "id": "18e49e11-456c-49bb-a6e4-62a42e67ffa1",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "144\n",
      "1728\n"
     ]
    }
   ],
   "source": [
    "def power(number, pow=2):\n",
    "    '''Raise number to the power of pow'''\n",
    "    new_value = number**pow\n",
    "    return new_value\n",
    "\n",
    "\n",
    "print(power(12))\n",
    "\n",
    "print(power(12,3))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "36c90c55-be7b-407c-890f-6ed5aedde928",
   "metadata": {},
   "source": [
    "# Let now look at flexible arguments, lets say that you want to write a function but arent sure how many arguments a user will want to pass it.  \n",
    "For example, a function that takes floats or ints and adds them up, irrespective of how many there are.  Enter flexible arguments. \n",
    "\n",
    "\n",
    "In below example, we write the function that sums up all the arguments passed to it.  In the function definition, we use the parameter str followed by args: this then turns all the arguments passed to a function call into a tuple called args in the function body.  (*can be used on unpacking also on packing)  Then in the function body, to write our desired function, we initialize our sum_all to 0, loop over the tuple args and add each element of it successively to aum_all and then return it.  \n",
    "\n",
    "We can now call our function add_all with any number of arguments to add them all up.  \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "id": "bc2d3c62-0333-4af4-8d8d-cddb9bc67ce5",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "133.7"
      ]
     },
     "execution_count": 61,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def add_all(*args):\n",
    "    '''Sum all values in *argstogether'''\n",
    "    \n",
    "    # Initialize sum\n",
    "    sum_all = 0\n",
    "    \n",
    "    # Accumulate the sum\n",
    "    for num in args:\n",
    "        sum_all += num\n",
    "        \n",
    "    return sum_all\n",
    "\n",
    "\n",
    "a = [1, 2, 3, 9.7, 23, 89, 6]\n",
    "add_all(*a)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4b70d5d2-7ef7-4904-9f19-6e911d49c2b1",
   "metadata": {},
   "source": [
    "\n",
    "**You can also use a double star to pass an arbitrary number keyword arguments, also called kwargs, that is arguments preceded by identifiers.  We'll write such a function called print_all that prints out the identifiers and the parameters passed to them as you see here.  \n",
    "\n",
    "Now, to write such function, we use the parameter kwargs preceded by double star.  This turns the identifier-keyword pairs into a dictionary withinn the function body.  Then in the function body all we need to do is to print out all the key-value pairs stored in the dictionary kwargs.  \n",
    "\n",
    "\n",
    "Note that is NOT the name args and kwargs that are important when using flexible arguments, but rather that they are processed by a single and double star respectively.  \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "id": "25e799f0-7548-4300-8db6-b722723806dc",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "name: jj\n",
      "employer: PP\n"
     ]
    }
   ],
   "source": [
    "def print_all(**kwargs):\n",
    "    '''Print out key-value pairs in **kwargs'''\n",
    "    \n",
    "    # Print out the key-value pairs\n",
    "    for key, value in kwargs.items():\n",
    "        print(key + \": \" + value)\n",
    "\n",
    "\n",
    "\n",
    "print_all(name='jj', employer='PP')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fda3b29a-b348-4c2d-85b0-728ed5e5f447",
   "metadata": {},
   "source": [
    "## Functions with one default argument\n",
    "\n",
    "In the previous chapter, you've learned to define functions with more than one parameter and then calling those functions by passing the required number of arguments. In the last video, Hugo built on this idea by showing you how to define functions with default arguments. You will practice that skill in this exercise by writing a function that uses a default argument and then calling the function a couple of times.\n",
    "Instructions\n",
    "100 XP\n",
    "\n",
    "    Complete the function header with the function name shout_echo. It accepts an argument word1 and a default argument echo with default value 1, in that order.\n",
    "    Use the * operator to concatenate echo copies of word1. Assign the result to echo_word.\n",
    "    Call shout_echo() with just the string, \"Hey\". Assign the result to no_echo.\n",
    "    Call shout_echo() with the string \"Hey\" and the value 5 for the default argument, echo. Assign the result to with_echo.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "id": "eb65cd75-82c9-4056-a87b-08f83dc8022d",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Hey!!!\n",
      "HeyHeyHeyHeyHey!!!\n"
     ]
    }
   ],
   "source": [
    "# Define shout_echo\n",
    "def shout_echo(word1, echo=1):\n",
    "    \"\"\"Concatenate echo copies of word1 and three\n",
    "     exclamation marks at the end of the string.\"\"\"\n",
    "\n",
    "    # Concatenate echo copies of word1 using *: echo_word\n",
    "    echo_word = word1 * echo\n",
    "\n",
    "    # Concatenate '!!!' to echo_word: shout_word\n",
    "    shout_word = echo_word + '!!!'\n",
    "\n",
    "    # Return shout_word\n",
    "    return shout_word\n",
    "\n",
    "# Call shout_echo() with \"Hey\": no_echo\n",
    "no_echo = shout_echo('Hey')\n",
    "\n",
    "# Call shout_echo() with \"Hey\" and echo=5: with_echo\n",
    "with_echo = shout_echo('Hey', echo=5)\n",
    "\n",
    "# Print no_echo and with_echo\n",
    "print(no_echo)\n",
    "print(with_echo)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "963c597a-8de8-447e-b00f-a2bf2b01d22e",
   "metadata": {},
   "source": [
    "## Functions with multiple default arguments\n",
    "\n",
    "You've now defined a function that uses a default argument - don't stop there just yet! You will now try your hand at defining a function with more than one default argument and then calling this function in various ways.\n",
    "\n",
    "After defining the function, you will call it by supplying values to all the default arguments of the function. Additionally, you will call the function by not passing a value to one of the default arguments - see how that changes the output of your function!\n",
    "Instructions\n",
    "100 XP\n",
    "\n",
    "    Complete the function header with the function name shout_echo. It accepts an argument word1, a default argument echo with default value 1 and a default argument intense with default value False, in that order.\n",
    "    In the body of the if statement, make the string object echo_word upper case by applying the method .upper() on it.\n",
    "    Call shout_echo() with the string, \"Hey\", the value 5 for echo and the value True for intense. Assign the result to with_big_echo.\n",
    "    Call shout_echo() with the string \"Hey\" and the value True for intense. Assign the result to big_no_echo.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "id": "0ee4698c-b07e-4d8b-8e84-3b9b8d01ff62",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "HEYHEYHEYHEYHEY!!!\n",
      "Hey!!!\n"
     ]
    }
   ],
   "source": [
    "# Define shout_echo\n",
    "def shout_echo(word1, echo=1, intense=False):\n",
    "    \"\"\"Concatenate echo copies of word1 and three\n",
    "    exclamation marks at the end of the string.\"\"\"\n",
    "\n",
    "    # Concatenate echo copies of word1 using *: echo_word\n",
    "    echo_word = word1 * echo\n",
    "\n",
    "    # Make echo_word uppercase if intense is True\n",
    "    if intense is True:\n",
    "        # Make uppercase and concatenate '!!!': echo_word_new\n",
    "        echo_word_new = echo_word.upper() + '!!!'\n",
    "    else:\n",
    "        # Concatenate '!!!' to echo_word: echo_word_new\n",
    "        echo_word_new = echo_word + '!!!'\n",
    "\n",
    "    # Return echo_word_new\n",
    "    return echo_word_new\n",
    "\n",
    "# Call shout_echo() with \"Hey\", echo=5 and intense=True: with_big_echo\n",
    "with_big_echo = shout_echo('Hey', echo=5, intense=True)\n",
    "\n",
    "# Call shout_echo() with \"Hey\" and intense=True: big_no_echo\n",
    "big_no_echo = shout_echo('Hey', True)\n",
    "\n",
    "# Print values\n",
    "print(with_big_echo)\n",
    "print(big_no_echo)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4d576bd1-517f-49f8-aa95-59f502914167",
   "metadata": {},
   "source": [
    "## Functions with variable-length arguments (*args)\n",
    "\n",
    "Flexible arguments enable you to pass a variable number of arguments to a function. In this exercise, you will practice defining a function that accepts a variable number of string arguments.\n",
    "\n",
    "The function you will define is gibberish() which can accept a variable number of string values. Its return value is a single string composed of all the string arguments concatenated together in the order they were passed to the function call. You will call the function with a single string argument and see how the output changes with another call using more than one string argument. Recall from the previous video that, within the function definition, args is a tuple.\n",
    "Instructions\n",
    "100 XP\n",
    "\n",
    "    Complete the function header with the function name gibberish. It accepts a single flexible argument *args.\n",
    "    Initialize a variable hodgepodge to an empty string.\n",
    "    Return the variable hodgepodge at the end of the function body.\n",
    "    Call gibberish() with the single string, \"luke\". Assign the result to one_word.\n",
    "    Hit the Submit button to call gibberish() with multiple arguments and to print the value to the Shell.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "id": "29f220b7-771d-4723-83fb-af31e07e16bf",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "luke, \n",
      "luke, leia, han, obi, darth, \n"
     ]
    }
   ],
   "source": [
    "# Define gibberish\n",
    "def gibberish(*args):\n",
    "    \"\"\"Concatenate strings in *args together.\"\"\"\n",
    "\n",
    "    # Initialize an empty string: hodgepodge\n",
    "    hodgepodge = str()\n",
    "\n",
    "    # Concatenate the strings in args\n",
    "    for word in args:\n",
    "        hodgepodge = hodgepodge + word + ', '\n",
    "\n",
    "    # Return hodgepodge\n",
    "    return hodgepodge\n",
    "\n",
    "# Call gibberish() with one string: one_word\n",
    "one_word = gibberish('luke')\n",
    "\n",
    "# Call gibberish() with five strings: many_words\n",
    "many_words = gibberish(\"luke\", \"leia\", \"han\", \"obi\", \"darth\")\n",
    "\n",
    "# Print one_word and many_words\n",
    "print(one_word)\n",
    "print(many_words)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f1d20036-5a5c-4567-a306-6428b0bbaa66",
   "metadata": {},
   "source": [
    "## Functions with variable-length keyword arguments (**kwargs)\n",
    "\n",
    "Let's push further on what you've learned about flexible arguments - you've used *args, you're now going to use **kwargs! What makes **kwargs different is that it allows you to pass a variable number of keyword arguments to functions. Recall from the previous video that, within the function definition, kwargs is a dictionary.\n",
    "\n",
    "To understand this idea better, you're going to use **kwargs in this exercise to define a function that accepts a variable number of keyword arguments. The function simulates a simple status report system that prints out the status of a character in a movie.\n",
    "Instructions\n",
    "100 XP\n",
    "\n",
    "    Complete the function header with the function name report_status. It accepts a single flexible argument **kwargs.\n",
    "    Iterate over the key-value pairs of kwargs to print out the keys and values, separated by a colon ':'.\n",
    "    In the first call to report_status(), pass the following keyword-value pairs: name=\"luke\", affiliation=\"jedi\" and status=\"missing\".\n",
    "    In the second call to report_status(), pass the following keyword-value pairs: name=\"anakin\", affiliation=\"sith lord\" and status=\"deceased\".\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "id": "d4a275b7-16c4-4698-bee6-e2188e854c05",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "****BEGIN: REPORT****\n",
      "name: luke\n",
      "affiliation: jedi\n",
      "status: missing\n",
      "****END REPORT****\n",
      "\n",
      "\n",
      "****BEGIN: REPORT****\n",
      "name: anakin\n",
      "affiliation: sith lord\n",
      "status: decreased\n",
      "****END REPORT****\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# Define report_status\n",
    "def report_status(**kwargs):\n",
    "    \"\"\"Print out the status of a movie character.\"\"\"\n",
    "\n",
    "    print(\"\\n****BEGIN: REPORT****\")\n",
    "\n",
    "    # Iterate over the key-value pairs of kwargs\n",
    "    for key, value in kwargs.items():\n",
    "        # Print out the keys and values, separated by a colon ':'\n",
    "        print(key + \": \" + value)\n",
    "\n",
    "    print(\"****END REPORT****\\n\")\n",
    "\n",
    "# First call to report_status()\n",
    "report_status(name='luke', affiliation='jedi', status='missing')\n",
    "\n",
    "# Second call to report_status()\n",
    "report_status(name='anakin', affiliation='sith lord', status='decreased')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d10b544c-33e9-4b3c-943a-405183cf67bf",
   "metadata": {},
   "source": [
    "## Bringing it all together\n",
    "\n",
    "\n",
    "\n",
    "**Its now time to get your hands dirty using you newly acquired skills by applying them to the wonderful world of DataScience: recall that, in the previous chapter, you wrote a function that would perform the following, it would take d DataFrame of Twitter data and return a dictionary containing langurage as keys and the number of times a tweet was written in a given language as values.  \n",
    "\n",
    "\n",
    "# In the following exercises, you will write a function that processes a DataFrame and return a dictionary with counts of occurances in any column at all.  By default however, it will process a column calledlang.  \n",
    "\n",
    "This generalizes the function that you wrote in previous exercises.  You will then generalize this further so that you can pass the function a DataFrame and any number of column names to perform the computation on an arbitrary number of columns.  \n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "622c3cc8-a314-4e78-9b44-28f4b2cdfbc4",
   "metadata": {},
   "source": [
    "## Bringing it all together (1)\n",
    "\n",
    "Recall the Bringing it all together exercise in the previous chapter where you did a simple Twitter analysis by developing a function that counts how many tweets are in certain languages. The output of your function was a dictionary that had the language as the keys and the counts of tweets in that language as the value.\n",
    "\n",
    "In this exercise, we will generalize the Twitter language analysis that you did in the previous chapter. You will do that by including a default argument that takes a column name.\n",
    "\n",
    "For your convenience, pandas has been imported as pd and the 'tweets.csv' file has been imported into the DataFrame tweets_df. Parts of the code from your previous work are also provided.\n",
    "Instructions\n",
    "100 XP\n",
    "\n",
    "    Complete the function header by supplying the parameter for a DataFrame df and the parameter col_name with a default value of 'lang' for the DataFrame column name.\n",
    "    Call count_entries() by passing the tweets_df DataFrame and the column name 'lang'. Assign the result to result1. Note that since 'lang' is the default value of the col_name parameter, you don't have to specify it here.\n",
    "    Call count_entries() by passing the tweets_df DataFrame and the column name 'source'. Assign the result to result2.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 100,
   "id": "daba800d-a48c-40b0-936c-8f1304f48fa0",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'en': 97, 'et': 1, 'und': 2}\n",
      "{'<a href=\"http://twitter.com\" rel=\"nofollow\">Twitter Web Client</a>': 24, '<a href=\"http://www.facebook.com/twitter\" rel=\"nofollow\">Facebook</a>': 1, '<a href=\"http://twitter.com/download/android\" rel=\"nofollow\">Twitter for Android</a>': 26, '<a href=\"http://twitter.com/download/iphone\" rel=\"nofollow\">Twitter for iPhone</a>': 33, '<a href=\"http://www.twitter.com\" rel=\"nofollow\">Twitter for BlackBerry</a>': 2, '<a href=\"http://www.google.com/\" rel=\"nofollow\">Google</a>': 2, '<a href=\"http://twitter.com/#!/download/ipad\" rel=\"nofollow\">Twitter for iPad</a>': 6, '<a href=\"http://linkis.com\" rel=\"nofollow\">Linkis.com</a>': 2, '<a href=\"http://rutracker.org/forum/viewforum.php?f=93\" rel=\"nofollow\">newzlasz</a>': 2, '<a href=\"http://ifttt.com\" rel=\"nofollow\">IFTTT</a>': 1, '<a href=\"http://www.myplume.com/\" rel=\"nofollow\">Plume\\xa0for\\xa0Android</a>': 1}\n"
     ]
    }
   ],
   "source": [
    "# Define count_entries()\n",
    "def count_entries(df, col_name='lang'):\n",
    "    \"\"\"Return a dictionary with counts of\n",
    "    occurrences as value for each key.\"\"\"\n",
    "\n",
    "    # Initialize an empty dictionary: cols_count\n",
    "    cols_count = {}\n",
    "\n",
    "    # Extract column from DataFrame: col\n",
    "    col = df[col_name]\n",
    "    \n",
    "    # Iterate over the column in DataFrame\n",
    "    for entry in col:\n",
    "\n",
    "        # If entry is in cols_count, add 1\n",
    "        if entry in cols_count.keys():\n",
    "            cols_count[entry] += 1\n",
    "\n",
    "        # Else add the entry to cols_count, set the value to 1\n",
    "        else:\n",
    "            cols_count[entry] = 1\n",
    "\n",
    "    # Return the cols_count dictionary\n",
    "    return cols_count\n",
    "\n",
    "\n",
    "\n",
    "import pandas as pd\n",
    "\n",
    "tweets_df = pd.read_csv('tweets.csv')\n",
    "\n",
    "\n",
    "# Call count_entries(): result1\n",
    "result1 = count_entries(tweets_df)\n",
    "\n",
    "# Call count_entries(): result2\n",
    "result2 = count_entries(tweets_df, 'source')\n",
    "\n",
    "# Print result1 and result2\n",
    "print(result1)\n",
    "print(result2)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4b1dbaf1-aa1a-4ae2-8f77-6e3c23a0f49c",
   "metadata": {},
   "source": [
    "## Bringing it all together (2)\n",
    "\n",
    "Wow, you've just generalized your Twitter language analysis that you did in the previous chapter to include a default argument for the column name. You're now going to generalize this function one step further by allowing the user to pass it a flexible argument, that is, in this case, as many column names as the user would like!\n",
    "\n",
    "Once again, for your convenience, pandas has been imported as pd and the 'tweets.csv' file has been imported into the DataFrame tweets_df. Parts of the code from your previous work are also provided.\n",
    "Instructions\n",
    "100 XP\n",
    "\n",
    "#    Complete the function header by supplying the parameter for the dataframe df and the flexible argument *args.\n",
    "    Complete the for loop within the function definition so that the loop occurs over the tuple args.\n",
    "    Call count_entries() by passing the tweets_df DataFrame and the column name 'lang'. Assign the result to result1.\n",
    "    Call count_entries() by passing the tweets_df DataFrame and the column names 'lang' and 'source'. Assign the result to result2.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 103,
   "id": "15274f5e-54aa-4e1b-b3e3-c1758d553276",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'en': 97, 'et': 1, 'und': 2}\n",
      "{'en': 97, 'et': 1, 'und': 2, '<a href=\"http://twitter.com\" rel=\"nofollow\">Twitter Web Client</a>': 24, '<a href=\"http://www.facebook.com/twitter\" rel=\"nofollow\">Facebook</a>': 1, '<a href=\"http://twitter.com/download/android\" rel=\"nofollow\">Twitter for Android</a>': 26, '<a href=\"http://twitter.com/download/iphone\" rel=\"nofollow\">Twitter for iPhone</a>': 33, '<a href=\"http://www.twitter.com\" rel=\"nofollow\">Twitter for BlackBerry</a>': 2, '<a href=\"http://www.google.com/\" rel=\"nofollow\">Google</a>': 2, '<a href=\"http://twitter.com/#!/download/ipad\" rel=\"nofollow\">Twitter for iPad</a>': 6, '<a href=\"http://linkis.com\" rel=\"nofollow\">Linkis.com</a>': 2, '<a href=\"http://rutracker.org/forum/viewforum.php?f=93\" rel=\"nofollow\">newzlasz</a>': 2, '<a href=\"http://ifttt.com\" rel=\"nofollow\">IFTTT</a>': 1, '<a href=\"http://www.myplume.com/\" rel=\"nofollow\">Plume\\xa0for\\xa0Android</a>': 1}\n"
     ]
    }
   ],
   "source": [
    "# Define count_entries()\n",
    "# *************************************************************************************************** #\n",
    "def count_entries(df, *args):\n",
    "    \"\"\"Return a dictionary with counts of\n",
    "    occurrences as value for each key.\"\"\"\n",
    "    \n",
    "    #Initialize an empty dictionary: cols_count\n",
    "    cols_count = {}\n",
    "    \n",
    "    # Iterate over column names in args\n",
    "    for col_name in args:\n",
    "    \n",
    "        # Extract column from DataFrame: col\n",
    "        col = df[col_name]\n",
    "    \n",
    "        # Iterate over the column in DataFrame\n",
    "        for entry in col:\n",
    "    \n",
    "            # If entry is in cols_count, add 1\n",
    "            if entry in cols_count.keys():\n",
    "                cols_count[entry] += 1\n",
    "    \n",
    "            # Else add the entry to cols_count, set the value to 1\n",
    "            else:\n",
    "                cols_count[entry] = 1\n",
    "\n",
    "    # Return the cols_count dictionary\n",
    "    return cols_count\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "import pandas as pd\n",
    "\n",
    "tweet_df = pd.read_csv('tweets.csv')\n",
    "#print(tweet_df.head())\n",
    "\n",
    "\n",
    "# Call count_entries(): result1\n",
    "result1 = count_entries(tweet_df, 'lang')\n",
    "\n",
    "# Call count_entries(): result2\n",
    "result2 = count_entries(tweet_df, 'lang', 'source')\n",
    "\n",
    "# Print result1 and result2\n",
    "print(result1)\n",
    "print(result2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "dafa8c59-7011-4822-9315-1292deb608f0",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "5fef83ca-6878-4b16-a807-e168076cc41c",
   "metadata": {},
   "source": [
    "## Lambda functions\n",
    "\n",
    "\n",
    "\n",
    "**We've written our very own Python functions using the def keyword, function headers, docstrings and function bodies.  \n",
    "\n",
    "# There is a quicker way to write functions on the fly, and these are called ambda functions because you use the keyword lambda.  \n",
    "\n",
    "\n",
    "\n",
    "# ****************************************************************************************************\n",
    "# Here we rewrite our function raise_to_power as a lambda function, to do so, after the keyword lambda, we specify the names of the arguments, then we use a colon followed by the expression that specifies what we wish the function to return.  \n",
    "\n",
    "\n",
    "# ****************************************************************************************************\n",
    "# **Lambda functions allow you to write functions in a quick and potentially dirty way so I wouldnt advice you to use them all the time but there are situations when they can come in very handy.  \n",
    "\n",
    "\n",
    "# ****************************************************************************************************\n",
    "For example, check out the map() function, which takes two arguments, a function and a sequence such as a list and applies the function over all elements of the sequence.  We can pass lambda functions to map without even naming them and in this case we refer to them as anonymous functions.  \n",
    "\n",
    "# ****************************************************************************************************\n",
    "In below example, we use map on a lambda function that squares all elements of a list and well store the result in square_all.  \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "id": "5ac7c988-580e-41ee-942c-687317ecb8a5",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "8"
      ]
     },
     "execution_count": 80,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "raise_to_power = lambda x, y: x**y\n",
    "\n",
    "\n",
    "raise_to_power(2, 3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 83,
   "id": "a8fbbd62-0df3-499a-b562-0d1185ea72e9",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[1, 4, 9, 16, 25, 36]"
      ]
     },
     "execution_count": 83,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nums = [1, 2, 3, 4, 5, 6]\n",
    "\n",
    "\n",
    "square_all = map(lambda x: x**2, nums)\n",
    "\n",
    "[*square_all]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2ce14ef7-2ae4-4913-bbee-3be3fef03825",
   "metadata": {},
   "source": [
    "## Pop quiz on lambda functions\n",
    "\n",
    "In this exercise, you will practice writing a simple lambda function and calling this function. Recall what you know about lambda functions and answer the following questions:\n",
    "\n",
    "    How would you write a lambda function add_bangs that adds three exclamation points '!!!' to the end of a string a?\n",
    "    How would you call add_bangs with the argument 'hello'?\n",
    "\n",
    "You may use the IPython shell to test your code.\n",
    "Instructions\n",
    "50 XP\n",
    "Possible Answers\n",
    "\n",
    "    The lambda function definition is: add_bangs = (a + '!!!'), and the function call is: add_bangs('hello').\n",
    "#    The lambda function definition is: add_bangs = (lambda a: a + '!!!'), and the function call is: add_bangs('hello').\n",
    "    The lambda function definition is: (lambda a: a + '!!!') = add_bangs, and the function call is: add_bangs('hello').\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 87,
   "id": "b9afe764-282a-47b6-bff5-285f81b11ff4",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Hello!!!\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "'Bonjue!!!'"
      ]
     },
     "execution_count": 87,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "add_bangs = (lambda a: a + '!!!')\n",
    "add_bangs2 = lambda a: a + '!!!'\n",
    "\n",
    "\n",
    "print(add_bangs('Hello'))\n",
    "add_bangs2('Bonjue')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "819e49a0-2d9d-4770-a21d-e3b5e2ce4481",
   "metadata": {},
   "source": [
    "## Writing a lambda function you already know\n",
    "\n",
    "# Some function definitions are simple enough that they can be converted to a lambda function. \n",
    "By doing this, you write less lines of code, which is pretty awesome and will come in handy, especially when you're writing and maintaining big programs. In this exercise, you will use what you know about lambda functions to convert a function that does a simple task into a lambda function. Take a look at this function definition:\n",
    "\n",
    "def echo_word(word1, echo):\n",
    "    \"\"\"Concatenate echo copies of word1.\"\"\"\n",
    "    words = word1 * echo\n",
    "    return words\n",
    "\n",
    "The function echo_word takes 2 parameters: a string value, word1 and an integer value, echo. It returns a string that is a concatenation of echo copies of word1. Your task is to convert this simple function into a lambda function.\n",
    "Instructions\n",
    "100 XP\n",
    "\n",
    "    Define the lambda function echo_word using the variables word1 and echo. Replicate what the original function definition for echo_word() does above.\n",
    "    Call echo_word() with the string argument 'hey' and the value 5, in that order. Assign the call to result.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 88,
   "id": "b0924032-cc98-4bc3-b4a5-bc10f88c372c",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "heyheyheyheyhey\n"
     ]
    }
   ],
   "source": [
    "# Define echo_word as a lambda function: echo_word\n",
    "echo_word = (lambda word1, echo: word1*echo)\n",
    "\n",
    "# Call echo_word: result\n",
    "result = echo_word('hey', 5)\n",
    "\n",
    "# Print result\n",
    "print(result)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "52c0438e-11e0-46b5-a6da-214a39e42759",
   "metadata": {},
   "source": [
    "## Map() and lambda functions\n",
    "\n",
    "So far, you've used lambda functions to write short, simple functions as well as to redefine functions with simple functionality. The best use case for lambda functions, however, are for when you want these simple functionalities to be anonymously embedded within larger expressions. What that means is that the functionality is not stored in the environment, unlike a function defined with def. To understand this idea better, you will use a lambda function in the context of the map() function.\n",
    "\n",
    "Recall from the video that map() applies a function over an object, such as a list. Here, you can use lambda functions to define the function that map() will use to process the object. For example:\n",
    "\n",
    "nums = [2, 4, 6, 8, 10]\n",
    "\n",
    "result = map(lambda a: a ** 2, nums)\n",
    "\n",
    "You can see here that a lambda function, which raises a value a to the power of 2, is passed to map() alongside a list of numbers, nums. The map object that results from the call to map() is stored in result. You will now practice the use of lambda functions with map(). For this exercise, you will map the functionality of the add_bangs() function you defined in previous exercises over a list of strings.\n",
    "Instructions\n",
    "100 XP\n",
    "\n",
    "    In the map() call, pass a lambda function that concatenates the string '!!!' to a string item; also pass the list of strings, spells. Assign the resulting map object to shout_spells.\n",
    "    Convert shout_spells to a list and print out the list.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 89,
   "id": "1e46e8f5-bfbf-43fa-86e2-a3ab86d99415",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['protego!!!', 'accio!!!', 'expecto patronum!!!', 'legilimens!!!']\n"
     ]
    }
   ],
   "source": [
    "# Create a list of strings: spells\n",
    "spells = [\"protego\", \"accio\", \"expecto patronum\", \"legilimens\"]\n",
    "\n",
    "# Use map() to apply a lambda function over spells: shout_spells\n",
    "shout_spells = map(lambda item: item + '!!!', spells)\n",
    "\n",
    "# Convert shout_spells to a list: shout_spells_list\n",
    "shout_spells_list = list(shout_spells)\n",
    "\n",
    "# Print the result\n",
    "print(shout_spells_list)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3b40228f-333b-4c4a-8917-6c19a389fe23",
   "metadata": {},
   "source": [
    "## Filter() and lambda functions\n",
    "\n",
    "In the previous exercise, you used lambda functions to anonymously embed an operation within map(). You will practice this again in this exercise by using a lambda function with filter(), which may be new to you! \n",
    "# The function filter() offers a way to filter out elements from a list that don't satisfy certain criteria.\n",
    "\n",
    "Your goal in this exercise is to use filter() to create, from an input list of strings, a new list that contains only strings that have more than 6 characters.\n",
    "Instructions\n",
    "100 XP\n",
    "\n",
    "# ****************************************************************************************************\n",
    "#    In the filter() call, pass a lambda function and the list of strings, fellowship. The lambda function should check if the number of characters in a string member is greater than 6; use the len() function to do this. Assign the resulting filter object to result.\n",
    "    Convert result to a list and print out the list.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 92,
   "id": "d2ca21cd-ac76-4a39-b63c-82eceeaaf918",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['samwise', 'aragorn', 'boromir', 'legolas', 'gandalf']\n",
      "Help on class filter in module builtins:\n",
      "\n",
      "class filter(object)\n",
      " |  filter(function or None, iterable) --> filter object\n",
      " |  \n",
      " |  Return an iterator yielding those items of iterable for which function(item)\n",
      " |  is true. If function is None, return the items that are true.\n",
      " |  \n",
      " |  Methods defined here:\n",
      " |  \n",
      " |  __getattribute__(self, name, /)\n",
      " |      Return getattr(self, name).\n",
      " |  \n",
      " |  __iter__(self, /)\n",
      " |      Implement iter(self).\n",
      " |  \n",
      " |  __next__(self, /)\n",
      " |      Implement next(self).\n",
      " |  \n",
      " |  __reduce__(...)\n",
      " |      Return state information for pickling.\n",
      " |  \n",
      " |  ----------------------------------------------------------------------\n",
      " |  Static methods defined here:\n",
      " |  \n",
      " |  __new__(*args, **kwargs) from builtins.type\n",
      " |      Create and return a new object.  See help(type) for accurate signature.\n",
      "\n",
      "None\n"
     ]
    }
   ],
   "source": [
    "# Create a list of strings: fellowship\n",
    "fellowship = ['frodo', 'samwise', 'merry', 'pippin', 'aragorn', 'boromir', 'legolas', 'gimli', 'gandalf']\n",
    "\n",
    "# Use filter() to apply a lambda function over fellowship: result\n",
    "# *************************************************************************************************** #\n",
    "result = filter(lambda member: len(member)>6, fellowship)\n",
    "\n",
    "# Convert result to a list: result_list\n",
    "result_list = list(result)\n",
    "\n",
    "# Print result_list\n",
    "print(result_list)\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "print(help(filter))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8861c262-29a3-4fab-8547-5f0826ad695a",
   "metadata": {},
   "source": [
    "## Reduce() and lambda functions\n",
    "\n",
    "# ***************************************************************************************************\n",
    "You're getting very good at using lambda functions! Here's one more function to add to your repertoire of skills. The reduce() function is useful for performing some computation on a list and, unlike map() and filter(), returns a single value as a result. To use reduce(), you must import it from the functools module.\n",
    "\n",
    "Remember gibberish() from a few exercises back?\n",
    "\n",
    "# Define gibberish\n",
    "def gibberish(*args):\n",
    "    \"\"\"Concatenate strings in *args together.\"\"\"\n",
    "    hodgepodge = ''\n",
    "    for word in args:\n",
    "        hodgepodge += word\n",
    "    return hodgepodge\n",
    "\n",
    "gibberish() simply takes a list of strings as an argument and returns, as a single-value result, the concatenation of all of these strings. In this exercise, you will replicate this functionality by using reduce() and a lambda function that concatenates strings together.\n",
    "Instructions\n",
    "100 XP\n",
    "\n",
    "    Import the reduce function from the functools module.\n",
    "    In the reduce() call, pass a lambda function that takes two string arguments item1 and item2 and concatenates them; also pass the list of strings, stark. Assign the result to result. The first argument to reduce() should be the lambda function and the second argument is the list stark.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 96,
   "id": "1d79e507-f185-49dc-a3da-231326ab05ae",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "robb, sansa, arya, brandon, rickon, \n",
      "Help on built-in function reduce in module _functools:\n",
      "\n",
      "reduce(...)\n",
      "    reduce(function, sequence[, initial]) -> value\n",
      "    \n",
      "    Apply a function of two arguments cumulatively to the items of a sequence,\n",
      "    from left to right, so as to reduce the sequence to a single value.\n",
      "    For example, reduce(lambda x, y: x+y, [1, 2, 3, 4, 5]) calculates\n",
      "    ((((1+2)+3)+4)+5).  If initial is present, it is placed before the items\n",
      "    of the sequence in the calculation, and serves as a default when the\n",
      "    sequence is empty.\n",
      "\n",
      "None\n"
     ]
    }
   ],
   "source": [
    "from functools import reduce\n",
    "\n",
    "# Import reduce from functools\n",
    "\n",
    "# Create a list of strings: stark\n",
    "stark = ['robb', 'sansa', 'arya', 'brandon', 'rickon']\n",
    "\n",
    "stark = [str(i+', ') for i in stark]\n",
    "\n",
    "# Use reduce() to apply a lambda function over stark: result\n",
    "result = reduce(lambda item1, item2: item1+item2, stark)\n",
    "\n",
    "# Print the result\n",
    "print(result)\n",
    "\n",
    "\n",
    "\n",
    "print(help(reduce))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a9a35171-2cfb-47ff-a84e-a21f8e03b3d6",
   "metadata": {},
   "source": [
    "## Introduction to error handling\n",
    "\n",
    "\n",
    "\n",
    "**When you use a function incorrectly, it should throw you an error.  \n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 98,
   "id": "de426890-9a07-46be-8653-e29f9ac0948d",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "123.0\n",
      "123.876\n"
     ]
    },
    {
     "ename": "ValueError",
     "evalue": "could not convert string to float: 'Hw'",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mValueError\u001b[0m                                Traceback (most recent call last)",
      "\u001b[0;32m/tmp/ipykernel_2050/1117571921.py\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[1;32m      7\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      8\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 9\u001b[0;31m \u001b[0mprint\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mfloat\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'Hw'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;31mValueError\u001b[0m: could not convert string to float: 'Hw'"
     ]
    }
   ],
   "source": [
    "print(float(123))\n",
    "\n",
    "\n",
    "\n",
    "print(float('123.876'))\n",
    "\n",
    "\n",
    "\n",
    "print(float('Hw'))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "90ea51bb-90b0-4adb-aa17-d37cc2924c62",
   "metadata": {},
   "source": [
    "\n",
    "When we write our own functions, we may wish to catch soecific problems and write specific error messages.  Lets check out this user-defined function that computes the square root of a number, it behaves as expected with integers.  What happens if we pass it a string such as 'bonjour sava'? \n",
    "\n",
    "\n",
    "Then it throws me an error corresponding to a line of code within the function definition.  This error says it was some sort of TypeError but the message may not be particularly useful to a user of our function, so we should endeavor to provide useful messages for the functions we write.  \n",
    "\n",
    "# This is an example of an error caught during execution, commonly called exceptions.  The main way to catch such exceptions is the try-except clause, in which Python tries to run the code following try and if it can, all is well.  If it can not, due to an exception, it runs the code following except.  \n",
    "\n",
    "\n",
    "Lets now rewrite our square root function but this time catch any exceptions raised.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 99,
   "id": "9d0ad8c8-5439-40ff-aaf6-47c42d57e5f1",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "3.4641016151377544\n"
     ]
    },
    {
     "ename": "TypeError",
     "evalue": "unsupported operand type(s) for ** or pow(): 'str' and 'float'",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mTypeError\u001b[0m                                 Traceback (most recent call last)",
      "\u001b[0;32m/tmp/ipykernel_2050/2374814177.py\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[1;32m      8\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      9\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 10\u001b[0;31m \u001b[0mprint\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0msqrt\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'bonjour sawa'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;32m/tmp/ipykernel_2050/2374814177.py\u001b[0m in \u001b[0;36msqrt\u001b[0;34m(x)\u001b[0m\n\u001b[1;32m      1\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0msqrt\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mx\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      2\u001b[0m     \u001b[0;34m'''Returns the square root of a value'''\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 3\u001b[0;31m     \u001b[0;32mreturn\u001b[0m \u001b[0mx\u001b[0m\u001b[0;34m**\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;36m0.5\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      4\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      5\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mTypeError\u001b[0m: unsupported operand type(s) for ** or pow(): 'str' and 'float'"
     ]
    }
   ],
   "source": [
    "def sqrt(x):\n",
    "    '''Returns the square root of a value'''\n",
    "    return x**(0.5)\n",
    "\n",
    "\n",
    "\n",
    "print(sqrt(12))\n",
    "\n",
    "\n",
    "print(sqrt('bonjour sawa'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 105,
   "id": "e821659c-4f9f-440e-8c56-b2cc35b77182",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "3.4641016151377544\n",
      "keep arg as int or float please! \n",
      "None\n"
     ]
    }
   ],
   "source": [
    "def sqrt(x):\n",
    "    '''Returns the square root of an value'''\n",
    "    try:\n",
    "        return x**(0.5)\n",
    "    except:\n",
    "        print('keep arg as int or float please! ')\n",
    "        \n",
    "        \n",
    "        \n",
    "print(sqrt(12))\n",
    "\n",
    "print(sqrt('bonjour sawa'))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9b59eeea-1677-4a80-8033-dc4be1fd0d26",
   "metadata": {},
   "source": [
    "**We may also wish to only catch TypeError and let other error pass through, in which case we would use except TypeError as you can see below.  \n",
    "\n",
    "There are many other types of exceptions that can be caught and you can get familiar with them by practice.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 115,
   "id": "37296b50-c9d1-45df-aa6f-dc0995ce4c15",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "lease keep arg x as int or float\n",
      "lease keep arg x as int or float\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "(6.121090488658852e-15+99.96499387285532j)"
      ]
     },
     "execution_count": 115,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def sqrt(x):\n",
    "    '''Return the square root of an value'''\n",
    "    \n",
    "    try:\n",
    "        return x**(0.5)\n",
    "    except TypeError:\n",
    "        print('lease keep arg x as int or float')\n",
    "        \n",
    "        \n",
    "        \n",
    "sqrt('bonjour')\n",
    "\n",
    "x = [1, 2, 3, 4, 5, 6]\n",
    "sqrt(x)\n",
    "\n",
    "sqrt(-9993)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ca6f4ab4-11f7-49bd-891c-2a7f34c256c1",
   "metadata": {},
   "source": [
    "# **More often than not, instead of merely printing an error message, we'll want to actually raise an error by using he keyward raise.  \n",
    "\n",
    "# ***************************************************************************************************\n",
    "For example, our square root function does something we may not desire, when applied to negatuive numbers.  It actually returns a complex number which we may not want.  In fact, lets say that we dont wish our function to work for negative numbers.  Then using an if clause, we can raise a ValueError for cases in which he user passes the function a negative number.  \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 119,
   "id": "a91977ee-9551-414b-90f6-4666523c8b2f",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "5.656854249492381\n"
     ]
    },
    {
     "ename": "ValueError",
     "evalue": "arg x must be positive value",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mValueError\u001b[0m                                Traceback (most recent call last)",
      "\u001b[0;32m/tmp/ipykernel_2050/2806178859.py\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[1;32m     18\u001b[0m \u001b[0;31m#print(sqrt('bonjour'))\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     19\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 20\u001b[0;31m \u001b[0mprint\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0msqrt\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m-\u001b[0m\u001b[0;36m9\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;32m/tmp/ipykernel_2050/2806178859.py\u001b[0m in \u001b[0;36msqrt\u001b[0;34m(x)\u001b[0m\n\u001b[1;32m      5\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      6\u001b[0m     \u001b[0;32melif\u001b[0m \u001b[0mx\u001b[0m\u001b[0;34m<\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 7\u001b[0;31m         \u001b[0;32mraise\u001b[0m \u001b[0mValueError\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'arg x must be positive value'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      8\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      9\u001b[0m     \u001b[0;32mtry\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mValueError\u001b[0m: arg x must be positive value"
     ]
    }
   ],
   "source": [
    "def sqrt(x):\n",
    "    '''Return the square root of given number'''\n",
    "    if isinstance(x, str):\n",
    "        raise TypeError('Keep arg x not str please')\n",
    "    \n",
    "    elif x<0:\n",
    "        raise ValueError('arg x must be positive value')\n",
    "        \n",
    "    try:\n",
    "        return x**(0.5)\n",
    "    except TypeError:\n",
    "        print('Please keep arg x in int or float')\n",
    "        \n",
    "        \n",
    "        \n",
    "print(sqrt(32))\n",
    "\n",
    "#print(sqrt('bonjour'))\n",
    "\n",
    "print(sqrt(-9))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "74190901-1f12-40c4-8fc2-1a16c1e5d88e",
   "metadata": {},
   "source": [
    "## Pop quiz about errors\n",
    "\n",
    "In the video, Hugo talked about how errors happen when functions are supplied arguments that they are unable to work with. In this exercise, you will identify which function call raises an error and what type of error is raised.\n",
    "\n",
    "Take a look at the following function calls to len():\n",
    "\n",
    "len('There is a beast in every man and it stirs when you put a sword in his hand.')\n",
    "\n",
    "len(['robb', 'sansa', 'arya', 'eddard', 'jon'])\n",
    "\n",
    "len(525600)\n",
    "\n",
    "len(('jaime', 'cersei', 'tywin', 'tyrion', 'joffrey'))\n",
    "\n",
    "Which of the function calls raises an error and what type of error is raised?\n",
    "Instructions\n",
    "50 XP\n",
    "Possible Answers\n",
    "\n",
    "    The call len('There is a beast in every man and it stirs when you put a sword in his hand.') raises a TypeError.\n",
    "    The call len(['robb', 'sansa', 'arya', 'eddard', 'jon']) raises an IndexError.\n",
    "    The call len(525600) raises a TypeError.\n",
    "    The call len(('jaime', 'cersei', 'tywin', 'tyrion', 'joffrey')) raises a NameError.\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 122,
   "id": "64e0825e-f910-46b5-81e3-e960828cf9ce",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "76\n",
      "5\n"
     ]
    },
    {
     "ename": "TypeError",
     "evalue": "object of type 'int' has no len()",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mTypeError\u001b[0m                                 Traceback (most recent call last)",
      "\u001b[0;32m/tmp/ipykernel_2050/2663335164.py\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[1;32m      3\u001b[0m \u001b[0mprint\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mlen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m'robb'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m'sansa'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m'arya'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m'eddard'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m'jon'\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      4\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 5\u001b[0;31m \u001b[0mprint\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mlen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;36m525600\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      6\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      7\u001b[0m \u001b[0mprint\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mlen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'jaime'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m'cersei'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m'tywin'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m'tyrion'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m'joffrey'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mTypeError\u001b[0m: object of type 'int' has no len()"
     ]
    }
   ],
   "source": [
    "print(len('There is a beast in every man and it stirs when you put a sword in his hand.'))\n",
    "\n",
    "print(len(['robb', 'sansa', 'arya', 'eddard', 'jon']))\n",
    "\n",
    "print(len(525600))\n",
    "\n",
    "print(len(('jaime', 'cersei', 'tywin', 'tyrion', 'joffrey')))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 125,
   "id": "26c40a0f-cdd1-43bb-8902-d448f469f3a3",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['__abs__', '__add__', '__and__', '__bool__', '__ceil__', '__class__', '__delattr__', '__dir__', '__divmod__', '__doc__', '__eq__', '__float__', '__floor__', '__floordiv__', '__format__', '__ge__', '__getattribute__', '__getnewargs__', '__gt__', '__hash__', '__index__', '__init__', '__init_subclass__', '__int__', '__invert__', '__le__', '__lshift__', '__lt__', '__mod__', '__mul__', '__ne__', '__neg__', '__new__', '__or__', '__pos__', '__pow__', '__radd__', '__rand__', '__rdivmod__', '__reduce__', '__reduce_ex__', '__repr__', '__rfloordiv__', '__rlshift__', '__rmod__', '__rmul__', '__ror__', '__round__', '__rpow__', '__rrshift__', '__rshift__', '__rsub__', '__rtruediv__', '__rxor__', '__setattr__', '__sizeof__', '__str__', '__sub__', '__subclasshook__', '__truediv__', '__trunc__', '__xor__', 'as_integer_ratio', 'bit_length', 'conjugate', 'denominator', 'from_bytes', 'imag', 'numerator', 'real', 'to_bytes']\n"
     ]
    }
   ],
   "source": [
    "print(dir(12345))\n",
    "\n",
    "\n",
    "#print(help(12345))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b76c2642-5dca-4aec-b780-af4ebd8da207",
   "metadata": {},
   "source": [
    "## Error handling with try-except\n",
    "\n",
    "A good practice in writing your own functions is also anticipating the ways in which other people (or yourself, if you accidentally misuse your own function) might use the function you defined.\n",
    "\n",
    "As in the previous exercise, you saw that the len() function is able to handle input arguments such as strings, lists, and tuples, but not int type ones and raises an appropriate error and error message when it encounters invalid input arguments. One way of doing this is through exception handling with the try-except block.\n",
    "\n",
    "# In this exercise, you will define a function as well as use a try-except block for handling cases when incorrect input arguments are passed to the function.\n",
    "\n",
    "Recall the shout_echo() function you defined in previous exercises; parts of the function definition are provided in the sample code. Your goal is to complete the exception handling code in the function definition and provide an appropriate error message when raising an error.\n",
    "Instructions\n",
    "100 XP\n",
    "\n",
    "    Initialize the variables echo_word and shout_words to empty strings.\n",
    "    Add the keywords try and except in the appropriate locations for the exception handling block.\n",
    "    Use the * operator to concatenate echo copies of word1. Assign the result to echo_word.\n",
    "    Concatenate the string '!!!' to echo_word. Assign the result to shout_words.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 128,
   "id": "31a133eb-b5bb-4290-bc6b-4685cf4eb0ce",
   "metadata": {},
   "outputs": [
    {
     "ename": "TypeError",
     "evalue": "Please keep first arg word1 in str and second arg echo in int",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mTypeError\u001b[0m                                 Traceback (most recent call last)",
      "\u001b[0;32m/tmp/ipykernel_2050/1076609540.py\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[1;32m     32\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     33\u001b[0m \u001b[0;31m# Call shout_echo\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 34\u001b[0;31m \u001b[0mshout_echo\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"particle\"\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mecho\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m\"accelerator\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;32m/tmp/ipykernel_2050/1076609540.py\u001b[0m in \u001b[0;36mshout_echo\u001b[0;34m(word1, echo)\u001b[0m\n\u001b[1;32m     11\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     12\u001b[0m     \u001b[0;32mif\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0misinstance\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mword1\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mstr\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mor\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0misinstance\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mecho\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mint\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 13\u001b[0;31m         \u001b[0;32mraise\u001b[0m \u001b[0mTypeError\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'Please keep first arg word1 in str and second arg echo in int'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     14\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     15\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mTypeError\u001b[0m: Please keep first arg word1 in str and second arg echo in int"
     ]
    }
   ],
   "source": [
    "# Define shout_echo\n",
    "def shout_echo(word1, echo=1):\n",
    "    \"\"\"Concatenate echo copies of word1 and three\n",
    "    exclamation marks at the end of the string.\"\"\"\n",
    "\n",
    "    # Initialize empty strings: echo_word, shout_words\n",
    "    echo_word = str()\n",
    "    shout_words = str()\n",
    "    \n",
    "    \n",
    "    \n",
    "    if not isinstance(word1, str) or not isinstance(echo, int):\n",
    "        raise TypeError('Please keep first arg word1 in str and second arg echo in int')\n",
    "    \n",
    "    \n",
    "    \n",
    "    # Add exception handling with try-except\n",
    "    try:\n",
    "        # Concatenate echo copies of word1 using *: echo_word\n",
    "        echo_word = word1 * echo\n",
    "\n",
    "        # Concatenate '!!!' to echo_word: shout_words\n",
    "        shout_words = echo_word + '!!!'\n",
    "    except ValueError:\n",
    "        # Print error message\n",
    "        print(\"word1 must be a string and echo must be an integer.\")\n",
    "\n",
    "    # Return shout_words\n",
    "    return shout_words\n",
    "\n",
    "\n",
    "\n",
    "# Call shout_echo\n",
    "shout_echo(\"particle\", echo=\"accelerator\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2e326198-f769-4a41-a242-f6a399a26491",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Solution:\n",
    "\n",
    "# Define shout_echo\n",
    "def shout_echo(word1, echo=1):\n",
    "    \"\"\"Concatenate echo copies of word1 and three\n",
    "    exclamation marks at the end of the string.\"\"\"\n",
    "\n",
    "    # Initialize empty strings: echo_word, shout_words\n",
    "    echo_word = ''\n",
    "    shout_words = ''\n",
    "\n",
    "    # Add exception handling with try-except\n",
    "    try:\n",
    "        # Concatenate echo copies of word1 using *: echo_word\n",
    "        echo_word = word1 * echo\n",
    "\n",
    "        # Concatenate '!!!' to echo_word: shout_words\n",
    "        shout_words = echo_word + '!!!'\n",
    "    except:\n",
    "        # Print error message\n",
    "        print(\"word1 must be a string and echo must be an integer.\")\n",
    "\n",
    "    # Return shout_words\n",
    "    return shout_words\n",
    "\n",
    "# Call shout_echo\n",
    "shout_echo(\"particle\", echo=\"accelerator\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3676f6a2-c948-4f5e-a7b0-1d1c525a641c",
   "metadata": {},
   "source": [
    "## Error handling by raising an error\n",
    "\n",
    "Another way to raise an error is by using raise. In this exercise, you will add a raise statement to the shout_echo() function you defined before to raise an error message when the value supplied by the user to the echo argument is less than 0.\n",
    "\n",
    "The call to shout_echo() uses valid argument values. To test and see how the raise statement works, simply change the value for the echo argument to a negative value. Don't forget to change it back to valid values to move on to the next exercise!\n",
    "Instructions\n",
    "100 XP\n",
    "\n",
    "    Complete the if statement by checking if the value of echo is less than 0.\n",
    "    In the body of the if statement, add a raise statement that raises a ValueError with message 'echo must be greater than or equal to 0' when the value supplied by the user to echo is less than 0.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "64b3b550-86d9-4967-9463-4b1015df7b5b",
   "metadata": {},
   "outputs": [
    {
     "ename": "ValueError",
     "evalue": "echo must be greater or equal to 0",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mValueError\u001b[0m                                Traceback (most recent call last)",
      "\u001b[0;32m/tmp/ipykernel_2050/925509444.py\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[1;32m     20\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     21\u001b[0m \u001b[0;31m# Call shout_echo\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 22\u001b[0;31m \u001b[0mshout_echo\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"particle\"\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mecho\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m-\u001b[0m\u001b[0;36m5\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;32m/tmp/ipykernel_2050/925509444.py\u001b[0m in \u001b[0;36mshout_echo\u001b[0;34m(word1, echo)\u001b[0m\n\u001b[1;32m      6\u001b[0m     \u001b[0;31m# Raise an error with raise\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      7\u001b[0m     \u001b[0;32mif\u001b[0m \u001b[0mecho\u001b[0m \u001b[0;34m<\u001b[0m \u001b[0;36m0\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 8\u001b[0;31m         \u001b[0;32mraise\u001b[0m \u001b[0mValueError\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'echo must be greater or equal to 0'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      9\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     10\u001b[0m     \u001b[0;31m# Concatenate echo copies of word1 using *: echo_word\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mValueError\u001b[0m: echo must be greater or equal to 0"
     ]
    }
   ],
   "source": [
    "# Define shout_echo\n",
    "def shout_echo(word1, echo=1):\n",
    "    \"\"\"Concatenate echo copies of word1 and three\n",
    "    exclamation marks at the end of the string.\"\"\"\n",
    "\n",
    "    # Raise an error with raise\n",
    "    if echo < 0:\n",
    "        raise ValueError('echo must be greater or equal to 0')\n",
    "\n",
    "    # Concatenate echo copies of word1 using *: echo_word\n",
    "    echo_word = word1 * echo\n",
    "\n",
    "    # Concatenate '!!!' to echo_word: shout_word\n",
    "    shout_word = echo_word + '!!!'\n",
    "\n",
    "    # Return shout_word\n",
    "    return shout_word\n",
    "\n",
    "\n",
    "\n",
    "# Call shout_echo\n",
    "shout_echo(\"particle\", echo=5)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "eec80a0e-3653-453b-ac5e-63c4d61df5b0",
   "metadata": {},
   "source": [
    "## Bringing it all together\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "**You're now tgoing to use your hard-earnt skills to write error messages into your DataFrame analyzer that you have been building up in previous chapters: lets say that a user of your function passes your function the name of a column that isnt a column in the DF that they passed it, you'll want them know.  \n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 130,
   "id": "e67c4c0f-c4bc-4e37-8a73-2a15bd4f9d2a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3.4641016151377544"
      ]
     },
     "execution_count": 130,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def sqrt(x):\n",
    "    '''Return squrt value of given arg'''\n",
    "    try:\n",
    "        return x**(0.5)\n",
    "    except:\n",
    "        print('x must be int or float')\n",
    "        \n",
    "        \n",
    "sqrt(12)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 134,
   "id": "1c6a309e-553e-4f36-a7e5-b79aa30c2332",
   "metadata": {},
   "outputs": [
    {
     "ename": "TypeError",
     "evalue": "please keep argx as a value",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mTypeError\u001b[0m                                 Traceback (most recent call last)",
      "\u001b[0;32m/tmp/ipykernel_2050/1935712354.py\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[1;32m     15\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     16\u001b[0m \u001b[0;31m#sqrt(-9)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 17\u001b[0;31m \u001b[0msqrt\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'bongour'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;32m/tmp/ipykernel_2050/1935712354.py\u001b[0m in \u001b[0;36msqrt\u001b[0;34m(x)\u001b[0m\n\u001b[1;32m      3\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      4\u001b[0m     \u001b[0;32mif\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0misinstance\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mx\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mint\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 5\u001b[0;31m         \u001b[0;32mraise\u001b[0m \u001b[0mTypeError\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'please keep argx as a value'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      6\u001b[0m     \u001b[0;32melif\u001b[0m \u001b[0mx\u001b[0m \u001b[0;34m<\u001b[0m \u001b[0;36m0\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      7\u001b[0m         \u001b[0;32mraise\u001b[0m \u001b[0mValueError\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'arg x must be non-negative'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mTypeError\u001b[0m: please keep argx as a value"
     ]
    }
   ],
   "source": [
    "def sqrt(x):\n",
    "    '''Return sqrt root of given number'''\n",
    "    \n",
    "    if not isinstance(x, int):\n",
    "        raise TypeError('please keep argx as a value')\n",
    "    elif x < 0:\n",
    "        raise ValueError('arg x must be non-negative')\n",
    "        \n",
    "    try:\n",
    "        return x**(0.5)\n",
    "    except TypeError:\n",
    "        print('x must be int or float')\n",
    "        \n",
    "        \n",
    "        \n",
    "#sqrt(-9)\n",
    "sqrt('bongour')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b85ec87e-0c07-45f5-82d5-d2505c65a36b",
   "metadata": {},
   "source": [
    "## Bringing it all together (1)\n",
    "\n",
    "This is awesome! You have now learned how to write anonymous functions using lambda, how to pass lambda functions as arguments to other functions such as map(), filter(), and reduce(), as well as how to write errors and output custom error messages within your functions. You will now put together these learnings to good use by working with a Twitter dataset. Before practicing your new error handling skills,in this exercise, you will write a lambda function and use filter() to select retweets, that is, tweets that begin with the string 'RT'.\n",
    "\n",
    "To help you accomplish this, the Twitter data has been imported into the DataFrame, tweets_df. Go for it!\n",
    "Instructions\n",
    "100 XP\n",
    "\n",
    "    In the filter() call, pass a lambda function and the sequence of tweets as strings, tweets_df['text']. The lambda function should check if the first 2 characters in a tweet x are 'RT'. Assign the resulting filter object to result. To get the first 2 characters in a tweet x, use x[0:2]. To check equality, use a Boolean filter with ==.\n",
    "    Convert result to a list and print out the list.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 137,
   "id": "d9beceae-174b-4fa1-9003-213f850d2ac3",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "RT @bpolitics: .@krollbondrating's Christopher Whalen says Clinton is the weakest Dem candidate in 50 years https://t.co/pLk7rvoRSn https:/…\n",
      "RT @HeidiAlpine: @dmartosko Cruz video found.....racing from the scene.... #cruzsexscandal https://t.co/zuAPZfQDk3\n",
      "RT @AlanLohner: The anti-American D.C. elites despise Trump for his America-first foreign policy. Trump threatens their gravy train. https:…\n",
      "RT @BIackPplTweets: Young Donald trump meets his neighbor  https://t.co/RFlu17Z1eE\n",
      "RT @trumpresearch: @WaitingInBagdad @thehill Trump supporters have selective amnisia.\n",
      "RT @HouseCracka: 29,000+ PEOPLE WATCHING TRUMP LIVE ON ONE STREAM!!!\n",
      "\n",
      "https://t.co/7QCFz9ehNe\n",
      "RT @urfavandtrump: RT for Brendon Urie\n",
      "Fav for Donald Trump https://t.co/PZ5vS94lOg\n",
      "RT @trapgrampa: This is how I see #Trump every time he speaks. https://t.co/fYSiHNS0nT\n",
      "RT @trumpresearch: @WaitingInBagdad @thehill Trump supporters have selective amnisia.\n",
      "RT @Pjw20161951: NO KIDDING: #SleazyDonald just attacked Scott Walker for NOT RAISING TAXES in WI! #LyinTrump\n",
      "#NeverTrump  #CruzCrew  https…\n",
      "RT @urfavandtrump: RT for Brendon Urie\n",
      "Fav for Donald Trump https://t.co/PZ5vS94lOg\n",
      "RT @ggreenwald: The media spent all day claiming @SusanSarandon said she might vote for Trump. A total fabrication, but whatever... https:/…\n",
      "RT @Pjw20161951: NO KIDDING: #SleazyDonald just attacked Scott Walker for NOT RAISING TAXES in WI! #LyinTrump\n",
      "#NeverTrump  #CruzCrew  https…\n",
      "RT @trapgrampa: This is how I see #Trump every time he speaks. https://t.co/fYSiHNS0nT\n",
      "RT @mitchellvii: So let me get this straight.  Any reporter can assault Mr Trump at any time and Corey can do nothing?  Michelle is clearly…\n",
      "RT @paulbenedict7: How #Trump Sacks RINO Strongholds by Hitting Positions Held by Dems and GOP https://t.co/D7ulnAJhis   #tcot #PJNET https…\n",
      "RT @DRUDGE_REPORT: VIDEO:  Trump emotional moment with Former Miss Wisconsin who has terminal illness... https://t.co/qt06aG9inT\n",
      "RT @ggreenwald: The media spent all day claiming @SusanSarandon said she might vote for Trump. A total fabrication, but whatever... https:/…\n",
      "RT @DennisApgar: Thank God I seen Trump at first stop in Wisconsin media doesn't know how great he is, advice watch live streaming https://…\n",
      "RT @paulbenedict7: How #Trump Sacks RINO Strongholds by Hitting Positions Held by Dems and GOP https://t.co/D7ulnAJhis   #tcot #PJNET https…\n",
      "RT @DRUDGE_REPORT: VIDEO:  Trump emotional moment with Former Miss Wisconsin who has terminal illness... https://t.co/qt06aG9inT\n",
      "RT @DennisApgar: Thank God I seen Trump at first stop in Wisconsin media doesn't know how great he is, advice watch live streaming https://…\n",
      "RT @mitchellvii: So let me get this straight.  Any reporter can assault Mr Trump at any time and Corey can do nothing?  Michelle is clearly…\n",
      "RT @sciam: Trump's idiosyncratic patterns of speech are why people tend either to love or hate him https://t.co/QXwquVgs3c https://t.co/P9N…\n",
      "RT @Norsu2: Nightmare WI poll for Ted Cruz has Kasich surging: Trump 29, Kasich 27, Cruz 25. https://t.co/lJsgbLYY1P #NeverTrump\n",
      "RT @thehill: WATCH: Protester pepper-sprayed point blank at Trump rally https://t.co/B5f65Al9ld https://t.co/skAfByXuQc\n",
      "RT @sciam: Trump's idiosyncratic patterns of speech are why people tend either to love or hate him https://t.co/QXwquVgs3c https://t.co/P9N…\n",
      "RT @ggreenwald: The media spent all day claiming @SusanSarandon said she might vote for Trump. A total fabrication, but whatever... https:/…\n",
      "RT @DebbieStout5: Wow! Last I checked it was just 12 points &amp; that wasn't more than a day ago. Oh boy Trump ppl might want to rethink🤔 http…\n",
      "RT @tyleroakley: i'm a messy bitch, but at least i'm not voting for trump\n",
      "RT @vandives: Trump supporters r tired of justice NOT being served. There's no justice anymore. Hardworking Americans get screwed. That's n…\n",
      "RT @AP: BREAKING: Trump vows to stand by campaign manager charged with battery, says he does not discard people.\n",
      "RT @AP: BREAKING: Trump vows to stand by campaign manager charged with battery, says he does not discard people.\n",
      "RT @urfavandtrump: RT for Jerrie (Little Mix)\n",
      "Fav for Donald Trump https://t.co/nEVxElW6iG\n",
      "RT @urfavandtrump: RT for Jerrie (Little Mix)\n",
      "Fav for Donald Trump https://t.co/nEVxElW6iG\n",
      "RT @NoahCRothman: When Walker was fighting for reforms, Trump was defending unions and collective bargaining privileges https://t.co/e1UWNN…\n",
      "RT @RedheadAndRight: Report: Secret Service Says Michelle Fields Touched Trump https://t.co/c5c2sD8VO2\n",
      "\n",
      "This is the only article you will n…\n",
      "RT @AIIAmericanGirI: VIDEO=&gt; Anti-Trump Protester SLUGS Elderly Trump Supporter in the Face\n",
      "https://t.co/GeEryMDuDY\n",
      "RT @NoahCRothman: When Walker was fighting for reforms, Trump was defending unions and collective bargaining privileges https://t.co/e1UWNN…\n",
      "RT @JusticeRanger1: @realDonaldTrump @Pudingtane @DanScavino @GOP @infowars @EricTrump \n",
      "URGENT PUBLIC TRUMP ALERT:\n",
      "COVERT KILL MEANS https:…\n",
      "RT @AIIAmericanGirI: VIDEO=&gt; Anti-Trump Protester SLUGS Elderly Trump Supporter in the Face\n",
      "https://t.co/GeEryMDuDY\n",
      "RT @RedheadAndRight: Report: Secret Service Says Michelle Fields Touched Trump https://t.co/c5c2sD8VO2\n",
      "\n",
      "This is the only article you will n…\n",
      "RT @JusticeRanger1: @realDonaldTrump @Pudingtane @DanScavino @GOP @infowars @EricTrump \n",
      "URGENT PUBLIC TRUMP ALERT:\n",
      "COVERT KILL MEANS https:…\n",
      "RT @Schneider_CM: Trump says nobody had ever heard of executive orders before Obama started signing them. Never heard of the Emancipation P…\n",
      "RT @RonBasler1: @DavidWhitDennis @realDonaldTrump @tedcruz \n",
      "\n",
      "CRUZ SCREWS HOOKERS\n",
      "\n",
      "CRUZ / CLINTON\n",
      "RT @DonaldsAngel: Former Ms. WI just said that she is terminally ill but because of Trump pageant, her 7 yr. old son has his college educat…\n",
      "RT @Schneider_CM: Trump says nobody had ever heard of executive orders before Obama started signing them. Never heard of the Emancipation P…\n",
      "RT @DonaldsAngel: Former Ms. WI just said that she is terminally ill but because of Trump pageant, her 7 yr. old son has his college educat…\n",
      "RT @Dodarey: @DR8801 @SykesCharlie Charlie, let's see you get a straight \"yes\" or \"no\" answer from Cruz a/b being unfaithful to his wife @T…\n",
      "RT @RonBasler1: @DavidWhitDennis @realDonaldTrump @tedcruz \n",
      "\n",
      "CRUZ SCREWS HOOKERS\n",
      "\n",
      "CRUZ / CLINTON\n",
      "RT @RockCliffOne: Remember when the idea of a diabolical moron holding the world hostage was an idea for a funny movie? #Trump #GOP https:/…\n",
      "RT @HillaryClinton: \"Every day, another Republican bemoans the rise of Donald Trump... but [he] didn’t come out of nowhere.\" —Hillary\n",
      "https…\n",
      "RT @Dodarey: @DR8801 @SykesCharlie Charlie, let's see you get a straight \"yes\" or \"no\" answer from Cruz a/b being unfaithful to his wife @T…\n",
      "RT @HillaryClinton: \"Every day, another Republican bemoans the rise of Donald Trump... but [he] didn’t come out of nowhere.\" —Hillary\n",
      "https…\n",
      "RT @RockCliffOne: Remember when the idea of a diabolical moron holding the world hostage was an idea for a funny movie? #Trump #GOP https:/…\n",
      "RT @immigrant4trump: @immigrant4trump msm, cable news attacking trump all day, from 8am to 10pm today, then the reruns come on, repeating t…\n",
      "RT @immigrant4trump: @immigrant4trump msm, cable news attacking trump all day, from 8am to 10pm today, then the reruns come on, repeating t…\n",
      "RT @GlendaJazzey: Donald Trump’s Campaign Financing Dodge, @rrotunda https://t.co/L8flI4lswG via @VerdictJustia\n",
      "RT @TUSK81: LOUDER FOR THE PEOPLE IN THE BACK https://t.co/hlPVyNLXzx\n",
      "RT @loopzoop: Well...put it back https://t.co/8Yb7BDT5VM\n",
      "RT @claytoncubitt: Stop asking Bernie supporters if they’ll vote for Hillary against Trump. We got a plan to beat Trump already. Called Ber…\n",
      "RT @akaMaude13: Seriously can't make this up. What a joke. #NeverTrump  https://t.co/JkTx6mdRgC\n"
     ]
    }
   ],
   "source": [
    "import pandas as pd\n",
    "\n",
    "tweet_df = pd.read_csv('tweets.csv')\n",
    "\n",
    "\n",
    "\n",
    "# Select retweets from the Twitter DataFrame: result\n",
    "result = filter(lambda x: x[:2] == 'RT', tweet_df['text'])\n",
    "\n",
    "# Create list from filter object result: res_list\n",
    "res_list = list(result)\n",
    "\n",
    "# Print all retweets in res_list\n",
    "for tweet in res_list:\n",
    "    print(tweet)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "acede6ba-b1cd-47d6-b61f-f707429ab456",
   "metadata": {},
   "source": [
    "## Bringing it all together (2)\n",
    "\n",
    "Sometimes, we make mistakes when calling functions - even ones you made yourself. But don't fret! In this exercise, you will improve on your previous work with the count_entries() function in the last chapter by adding a try-except block to it. This will allow your function to provide a helpful message when the user calls your count_entries() function but provides a column name that isn't in the DataFrame.\n",
    "\n",
    "Once again, for your convenience, pandas has been imported as pd and the 'tweets.csv' file has been imported into the DataFrame tweets_df. Parts of the code from your previous work are also provided.\n",
    "Instructions\n",
    "100 XP\n",
    "\n",
    "    Add a try block so that when the function is called with the correct arguments, it processes the DataFrame and returns a dictionary of results.\n",
    "    Add an except block so that when the function is called incorrectly, it displays the following error message: 'The DataFrame does not have a ' + col_name + ' column.'.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 139,
   "id": "77f456fc-85fb-484e-83d1-43cadee1deb7",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'en': 97, 'et': 1, 'und': 2}\n",
      "The DataFrame does not have a hello column.\n",
      "None\n"
     ]
    }
   ],
   "source": [
    "# Define count_entries()\n",
    "def count_entries(df, col_name='lang'):\n",
    "    \"\"\"Return a dictionary with counts of\n",
    "    occurrences as value for each key.\"\"\"\n",
    "\n",
    "    # Initialize an empty dictionary: cols_count\n",
    "    cols_count = {}\n",
    "\n",
    "    # Add try block\n",
    "    try:\n",
    "        # Extract column from DataFrame: col\n",
    "        col = df[col_name]\n",
    "        \n",
    "        # Iterate over the column in dataframe\n",
    "        for entry in col:\n",
    "    \n",
    "            # If entry is in cols_count, add 1\n",
    "            if entry in cols_count.keys():\n",
    "                cols_count[entry] += 1\n",
    "            # Else add the entry to cols_count, set the value to 1\n",
    "            else:\n",
    "                cols_count[entry] = 1\n",
    "    \n",
    "        # Return the cols_count dictionary\n",
    "        return cols_count\n",
    "\n",
    "    # Add except block\n",
    "    except:\n",
    "        print('The DataFrame does not have a ' + col_name + ' column.')\n",
    "\n",
    "# Call count_entries(): result1\n",
    "result1 = count_entries(tweets_df, 'lang')\n",
    "\n",
    "# Print result1\n",
    "print(result1)\n",
    "\n",
    "\n",
    "\n",
    "result2 = count_entries(tweets_df, 'hello')\n",
    "print(result2)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8272f43f-f217-4497-a5dd-9916702a7c3e",
   "metadata": {},
   "source": [
    "## Bringing it all together (3)\n",
    "\n",
    "In the previous exercise, you built on your function count_entries() to add a try-except block. This was so that users would get helpful messages when calling your count_entries() function and providing a column name that isn't in the DataFrame. In this exercise, you'll instead raise a ValueError in the case that the user provides a column name that isn't in the DataFrame.\n",
    "\n",
    "Once again, for your convenience, pandas has been imported as pd and the 'tweets.csv' file has been imported into the DataFrame tweets_df. Parts of the code from your previous work are also provided.\n",
    "Instructions\n",
    "100 XP\n",
    "\n",
    "    If col_name is not a column in the DataFrame df, raise a ValueError 'The DataFrame does not have a ' + col_name + ' column.'.\n",
    "    Call your new function count_entries() to analyze the 'lang' column of tweets_df. Store the result in result1.\n",
    "    Print result1. This has been done for you, so hit 'Submit Answer' to check out the result. In the next exercise, you'll see that it raises the necessary ValueErrors.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 141,
   "id": "66dd29c0-6e67-46b0-9538-0fe5ff4dbd35",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'en': 97, 'et': 1, 'und': 2}\n"
     ]
    }
   ],
   "source": [
    "# Define count_entries()\n",
    "def count_entries(df, col_name='lang'):\n",
    "    \"\"\"Return a dictionary with counts of\n",
    "    occurrences as value for each key.\"\"\"\n",
    "    \n",
    "    # Raise a ValueError if col_name is NOT in DataFrame\n",
    "    if col_name not in df.columns:\n",
    "        raise ValueError('The DataFrame does not have a ' + col_name + ' column.')\n",
    "\n",
    "    # Initialize an empty dictionary: cols_count\n",
    "    cols_count = {}\n",
    "    \n",
    "    # Extract column from DataFrame: col\n",
    "    col = df[col_name]\n",
    "    \n",
    "    # Iterate over the column in DataFrame\n",
    "    for entry in col:\n",
    "\n",
    "        # If entry is in cols_count, add 1\n",
    "        if entry in cols_count.keys():\n",
    "            cols_count[entry] += 1\n",
    "            # Else add the entry to cols_count, set the value to 1\n",
    "        else:\n",
    "            cols_count[entry] = 1\n",
    "        \n",
    "        # Return the cols_count dictionary\n",
    "    return cols_count\n",
    "\n",
    "# Call count_entries(): result1\n",
    "result1 = count_entries(tweets_df, 'lang')\n",
    "\n",
    "# Print result1\n",
    "print(result1)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e6bb1717-02e2-4de8-9b40-0af55bd68f19",
   "metadata": {},
   "source": [
    "## Bringing it all together: testing your error handling skills\n",
    "\n",
    "You have just written error handling into your count_entries() function so that, when the user passes the function a column (as 2nd argument) NOT contained in the DataFrame (1st argument), a ValueError is thrown. You're now going to play with this function: it is loaded into pre-exercise code, as is the DataFrame tweets_df. Try calling count_entries(tweets_df, 'lang') to confirm that the function behaves as it should. Then call count_entries(tweets_df, 'lang1'): what is the last line of the output?\n",
    "Instructions\n",
    "50 XP\n",
    "Possible Answers\n",
    "\n",
    "    'ValueError: The DataFrame does not have the requested column.'\n",
    "    'ValueError: The DataFrame does not have a lang1 column.'\n",
    "    'TypeError: The DataFrame does not have the requested column.'\n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0214c1d7-5a70-4757-879f-5872a4ee8a77",
   "metadata": {},
   "source": [
    "## Congratulations!\n",
    "\n",
    "\n",
    "\n",
    "**You are now able to write functions in Python that accept single and multiple arguments and can return as many values as you please.  You're also adept at using default and flexible arguments and keyword arguments.  You've gained insight into scoping in Python, can write lambda functions and handle errors in your very own function writing practice.  You've also gained invaluable practice in using all of these techniques to write functions that are useful in a DataScience context.  You have come a long way in your developing practice as a budding Pythonista Data Scientist.  \n",
    "\n",
    "There are more basic skills that you will need to learn in Oython to be valuable as a working Data Scientist, and many of these we'll cover in the sequel to this course, so if you're finding youself still thirsty for more Pythonista Data Science chops, I'd head over there right now.  There you'll learnall about list comprehensions, which allow you to wrangle data in lists to create other lists, a tool to utilized by all Data Scientist working in Python today.  You'll also learn about iterators, which you have already seen in the context of for loops without having necessarily known it, iterators are everywhere in PythonLand, and to put it simply, allow you to rapidly iterate DataScience protocols and procedures over sets of objects.  These are a couple of cool functionalities in Pythonland you'll encounter in the sequel to this course, which will conclude with an entire chapter devoted to a case study in which you'll apply time and time again techniques learnt in both od these courses.  \n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "498e3761-ca39-417c-9fa3-86c1039f3282",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c6c773d7-2bee-447c-a3e3-e85df68a49cd",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d363be4c-52c4-4cb2-85ef-c91a0ee2c65a",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6392d40a-6567-47a7-8a68-a9b970f280b3",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "588f6897-a599-4718-bce5-3929361dde27",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1816d10f-7070-4749-9e33-11ee2103efbb",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4fd88ffd-a893-447b-9475-e308153bff2a",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "545f58f2-db2b-46dc-bad9-7d0ceb0c69a8",
   "metadata": {},
   "source": [
    "## Introduction to iterators\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "**Welcome to the course.  In this course, the second of Python Data Science Toolbox course, you'll learn about iterables, iterators, list comprehensions and generators - all essential components in the Pythonista's Data Science Toolbox.  We will conclude with an entire chapter devoted to a case study in which you'll apply time and time again techniques learnt on both of these courses.  \n",
    "\n",
    "\n",
    "# ***************************************************************************************************\n",
    "When you use a for loop to print our each element of a list, you're iterating over the list, or a string elements, or q sequence number in range() object.  The reason that we can loop over such objects is that they are special objects called iterables.  lists, strings, and range objects are all iterables, as are many other Python objects, such as dictionaries, and file connections.  \n",
    "\n",
    "\n",
    "# ***************************************************************************************************\n",
    "**The actual definition of an iterable is an object that has an associated iter() methods.  Once this iter() method is applied to an iterable, an iterator object is created.  '__iter__()' method in Py 3.9\n",
    "\n",
    "Under the hood, this is actually what a for loop is doing: it takes an iterable, creates the associated iterator object, and iterates over it.  \n",
    "\n",
    "# ***************************************************************************************************\n",
    "**An iteraor is defined as an object that has an associated next() method that produces the consecutive values.  '__next__()' method in Py 3.9\n",
    "\n",
    "To create an iterator from an iterable, all we need to do is use the function iter() and pass it iterable.  Once we have the iterator defined, we pass it to the function next() and this returns the first value.  Calling next() again on the iterator returns the next value untill so on, untill there are no values left to return and then it throws us a StopIteration error.  \n",
    "\n",
    "You can also print all valuesof an iterator in one fell swoop, using the * operator, referred to as the splat operator in some circles.  \n",
    "# This * operator unparks all elements of an iterator or an iterable.  \n",
    "Be warned, however, once you do so, you cannot do it again as there are no more values to iterate through.  We would have to redefine our iterator to do so.  \n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b79ef55d-d4b3-4c68-b3f3-9c353db261bb",
   "metadata": {},
   "source": [
    "Iterator in Python is simply an object that can be iterated upon. An object which will return data, one element at a time.\n",
    "\n",
    "# Technically speaking, a Python iterator object must implement two special methods, __iter__() and __next__(), collectively called the iterator protocol.\n",
    "\n",
    "\n",
    "# An object is called iterable if we can get an iterator from it. Most built-in containers in Python like: list, tuple, string etc. are iterables.\n",
    "\n",
    "The iter() function (which in turn calls the __iter__() method) returns an iterator from them."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 155,
   "id": "c1da0f52-e23a-418c-881a-3aa2085b313a",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "True\n",
      "True\n",
      "True\n",
      "['__add__', '__class__', '__class_getitem__', '__contains__', '__delattr__', '__delitem__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__getitem__', '__gt__', '__hash__', '__iadd__', '__imul__', '__init__', '__init_subclass__', '__iter__', '__le__', '__len__', '__lt__', '__mul__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__reversed__', '__rmul__', '__setattr__', '__setitem__', '__sizeof__', '__str__', '__subclasshook__', 'append', 'clear', 'copy', 'count', 'extend', 'index', 'insert', 'pop', 'remove', 'reverse', 'sort']\n"
     ]
    }
   ],
   "source": [
    "print('__iter__' in dir(str))\n",
    "\n",
    "\n",
    "print('__iter__' in dir(list))\n",
    "\n",
    "\n",
    "print('__iter__' in dir(dict))\n",
    "\n",
    "\n",
    "print(dir(list))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 179,
   "id": "c0efa3c4-5c73-4908-bd7d-e7116ec693fe",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<list_iterator object at 0x7f9fb1909430>\n",
      "<class 'list_iterator'>\n",
      "['__class__', '__delattr__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__iter__', '__le__', '__length_hint__', '__lt__', '__ne__', '__new__', '__next__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__setstate__', '__sizeof__', '__str__', '__subclasshook__']\n",
      "\n",
      "2 3 4 5 6\n"
     ]
    },
    {
     "ename": "StopIteration",
     "evalue": "",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mStopIteration\u001b[0m                             Traceback (most recent call last)",
      "\u001b[0;32m/tmp/ipykernel_2050/2024799334.py\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[1;32m     17\u001b[0m \u001b[0;31m# *************************************************************************************************** #\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     18\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 19\u001b[0;31m \u001b[0mprint\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0moutcome\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m__next__\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     20\u001b[0m \u001b[0mprint\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0moutcome\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m__next__\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     21\u001b[0m \u001b[0mprint\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0moutcome\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m__next__\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mStopIteration\u001b[0m: "
     ]
    }
   ],
   "source": [
    "my_list = [2, 3, 4, 5, 6]\n",
    "\n",
    "\n",
    "outcome = my_list.__iter__()\n",
    "\n",
    "print(outcome)\n",
    "print(type(outcome))\n",
    "\n",
    "print(dir(outcome))\n",
    "print()\n",
    "\n",
    "\n",
    "print(*outcome)\n",
    "\n",
    "# Seems its a generator, unpacking once, the __next__() does not work after * unparking\n",
    "# *************************************************************************************************** #\n",
    "# *************************************************************************************************** #\n",
    "\n",
    "print(outcome.__next__())\n",
    "print(outcome.__next__())\n",
    "print(outcome.__next__())\n",
    "print(outcome.__next__())\n",
    "print(outcome.__next__())\n",
    "print(outcome.__next__())\n",
    "\n",
    "\n",
    "\n",
    "for i in outcome:\n",
    "    print(i.__call__())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "046a3865-fd24-42d1-8c1a-6bde75e8a12c",
   "metadata": {},
   "source": [
    "\n",
    "We mentioned before that dictionaries and file connections are iterablesas well.  To iterate over the key-value pairs of a Python dictionary, we need to unpark them by applying .items() method to the dictionary as you can see here. \n",
    "\n",
    "\n",
    "With respect to file connections, here you can see how to use the iter() and next() ('__iter__()' and '__Next__()' method in Python 3.9) methods to return the lines from a file.  \n",
    "\n",
    "\n",
    "\n",
    "# This has been your crash course in the fundamentals of iterables and iterators. \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 180,
   "id": "f739097f-c70b-4e86-8f30-67a2d250f552",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "nam Jhhu\n",
      "title handsome\n",
      "age 20\n"
     ]
    }
   ],
   "source": [
    "bonjour_sava = {'nam': 'Jhhu', 'title': 'handsome', 'age': 20}\n",
    "\n",
    "\n",
    "for key, item in bonjour_sava.items():\n",
    "    print(key, item)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 121,
   "id": "cb7c5355-9bbf-43f7-9733-e83306124e83",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "﻿The\n",
      "Project\n",
      "Gutenberg\n",
      "eBook\n",
      "of\n",
      "Pride\n"
     ]
    }
   ],
   "source": [
    "with open('Pride and Prejudice, by Jane Austen.txt', 'r') as file:\n",
    "    content = file.read()\n",
    "    text = content.split()\n",
    "    #print(text)\n",
    "\n",
    "# *************************************************************************************************** #\n",
    "# *************************************************************************************************** #\n",
    "# Always use a context manager to do IO operations(file r/w/a, database connection etc. )\n",
    "# with open('Pride and Prejudice, by Jane Austen.txt', 'r') as file:\n",
    "#    \n",
    "\n",
    "\n",
    "it = text.__iter__()\n",
    "\n",
    "\n",
    "print(next(it))\n",
    "print(next(it))\n",
    "print(next(it))\n",
    "print(next(it))\n",
    "print(next(it))\n",
    "\n",
    "\n",
    "\n",
    "file.close()\n",
    "print(next(it))\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 190,
   "id": "4f195981-b0cc-41aa-8875-91b07c2ad6e6",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "﻿The Project Gutenberg eBook of Pride and Prejudice, by Jane Austen\n",
      "\n",
      "<_io.TextIOWrapper name='Pride and Prejudice, by Jane Austen.txt' mode='r' encoding='UTF-8'>\n",
      "﻿The Project Gutenberg eBook of Pride and Prejudice, by Jane Austen\n",
      "\n"
     ]
    }
   ],
   "source": [
    "with open('Pride and Prejudice, by Jane Austen.txt', 'r') as file:\n",
    "    content = file.readline()\n",
    "    print(content)\n",
    "    \n",
    "    \n",
    "# *************************************************************************************************** #\n",
    "# *************************************************************************************************** #\n",
    "print(file)   # The file IO stopped, but file obj kept in memory?\n",
    "print(content)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4e20a12f-e654-4195-afd2-4c217852a06a",
   "metadata": {},
   "source": [
    "## Iterators vs Iterables\n",
    "\n",
    "# ****************************************************************************************************\n",
    "# Let's do a quick recall of what you've learned about iterables and iterators. Recall from the video that an iterable is an object that can return an iterator, while an iterator is an object that keeps state and produces the next value when you call next() on it. In this exercise, you will identify which object is an iterable and which is an iterator.\n",
    "\n",
    "The environment has been pre-loaded with the variables flash1 and flash2. Try printing out their values with print() and next() to figure out which is an iterable and which is an iterator.\n",
    "Instructions\n",
    "50 XP\n",
    "Possible Answers\n",
    "\n",
    "    Both flash1 and flash2 are iterators.\n",
    "    Both flash1 and flash2 are iterables.\n",
    "#    flash1 is an iterable and flash2 is an iterator.\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f997762a-8f87-4e61-a1db-bdbd512d4df7",
   "metadata": {},
   "outputs": [],
   "source": [
    "In [1]:\n",
    "flash1\n",
    "Out[1]:\n",
    "['jay garrick', 'barry allen', 'wally west', 'bart allen']\n",
    "In [2]:\n",
    "flash2\n",
    "Out[2]:\n",
    "<list_iterator at 0x7effaec77400>\n",
    "In [3]:\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b10c409a-c8e8-4903-bee5-94211145672e",
   "metadata": {},
   "source": [
    "## Iterating over iterables (1)\n",
    "\n",
    "Great, you're familiar with what iterables and iterators are! In this exercise, you will reinforce your knowledge about these by iterating over and printing from iterables and iterators.\n",
    "\n",
    "You are provided with a list of strings flash. You will practice iterating over the list by using a for loop. You will also create an iterator for the list and access the values from the iterator.\n",
    "Instructions\n",
    "100 XP\n",
    "\n",
    "    Create a for loop to loop over flash and print the values in the list. \n",
    "#    Use person as the loop variable.\n",
    "    Create an iterator for the list flash and assign the result to superhero.\n",
    "    Print each of the items from superhero using next() 4 times.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 192,
   "id": "35771f7f-f9e9-4436-92cf-727c6b573fa6",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "jay garrick\n",
      "barry allen\n",
      "wally west\n",
      "bart allen\n",
      "jay garrick\n",
      "barry allen\n",
      "wally west\n",
      "bart allen\n"
     ]
    }
   ],
   "source": [
    "# Create a list of strings: flash\n",
    "flash = ['jay garrick', 'barry allen', 'wally west', 'bart allen']\n",
    "\n",
    "# Print each list item in flash using a for loop\n",
    "for person in flash:\n",
    "    print(person)\n",
    "\n",
    "\n",
    "# Create an iterator for flash: superhero\n",
    "superhero = flash.__iter__()    # My Python enviroment is Python 3.9\n",
    "\n",
    "# Print each item from the iterator\n",
    "print(superhero.__next__())\n",
    "print(superhero.__next__())\n",
    "print(superhero.__next__())\n",
    "print(superhero.__next__())\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "694ad3d2-fb2e-48ff-8caf-e1f7655aaf2d",
   "metadata": {},
   "source": [
    "## Iterating over iterables (2)\n",
    "\n",
    "One of the things you learned about in this chapter is that not all iterables are actual lists. A couple of examples that we looked at are strings and the use of the range() function. In this exercise, we will focus on the range() function.\n",
    "\n",
    "You can use range() in a for loop as if it's a list to be iterated over:\n",
    "\n",
    "for i in range(5):\n",
    "    print(i)\n",
    "\n",
    "# ****************************************************************************************************\n",
    "# Recall that range() doesn't actually create the list; instead, it creates a range object with an iterator that produces the values until it reaches the limit (in the example, until the value 4). If range() created the actual list, calling it with a value of 10^100 may not work, especially since a number as big as that may go over a regular computer's memory. The value 10^100 is actually what's called a Googol which is a 1 followed by a hundred 0s. That's a huge number!\n",
    "\n",
    "Your task for this exercise is to show that calling range() with\n",
    "\n",
    "won't actually pre-create the list.\n",
    "Instructions\n",
    "100 XP\n",
    "\n",
    "    Create an iterator object small_value over range(3) using the function iter().\n",
    "    Using a for loop, iterate over range(3), printing the value for every iteration. Use num as the loop variable.\n",
    "    Create an iterator object googol over range(10 ** 100).\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "35ab53d1-8878-4c50-acab-1f689f82a819",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'range'>\n",
      "<class 'range_iterator'>\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "3"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "r_4 = range(4)\n",
    "print(type(r_4))\n",
    "\n",
    "r_4_int = r_4.__iter__()\n",
    "print(type(r_4_int))\n",
    "\n",
    "\n",
    "# *************************************************************************************************** #\n",
    "# next(r_4)     # TypeError: 'range' object is not an iterator\n",
    "next(r_4_int)\n",
    "next(r_4_int)\n",
    "next(r_4_int)\n",
    "next(r_4_int)\n",
    "#next(r_4_int)   # StopIteration: \n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8dcaf0be-93b8-47e7-bc05-4b4d21c8a03a",
   "metadata": {},
   "source": [
    "\n",
    "# Check above results, range(n) returns a range object, when applying __iter__() method or iter() function range object, it returns a iterator of range object ????????????????\n",
    "# ***************************************************************************************************\n",
    "<class 'range'>\n",
    "<class 'range_iterator'>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 206,
   "id": "410b4fbd-e01b-460a-bd7f-61747081e592",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0\n",
      "1\n",
      "2\n",
      "0\n",
      "1\n",
      "2\n",
      "0\n",
      "1\n",
      "2\n",
      "3\n",
      "4\n"
     ]
    }
   ],
   "source": [
    "# Create an iterator for range(3): small_value\n",
    "small_value = range(3).__iter__()\n",
    "\n",
    "# Print the values in small_value\n",
    "print(next(small_value))\n",
    "print(next(small_value))\n",
    "print(next(small_value))\n",
    "\n",
    "# Loop over range(3) and print the values\n",
    "for num in range(3).__iter__():\n",
    "    print(num)\n",
    "\n",
    "\n",
    "# Create an iterator for range(10 ** 100): googol\n",
    "googol = range(10**100).__iter__()\n",
    "\n",
    "# Print the first 5 values from googol\n",
    "print(next(googol))\n",
    "print(next(googol))\n",
    "print(next(googol))\n",
    "print(next(googol))\n",
    "print(next(googol))\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 210,
   "id": "c53054bd-2f3b-4dba-b892-2270cb6214a8",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Help on built-in function iter in module builtins:\n",
      "\n",
      "iter(...)\n",
      "    iter(iterable) -> iterator\n",
      "    iter(callable, sentinel) -> iterator\n",
      "    \n",
      "    Get an iterator from an object.  In the first form, the argument must\n",
      "    supply its own iterator, or be a sequence.\n",
      "    In the second form, the callable is called until it returns the sentinel.\n",
      "\n",
      "None\n",
      "\n",
      "Help on method-wrapper:\n",
      "\n",
      "__iter__ = <method-wrapper '__iter__' of range_iterator object>\n",
      "    Implement iter(self).\n",
      "\n",
      "None\n"
     ]
    }
   ],
   "source": [
    "print(help(iter))\n",
    "print()\n",
    "\n",
    "\n",
    "print(help(small_value.__iter__))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "a3b03a49-5df7-451d-bfd2-869ff9acf38f",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0\n",
      "1\n",
      "2\n",
      "0\n",
      "1\n",
      "2\n",
      "<longrange_iterator object at 0x7f57a65c8810>\n",
      "[999990, 999991, 999992, 999993, 999994, 999995, 999996, 999997, 999998]\n",
      "0\n",
      "1\n",
      "2\n",
      "3\n",
      "4\n",
      "5\n",
      "6\n"
     ]
    }
   ],
   "source": [
    "# Create an iterator for range(3): small_value\n",
    "small_value = iter(range(3))\n",
    "\n",
    "# Print the values in small_value\n",
    "print(next(small_value))\n",
    "print(next(small_value))\n",
    "print(next(small_value))\n",
    "\n",
    "# Loop over range(3) and print the values\n",
    "for num in iter(range(3)):\n",
    "    print(num)\n",
    "\n",
    "\n",
    "# Create an iterator for range(10 ** 100): googol\n",
    "googol = iter(range(100**100000))\n",
    "\n",
    "\n",
    "print(googol)\n",
    "print([*range(10**6)][-10:-1])\n",
    "\n",
    "\n",
    "# Print the first 5 values from googol\n",
    "print(next(googol))\n",
    "print(next(googol))\n",
    "print(next(googol))\n",
    "print(next(googol))\n",
    "print(next(googol))\n",
    "print(next(googol))\n",
    "print(next(googol))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "77e6b366-516e-4be7-8795-adee2b0a6c1a",
   "metadata": {},
   "source": [
    "## Iterators as function arguments\n",
    "\n",
    "# It seems Python provides iter() and next() method out of the box, and any iterable object also provides iter() and next() method as well? what s the point? (__iter__() and __next__() in Python 3.9)\n",
    "# ****************************************************************************************************\n",
    "# ****************************************************************************************************\n",
    "# You've been using the iter() function to get an iterator object, as well as the next() function to retrieve the values one by one from the iterator object.\n",
    "\n",
    "There are also functions that take iterators and iterables as arguments. For example, the list() and sum() functions return a list and the sum of elements, respectively.\n",
    "\n",
    "In this exercise, you will use these functions by passing an iterable from range() and then printing the results of the function calls.\n",
    "Instructions\n",
    "100 XP\n",
    "\n",
    "    Create a range object that would produce the values from 10 to 20 using range(). Assign the result to values.\n",
    "    Use the list() function to create a list of values from the range object values. Assign the result to values_list.\n",
    "    Use the sum() function to get the sum of the values from 10 to 20 from the range object values. Assign the result to values_sum.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 212,
   "id": "5869094a-f05e-4206-bd09-0807394890c4",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "range(10, 21)\n",
      "[10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20]\n",
      "165\n"
     ]
    }
   ],
   "source": [
    "# Create a range object: values\n",
    "values = range(10,21)\n",
    "\n",
    "# Print the range object\n",
    "print(values)\n",
    "\n",
    "# Create a list of integers: values_list\n",
    "values_list = list(values)\n",
    "\n",
    "# Print values_list\n",
    "print(values_list)\n",
    "\n",
    "# Get the sum of values: values_sum\n",
    "#values_sum = sum(values_list)\n",
    "value_sum = sum(values)\n",
    "\n",
    "# Print values_sum\n",
    "print(values_sum)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 246,
   "id": "8ace908a-40ed-42e1-a0eb-c2e8657507b6",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20]\n",
      "['__bool__', '__class__', '__contains__', '__delattr__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__getitem__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__iter__', '__le__', '__len__', '__lt__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__reversed__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', 'count', 'index', 'start', 'step', 'stop']\n",
      "\n",
      "['__class__', '__delattr__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__iter__', '__le__', '__length_hint__', '__lt__', '__ne__', '__new__', '__next__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__setstate__', '__sizeof__', '__str__', '__subclasshook__']\n"
     ]
    }
   ],
   "source": [
    "# Create a range object: values\n",
    "values = range(10,21)\n",
    "\n",
    "\n",
    "list_values = list(values)\n",
    "print(list_values)\n",
    "\n",
    "\n",
    "# *************************************************************************************************** #\n",
    "# *************************************************************************************************** #\n",
    "print(dir(values))             # Range object is an iterable, thus have __iter__() but NO __next__()\n",
    "#print(dir(list_values))       # Iterable have __iter__() method\n",
    "print()\n",
    "\n",
    "\n",
    "iter_values = iter(values)\n",
    "print(dir(iter_values))        # Iterator have __iter__() and __next__() methods"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8c32e407-c8ad-4208-bbd4-110f30a97b72",
   "metadata": {},
   "source": [
    "## Playing with iterators\n",
    "\n",
    "\n",
    "\n",
    "**We are now going to dive a bit deeper into the world of iterables and iterators by checking out some very cool, very useful functions.  The first function, enumerate(), will allow us to add a counter(index actually) to any iterable, while the second function zip() will allow us to stitch together an arbitaray number of iterables.  Lets begin.  \n",
    "\n",
    "\n",
    "# ***************************************************************************************************\n",
    "Enumerate() is a function that takes any itrable as argument, such as a list, and returns a special enumerate object, which consists of pairs containing the elements of original iterable, along with their index within the iterable.  We can use the function list() to turn this enumerate object into a list of tuples, and print to see what it contains.  \n",
    "\n",
    "\n",
    "\n",
    "# ***************************************************************************************************\n",
    "#                                        iterator based on dir(), have both __next__() & __iter__()\n",
    "# The enumerate object itself is also an iterable, and we can loop over it while unparking its elements using the clause for index, value in enumerate(avengers) for example.   \n",
    "\n",
    "It is the default behavior of enumerate to begin indexing at zero, however you can alter this with second argument start as you see below. \n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "id": "8fa12a12-247c-4db1-8276-64ddd9341ec0",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0 hawkeye\n",
      "1 iron man\n",
      "2 thor\n",
      "3 quicksilver\n",
      "12 hawkeye\n",
      "13 iron man\n",
      "14 thor\n",
      "15 quicksilver\n",
      "\n",
      "<enumerate object at 0x7f57a6587e80>\n",
      "\n",
      "['__class__', '__class_getitem__', '__delattr__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__iter__', '__le__', '__lt__', '__ne__', '__new__', '__next__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__']\n",
      "<class 'enumerate'>\n",
      "\n",
      "['__class__', '__class_getitem__', '__delattr__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__iter__', '__le__', '__lt__', '__ne__', '__new__', '__next__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__']\n",
      "<class 'enumerate'>\n",
      "\n",
      "['__class__', '__class_getitem__', '__delattr__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__iter__', '__le__', '__lt__', '__ne__', '__new__', '__next__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__']\n",
      "<class 'enumerate'>\n"
     ]
    }
   ],
   "source": [
    "avengers = ['hawkeye', 'iron man', 'thor', 'quicksilver']\n",
    "\n",
    "\n",
    "for index, value in enumerate(avengers):\n",
    "    print(index, value)\n",
    "    \n",
    "\n",
    "for index, value in enumerate(avengers, start=12):\n",
    "    print(index, value)\n",
    "print()\n",
    "    \n",
    "    \n",
    "enu_avenger = enumerate(avengers)\n",
    "print(enu_avenger)\n",
    "print()\n",
    "\n",
    "\n",
    "\n",
    "# *************************************************************************************************** #\n",
    "# *************************************************************************************************** #\n",
    "print(dir(enu_avenger))                   # enumerate object is an iterator, have __next__() __iter__()\n",
    "print(type(enu_avenger))\n",
    "print()\n",
    "\n",
    "# *************************************************************************************************** #\n",
    "iter_avenger = enu_avenger.__iter__()     # Cause you cant iter() an iterator into an iterator\n",
    "\n",
    "# *************************************************************************************************** #\n",
    "iter_again_avenger = iter(enu_avenger)    # You can only iter() an iterable into an iterator\n",
    "\n",
    "\n",
    "print(dir(iter_avenger))\n",
    "print(type(iter_avenger))\n",
    "print()\n",
    "\n",
    "print(dir(iter_again_avenger))\n",
    "print(type(iter_again_avenger))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "04d534a1-d5eb-45c3-ba88-04ba87e2b8c1",
   "metadata": {},
   "source": [
    "\n",
    "# ***************************************************************************************************\n",
    "Now lets move on the zip(), which accepts an arbitrary number or iterables and returns iterator of tuples.  Here we have two lists, one of the avengers, another of their names.  \n",
    "\n",
    "# Zipping them together creates a zip object, which is an iterator of tuples.  \n",
    "# (zip object got a __iter__, __next__ in its dir(), so its an iterator)\n",
    "\n",
    "We can turn this iterator of tuples into a list and print the list.  The first element is a tuple containing the first element of each list that we zipped, the second element is a tuple containing the second element of each list we zipped, and so on.  \n",
    "\n",
    "\n",
    "# Alternatively, we can use a for loop to iterator over the zip object and print the tuples.  We could also have used the splat operator to print all the elements.  \n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "348271a0-410a-45f7-bbe1-823a5e6bd6bf",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "('hawkeye', 'barton') ('iron man', 'stark') ('thor', 'odinson') ('quicksilver', 'maximoff')\n",
      "<class 'zip'>\n",
      "['__class__', '__delattr__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__iter__', '__le__', '__lt__', '__ne__', '__new__', '__next__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__']\n",
      "True\n"
     ]
    }
   ],
   "source": [
    "avengers = ['hawkeye', 'iron man', 'thor', 'quicksilver']\n",
    "names = ['barton', 'stark', 'odinson', 'maximoff']\n",
    "\n",
    "\n",
    "z = zip(avengers, names)\n",
    "\n",
    "print(*z)\n",
    "print(type(z))\n",
    "\n",
    "print(dir(z))\n",
    "print('__iter__' and '__next__' in dir(z))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0195d0c9-1169-4df0-9f34-e0757351ae35",
   "metadata": {},
   "source": [
    "## Using enumerate\n",
    "\n",
    "You're really getting the hang of using iterators, great job!\n",
    "\n",
    "You've just gained several new ideas on iterators from the last video and one of them is the enumerate() function. \n",
    "# Recall that enumerate() returns an enumerate object that produces a sequence of tuples, and each of the tuples is an index-value pair.\n",
    "\n",
    "In this exercise, you are given a list of strings mutants and you will practice using enumerate() on it by printing out a list of tuples and unpacking the tuples using a for loop.\n",
    "Instructions\n",
    "100 XP\n",
    "\n",
    "    Create a list of tuples from mutants and assign the result to mutant_list. Make sure you generate the tuples using enumerate() and turn the result from it into a list using list().\n",
    "    Complete the first for loop by unpacking the tuples generated by calling enumerate() on mutants. Use index1 for the index and value1 for the value when unpacking the tuple.\n",
    "    Complete the second for loop similarly as with the first, but this time change the starting index to start from 1 by passing it in as an argument to the start parameter of enumerate(). Use index2 for the index and value2 for the value when unpacking the tuple.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 250,
   "id": "64c1c53c-9d11-4122-9d4b-a04995a347f1",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[(0, 'charles xavier'), (1, 'bobby drake'), (2, 'kurt wagner'), (3, 'max eisenhardt'), (4, 'kitty pryde')]\n",
      "0 charles xavier\n",
      "1 bobby drake\n",
      "2 kurt wagner\n",
      "3 max eisenhardt\n",
      "4 kitty pryde\n",
      "\n",
      "1 charles xavier\n",
      "2 bobby drake\n",
      "3 kurt wagner\n",
      "4 max eisenhardt\n",
      "5 kitty pryde\n"
     ]
    }
   ],
   "source": [
    "# Create a list of strings: mutants\n",
    "mutants = ['charles xavier', \n",
    "            'bobby drake', \n",
    "            'kurt wagner', \n",
    "            'max eisenhardt', \n",
    "            'kitty pryde']\n",
    "\n",
    "# Create a list of tuples: mutant_list\n",
    "mutant_list = list(enumerate(mutants))\n",
    "\n",
    "# Print the list of tuples\n",
    "print(mutant_list)\n",
    "\n",
    "# Unpack and print the tuple pairs\n",
    "for index1, value1 in mutant_list:\n",
    "    print(index1, value1)\n",
    "    \n",
    "    \n",
    "print()\n",
    "    \n",
    "    \n",
    "# Change the start index\n",
    "for index2, value2 in enumerate(mutants, start=1):\n",
    "    print(index2, value2)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ebf77294-4b1a-4297-98bb-bfd3653769e6",
   "metadata": {},
   "source": [
    "## Using zip\n",
    "\n",
    "# Another interesting function that you've learned is zip(), which takes any number of iterables and returns a zip object that is an iterator of tuples. If you wanted to print the values of a zip object, you can convert it into a list and then print it. Printing just a zip object will not return the values unless you unpack it first. In this exercise, you will explore this for yourself.\n",
    "\n",
    "Three lists of strings are pre-loaded: mutants, aliases, and powers. First, you will use list() and zip() on these lists to generate a list of tuples. Then, you will create a zip object using zip(). Finally, you will unpack this zip object in a for loop to print the values in each tuple. Observe the different output generated by printing the list of tuples, then the zip object, and finally, the tuple values in the for loop.\n",
    "Instructions\n",
    "100 XP\n",
    "\n",
    "    Using zip() with list(), create a list of tuples from the three lists mutants, aliases, and powers (in that order) and assign the result to mutant_data.\n",
    "    Using zip(), create a zip object called mutant_zip from the three lists mutants, aliases, and powers.\n",
    "    Complete the for loop by unpacking the zip object you created and printing the tuple values. Use value1, value2, value3 for the values from each of mutants, aliases, and powers, in that order.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "46673100-40b2-44c7-8e1f-258846459bcb",
   "metadata": {},
   "outputs": [],
   "source": [
    "In [1]:\n",
    "mutants\n",
    "Out[1]:\n",
    "\n",
    "['charles xavier',\n",
    " 'bobby drake',\n",
    " 'kurt wagner',\n",
    " 'max eisenhardt',\n",
    " 'kitty pryde']\n",
    "In [2]:\n",
    "aliases\n",
    "Out[2]:\n",
    "['prof x', 'iceman', 'nightcrawler', 'magneto', 'shadowcat']\n",
    "In [3]:\n",
    "powers\n",
    "Out[3]:\n",
    "\n",
    "['telepathy',\n",
    " 'thermokinesis',\n",
    " 'teleportation',\n",
    " 'magnetokinesis',\n",
    " 'intangibility']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 253,
   "id": "55097180-e62a-421b-8d04-81d54cd52ae8",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[('charles xavier', 'prof x', 'telepathy'), ('bobby drake', 'iceman', 'thermokinesis'), ('kurt wagner', 'nightcrawler', 'teleportation'), ('max eisenhardt', 'magneto', 'magnetokinesis'), ('kitty pryde', 'shadowcat', 'intangibility')]\n",
      "\n",
      "<zip object at 0x7f9fab18b980>\n",
      "\n",
      "charles xavier prof x telepathy\n",
      "bobby drake iceman thermokinesis\n",
      "kurt wagner nightcrawler teleportation\n",
      "max eisenhardt magneto magnetokinesis\n",
      "kitty pryde shadowcat intangibility\n"
     ]
    }
   ],
   "source": [
    "mutants = ['charles xavier', 'bobby drake', 'kurt wagner', 'max eisenhardt', 'kitty pryde']\n",
    "\n",
    "aliases = ['prof x', 'iceman', 'nightcrawler', 'magneto', 'shadowcat']\n",
    "\n",
    "powers = ['telepathy', 'thermokinesis', 'teleportation', 'magnetokinesis', 'intangibility']\n",
    "\n",
    "\n",
    "\n",
    "# Create a list of tuples: mutant_data\n",
    "mutant_data = list(zip(mutants, aliases, powers))\n",
    "\n",
    "# Print the list of tuples\n",
    "print(mutant_data)\n",
    "print()\n",
    "\n",
    "# Create a zip object using the three lists: mutant_zip\n",
    "mutant_zip = zip(mutants, aliases, powers)\n",
    "\n",
    "# Print the zip object\n",
    "print(mutant_zip)\n",
    "print()\n",
    "\n",
    "# Unpack the zip object and print the tuple values\n",
    "for value1, value2, value3 in mutant_zip:\n",
    "    print(value1, value2, value3)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3d8398f8-88eb-49c2-a53a-81a21de210b4",
   "metadata": {},
   "source": [
    "## Using * and zip to 'unzip'\n",
    "\n",
    "You know how to use zip() as well as how to print out values from a zip object. Excellent!\n",
    "\n",
    "Let's play around with zip() a little more. There is no unzip function for doing the reverse of what zip() does. We can, however, reverse what has been zipped together by using zip() with a little help from *! * unpacks an iterable such as a list or a tuple into positional arguments in a function call.\n",
    "\n",
    "In this exercise, you will use * in a call to zip() to unpack the tuples produced by zip().\n",
    "\n",
    "Two tuples of strings, mutants and powers have been pre-loaded.\n",
    "Instructions\n",
    "100 XP\n",
    "\n",
    "    Create a zip object by using zip() on mutants and powers, in that order. Assign the result to z1.\n",
    "    Print the tuples in z1 by unpacking them into positional arguments using the * operator in a print() call.\n",
    "    Because the previous print() call would have exhausted the elements in z1, recreate the zip object you defined earlier and assign the result again to z1.\n",
    "#    'Unzip' the tuples in z1 by unpacking them into positional arguments using the * operator in a zip() call. Assign the results to result1 and result2, in that order.\n",
    "    The last print() statements prints the output of comparing result1 to mutants and result2 to powers. Click Submit Answer to see if the unpacked result1 and result2 are equivalent to mutants and powers, respectively.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 261,
   "id": "578a1a84-6356-416e-b931-547f9acce4b5",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "('charles xavier', 'telepathy') ('bobby drake', 'thermokinesis') ('kurt wagner', 'teleportation') ('max eisenhardt', 'magnetokinesis') ('kitty pryde', 'intangibility')\n",
      "True\n",
      "True\n",
      "('charles xavier', 'bobby drake', 'kurt wagner', 'max eisenhardt', 'kitty pryde')\n",
      "('telepathy', 'thermokinesis', 'teleportation', 'magnetokinesis', 'intangibility')\n"
     ]
    }
   ],
   "source": [
    "mutants = ('charles xavier', 'bobby drake', 'kurt wagner', 'max eisenhardt', 'kitty pryde')\n",
    "\n",
    "powers = ('telepathy', 'thermokinesis', 'teleportation', 'magnetokinesis', 'intangibility')\n",
    "\n",
    "\n",
    "\n",
    "# Create a zip object from mutants and powers: z1\n",
    "z1 = zip(mutants, powers)\n",
    "\n",
    "# Print the tuples in z1 by unpacking with *\n",
    "print(*z1)\n",
    "\n",
    "# Re-create a zip object from mutants and powers: z1\n",
    "z1 = zip(mutants, powers)\n",
    "\n",
    "# 'Unzip' the tuples in z1 by unpacking with * and zip(): result1, result2\n",
    "result1, result2 = zip(*z1)\n",
    "# *************************************************************************************************** #\n",
    "# *************************************************************************************************** #\n",
    "\n",
    "# Check if unpacked tuples are equivalent to original tuples\n",
    "print(result1 == mutants)\n",
    "print(result2 == powers)\n",
    "\n",
    "\n",
    "\n",
    "print(result1)\n",
    "print(result2)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "490da894-87ca-4277-b7ea-c46cd2f1654c",
   "metadata": {},
   "source": [
    "## Using iterators to load large files into memory\n",
    "\n",
    "# *****************************************************************************************************\n",
    "\n",
    "\n",
    "\n",
    "**Now if you are more comfortable with iterables, iterators and how they work, we are going to check out a particular use case that is pertinent to the world of Data Science.  Dealing with large amounts of data.  \n",
    "\n",
    "# ***************************************************************************************************\n",
    "Lets say you are pulling data from a file, database or API and there's so much of it, just so much data, that you can't hold it in memory.  One solution is to load the data in chunks, perform the desired operation or operations on each chunk, store the results, discard the chunk and then load the next chunk of data.  This sounds like a place where iterator could be useful.  \n",
    "\n",
    "\n",
    "# ***************************************************************************************************\n",
    "To surmount this challenge, we are going to use Pandas function read_csv(), which provides a wonderful option whereby you can load data in chunks and iterator over them.  All we need to do is specify the chunk using the argument - chunksize.  \n",
    "\n",
    "As with much of what we do in DataScience, this best illustrated by an example, lets say we have a csv with a column called x containing numbers and I want compute the sum of all the numbers in that column.  However the file is just too large to store in memory.  \n",
    "\n",
    "\n",
    "# ***************************************************************************************************\n",
    "# ***************************************************************************************************\n",
    "# ***************************************************************************************************\n",
    "We first import Pandas and then initialize an empty list to hold the result of each iteration.  We then use the read_csv() function utilizing the argument chunk_size, setting it to the size of the chunk I want to read in.  In below example, we use a chunksize=1000.  The object created by read_csv is an iterable, so I can iterate over it, using a for loop, in which each chunk will be a DataFrame.  Within the for loop, thats on each iteration, we can compute the sum of the column of interest, and we append it to the list result.  Once this is executed, we can take the sum of the list result and this gives us our total sum of the column of interest.  Iterators to the rescue. \n",
    "\n",
    "\n",
    "Also note that we need not have used a list to store the result, we could initialized total_x to 0 before iterating over the file and added each sum during the iteration procedure, as you see below.  \n",
    "\n",
    "\n",
    "\n",
    "# ***************************************************************************************************\n",
    "Now things get realy cool, you are going to use a iterator to load Twitter data in chunks and perform a similar computation that you did in prequel to this course.  Then you're going to write a function that does the same.  \n",
    "\n",
    "# Happy iterating friend. \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "37f114cd-0961-4443-83d5-b3f7026406e5",
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "\n",
    "\n",
    "#print(help(pd.read_csv))     # chunksize : int, optional\n",
    "\n",
    "# print(dir(pd.read_csv))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cb10a136-4c9f-4047-ac77-c7a2996270c7",
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "\n",
    "\n",
    "result = []\n",
    "\n",
    "for chunk in pd.read_csv('data.csv', chunksize=1000):\n",
    "    result.append(sum(chunk['x']))\n",
    "    \n",
    "total_x = sum(result)\n",
    "\n",
    "print(total_x)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5c214fc2-a925-4fe5-9513-66be6c397622",
   "metadata": {},
   "source": [
    "# Are there any better solutions?  Cause applying chunksize needs to use for loop, and its inefficient"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fdee24a7-af0e-46f1-ad09-22aeb38ab891",
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "\n",
    "total_x = 0\n",
    "\n",
    "for chunk in pd.read_csv('data.csv', chunksize=1000):\n",
    "    total_x += sum(chunk['x'])\n",
    "    \n",
    "    \n",
    "print(total_x)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "94e43955-d5ba-4415-af21-8efce3e5a190",
   "metadata": {},
   "source": [
    "## Processing large amounts of Twitter data\n",
    "\n",
    "\n",
    "# ****************************************************************************************************\n",
    "# Does this means we dont need to consider the efficiency?  Cause for loop does not efficient, or thats # because we have no other options but only this one? \n",
    "\n",
    "# ****************************************************************************************************\n",
    "# Sometimes, the data we have to process reaches a size that is too much for a computer's memory to handle. This is a common problem faced by data scientists. A solution to this is to process an entire data source chunk by chunk, instead of a single go all at once.\n",
    "\n",
    "In this exercise, you will do just that. You will process a large csv file of Twitter data in the same way that you processed 'tweets.csv' in Bringing it all together exercises of the prequel course, but this time, working on it in chunks of 10 entries at a time.\n",
    "\n",
    "If you are interested in learning how to access Twitter data so you can work with it on your own system, refer to Part 2 [Intermediate Importing Data in Python] of the DataCamp course on Importing Data in Python.\n",
    "\n",
    "The pandas package has been imported as pd and the file 'tweets.csv' is in your current directory for your use.\n",
    "\n",
    "Be aware that this is real data from Twitter and as such there is always a risk that it may contain profanity or other offensive content (in this exercise, and any following exercises that also use real Twitter data).\n",
    "Instructions\n",
    "100 XP\n",
    "\n",
    "    Initialize an empty dictionary counts_dict for storing the results of processing the Twitter data.\n",
    "    Iterate over the 'tweets.csv' file by using a for loop. Use the loop variable chunk and iterate over the call to pd.read_csv() with a chunksize of 10.\n",
    "    In the inner loop, iterate over the column 'lang' in chunk by using a for loop. Use the loop variable entry.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 263,
   "id": "a8bfaf70-8495-4482-81df-5b55c4b3dac1",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'en': 97, 'et': 1, 'und': 2}\n"
     ]
    }
   ],
   "source": [
    "import pandas as pd\n",
    "\n",
    "\n",
    "# Initialize an empty dictionary: counts_dict\n",
    "counts_dict = {}\n",
    "\n",
    "# Iterate over the file chunk by chunk\n",
    "for chunk in pd.read_csv('tweets.csv', chunksize=10):\n",
    "\n",
    "    # Iterate over the column in DataFrame\n",
    "    for entry in chunk['lang']:\n",
    "        if entry in counts_dict.keys():\n",
    "            counts_dict[entry] += 1\n",
    "        else:\n",
    "            counts_dict[entry] = 1\n",
    "\n",
    "# Print the populated dictionary\n",
    "print(counts_dict)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2f2210b9-f8ea-481c-a186-a73afbe7a7ac",
   "metadata": {},
   "source": [
    "## Extracting information for large amounts of Twitter data\n",
    "\n",
    "Great job chunking out that file in the previous exercise. You now know how to deal with situations where you need to process a very large file and that's a very useful skill to have!\n",
    "\n",
    "It's good to know how to process a file in smaller, more manageable chunks, but it can become very tedious having to write and rewrite the same code for the same task each time. In this exercise, you will be making your code more reusable by putting your work in the last exercise in a function definition.\n",
    "\n",
    "The pandas package has been imported as pd and the file 'tweets.csv' is in your current directory for your use.\n",
    "Instructions\n",
    "100 XP\n",
    "\n",
    "    Define the function count_entries(), which has 3 parameters. The first parameter is csv_file for the filename, the second is c_size for the chunk size, and the last is colname for the column name.\n",
    "    Iterate over the file in csv_file file by using a for loop. Use the loop variable chunk and iterate over the call to pd.read_csv(), passing c_size to chunksize.\n",
    "    In the inner loop, iterate over the column given by colname in chunk by using a for loop. Use the loop variable entry.\n",
    "    Call the count_entries() function by passing to it the filename 'tweets.csv', the size of chunks 10, and the name of the column to count, 'lang'. Assign the result of the call to the variable result_counts.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 268,
   "id": "748b3696-e884-447c-adb7-ff23f89a714d",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'en': 97, 'et': 1, 'und': 2}\n"
     ]
    }
   ],
   "source": [
    "# Define count_entries()\n",
    "# *************************************************************************************************** #\n",
    "def count_entries(csv_file, c_size, colname):\n",
    "    \"\"\"Return a dictionary with counts of\n",
    "    occurrences as value for each key.\"\"\"\n",
    "    \n",
    "    # Initialize an empty dictionary: counts_dict\n",
    "    counts_dict = {}\n",
    "\n",
    "    # Iterate over the file chunk by chunk\n",
    "    for chunk in pd.read_csv(csv_file, chunksize=c_size):\n",
    "\n",
    "        # Iterate over the column in DataFrame\n",
    "        for entry in chunk[colname]:\n",
    "            if entry in counts_dict.keys():\n",
    "                counts_dict[entry] += 1\n",
    "            else:\n",
    "                counts_dict[entry] = 1\n",
    "\n",
    "    # Return counts_dict\n",
    "    return counts_dict\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "# Call count_entries(): result_counts\n",
    "result_counts = count_entries('tweets.csv', 10, 'lang')\n",
    "\n",
    "# Print result_counts\n",
    "print(result_counts)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2d7e1ccf-d24b-4f6e-9c1c-b46ec361023b",
   "metadata": {},
   "source": [
    "## Congratulations!\n",
    "\n",
    "\n",
    "\n",
    "# **Congratulations, you've now grappled with nuts and bolts of iterators in PythonLand and surived.  You even applied your newfound skills to an example from the world of big data: loading and working on a file chunk by chunk.  Whats up next, I hear you ask.  Great question.  We've got some of the most important Pythonista tools in store for you, comprehensions and generators.  \n",
    "\n",
    "\n",
    "**These terms may not mean much to you yet, but soon you wont be able to forget them.  In fact you may even want more, because they are that useful.   \n",
    "\n",
    "\n",
    "List comprehensions, for example, allow us to create lists from other lists or from the columns of DataFrames, among many other objects.  Why would we want to do this?  Check this out for example.  \n",
    "\n",
    "# ****************************************************************************************************\n",
    "Lets say we have a DataFrame, with a column of time-stamped data and we want to extract the year from it.  We could extract the column from the DataFrame, loop over its entries using for example a for loop, then within a for loop, extract the desired information.  In that case we probably would have to initialize an empty list in which to store the retrieved data.   \n",
    "\n",
    "A list comprehension however, allows us to perform this entire operation, in a single line of code.  And its waaaay more efficient than the equivalent for loop code.  \n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 301,
   "id": "eb979675-6069-4b00-b1f7-e2dbe53dea2e",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'str'>\n",
      "Tue Mar 29 23:40:17 +0000 2016\n",
      "Index(['contributors', 'coordinates', 'created_at', 'entities',\n",
      "       'extended_entities', 'favorite_count', 'favorited', 'filter_level',\n",
      "       'geo', 'id', 'id_str', 'in_reply_to_screen_name',\n",
      "       'in_reply_to_status_id', 'in_reply_to_status_id_str',\n",
      "       'in_reply_to_user_id', 'in_reply_to_user_id_str', 'is_quote_status',\n",
      "       'lang', 'place', 'possibly_sensitive', 'quoted_status',\n",
      "       'quoted_status_id', 'quoted_status_id_str', 'retweet_count',\n",
      "       'retweeted', 'retweeted_status', 'source', 'text', 'timestamp_ms',\n",
      "       'truncated', 'user'],\n",
      "      dtype='object')\n",
      "2021-11-26 11:38:12.722814\n",
      "\n",
      "[2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016]\n"
     ]
    }
   ],
   "source": [
    "import pandas as pd\n",
    "\n",
    "\n",
    "#tweet_df = pd.read_csv('tweets.csv', chunksize=10)    # In this case, we cant extract the column of DF\n",
    "tweets_df = pd.read_csv('tweets.csv')\n",
    "\n",
    "\n",
    "print(type(tweets_df['created_at'][2]))\n",
    "print(tweets_df['created_at'][2])       #### Tue Mar 29 23:40:17 +0000 2016   Using %d/%m/%Y\n",
    "\n",
    "\n",
    "\n",
    "#######################################################################################################\n",
    "#s = \"2016-03-26T09:25:55.000Z\"\n",
    "#f = \"%Y-%m-%dT%H:%M:%S.%fZ\"\n",
    "#out = datetime.strptime(s, f)\n",
    "#print(out)\n",
    "#output:\n",
    "#2016-03-26 09:25:55\n",
    "#######################################################################################################\n",
    "\n",
    "\n",
    "#######################################################################################################\n",
    "# dtime = tweet['created_at']\n",
    "#dtime = 'Fri Oct 09 10:01:41 +0000 2015'\n",
    "#new_datetime = datetime.strftime(datetime.strptime(dtime,'%a %b %d %H:%M:%S +0000 %Y'), '%Y-%m-%d %H:%M:%S')\n",
    "#print((new_datetime))\n",
    "#######################################################################################################\n",
    "\n",
    "\n",
    "\n",
    "print(tweets_df.columns)\n",
    "\n",
    "\n",
    "\n",
    "from datetime import datetime\n",
    "\n",
    "print(datetime.now())\n",
    "print()\n",
    "\n",
    "\n",
    "\n",
    "years = []\n",
    "\n",
    "for date in tweets_df['created_at']:\n",
    "    datatimeinfo = datetime.strptime(date, '%a %b %d %H:%M:%S +0000 %Y')\n",
    "    years.append(datatimeinfo.year)\n",
    "    #print(datatimeinfo)\n",
    "    #print(datatimeinfo.year)\n",
    "    \n",
    "print(years[:30])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 310,
   "id": "b92a9779-3ef7-43ad-bf6b-6b78d9b8199b",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "deque([2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016])\n"
     ]
    }
   ],
   "source": [
    "def collect_year(csvfile, yearcol):#, csize=10):\n",
    "    '''Returns a list of all year appeared in yearcol of given csvfile\n",
    "    \n",
    "    Args:\n",
    "      csvfile: csv file name\n",
    "      yearcol: the column containing year info\n",
    "      csize: chunksize, incase the file is too large, default 10\n",
    "      \n",
    "    Return:\n",
    "      a deque list containing all the year info of yearcol in given csv file\n",
    "    '''\n",
    "    import pandas as pd\n",
    "    from datetime import datetime\n",
    "    from collections import deque\n",
    "    \n",
    "    outcome = deque()\n",
    "    \n",
    "    df = pd.read_csv(csvfile)\n",
    "    #df = pd.read_csv(csvfile, chunksize=csize)\n",
    "    for i in df[yearcol]:\n",
    "        date = datetime.strptime(i, '%a %b %d %H:%M:%S +0000 %Y')\n",
    "        outcome.append(date.year)\n",
    "        \n",
    "    #date = datetime.strptime(df[yearcol], '%a %b %d %H:%M:%S +0000 %Y')\n",
    "    #outcome.append(date.year)\n",
    "        \n",
    "    return outcome\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "year_list = collect_year('tweets.csv', 'created_at')\n",
    "print(year_list)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 315,
   "id": "fe5d0112-479b-4e7d-8697-795379b41773",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "deque([2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016])\n"
     ]
    }
   ],
   "source": [
    "def collect_year(csvfile, yearcol, csize=10):\n",
    "    '''Returns a list of all year appeared in yearcol of given csvfile\n",
    "    \n",
    "    Args:\n",
    "      csvfile: csv file name\n",
    "      yearcol: the column containing year info\n",
    "      csize: chunksize, incase the file is too large, default 10\n",
    "      \n",
    "    Return:\n",
    "      a deque list containing all the year info of yearcol in given csv file\n",
    "    '''\n",
    "    import pandas as pd\n",
    "    from datetime import datetime\n",
    "    from collections import deque\n",
    "    \n",
    "    outcome = deque()\n",
    "        \n",
    "    ###################################################################################################\n",
    "    #for chunk in pd.read_csv(csv_file, chunksize=c_size):\n",
    "    \n",
    "    for chunk in pd.read_csv(csvfile, chunksize=csize):\n",
    "        for entry in chunk[yearcol]:\n",
    "            date = datetime.strptime(entry, '%a %b %d %H:%M:%S +0000 %Y')\n",
    "            outcome.append(date.year)\n",
    "        \n",
    "    return outcome\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "year_list = collect_year('tweets.csv', 'created_at')\n",
    "print(year_list)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 329,
   "id": "a4cc541f-fc9f-4f04-8c6a-b0cd620119af",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['__add__', '__bool__', '__class__', '__class_getitem__', '__contains__', '__copy__', '__delattr__', '__delitem__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__getitem__', '__gt__', '__hash__', '__iadd__', '__imul__', '__init__', '__init_subclass__', '__iter__', '__le__', '__len__', '__lt__', '__mul__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__reversed__', '__rmul__', '__setattr__', '__setitem__', '__sizeof__', '__str__', '__subclasshook__', 'append', 'appendleft', 'clear', 'copy', 'count', 'extend', 'extendleft', 'index', 'insert', 'maxlen', 'pop', 'popleft', 'remove', 'reverse', 'rotate']\n",
      "[2016, 2016, 2016, 2016, 2016, 2016, 2016]\n"
     ]
    }
   ],
   "source": [
    "year_list[4]\n",
    "\n",
    "print(dir(year_list))\n",
    "\n",
    "#print(year_list.index(20[:20]))\n",
    "#print(help(year_list.index))\n",
    "\n",
    "\n",
    "\n",
    "import itertools\n",
    "\n",
    "new_list = list(itertools.islice(year_list, 0,7))\n",
    "print(new_list)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 319,
   "id": "2315a13d-7b5c-4e00-b8e3-e470777c1ef0",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016]\n",
      "<class 'str'>\n",
      "<class 'str'>\n",
      "<class 'str'>\n",
      "<class 'str'>\n",
      "<class 'str'>\n",
      "<class 'str'>\n",
      "<class 'str'>\n",
      "<class 'str'>\n",
      "<class 'str'>\n",
      "<class 'str'>\n"
     ]
    }
   ],
   "source": [
    "import pandas as pd\n",
    "from datetime import datetime\n",
    "\n",
    "\n",
    "outcome = [datetime.strptime(i, '%a %b %d %H:%M:%S +0000 %Y').year\n",
    " for i in pd.read_csv('tweets.csv')['created_at']]\n",
    "\n",
    "\n",
    "print(outcome[:10])\n",
    "\n",
    "\n",
    "\n",
    "for i in pd.read_csv('tweets.csv')['created_at'][:10]:\n",
    "    print(type(i))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ac8e9e5a-ccdc-447c-aaa5-558134c9b2e6",
   "metadata": {},
   "source": [
    "# ****************************************************************************************************\n",
    "Lets say we have a DataFrame, with a column of time-stamped data and we want to extract the year from it.  We could extract the column from the DataFrame, loop over its entries using for example a for loop, then within a for loop, extract the desired information.  In that case we probably would have to initialize an empty list in which to store the retrieved data.   \n",
    "\n",
    "A list comprehension however, allows us to perform this entire operation, in a single line of code.  And its waaaay more efficient than the equivalent for loop code.  \n",
    "\n",
    "\n",
    "\n",
    "This is one of a number Pythonista Data Science chops that you'll learn in the rest of this course.  \n",
    "\n",
    "Moreover, the entire final chapter will be dedicated to using the skills you're developing, to extract meainingful information from a real-world dataset.  The world abnk development indicators dataset, which according to the world bank, presents the most current and accurate global development data available, and includes national, regional and global estimates. \n",
    "\n",
    "See you in the next chapter. \n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "68e17029-fd1d-4e77-877a-4198596cc602",
   "metadata": {},
   "source": [
    "## List comprehensions\n",
    "\n",
    "\n",
    "\n",
    "**Lets say you have a list of numbers and you want to create a new list of numbers thats the sam eas the old list, except that each number has a 1 added to it.  \n",
    "\n",
    "You could initialize a new empty list, loop through the old list, add 1 to each entry and append all new values to the new list.  But for loop is inefficient, both computationally and in terms of coding time and space, particularly when you could do this in one line of code.  \n",
    "\n",
    "\n",
    "# One line of code\n",
    "\n",
    "**Welcome to the wonderful world of list comprehensions.  The syntax is as follows: within square brackets, you write the values you wish to create, otherwise know as the output expression, followed by the for clause refering to the original list.  \n",
    "\n",
    "So in our case, you open the square bracket, followed by num + 1 for num in nums, and then you close the square bracket.  This is a list comprehenion and creates precisely the desired list.  \n",
    "\n",
    "See below, the relationship between the for loop syntax and list comprehension suntax.  \n",
    "\n",
    "The power of list comprehension is not merely relegated to the world of lists however.  You can write a list comprehension over any iterable.  Below is another example of list comprehension using a range object.  \n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 331,
   "id": "babe7512-4a51-4765-b379-eb84bcd7b848",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[2, 3, 4, 5, 6, 7, 8]"
      ]
     },
     "execution_count": 331,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nums = [1, 2, 3, 4, 5, 6, 7]\n",
    "\n",
    "\n",
    "[num+1 for num in nums]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 332,
   "id": "4cc3c63e-0aa9-4820-943a-5875bb9418d2",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[2, 3, 4, 5, 6, 7, 8]\n"
     ]
    }
   ],
   "source": [
    "new_nums = []\n",
    "\n",
    "for num in nums:\n",
    "    new_nums.append(num + 1)\n",
    "    \n",
    "    \n",
    "print(new_nums)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 333,
   "id": "f5de81f1-d472-4ef4-a83a-d055f7ef6a58",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16]"
      ]
     },
     "execution_count": 333,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "[num+5 for num in range(12)]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4d082e93-1422-4501-8297-7fbdff60958a",
   "metadata": {},
   "source": [
    "# ****************************************************************************************************\n",
    "# To summarize, list comprehensions collapse for loop for building lists into a single line and required componentsre \n",
    "#  1, an iterable, \n",
    "#  2, a iterator variable that represents the members of the iterable, \n",
    "#  3, the output expression.  \n",
    "\n",
    "**You can also use list comprehensions in place of nested for loops, for example, lets say that we want to create a list of all pairs of integers where the first integer is 0 and 1, and second between 6 and 7, below nested for loop would produce the required result.  \n",
    "\n",
    "\n",
    "Can we do the same with a list comprehension? \n",
    "\n",
    "# Once again, within the square brackets, place the desired output expression followed by the two required loop clauses.  \n",
    "\n",
    "You may observe that wile it keeps to a single line of code, we sacrifice some readability of the code as a tradeoff, so you'll have to consider if you'd like to use list comprehensions in cases such as this.  The more often you use this, the more you get used to reading this comprehensions, so readability may not be a problem for you later on.  But do remember that others may have to read your code as well.  \n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 335,
   "id": "3d22b7a3-274d-4656-ab89-058a038013d6",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[(0, 6), (0, 7), (1, 6), (1, 7)]\n"
     ]
    }
   ],
   "source": [
    "pairs = []\n",
    "\n",
    "for i in range(0,2):\n",
    "    for j in range(6,8):\n",
    "        pairs.append((i,j))\n",
    "        \n",
    "        \n",
    "print(pairs)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 336,
   "id": "53cc8ad6-29cd-4f63-974f-7c65c150bc50",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[(0, 6), (0, 7), (1, 6), (1, 7)]"
      ]
     },
     "execution_count": 336,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "[(i,j) for i in range(0,2) for j in range(6,8)]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "600c0c79-5e4c-4fba-97d1-c5c3ed90aedc",
   "metadata": {},
   "source": [
    "## Write a basic list comprehension\n",
    "\n",
    "In this exercise, you will practice what you've learned from the video about writing list comprehensions. You will write a list comprehension and identify the output that will be produced.\n",
    "\n",
    "The following list has been pre-loaded in the environment.\n",
    "\n",
    "doctor = ['house', 'cuddy', 'chase', 'thirteen', 'wilson']\n",
    "\n",
    "How would a list comprehension that produces a list of the first character of each string in doctor look like? Note that the list comprehension uses doc as the iterator variable. What will the output be?\n",
    "Instructions\n",
    "50 XP\n",
    "Possible Answers\n",
    "\n",
    "    The list comprehension is [for doc in doctor: doc[0]] and produces the list ['h', 'c', 'c', 't', 'w'].\n",
    "#    The list comprehension is [doc[0] for doc in doctor] and produces the list ['h', 'c', 'c', 't', 'w'].\n",
    "    The list comprehension is [doc[0] in doctor] and produces the list ['h', 'c', 'c', 't', 'w'].\n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "44fcd5de-ac8d-425d-895a-d3dd97c92e40",
   "metadata": {},
   "source": [
    "## List comprehension over iterables\n",
    "\n",
    "You know that list comprehensions can be built over iterables. Given the following objects below, which of these can we build list comprehensions over?\n",
    "\n",
    "doctor = ['house', 'cuddy', 'chase', 'thirteen', 'wilson']\n",
    "\n",
    "range(50)\n",
    "\n",
    "underwood = 'After all, we are nothing more or less than what we choose to reveal.'\n",
    "\n",
    "jean = '24601'\n",
    "\n",
    "flash = ['jay garrick', 'barry allen', 'wally west', 'bart allen']\n",
    "\n",
    "valjean = 24601\n",
    "\n",
    "Instructions\n",
    "50 XP\n",
    "Possible Answers\n",
    "\n",
    "    You can build list comprehensions over all the objects except the string of number characters jean.\n",
    "    You can build list comprehensions over all the objects except the string lists doctor and flash.\n",
    "    You can build list comprehensions over all the objects except range(50).\n",
    "#    You can build list comprehensions over all the objects except the integer object valjean.\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 337,
   "id": "5174061c-f6cc-4687-8458-09ccb4b5bfee",
   "metadata": {},
   "outputs": [
    {
     "ename": "TypeError",
     "evalue": "'int' object is not iterable",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mTypeError\u001b[0m                                 Traceback (most recent call last)",
      "\u001b[0;32m/tmp/ipykernel_2050/3030663192.py\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[1;32m      2\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      3\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 4\u001b[0;31m \u001b[0;34m[\u001b[0m\u001b[0mi\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mi\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mvaljean\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;31mTypeError\u001b[0m: 'int' object is not iterable"
     ]
    }
   ],
   "source": [
    "valjean = 24601\n",
    "\n",
    "\n",
    "[i[1] for i in valjean]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1eb2b6e8-8aff-4d0f-9467-37fa75d0fdbf",
   "metadata": {},
   "source": [
    "## Writing list comprehensions\n",
    "\n",
    "You now have all the knowledge necessary to begin writing list comprehensions! Your job in this exercise is to write a list comprehension that produces a list of the squares of the numbers ranging from 0 to 9.\n",
    "Instructions\n",
    "100 XP\n",
    "\n",
    "    Using the range of numbers from 0 to 9 as your iterable and i as your iterator variable, write a list comprehension that produces a list of numbers consisting of the squared values of i.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 339,
   "id": "c6e247d9-ea6b-41f1-8f85-123434db9672",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0, 1, 4, 9, 16, 25, 36, 49, 64, 81]\n"
     ]
    }
   ],
   "source": [
    "# Create list comprehension: squares\n",
    "squares = [i**2 for i in range(0,10)]\n",
    "\n",
    "\n",
    "print(squares)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f2417c13-fef3-4e87-b900-a09157c2b142",
   "metadata": {},
   "source": [
    "## Nested list comprehensions\n",
    "\n",
    "Great! At this point, you have a good grasp of the basic syntax of list comprehensions. Let's push your code-writing skills a little further. In this exercise, you will be writing a list comprehension within another list comprehension, or nested list comprehensions. It sounds a little tricky, but you can do it!\n",
    "\n",
    "Let's step aside for a while from strings. One of the ways in which lists can be used are in representing multi-dimension objects such as matrices. Matrices can be represented as a list of lists in Python. For example a 5 x 5 matrix with values 0 to 4 in each row can be written as:\n",
    "\n",
    "matrix = [[0, 1, 2, 3, 4],\n",
    "          [0, 1, 2, 3, 4],\n",
    "          [0, 1, 2, 3, 4],\n",
    "          [0, 1, 2, 3, 4],\n",
    "          [0, 1, 2, 3, 4]]\n",
    "\n",
    "Your task is to recreate this matrix by using nested listed comprehensions. Recall that you can create one of the rows of the matrix with a single list comprehension. To create the list of lists, you simply have to supply the list comprehension as the output expression of the overall list comprehension:\n",
    "\n",
    "[[output expression] for iterator variable in iterable]\n",
    "\n",
    "Note that here, the output expression is itself a list comprehension.\n",
    "Instructions\n",
    "100 XP\n",
    "\n",
    "    In the inner list comprehension - that is, the output expression of the nested list comprehension - create a list of values from 0 to 4 using range(). Use col as the iterator variable.\n",
    "    In the iterable part of your nested list comprehension, use range() to count 5 rows - that is, create a list of values from 0 to 4. Use row as the iterator variable; note that you won't be needing this variable to create values in the list of lists.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 344,
   "id": "759acdd8-22a0-4ae0-a08f-2baf743d8591",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0, 1, 2, 3, 4]\n",
      "[0, 1, 2, 3, 4]\n",
      "[0, 1, 2, 3, 4]\n",
      "[0, 1, 2, 3, 4]\n",
      "[0, 1, 2, 3, 4]\n",
      "[[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]]\n"
     ]
    }
   ],
   "source": [
    "# Create a 5 x 5 matrix using a list of lists: matrix\n",
    "matrix = [[*range(5)] for i in range(5)]\n",
    "\n",
    "# Print the matrix\n",
    "for row in matrix:\n",
    "    print(row)\n",
    "\n",
    "    \n",
    "    \n",
    "print(matrix)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 346,
   "id": "6e27ad04-2a1f-4be1-8528-b847ad79da0c",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0, 1, 2, 3, 4]\n",
      "[0, 1, 2, 3, 4]\n",
      "[0, 1, 2, 3, 4]\n",
      "[0, 1, 2, 3, 4]\n",
      "[0, 1, 2, 3, 4]\n"
     ]
    }
   ],
   "source": [
    "# Create a 5 x 5 matrix using a list of lists: matrix\n",
    "matrix = [[i for i in range(5)] for row in range(5)]\n",
    "\n",
    "# Print the matrix\n",
    "for row in matrix:\n",
    "    print(row)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2ec860b1-a060-4cfc-8e90-e96cc629c062",
   "metadata": {},
   "source": [
    "## Advanced comprehensions\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "**Now that you know the basic of list comprehension, lets check some more advanced comprehension capabilities.  Such as conditionals.  Here we see that we can \n",
    "# filter the output of list comprehension using a conditional on the iterable.  \n",
    "In this example, the resulting list is the square of the values in range(10) under the condition that the value itself is even.  \n",
    "\n",
    "If you have not seen it before, the percentage sign opeartion that you see being used in the comprehension is called the modulo operator.  It produces the remainder from the division of the first argument by the second.  Thus an integer modulo 2 is 0 if and only if the integer is even.  \n",
    "\n",
    "\n",
    "# We can also condition the list comprehension on the output expression.  \n",
    "\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 352,
   "id": "a8894ebc-ea5e-42d2-b07a-3e19514aa2c4",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[0, 4, 16, 36, 64]"
      ]
     },
     "execution_count": 352,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "\n",
    "\n",
    "#                        Filter the output of comprehension using a conditional on the iterable\n",
    "[i**2 for i in range(10) if i%2 == 0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 351,
   "id": "cb286708-6d97-488e-ba45-9a427f65c70e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[0, 0, 8, 0, 64, 0, 216, 0, 512, 0]"
      ]
     },
     "execution_count": 351,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "\n",
    "\n",
    "#     Condition the list comprehension on the output expression\n",
    "[i**3 if i%2 == 0 else 0 for i in range(10)]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "73247eb0-9a78-4962-b604-64b65ded47da",
   "metadata": {},
   "source": [
    "**Now we can also write dictionary comprehensions to create new dictionaries from iterables.  The syntax is almost the same as in list comprehension, and there are 2 differences.  \n",
    "\n",
    "1, we use curly braces instead of square brackets, \n",
    "2, the key and value are separated by a colon in the output expression. \n",
    "\n",
    "\n",
    "Now that we have a grasp on advanced list comprehensions and idctionary comprehensions, its time to practice.  \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 353,
   "id": "ec7a419b-842d-4da7-8b17-40a790e5c970",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{0: 1, 1: 2, 2: 3, 3: 4, 4: 5, 5: 6, 6: 7, 7: 8, 8: 9, 9: 10}"
      ]
     },
     "execution_count": 353,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "{i:i+1 for i in range(10)}"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "33285054-054f-4879-ad28-e2266719c873",
   "metadata": {},
   "source": [
    "## Using conditionals in comprehensions (1)\n",
    "\n",
    "You've been using list comprehensions to build lists of values, sometimes using operations to create these values.\n",
    "\n",
    "# An interesting mechanism in list comprehensions is that you can also create lists with values that meet only a certain condition. One way of doing this is by using conditionals on iterator variables. In this exercise, you will do exactly that!\n",
    "\n",
    "Recall from the video that you can apply a conditional statement to test the iterator variable by adding an if statement in the optional predicate expression part after the for statement in the comprehension:\n",
    "\n",
    "# ****************************************************************************************************\n",
    "\n",
    "[ output expression for iterator variable in iterable if predicate expression ].\n",
    "# ****************************************************************************************************\n",
    "\n",
    "\n",
    "You will use this recipe to write a list comprehension for this exercise. You are given a list of strings fellowship and, using a list comprehension, you will create a list that only includes the members of fellowship that have 7 characters or more.\n",
    "Instructions\n",
    "100 XP\n",
    "\n",
    "    Use member as the iterator variable in the list comprehension. For the conditional, use len() to evaluate the iterator variable. Note that you only want strings with 7 characters or more.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 356,
   "id": "876f1463-bcb3-4920-8bae-26e714b90bf4",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['samwise', 'aragorn', 'legolas', 'boromir']\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "{'frodo': 5,\n",
       " 'samwise': 7,\n",
       " 'merry': 5,\n",
       " 'aragorn': 7,\n",
       " 'legolas': 7,\n",
       " 'boromir': 7,\n",
       " 'gimli': 5}"
      ]
     },
     "execution_count": 356,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Create a list of strings: fellowship\n",
    "fellowship = ['frodo', 'samwise', 'merry', 'aragorn', 'legolas', 'boromir', 'gimli']\n",
    "\n",
    "# Create list comprehension: new_fellowship\n",
    "new_fellowship = [member for member in fellowship if len(member)>6]\n",
    "\n",
    "# Print the new list\n",
    "print(new_fellowship)\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "{i:len(i) for i in fellowship}"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b9c00515-151c-4d66-a53b-e72996c0b599",
   "metadata": {},
   "source": [
    "## Using conditionals in comprehensions (2)\n",
    "\n",
    "In the previous exercise, you used an if conditional statement in the predicate expression part of a list comprehension to evaluate an iterator variable. In this exercise, you will use an if-else statement on the output expression of the list.\n",
    "\n",
    "You will work on the same list, fellowship and, using a list comprehension and an if-else conditional statement in the output expression, create a list that keeps members of fellowship with 7 or more characters and replaces others with an empty string. Use member as the iterator variable in the list comprehension.\n",
    "Instructions\n",
    "100 XP\n",
    "\n",
    "    In the output expression, keep the string as-is if the number of characters is >= 7, else replace it with an empty string - that is, '' or \"\".\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 357,
   "id": "9e0af4af-8022-48d0-a5ea-f235ec574546",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['', 'samwise', '', 'aragorn', 'legolas', 'boromir', '']\n"
     ]
    }
   ],
   "source": [
    "# Create a list of strings: fellowship\n",
    "fellowship = ['frodo', 'samwise', 'merry', 'aragorn', 'legolas', 'boromir', 'gimli']\n",
    "\n",
    "# Create list comprehension: new_fellowship\n",
    "new_fellowship = [member if len(member)>=7 else '' for member in fellowship]\n",
    "\n",
    "# Print the new list\n",
    "print(new_fellowship)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d078870e-f96a-4309-b21b-64933f1651a8",
   "metadata": {},
   "source": [
    "## Dict comprehensions\n",
    "\n",
    "Comprehensions aren't relegated merely to the world of lists. There are many other objects you can build using comprehensions, such as dictionaries, pervasive objects in Data Science. You will create a dictionary using the comprehension syntax for this exercise. In this case, the comprehension is called a dict comprehension.\n",
    "\n",
    "Recall that the main difference between a list comprehension and a dict comprehension is the use of curly braces {} instead of []. Additionally, members of the dictionary are created using a colon :, as in <key> : <value>.\n",
    "\n",
    "You are given a list of strings fellowship and, using a dict comprehension, create a dictionary with the members of the list as the keys and the length of each string as the corresponding values.\n",
    "Instructions\n",
    "100 XP\n",
    "\n",
    "Create a dict comprehension where the key is a string in fellowship and the value is the length of the string. Remember to use the syntax <key> : <value> in the output expression part of the comprehension to create the members of the dictionary. Use member as the iterator variable.\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 358,
   "id": "51a0efaa-4bf6-45e3-9aaa-ffb134665556",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'frodo': 5, 'samwise': 7, 'merry': 5, 'aragorn': 7, 'legolas': 7, 'boromir': 7, 'gimli': 5}\n"
     ]
    }
   ],
   "source": [
    "# Create a list of strings: fellowship\n",
    "fellowship = ['frodo', 'samwise', 'merry', 'aragorn', 'legolas', 'boromir', 'gimli']\n",
    "\n",
    "# Create dict comprehension: new_fellowship\n",
    "new_fellowship = {member:len(member) for member in fellowship}\n",
    "\n",
    "# Print the new dictionary\n",
    "print(new_fellowship)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "af680753-812a-4200-b7dc-f487db542788",
   "metadata": {},
   "source": [
    "## Introduction to generator expressions\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "**Now that you're getting a bit more comfortable with compreshenions, we are going to check generators.  Which are related to comprehensions in a way that will soon become evident.  Recall that this list comprehension will create a list of the first 10 even numbers.  \n",
    "\n",
    "# ***************************************************************************************************\n",
    "Now lets replace the square brackets with round parentheses and voila.  Something called a generator object has been cretaed.  So what is a generator object?  \n",
    "\n",
    "# A generator is like a list comprehension, except it does not store the list in memory, it does not construct the list, but is an object we can iterate over to product elements of the list as required.  \n",
    "\n",
    "Here we can see that looping over a generator expression produces elements of the analogous list.  We can also pass a generator to the function list() to create a list.  Moreover, like any other iterator, we can pass a generator to the function next() in order to iterate through its elements.  \n",
    "\n",
    "\n",
    "\n",
    "\n",
    "# ***************************************************************************************************\n",
    "This is an example of something called lazy evaluation, whereby the evaluation of the expression is deplayed untill its value is needed.  \n",
    "This can help a great deal when working with extremely large sequences as you dont want to store the entire list in memory, which is what comprehensions would do.  You want to generate elements of the sequence on the fly.  \n",
    "\n",
    "\n",
    "Lets say that we want to iterate over a very larget sequence of numbers, such as the integers from 0 upto 10 to the power of a million, or at least want to do so untill another condition was satisfied.  \n",
    "\n",
    "Check what happens when we try to build such an iterable list using a comprehension?  (range() will give you what you want? we can use it when needed, why have to build such a list?)\n",
    "\n",
    "However, we can easily create the analogousgenerator object, because it does not yet create the entire list.  Whats really cool is that anything we can do in a list comprehension, such as filtering and apply conditionals, we can also do in a generator expression.  \n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 369,
   "id": "e0ca601a-0ccd-4405-97ee-7e97a116c892",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<generator object <genexpr> at 0x7f9fab1b7040>\n",
      "0 1 2 3 4 5 6 7 8 9 10 11\n",
      "\n",
      "0 4 16 36 64 100\n",
      "0\n",
      "4\n",
      "16\n",
      "36\n",
      "64\n",
      "100\n",
      "\n",
      "0\n",
      "4\n",
      "16\n"
     ]
    }
   ],
   "source": [
    "(i for i in range(12))\n",
    "\n",
    "\n",
    "outcome = (i for i in range(12))\n",
    "print(outcome)\n",
    "print(*outcome)\n",
    "print()\n",
    "\n",
    "\n",
    "outcome2 = (i**2 for i in range(12) if i%2 == 0)\n",
    "print(*outcome2)\n",
    "\n",
    "for i in (i**2 for i in range(12) if i%2 == 0):\n",
    "    print(i)\n",
    "print()\n",
    "    \n",
    "    \n",
    "    \n",
    "outcome2 = (i**2 for i in range(12) if i%2 == 0)\n",
    "print(next(outcome2))\n",
    "print(next(outcome2))\n",
    "print(next(outcome2))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "524fa314-982e-4305-bc5a-83ec3cacb96f",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[99999840000064, 999997900001469999657, 99999880000036, 999998500000749999875, 99999920000016, 999999100000269999973, 99999960000004, 999999700000029999999]\n",
      "[9999982, 9999984, 9999986, 9999988, 9999990, 9999992, 9999994, 9999996, 9999998]\n"
     ]
    }
   ],
   "source": [
    "large_list = [i for i in range(10*1000000) if i%2==0]   \n",
    "# trying range(10*1000000) will not work, too larg to fit\n",
    "\n",
    "\n",
    "large_generator = (i**2 if i%2==0 else i**3 for i in range(10*1000000))\n",
    "print(list(large_generator)[-8:])\n",
    "\n",
    "\n",
    "\n",
    "print(large_list[-9:])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "dfeb785f-edff-4a64-9c29-51cf3e3a31b5",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0, 1, 2, 3, 4, 5, 6, 7, 8]\n"
     ]
    }
   ],
   "source": [
    "outcome = range(10*100000000)\n",
    "\n",
    "\n",
    "import itertools \n",
    "print(list(itertools.islice(outcome, 0, 9)))\n",
    "\n",
    "\n",
    "\n",
    "#outcome_list = [*outcome]\n",
    "#print(outcome_list[-9:])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3982dc65-f1cb-4a60-8f4f-2afaf1e67490",
   "metadata": {},
   "source": [
    "# ***************************************************************************************************\n",
    "# ***************************************************************************************************\n",
    "# The last to discuss before you get coding is the ability to write a generator functions.  \n",
    "\n",
    "**Generator functions are functions that when called, produce generator objects.  Generator functions are written with syntax of any other user-defined function, however instead of returning values using keyword return, they yield sequence of values using the keyword yield.  \n",
    "\n",
    "Below I have defined a generator function that whenever called with a number n, produces a generator object that generates integers 0 through n.  We can see within the function definition that i  initialized to 0, and that first time the generator object is called, it yields i equal to 0.  It then adds 1 to i and will then yield 1 on the next iteration and so on.  The wile loop is true untill i is equal to n and then the generator ceases to yielld values.  \n",
    "\n",
    "This generator function can be called as you do any other function.  Below I call the generator function with the argument 5.  We see that it prodeces a generator object that we can iterate over this generator object with a for loop to print the values it yields.  \n",
    "\n",
    "\n",
    "# ***************************************************************************************************\n",
    "Generator functions are a powerful and customizable way to create generators.  \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "2ec6d7b9-4068-4f01-b5aa-1f4e07955092",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<generator object num_sequence at 0x7f57ad8809e0>\n",
      "0\n",
      "1\n",
      "2\n",
      "3\n",
      "4\n"
     ]
    }
   ],
   "source": [
    "def num_sequence(n):\n",
    "    '''Generate valuer from 0 to n'''\n",
    "    i = 0\n",
    "    while i < n:\n",
    "        yield i\n",
    "        i += 1\n",
    "        \n",
    "        \n",
    "print(num_sequence(5))\n",
    "\n",
    "for i in num_sequence(5):\n",
    "    print(i)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "98f9acac-61fe-4bbc-96a8-81817386a189",
   "metadata": {},
   "source": [
    "## List comprehensions vs generators\n",
    "\n",
    "You've seen from the videos that list comprehensions and generator expressions look very similar in their syntax, except for the use of parentheses () in generator expressions and brackets [] in list comprehensions.\n",
    "\n",
    "In this exercise, you will recall the difference between list comprehensions and generators. To help with that task, the following code has been pre-loaded in the environment:\n",
    "\n",
    "# List of strings\n",
    "fellowship = ['frodo', 'samwise', 'merry', 'aragorn', 'legolas', 'boromir', 'gimli']\n",
    "\n",
    "# List comprehension\n",
    "fellow1 = [member for member in fellowship if len(member) >= 7]\n",
    "\n",
    "# Generator expression\n",
    "fellow2 = (member for member in fellowship if len(member) >= 7)\n",
    "\n",
    "Try to play around with fellow1 and fellow2 by figuring out their types and printing out their values. Based on your observations and what you can recall from the video, select from the options below the best description for the difference between list comprehensions and generators.\n",
    "Instructions\n",
    "50 XP\n",
    "Possible Answers\n",
    "\n",
    "    List comprehensions and generators are not different at all; they are just different ways of writing the same thing.\n",
    "#    A list comprehension produces a list as output, a generator produces a generator object.\n",
    "    A list comprehension produces a list as output that can be iterated over, a generator produces a generator object that can't be iterated over.\n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "81418bbb-be1b-4afe-a4c9-9894d154d2b9",
   "metadata": {},
   "source": [
    "## Write your own generator expressions\n",
    "\n",
    "You are familiar with what generators and generator expressions are, as well as its difference from list comprehensions. In this exercise, you will practice building generator expressions on your own.\n",
    "\n",
    "# Recall that generator expressions basically have the same syntax as list comprehensions, except that it uses parentheses () instead of brackets []; \n",
    "this should make things feel familiar! Furthermore, if you have ever iterated over a dictionary with .items(), or used the range() function, for example, you have already encountered and used generators before, without knowing it! When you use these functions, Python creates generators for you behind the scenes.\n",
    "\n",
    "Now, you will start simple by creating a generator object that produces numeric values.\n",
    "Instructions\n",
    "100 XP\n",
    "\n",
    "    Create a generator object that will produce values from 0 to 30. Assign the result to result and use num as the iterator variable in the generator expression.\n",
    "    Print the first 5 values by using next() appropriately in print().\n",
    "    Print the rest of the values by using a for loop to iterate over the generator object.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "id": "b337d68b-8ee3-4b63-acdb-a7502950ad45",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0\n",
      "1\n",
      "2\n",
      "3\n",
      "4\n",
      "\n",
      "5\n",
      "6\n",
      "7\n",
      "8\n",
      "9\n",
      "10\n",
      "11\n",
      "12\n",
      "13\n",
      "14\n",
      "15\n",
      "16\n",
      "17\n",
      "18\n",
      "19\n",
      "20\n",
      "21\n",
      "22\n",
      "23\n",
      "24\n",
      "25\n",
      "26\n",
      "27\n",
      "28\n",
      "29\n",
      "30\n"
     ]
    }
   ],
   "source": [
    "# Create generator object: result\n",
    "result = (num for num in range(31))\n",
    "\n",
    "# Print the first 5 values\n",
    "print(result.__next__())\n",
    "print(result.__next__())\n",
    "print(result.__next__())\n",
    "print(result.__next__())\n",
    "print(result.__next__())\n",
    "\n",
    "print()\n",
    "\n",
    "# Cause generator object unpacking once, or used only once\n",
    "# *************************************************************************************************** #\n",
    "# Print the rest of the values\n",
    "for value in result:\n",
    "    print(value)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "93023809-6350-4a2a-bc9c-7ad01840b032",
   "metadata": {},
   "source": [
    "## Changing the output in generator expressions\n",
    "\n",
    "Great! At this point, you already know how to write a basic generator expression. In this exercise, you will push this idea a little further by adding to the output expression of a generator expression. Because generator expressions and list comprehensions are so alike in syntax, this should be a familiar task for you!\n",
    "\n",
    "You are given a list of strings lannister and, using a generator expression, create a generator object that you will iterate over to print its values.\n",
    "Instructions\n",
    "100 XP\n",
    "\n",
    "#    Write a generator expression that will generate the lengths of each string in lannister. Use person as the iterator variable. Assign the result to lengths.\n",
    "    Supply the correct iterable in the for loop for printing the values in the generator object.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "id": "df3c34cf-2264-4ad4-806a-6b7535b6d768",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "6\n",
      "5\n",
      "5\n",
      "6\n",
      "7\n"
     ]
    }
   ],
   "source": [
    "# Create a list of strings: lannister\n",
    "lannister = ['cersei', 'jaime', 'tywin', 'tyrion', 'joffrey']\n",
    "\n",
    "# Create a generator object: lengths\n",
    "lengths = (len(person) for person in lannister)\n",
    "\n",
    "# Iterate over and print the values in lengths\n",
    "for value in lengths:\n",
    "    print(value)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f96df9ad-b5fd-442b-847f-019501f2ac47",
   "metadata": {},
   "source": [
    "## Build a generator\n",
    "\n",
    "In previous exercises, you've dealt mainly with writing generator expressions, which uses comprehension syntax. Being able to use comprehension syntax for generator expressions made your work so much easier!\n",
    "\n",
    "# Now, recall from the video that not only are there generator expressions, there are generator functions as well. Generator functions are functions that, like generator expressions, yield a series of values, instead of returning a single value. A generator function is defined as you do a regular function, but whenever it generates a value, it uses the keyword yield instead of return.\n",
    "\n",
    "In this exercise, you will create a generator function with a similar mechanism as the generator expression you defined in the previous exercise:\n",
    "\n",
    "lengths = (len(person) for person in lannister)\n",
    "\n",
    "Instructions\n",
    "100 XP\n",
    "\n",
    "    Complete the function header for the function get_lengths() that has a single parameter, input_list.\n",
    "    In the for loop in the function definition, yield the length of the strings in input_list.\n",
    "    Complete the iterable part of the for loop for printing the values generated by the get_lengths() generator function. Supply the call to get_lengths(), passing in the list lannister.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "id": "2a28553e-075f-4fce-80eb-6d9769b6e4e5",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "6\n",
      "5\n",
      "5\n",
      "6\n",
      "7\n"
     ]
    }
   ],
   "source": [
    "# Create a list of strings\n",
    "lannister = ['cersei', 'jaime', 'tywin', 'tyrion', 'joffrey']\n",
    "\n",
    "# Define generator function get_lengths\n",
    "def get_lengths(input_list):\n",
    "    \"\"\"Generator function that yields the\n",
    "    length of the strings in input_list.\"\"\"\n",
    "\n",
    "    # Yield the length of a string\n",
    "    for person in input_list:\n",
    "        yield len(person)\n",
    "\n",
    "# Print the values generated by get_lengths()\n",
    "for value in get_lengths(lannister):\n",
    "    print(value)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1fab3ae4-d2c1-4c42-b2df-5623c60b874d",
   "metadata": {},
   "source": [
    "## Wrapping up comprehensions and generators.\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "**Its now time to apply your new and hard-won skills to the Data Science domain.  In the following exercises, you'll use list comprehensions to wrangle important information from the time-stamped data of real tweets.  I want to remind you of the structure and syntax of a list comprehension: \n",
    "\n",
    "In its most basic form, it is enclosed in square brackets and is structed as output expression for iterator variable in iterable.  \n",
    "More advanced comprehensions an include conditionals on the output expression and/or conditionals on the iterable.  \n",
    "\n",
    "\n",
    "After you complete the upcoming exercises, I'll see you in the next chapter.  Therein, wewill not teach any new material.  We'll be consolidating everything that we've learned so far in these 2 Python Date Science Toolbox courses using an extended case study.  This chapter will be dedicated to using the skills you're developing to extract meaningful information from a real-world dataset, the world bank world development indicators dataset.  Happy hacking and see you there.  \n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "64b993d0-3f11-45e1-8a45-5a6d3ad7a152",
   "metadata": {},
   "source": [
    "## List comprehensions for time-stamped data\n",
    "\n",
    "You will now make use of what you've learned from this chapter to solve a simple data extraction problem. You will also be introduced to a data structure, the pandas Series, in this exercise. We won't elaborate on it much here, but what you should know is that it is a data structure that you will be working with a lot of times when analyzing data from pandas DataFrames. You can think of DataFrame columns as single-dimension arrays called Series.\n",
    "\n",
    "In this exercise, you will be using a list comprehension to extract the time from time-stamped Twitter data. The pandas package has been imported as pd and the file 'tweets.csv' has been imported as the df DataFrame for your use.\n",
    "Instructions\n",
    "100 XP\n",
    "\n",
    "    Extract the column 'created_at' from df and assign the result to tweet_time. Fun fact: the extracted column in tweet_time here is a Series data structure!\n",
    "#    Create a list comprehension that extracts the time from each row in tweet_time. Each row is a string that represents a timestamp, and you will access the 12th to 19th characters in the string to extract the time. Use entry as the iterator variable and assign the result to tweet_clock_time. Remember that Python uses 0-based indexing!\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "id": "568501ff-9fbb-47b3-88c1-fd8f96988aa6",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['23:40:17', '23:40:17', '23:40:17', '23:40:17', '23:40:17', '23:40:17', '23:40:18', '23:40:17', '23:40:18', '23:40:18', '23:40:18', '23:40:17', '23:40:18', '23:40:18', '23:40:17', '23:40:18', '23:40:18', '23:40:17', '23:40:18', '23:40:17', '23:40:18', '23:40:18', '23:40:18', '23:40:18', '23:40:17', '23:40:18', '23:40:18', '23:40:17', '23:40:18', '23:40:18', '23:40:18', '23:40:18', '23:40:18', '23:40:18', '23:40:18', '23:40:18', '23:40:18', '23:40:18', '23:40:18', '23:40:18', '23:40:18', '23:40:18', '23:40:18', '23:40:18', '23:40:18', '23:40:18', '23:40:18', '23:40:18', '23:40:18', '23:40:18', '23:40:18', '23:40:18', '23:40:18', '23:40:18', '23:40:18', '23:40:18', '23:40:18', '23:40:18', '23:40:18', '23:40:18', '23:40:19', '23:40:18', '23:40:18', '23:40:18', '23:40:19', '23:40:19', '23:40:19', '23:40:18', '23:40:19', '23:40:19', '23:40:19', '23:40:18', '23:40:19', '23:40:19', '23:40:19', '23:40:18', '23:40:19', '23:40:19', '23:40:19', '23:40:19', '23:40:19', '23:40:19', '23:40:19', '23:40:19', '23:40:19', '23:40:19', '23:40:19', '23:40:19', '23:40:19', '23:40:19', '23:40:19', '23:40:19', '23:40:19', '23:40:19', '23:40:19', '23:40:19', '23:40:19', '23:40:19', '23:40:19', '23:40:19']\n"
     ]
    }
   ],
   "source": [
    "import pandas as pd\n",
    "\n",
    "df = pd.read_csv('tweets.csv')\n",
    "\n",
    "\n",
    "# Extract the created_at column from df: tweet_time\n",
    "tweet_time = df['created_at']\n",
    "\n",
    "# Extract the clock time: tweet_clock_time\n",
    "tweet_clock_time = [entry[11:19] for entry in tweet_time]\n",
    "\n",
    "# Print the extracted times\n",
    "print(tweet_clock_time)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8e060ec6-b445-48e3-a802-ce5b34a744a9",
   "metadata": {},
   "source": [
    "## Conditional list comprehensions for time-stamped data\n",
    "\n",
    "Great, you've successfully extracted the data of interest, the time, from a pandas DataFrame! Let's tweak your work further by adding a conditional that further specifies which entries to select.\n",
    "\n",
    "In this exercise, you will be using a list comprehension to extract the time from time-stamped Twitter data. You will add a conditional expression to the list comprehension so that you only select the times in which entry[17:19] is equal to '19'. The pandas package has been imported as pd and the file 'tweets.csv' has been imported as the df DataFrame for your use.\n",
    "Instructions\n",
    "100 XP\n",
    "\n",
    "    Extract the column 'created_at' from df and assign the result to tweet_time.\n",
    "    Create a list comprehension that extracts the time from each row in tweet_time. Each row is a string that represents a timestamp, and you will access the 12th to 19th characters in the string to extract the time. Use entry as the iterator variable and assign the result to tweet_clock_time. Additionally, add a conditional expression that checks whether entry[17:19] is equal to '19'.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "id": "0b9ea89d-8ea7-45b0-910f-ccea13d72788",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['23:40:19', '23:40:19', '23:40:19', '23:40:19', '23:40:19', '23:40:19', '23:40:19', '23:40:19', '23:40:19', '23:40:19', '23:40:19', '23:40:19', '23:40:19', '23:40:19', '23:40:19', '23:40:19', '23:40:19', '23:40:19', '23:40:19', '23:40:19', '23:40:19', '23:40:19', '23:40:19', '23:40:19', '23:40:19', '23:40:19', '23:40:19', '23:40:19', '23:40:19', '23:40:19', '23:40:19', '23:40:19', '23:40:19', '23:40:19']\n"
     ]
    }
   ],
   "source": [
    "import pandas as pd\n",
    "\n",
    "df = pd.read_csv('tweets.csv')\n",
    "\n",
    "\n",
    "# Extract the created_at column from df: tweet_time\n",
    "tweet_time = df['created_at']\n",
    "\n",
    "# Extract the clock time: tweet_clock_time\n",
    "tweet_clock_time = [entry[11:19] for entry in tweet_time if entry[17:19] == '19']\n",
    "\n",
    "# Print the extracted times\n",
    "print(tweet_clock_time)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "936f40bb-4ca6-4845-bbb5-ceb3e800a6aa",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "28ba21c9-8a00-43bc-8d72-fc03e8864068",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "b67b7b45-4153-4355-85df-90c040ba69f6",
   "metadata": {},
   "source": [
    "## Welcome to the case study!\n",
    "# *****************************************************************************************************\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "**So in this course along with its prequel,  you have learnt a formidable number of series Pythonista Data  Science chops.  Its now time to flex those muscles and see what metal you're made of. \n",
    "\n",
    "\n",
    "In this chapter, you'll use you recently-gained skills of writing user-defined functions, iterators, list comprehensions and generators to wrangle and extract meaningful information from a real-world dataset the world bank world development indicators dataset.  This dataset contains data on 2017 world economies for over half a century, from 1960 up untill 2015.  The data contains hundreads of indicators from population, electricity consumption and co2 emissions to literacy rates, unemployment and mortality rates.  In the following exercises, you'll use dictionaries, zipping, user-defined functions and list comprehensions to wrangle some of this wonderfully rich dataset.  \n",
    "\n",
    "\n",
    "# ****************************************************************************************************\n",
    "# Lets quickly recap these tools: \n",
    "\n",
    "The zip() function accepts an arbitrary number of iterables and return an iterator of tuples, we bring back our two list from before, one of the avengers, the other of their names.  Passing these to the zip function creates a zip object which is an iterator of tuples.  We can turnthem into a list and then print this list.  \n",
    "For writing functions, function headers begin with the keyword def, followed by the function name, arguments inside the parentheses and a colon.  We then have the function body, with the docstrings enclosed in triple quotation marks, the rest of the function body performs the computation that the function does and closes with the keyword return, followed by the value or values to return.  \n",
    "     A quick word on parameters and arguments: when you define a function, you write parameters in the      function header.  When you call the function, you pass arguments into the function.  \n",
    "Comprehensions now, in their most basic forms, are enclosed in square brackets and are atructured as output expression fr iterator variable in iterable.  More advanced comprehensions can include conditionals on the output expression and/or conditionals on the iterable.  \n",
    "\n",
    "Now that we've recapped all the tools that you'll need to hack away at this dataset in the next few exercises, lets get hacking.  \n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "862bac7f-1c27-4f17-98db-83247d2d3bff",
   "metadata": {},
   "source": [
    "## Dictionaries for data science\n",
    "\n",
    "For this exercise, you'll use what you've learned about the zip() function and combine two lists into a dictionary.\n",
    "\n",
    "These lists are actually extracted from a bigger dataset file of world development indicators from the World Bank. For pedagogical purposes, we have pre-processed this dataset into the lists that you'll be working with.\n",
    "\n",
    "The first list feature_names contains header names of the dataset and the second list row_vals contains actual values of a row from the dataset, corresponding to each of the header names.\n",
    "Instructions\n",
    "100 XP\n",
    "\n",
    "    Create a zip object by calling zip() and passing to it feature_names and row_vals. Assign the result to zipped_lists.\n",
    "    Create a dictionary from the zipped_lists zip object by calling dict() with zipped_lists. Assign the resulting dictionary to rs_dict.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "401c9866-8a49-41f9-8749-67548768c23c",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Zip lists: zipped_lists\n",
    "zipped_lists = zip(feature_names, row_vals)\n",
    "\n",
    "# Create a dictionary: rs_dict\n",
    "rs_dict = dict(zipped_lists)\n",
    "\n",
    "# Print the dictionary\n",
    "print(rs_dict)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ad154af0-c2c9-4045-8ab1-0c7fbee580bd",
   "metadata": {},
   "source": [
    "## Writing a function to help you\n",
    "\n",
    "Suppose you needed to repeat the same process done in the previous exercise to many, many rows of data. Rewriting your code again and again could become very tedious, repetitive, and unmaintainable.\n",
    "\n",
    "In this exercise, you will create a function to house the code you wrote earlier to make things easier and much more concise. Why? This way, you only need to call the function and supply the appropriate lists to create your dictionaries! Again, the lists feature_names and row_vals are preloaded and these contain the header names of the dataset and actual values of a row from the dataset, respectively.\n",
    "Instructions\n",
    "100 XP\n",
    "\n",
    "    Define the function lists2dict() with two parameters: first is list1 and second is list2.\n",
    "    Return the resulting dictionary rs_dict in lists2dict().\n",
    "    Call the lists2dict() function with the arguments feature_names and row_vals. Assign the result of the function call to rs_fxn.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "da1af512-6dbb-4404-a112-b5f39bdcfdec",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Define lists2dict()\n",
    "def lists2dict(list1, list2):\n",
    "    \"\"\"Return a dictionary where list1 provides\n",
    "    the keys and list2 provides the values.\"\"\"\n",
    "\n",
    "    # Zip lists: zipped_lists\n",
    "    zipped_lists = zip(list1, list2)   # zip object, an iterator contain (list1_ele, list2_ele) pairs\n",
    "\n",
    "    # Create a dictionary: rs_dict\n",
    "    rs_dict = dict(zipped_lists)       # {list1_ele: list2_ele, list1_ele: list2_ele, ...}\n",
    "\n",
    "    # Return the dictionary\n",
    "    return rs_dict\n",
    "\n",
    "# Call lists2dict: rs_fxn\n",
    "rs_fxn = lists2dict(feature_names, row_vals)\n",
    "\n",
    "# Print rs_fxn\n",
    "print(rs_fxn)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "id": "67388dca-2e98-4bb9-9875-e0db119b86f9",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "        country    capital    area  population\n",
      "0        Brazil   Brasilia   8.516      200.40\n",
      "1        Russia     Moscow  17.100      143.50\n",
      "2         India  New Delhi   3.286     1252.00\n",
      "3         China    Beijing   9.597     1357.00\n",
      "4  South Africa   Pretoria   1.221       52.98\n"
     ]
    }
   ],
   "source": [
    "list_dict = [{'country': 'Brazil', 'capital': 'Brasilia', 'area': 8.516, 'population': 200.4}, \n",
    "             {'country':'Russia', 'capital':'Moscow', 'area':17.10, 'population':143.5}, \n",
    "             {'country':'India', 'capital':'New Delhi', 'area':3.286, 'population':1252}, \n",
    "             {'country':'China', 'capital':'Beijing', 'area':9.597, 'population':1357}, \n",
    "             {'country':'South Africa','capital':'Pretoria','area':1.221, 'population':52.98}]\n",
    "\n",
    "\n",
    "import pandas as pd\n",
    "putcome = pd.DataFrame(list_dict)\n",
    "print(putcome)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f6157dd4-2dca-4557-ab9e-cf05128f148c",
   "metadata": {},
   "source": [
    "## Using a list comprehension\n",
    "\n",
    "This time, you're going to use the lists2dict() function you defined in the last exercise to turn a bunch of lists into a list of dictionaries with the help of a list comprehension.\n",
    "\n",
    "The lists2dict() function has already been preloaded, together with a couple of lists, feature_names and row_lists. feature_names contains the header names of the World Bank dataset and row_lists is a list of lists, where each sublist is a list of actual values of a row from the dataset.\n",
    "\n",
    "# Your goal is to use a list comprehension to generate a list of dicts, where the keys are the header names and the values are the row entries.\n",
    "Instructions\n",
    "100 XP\n",
    "\n",
    "    Inspect the contents of row_lists by printing the first two lists in row_lists.\n",
    "#    Create a list comprehension that generates a dictionary using lists2dict() for each sublist in row_lists. The keys are from the feature_names list and the values are the row entries in row_lists. Use sublist as your iterator variable and assign the resulting list of dictionaries to list_of_dicts.\n",
    "    Look at the first two dictionaries in list_of_dicts by printing them out.\n",
    "\n",
    "Hint\n",
    "\n",
    "    To access the yth element of a list x, do: x[y]. Remember that indexing starts at 0 in Python!\n",
    "    The output expression should be a call to lists2dict(), with feature_names as the first argument and sublist as the second argument. Make sure you iterate over the list of lists row_lists.\n",
    "    You can access the elements of a list of dictionaries the same way as you would any other list.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f260ce83-e10f-4692-bba9-f4691f5ce909",
   "metadata": {},
   "source": [
    "# Below is the wrong approach, think carefully, do not automatically guessing it\n",
    "\n",
    "# ****************************************************************************************************\n",
    "\n",
    "Yes, list do not support vector operation, but we are actually apply() a function - lists2dict() to those list and sublist, and the lists2dict() function takes two list as arguments, so its okay. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "28eacf3a-58b4-48a5-b90f-c08e50162d35",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Print the first two lists in row_lists\n",
    "print(row_lists[0])\n",
    "print(row_lists[1])\n",
    "\n",
    "# Turn list of lists into list of dicts: list_of_dicts\n",
    "list_of_dicts = [lists2dict(key,entry) for key,entry in feature_names,row_lists]\n",
    "#                Here, I should recall what does lists2dict() function takes as arguments\n",
    "# *************************************************************************************************** #\n",
    "\n",
    "\n",
    "# Print the first two dictionaries in list_of_dicts\n",
    "print(list_of_dicts[0])\n",
    "print(list_of_dicts[1])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c86628f5-c98f-4c0c-9ea2-040c269b42aa",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Print the first two lists in row_lists\n",
    "print(row_lists[0])\n",
    "print(row_lists[1])\n",
    "\n",
    "# Turn list of lists into list of dicts: list_of_dicts\n",
    "list_of_dicts = [lists2dict(feature_names, sublist) for sublist in row_lists]\n",
    "\n",
    "# Print the first two dictionaries in list_of_dicts\n",
    "print(list_of_dicts[0])\n",
    "print(list_of_dicts[1])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "71bbea9f-76e0-42fa-be5b-8f52ec4236ba",
   "metadata": {},
   "source": [
    "## Turning this all into a DataFrame\n",
    "\n",
    "You've zipped lists together, created a function to house your code, and even used the function in a list comprehension to generate a list of dictionaries. That was a lot of work and you did a great job!\n",
    "\n",
    "You will now use all of these to convert the list of dictionaries into a pandas DataFrame. You will see how convenient it is to generate a DataFrame from dictionaries with the DataFrame() function from the pandas package.\n",
    "\n",
    "The lists2dict() function, feature_names list, and row_lists list have been preloaded for this exercise.\n",
    "\n",
    "Go for it!\n",
    "Instructions\n",
    "100 XP\n",
    "\n",
    "    To use the DataFrame() function you need, first import the pandas package with the alias pd.\n",
    "    Create a DataFrame from the list of dictionaries in list_of_dicts by calling pd.DataFrame(). Assign the resulting DataFrame to df.\n",
    "    Inspect the contents of df printing the head of the DataFrame. Head of the DataFrame df can be accessed by calling df.head().\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "24c4e6f4-84f8-45b5-a96d-a7d89be2da2c",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Import the pandas package\n",
    "import pandas as pd\n",
    "\n",
    "# Turn list of lists into list of dicts: list_of_dicts\n",
    "list_of_dicts = [lists2dict(feature_names, sublist) for sublist in row_lists]\n",
    "\n",
    "# Turn list of dicts into a DataFrame: df\n",
    "df = pd.DataFrame(list_of_dicts)\n",
    "\n",
    "# Print the head of the DataFrame\n",
    "print(df.head())\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bf957b1b-107e-4f20-8103-ac236d1854c0",
   "metadata": {},
   "source": [
    "## Using Python generators for streaming data\n",
    "\n",
    "# ****************************************************************************************************\n",
    "\n",
    "\n",
    "**Having played arounded with some of the data, we're now going to back a bit and look at importing the dataset, which is pretty large.  We already saw that, for a large dataset, we can use an iterator to load it in chunks.  \n",
    "\n",
    "**We can also write a generator to load it in line by line and one of the really cool things about this method is that if the data is streaming, that is if new lines are being written to the file you're reading , this method will keep on reading and processing the file untill there are no lines left for it to read.  \n",
    "\n",
    "In the following exercises, you'll be writing a generator function that can process large amounts of data, so lets me remind you our typical example of a generator function: Below is a generator function thatn when called with a number n, produces a generator object that generates integers 0 though n.  \n",
    "\n",
    "\n",
    "\n",
    "# ****************************************************************************************************\n",
    "# In the exercises, you'll be writing a generator function that , on each iteration, reads a new line of the file.  \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "e79ceee4-a8bf-4384-b888-582075ec78be",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<generator object num_sequence at 0x7f2ac86ab9e0>\n",
      "['__class__', '__del__', '__delattr__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__iter__', '__le__', '__lt__', '__name__', '__ne__', '__new__', '__next__', '__qualname__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', 'close', 'gi_code', 'gi_frame', 'gi_running', 'gi_yieldfrom', 'send', 'throw']\n",
      "0\n",
      "1\n",
      "2\n",
      "3\n",
      "4\n"
     ]
    }
   ],
   "source": [
    "def num_sequence(n):\n",
    "    '''Generate values though 0 to n'''\n",
    "    i = 0\n",
    "    while i < n:\n",
    "        yield i\n",
    "        i += 1\n",
    "        \n",
    "        \n",
    "        \n",
    "print(num_sequence(8))\n",
    "print(dir(num_sequence(8)))\n",
    "\n",
    "seq_8 = num_sequence(8)\n",
    "print(seq_8.__next__())\n",
    "print(seq_8.__next__())\n",
    "print(seq_8.__next__())\n",
    "print(seq_8.__next__())\n",
    "print(seq_8.__next__())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6983e26a-cbc0-4545-854f-7d693f09e154",
   "metadata": {},
   "source": [
    "## Processing data in chunks (1)\n",
    "\n",
    "# Sometimes, data sources can be so large in size that storing the entire dataset in memory becomes too resource-intensive. In this exercise, you will process the first 1000 rows of a file line by line, to create a dictionary of the counts of how many times each country appears in a column in the dataset.\n",
    "\n",
    "# The csv file 'world_dev_ind.csv' is in your current directory for your use. To begin, you need to open a connection to this file using what is known as a context manager. For example, the command with open('datacamp.csv') as datacamp binds the csv file 'datacamp.csv' as datacamp in the context manager. Here, the with statement is the context manager, and its purpose is to ensure that resources are efficiently allocated when opening a connection to a file.\n",
    "\n",
    "# If you'd like to learn more about context managers, refer to the DataCamp course on Importing Data in Python.\n",
    "Instructions\n",
    "100 XP\n",
    "\n",
    "    Use open() to bind the csv file 'world_dev_ind.csv' as file in the context manager.\n",
    "    Complete the for loop so that it iterates 1000 times to perform the loop body and process only the first 1000 rows of data of the file.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d33b2071-5136-4dea-adfb-34e697dbd40c",
   "metadata": {},
   "source": [
    "# Our world_dev_ind.csv dataset looks like this, we'll pick country column and count their appearance\n",
    "\n",
    "---------------------------------------------------------------------------------------------------\n",
    "CountryName,            | CountryCode, | Year, | Total Population, | Urban population (% of total)\n",
    "Arab World,             | ARB,         | 1960, | 92495902.         | 0,31.285384211605397\n",
    "Caribbean small states, | CSS,         | 1960, | 4190810.          | 0,31.5974898513652"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "85c3db44-9ac6-4e28-83a9-218f36c7bc69",
   "metadata": {},
   "source": [
    "# The .split(',') method applied on a string, separest them into a list, check in below\n",
    "# *****************************************************************************************************\n",
    "# and file_obj.readline() method returns a string\n",
    "\n",
    "\n",
    "\n",
    "#######################################################################################################\n",
    "# You must READ, READ, READ, and image what outcome will return after the code run"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "id": "11b2200a-5f31-4438-b8ca-417fa0d16cff",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Europe & Central Asia (all income levels),ECS,1960,667489033.0,55.378977445327905\n",
      "\n",
      "<class 'str'>\n",
      "['Europe & Central Asia (all income levels)', 'ECS', '1960', '667489033.0', '55.378977445327905\\n']\n"
     ]
    }
   ],
   "source": [
    "with open('world_dev_ind.csv') as file:\n",
    "    file.readline()\n",
    "    for i in range(7):\n",
    "        line = file.readline()\n",
    "    print(line)\n",
    "    print(type(line))\n",
    "        \n",
    "        \n",
    "    out_list = line.split(',')\n",
    "    print(out_list)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "id": "eb0add79-ecb8-440f-b163-d9d10d1fd899",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'Arab World': 1, 'Caribbean small states': 1, 'Central Europe and the Baltics': 1, 'East Asia & Pacific (all income levels)': 1, 'East Asia & Pacific (developing only)': 1, 'Euro area': 1, 'Europe & Central Asia (all income levels)': 1, 'Europe & Central Asia (developing only)': 1, 'European Union': 1, 'Fragile and conflict affected situations': 1, 'Heavily indebted poor countries (HIPC)': 1, 'High income': 1, 'High income: nonOECD': 1, 'High income: OECD': 1, 'Latin America & Caribbean (all income levels)': 1, 'Latin America & Caribbean (developing only)': 1, 'Least developed countries: UN classification': 1, 'Low & middle income': 1, 'Low income': 1, 'Lower middle income': 1, 'Middle East & North Africa (all income levels)': 1, 'Middle East & North Africa (developing only)': 1, 'Middle income': 1, 'North America': 1, 'OECD members': 1, 'Other small states': 1, 'Pacific island small states': 1, 'Small states': 1, 'South Asia': 1, 'Sub-Saharan Africa (all income levels)': 1, 'Sub-Saharan Africa (developing only)': 1, 'Upper middle income': 1, 'World': 1, 'Afghanistan': 1, 'Albania': 1, 'Algeria': 1, 'American Samoa': 1, 'Andorra': 1, 'Angola': 1, 'Antigua and Barbuda': 1, 'Argentina': 1, 'Armenia': 1, 'Aruba': 1, 'Australia': 1, 'Austria': 1, 'Azerbaijan': 1, '\"Bahamas': 1, 'Bahrain': 1, 'Bangladesh': 1, 'Barbados': 1, 'Belarus': 1, 'Belgium': 1, 'Belize': 1, 'Benin': 1, 'Bermuda': 1, 'Bhutan': 1, 'Bolivia': 1, 'Bosnia and Herzegovina': 1, 'Botswana': 1, 'Brazil': 1, 'Brunei Darussalam': 1, 'Bulgaria': 1, 'Burkina Faso': 1, 'Burundi': 1, 'Cabo Verde': 1, 'Cambodia': 1, 'Cameroon': 1, 'Canada': 1, 'Cayman Islands': 1, 'Central African Republic': 1, 'Chad': 1, 'Channel Islands': 1, 'Chile': 1, 'China': 1, 'Colombia': 1, 'Comoros': 1, '\"Congo': 2, 'Costa Rica': 1, \"Cote d'Ivoire\": 1, 'Croatia': 1, 'Cuba': 1, 'Curacao': 1, 'Cyprus': 1, 'Czech Republic': 1, 'Denmark': 1, 'Djibouti': 1, 'Dominica': 1, 'Dominican Republic': 1, 'Ecuador': 1, '\"Egypt': 1, 'El Salvador': 1, 'Equatorial Guinea': 1, 'Eritrea': 1, 'Estonia': 1, 'Ethiopia': 1, 'Faeroe Islands': 1, 'Fiji': 1, 'Finland': 1, 'France': 1}\n"
     ]
    }
   ],
   "source": [
    "# Open a connection to the file\n",
    "with open('world_dev_ind.csv') as file:\n",
    "\n",
    "    # Skip the column names   ##################################################\n",
    "    file.readline()     # Why not using next(file) to skip the first column, check in below code box\n",
    "\n",
    "    # Initialize an empty dictionary: counts_dict\n",
    "    counts_dict = {}\n",
    "\n",
    "    # Process only the first 1000 rows\n",
    "    for j in range(100):\n",
    "\n",
    "        # Split the current line into a list: line   # *******************************************\n",
    "        line_list = file.readline().split(',')            # readline() method used on file object\n",
    "        ###print(line)   ### ['Arab World', 'ARB', '1960', '92495902.0', '31.285384211605397\\n']\n",
    "        # Get the value for the first column: first_col\n",
    "        first_col = line_list[0]  # file_obj.readline() -> string, string.split(',') -> list of str \n",
    "                                  ##################################################################\n",
    "        # If the column value is in the dict, increment its value\n",
    "        if first_col in counts_dict.keys():\n",
    "            counts_dict[first_col] += 1\n",
    "\n",
    "        # Else, add to the dict and set value to 1\n",
    "        else:\n",
    "            counts_dict[first_col] = 1\n",
    "\n",
    "# Print the resulting dictionary\n",
    "print(counts_dict)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "id": "49e56355-e125-45f4-b311-c077675299cc",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'Arab World': 2, 'Caribbean small states': 2, 'Central Europe and the Baltics': 2, 'East Asia & Pacific (all income levels)': 2, 'East Asia & Pacific (developing only)': 2, 'Euro area': 2, 'Europe & Central Asia (all income levels)': 2, 'Europe & Central Asia (developing only)': 2, 'European Union': 2, 'Fragile and conflict affected situations': 2}\n"
     ]
    }
   ],
   "source": [
    "with open('world_dev_ind.csv') as file:\n",
    "    file.readline() # or using next(file)\n",
    "    \n",
    "    counts_dict = {}\n",
    "    for j in range(10):\n",
    "\n",
    "        line_list = file.readline().split(',')\n",
    "        first_col = line_list[0]\n",
    "\n",
    "        if first_col not in counts_dict.keys():\n",
    "            counts_dict[first_col] = 1\n",
    "\n",
    "        counts_dict[first_col] += 1\n",
    "\n",
    "\n",
    "\n",
    "print(counts_dict)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "385f977f-c6c1-4a5a-b0f1-81cae58fb15a",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[('1', 'Bulbasaur', 'Grass', 'Poison'), ('2', 'Ivysaur', 'Grass', 'Poison'), ('3', 'Venusaur', 'Grass', 'Poison'), ('4', 'Charmander', 'Fire', 'Nan'), ('5', 'Charmeleon', 'Fire', 'Nan')]\n"
     ]
    }
   ],
   "source": [
    "import csv\n",
    "\n",
    "\n",
    "with open('PokeMon_List2.csv', 'r') as file_obj:\n",
    "    csv_reader = csv.reader(file_obj)\n",
    "    next(csv_reader)   # Working code, skips the first row of the csv file\n",
    "    list_of_tuples = list(map(tuple, csv_reader))\n",
    "\n",
    "\n",
    "print(list_of_tuples[:5])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "09b25701-aeee-4806-a760-6e0e97c0212f",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2,Ivysaur,Grass,Poison\n",
      "\n",
      "[('3', 'Venusaur', 'Grass', 'Poison'), ('4', 'Charmander', 'Fire', 'Nan'), ('5', 'Charmeleon', 'Fire', 'Nan'), ('6', 'Charizard', 'Fire', 'Flying'), ('7', 'Squirtle', 'Water', 'Nan')]\n"
     ]
    }
   ],
   "source": [
    "import csv\n",
    "\n",
    "\n",
    "with open('PokeMon_List2.csv', 'r') as file_obj:\n",
    "    file_obj.readline()   # everytime we call file_obj.readline(), it moves one line after, like next()\n",
    "    file_obj.readline()\n",
    "    ff = file_obj.readline()\n",
    "    print(ff)   # Idex,Pokemon,Type,Secondary Type\n",
    "    csv_reader = csv.reader(file_obj)\n",
    "    #next(csv_reader)   # Working code, skips the first row of the csv file\n",
    "    list_of_tuples = list(map(tuple, csv_reader))\n",
    "\n",
    "\n",
    "print(list_of_tuples[:5])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "150d4927-adc9-462e-be68-3d5b00316c16",
   "metadata": {},
   "source": [
    "## Writing a generator to load data in chunks (2)\n",
    "\n",
    "In the previous exercise, you processed a file line by line for a given number of lines. What if, however, you want to do this for the entire file?\n",
    "\n",
    "In this case, it would be useful to use generators. Generators allow users to lazily evaluate data. This concept of lazy evaluation is useful when you have to deal with very large datasets because it lets you generate values in an efficient manner by yielding only chunks of data at a time instead of the whole thing at once.\n",
    "\n",
    "In this exercise, you will define a generator function read_large_file() that produces a generator object which yields a single line from a file each time next() is called on it. The csv file 'world_dev_ind.csv' is in your current directory for your use.\n",
    "\n",
    "Note that when you open a connection to a file, the resulting file object is already a generator! So out in the wild, you won't have to explicitly create generator objects in cases such as this. However, for pedagogical reasons, we are having you practice how to do this here with the read_large_file() function. Go for it!\n",
    "Instructions\n",
    "100 XP\n",
    "\n",
    "    In the function read_large_file(), read a line from file_object by using the method readline(). Assign the result to data.\n",
    "    In the function read_large_file(), yield the line read from the file data.\n",
    "#    In the context manager, create a generator object gen_file by calling your generator function read_large_file() and passing file to it.\n",
    "    Print the first three lines produced by the generator object gen_file using next().\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "3ea55cd9-37af-4e72-8372-bdad8de3cd67",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CountryName,CountryCode,Year,Total Population,Urban population (% of total)\n",
      "\n",
      "Arab World,ARB,1960,92495902.0,31.285384211605397\n",
      "\n",
      "Caribbean small states,CSS,1960,4190810.0,31.5974898513652\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# Define read_large_file()\n",
    "def read_large_file(file_object):\n",
    "    \"\"\"A generator function to read a large file lazily.\"\"\"\n",
    "\n",
    "    # Loop indefinitely until the end of the file\n",
    "    while True:\n",
    "\n",
    "        # Read a line from the file: data\n",
    "        data = file_object.readline()\n",
    "\n",
    "        # Break if this is the end of the file\n",
    "        if not data:\n",
    "            break\n",
    "\n",
    "        # Yield the line of data\n",
    "        yield data\n",
    "        \n",
    "# Open a connection to the file\n",
    "with open('world_dev_ind.csv') as file:\n",
    "\n",
    "    # Create a generator object for the file: gen_file\n",
    "    gen_file = read_large_file(file)\n",
    "\n",
    "    # Print the first three lines of the file\n",
    "    print(gen_file.__next__())\n",
    "    print(gen_file.__next__())\n",
    "    print(gen_file.__next__())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8caa2d13-3fb8-4558-a739-273e4bc3d0b5",
   "metadata": {},
   "source": [
    "## Writing a generator to load data in chunks (3)\n",
    "\n",
    "Great! You've just created a generator function that you can use to help you process large files.\n",
    "\n",
    "Now let's use your generator function to process the World Bank dataset like you did previously. You will process the file line by line, to create a dictionary of the counts of how many times each country appears in a column in the dataset. For this exercise, however, you won't process just 1000 rows of data, you'll process the entire dataset!\n",
    "\n",
    "The generator function read_large_file() and the csv file 'world_dev_ind.csv' are preloaded and ready for your use. Go for it!\n",
    "Instructions\n",
    "100 XP\n",
    "\n",
    "    Bind the file 'world_dev_ind.csv' to file in the context manager with open().\n",
    "    Complete the for loop so that it iterates over the generator from the call to read_large_file() to process all the rows of the file.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 97,
   "id": "3edff6db-6f11-4994-bde1-3adee48a12f6",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'CountryName': 1, 'Arab World': 55, 'Caribbean small states': 55, 'Central Europe and the Baltics': 55, 'East Asia & Pacific (all income levels)': 55, 'East Asia & Pacific (developing only)': 55, 'Euro area': 55, 'Europe & Central Asia (all income levels)': 55, 'Europe & Central Asia (developing only)': 55, 'European Union': 55, 'Fragile and conflict affected situations': 55, 'Heavily indebted poor countries (HIPC)': 55, 'High income': 55, 'High income: nonOECD': 55, 'High income: OECD': 55, 'Latin America & Caribbean (all income levels)': 55, 'Latin America & Caribbean (developing only)': 55, 'Least developed countries: UN classification': 55, 'Low & middle income': 55, 'Low income': 55, 'Lower middle income': 55, 'Middle East & North Africa (all income levels)': 55, 'Middle East & North Africa (developing only)': 55, 'Middle income': 55, 'North America': 55, 'OECD members': 55, 'Other small states': 55, 'Pacific island small states': 55, 'Small states': 55, 'South Asia': 55, 'Sub-Saharan Africa (all income levels)': 55, 'Sub-Saharan Africa (developing only)': 55, 'Upper middle income': 55, 'World': 55, 'Afghanistan': 55, 'Albania': 55, 'Algeria': 55, 'American Samoa': 55, 'Andorra': 55, 'Angola': 55, 'Antigua and Barbuda': 55, 'Argentina': 55, 'Armenia': 55, 'Aruba': 55, 'Australia': 55, 'Austria': 55, 'Azerbaijan': 55, '\"Bahamas': 55, 'Bahrain': 55, 'Bangladesh': 55, 'Barbados': 55, 'Belarus': 55, 'Belgium': 55, 'Belize': 55, 'Benin': 55, 'Bermuda': 55, 'Bhutan': 55, 'Bolivia': 55, 'Bosnia and Herzegovina': 55, 'Botswana': 55, 'Brazil': 55, 'Brunei Darussalam': 55, 'Bulgaria': 55, 'Burkina Faso': 55, 'Burundi': 55, 'Cabo Verde': 55, 'Cambodia': 55, 'Cameroon': 55, 'Canada': 55, 'Cayman Islands': 55, 'Central African Republic': 55, 'Chad': 55, 'Channel Islands': 55, 'Chile': 55, 'China': 55, 'Colombia': 55, 'Comoros': 55, '\"Congo': 110, 'Costa Rica': 55, \"Cote d'Ivoire\": 55, 'Croatia': 55, 'Cuba': 55, 'Curacao': 55, 'Cyprus': 55, 'Czech Republic': 55, 'Denmark': 55, 'Djibouti': 55, 'Dominica': 55, 'Dominican Republic': 55, 'Ecuador': 55, '\"Egypt': 55, 'El Salvador': 55, 'Equatorial Guinea': 55, 'Eritrea': 55, 'Estonia': 55, 'Ethiopia': 55, 'Faeroe Islands': 55, 'Fiji': 55, 'Finland': 55, 'France': 55, 'French Polynesia': 55, 'Gabon': 55, '\"Gambia': 55, 'Georgia': 55, 'Germany': 55, 'Ghana': 55, 'Greece': 55, 'Greenland': 55, 'Grenada': 55, 'Guam': 55, 'Guatemala': 55, 'Guinea': 55, 'Guinea-Bissau': 55, 'Guyana': 55, 'Haiti': 55, 'Honduras': 55, '\"Hong Kong SAR': 55, 'Hungary': 55, 'Iceland': 55, 'India': 55, 'Indonesia': 55, '\"Iran': 55, 'Iraq': 55, 'Ireland': 55, 'Isle of Man': 55, 'Israel': 55, 'Italy': 55, 'Jamaica': 55, 'Japan': 55, 'Jordan': 55, 'Kazakhstan': 55, 'Kenya': 55, 'Kiribati': 55, '\"Korea': 110, 'Kuwait': 52, 'Kyrgyz Republic': 55, 'Lao PDR': 55, 'Latvia': 55, 'Lebanon': 55, 'Lesotho': 55, 'Liberia': 55, 'Libya': 55, 'Liechtenstein': 55, 'Lithuania': 55, 'Luxembourg': 55, '\"Macao SAR': 55, '\"Macedonia': 55, 'Madagascar': 55, 'Malawi': 55, 'Malaysia': 55, 'Maldives': 55, 'Mali': 55, 'Malta': 55, 'Marshall Islands': 55, 'Mauritania': 55, 'Mauritius': 55, 'Mexico': 55, '\"Micronesia': 55, 'Moldova': 55, 'Monaco': 55, 'Mongolia': 55, 'Montenegro': 55, 'Morocco': 55, 'Mozambique': 55, 'Myanmar': 55, 'Namibia': 55, 'Nepal': 55, 'Netherlands': 55, 'New Caledonia': 55, 'New Zealand': 55, 'Nicaragua': 55, 'Niger': 55, 'Nigeria': 55, 'Northern Mariana Islands': 55, 'Norway': 55, 'Oman': 55, 'Pakistan': 55, 'Palau': 55, 'Panama': 55, 'Papua New Guinea': 55, 'Paraguay': 55, 'Peru': 55, 'Philippines': 55, 'Poland': 55, 'Portugal': 55, 'Puerto Rico': 55, 'Qatar': 55, 'Romania': 55, 'Russian Federation': 55, 'Rwanda': 55, 'Samoa': 55, 'San Marino': 55, 'Sao Tome and Principe': 55, 'Saudi Arabia': 55, 'Senegal': 55, 'Seychelles': 55, 'Sierra Leone': 55, 'Singapore': 55, 'Slovak Republic': 55, 'Slovenia': 55, 'Solomon Islands': 55, 'Somalia': 55, 'South Africa': 55, 'South Sudan': 55, 'Spain': 55, 'Sri Lanka': 55, 'St. Kitts and Nevis': 55, 'St. Lucia': 55, 'St. Vincent and the Grenadines': 55, 'Sudan': 55, 'Suriname': 55, 'Swaziland': 55, 'Sweden': 55, 'Switzerland': 55, 'Syrian Arab Republic': 55, 'Tajikistan': 55, 'Tanzania': 55, 'Thailand': 55, 'Timor-Leste': 55, 'Togo': 55, 'Tonga': 55, 'Trinidad and Tobago': 55, 'Tunisia': 55, 'Turkey': 55, 'Turkmenistan': 55, 'Turks and Caicos Islands': 55, 'Tuvalu': 55, 'Uganda': 55, 'Ukraine': 55, 'United Arab Emirates': 55, 'United Kingdom': 55, 'United States': 55, 'Uruguay': 55, 'Uzbekistan': 55, 'Vanuatu': 55, '\"Venezuela': 55, 'Vietnam': 55, 'Virgin Islands (U.S.)': 55, '\"Yemen': 55, 'Zambia': 55, 'Zimbabwe': 55, 'Serbia': 25, 'West Bank and Gaza': 25, 'Sint Maarten (Dutch part)': 17}\n",
      "Help on function most_common in module collections:\n",
      "\n",
      "most_common(self, n=None)\n",
      "    List the n most common elements and their counts from the most\n",
      "    common to the least.  If n is None, then list all element counts.\n",
      "    \n",
      "    >>> Counter('abracadabra').most_common(3)\n",
      "    [('a', 5), ('b', 2), ('r', 2)]\n",
      "\n",
      "None\n"
     ]
    }
   ],
   "source": [
    "# Initialize an empty dictionary: counts_dict\n",
    "counts_dict = {}\n",
    "\n",
    "# Open a connection to the file\n",
    "with open('world_dev_ind.csv') as file:\n",
    "\n",
    "    # Iterate over the generator from read_large_file()\n",
    "    for line in read_large_file(file):\n",
    "\n",
    "        row = line.split(',')   # Here str.solit() method convert line data into a list\n",
    "        first_col = row[0]\n",
    "\n",
    "        if first_col in counts_dict.keys():\n",
    "            counts_dict[first_col] += 1\n",
    "        else:\n",
    "            counts_dict[first_col] = 1\n",
    "\n",
    "# Print            \n",
    "print(counts_dict)\n",
    "\n",
    "\n",
    "from collections import Counter\n",
    "print(help(Counter.most_common))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 96,
   "id": "ba6f2aef-5747-49e2-9d13-b4275bf86cb5",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['__add__', '__class__', '__contains__', '__delattr__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__getitem__', '__getnewargs__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__iter__', '__le__', '__len__', '__lt__', '__mod__', '__mul__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__rmod__', '__rmul__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', 'capitalize', 'casefold', 'center', 'count', 'encode', 'endswith', 'expandtabs', 'find', 'format', 'format_map', 'index', 'isalnum', 'isalpha', 'isascii', 'isdecimal', 'isdigit', 'isidentifier', 'islower', 'isnumeric', 'isprintable', 'isspace', 'istitle', 'isupper', 'join', 'ljust', 'lower', 'lstrip', 'maketrans', 'partition', 'removeprefix', 'removesuffix', 'replace', 'rfind', 'rindex', 'rjust', 'rpartition', 'rsplit', 'rstrip', 'split', 'splitlines', 'startswith', 'strip', 'swapcase', 'title', 'translate', 'upper', 'zfill']\n",
      "True\n",
      "Help on method_descriptor:\n",
      "\n",
      "split(self, /, sep=None, maxsplit=-1)\n",
      "    Return a list of the words in the string, using sep as the delimiter string.\n",
      "    \n",
      "    sep\n",
      "      The delimiter according which to split the string.\n",
      "      None (the default value) means split according to any whitespace,\n",
      "      and discard empty strings from the result.\n",
      "    maxsplit\n",
      "      Maximum number of splits to do.\n",
      "      -1 (the default value) means no limit.\n",
      "\n",
      "None\n"
     ]
    }
   ],
   "source": [
    "print(dir(str))\n",
    "\n",
    "\n",
    "print('split' in dir(str))\n",
    "\n",
    "print(help(str.split))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "29977349-180b-49c4-9f68-aa01080da921",
   "metadata": {},
   "source": [
    "# Do you still remember which function comes from which module?\n",
    "We have collections, itertools, and??????"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f09380de-bab8-411e-92bd-126f97ef4dad",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "6b56dade-2e14-4120-ae58-e21a5cb3d267",
   "metadata": {},
   "source": [
    "## Using pandas' read_csv iterator for streaming data\n",
    "\n",
    "\n",
    "\n",
    "**You are on the howe stretch now.  That was some great work with generators to read in large and/or streaming files.  In the remaining exercises, you'll revisit the Pandas function read_csv() along with its chunksize argument in order to read in dataset in chunks, process each chunk and aggregate.  \n",
    "\n",
    "**Herein, you'll do so in order to look at specific indicators in specific countries and to plot the results over all the years for which the relevant data exists.  You'll then try your hand at writing a function to perform the same task which will save so much work when you want to look at the data for a different country.  \n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "177311e8-775b-4498-be5a-86d364f89f53",
   "metadata": {},
   "source": [
    "## Writing an iterator to load data in chunks (1)\n",
    "\n",
    "Another way to read data too large to store in memory in chunks is to read the file in as DataFrames of a certain length, say, 100. For example, with the pandas package (imported as pd), you can do pd.read_csv(filename, chunksize=100). This creates an iterable reader object, which means that you can use next() on it.\n",
    "\n",
    "In this exercise, you will read a file in small DataFrame chunks with read_csv(). You're going to use the World Bank Indicators data 'ind_pop.csv', available in your current directory, to look at the urban population indicator for numerous countries and years.\n",
    "Instructions\n",
    "100 XP\n",
    "\n",
    "    Use pd.read_csv() to read in 'ind_pop.csv' in chunks of size 10. Assign the result to df_reader.\n",
    "    Print the first two chunks from df_reader.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "id": "f373a3d3-89a3-4b23-a314-db4baa7a766f",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<pandas.io.parsers.readers.TextFileReader object at 0x7f2a8e148670>\n",
      "<pandas.io.parsers.readers.TextFileReader object at 0x7f2a8e148670>\n",
      "\n",
      "True\n",
      "                               CountryName CountryCode  Year  \\\n",
      "0                               Arab World         ARB  1960   \n",
      "1                   Caribbean small states         CSS  1960   \n",
      "2           Central Europe and the Baltics         CEB  1960   \n",
      "3  East Asia & Pacific (all income levels)         EAS  1960   \n",
      "4    East Asia & Pacific (developing only)         EAP  1960   \n",
      "\n",
      "   Total Population  Urban population (% of total)  \n",
      "0      9.249590e+07                      31.285384  \n",
      "1      4.190810e+06                      31.597490  \n",
      "2      9.140158e+07                      44.507921  \n",
      "3      1.042475e+09                      22.471132  \n",
      "4      8.964930e+08                      16.917679  \n",
      "                                 CountryName CountryCode  Year  \\\n",
      "5                                  Euro area         EMU  1960   \n",
      "6  Europe & Central Asia (all income levels)         ECS  1960   \n",
      "7    Europe & Central Asia (developing only)         ECA  1960   \n",
      "8                             European Union         EUU  1960   \n",
      "9   Fragile and conflict affected situations         FCS  1960   \n",
      "\n",
      "   Total Population  Urban population (% of total)  \n",
      "5       265396501.0                      62.096947  \n",
      "6       667489033.0                      55.378977  \n",
      "7       155317369.0                      38.066129  \n",
      "8       409498462.0                      61.212898  \n",
      "9       120354582.0                      17.891972  \n"
     ]
    }
   ],
   "source": [
    "# Import the pandas package\n",
    "import pandas as pd\n",
    "\n",
    "# Initialize reader object: df_reader\n",
    "df_reader = pd.read_csv('world_dev_ind.csv', chunksize=5)\n",
    "\n",
    "# Print two chunks   ##############################################################################\n",
    "print(df_reader)     # Cause when you iterate it, it read a(another) chunk of data, lazy iterator\n",
    "print(df_reader)\n",
    "print()\n",
    "print('__iter__' and '__next__' in dir(df_reader))   # Its a iterator\n",
    "\n",
    "print(next(df_reader))\n",
    "print(next(df_reader))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3ff4f147-13fa-4976-9243-1f31c28cfeea",
   "metadata": {},
   "source": [
    "## Writing an iterator to load data in chunks (2)\n",
    "\n",
    "In the previous exercise, you used read_csv() to read in DataFrame chunks from a large dataset. In this exercise, you will read in a file using a bigger DataFrame chunk size and then process the data from the first chunk.\n",
    "\n",
    "To process the data, you will create another DataFrame composed of only the rows from a specific country. You will then zip together two of the columns from the new DataFrame, 'Total Population' and 'Urban population (% of total)'. Finally, you will create a list of tuples from the zip object, where each tuple is composed of a value from each of the two columns mentioned.\n",
    "\n",
    "You're going to use the data from 'ind_pop_data.csv', available in your current directory. Pandas has been imported as pd.\n",
    "Instructions\n",
    "100 XP\n",
    "\n",
    "    Use pd.read_csv() to read in the file in 'ind_pop_data.csv' in chunks of size 1000. Assign the result to urb_pop_reader.\n",
    "    Get the first DataFrame chunk from the iterable urb_pop_reader and assign this to df_urb_pop.\n",
    "#    Select only the rows of df_urb_pop that have a 'CountryCode' of 'CEB'. To do this, compare whether df_urb_pop['CountryCode'] is equal to 'CEB' within the square brackets in df_urb_pop[____].\n",
    "    Using zip(), zip together the 'Total Population' and 'Urban population (% of total)' columns of df_pop_ceb. Assign the resulting zip object to pops.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "14df1cbf-bd72-4844-ad08-4272d270d0d5",
   "metadata": {},
   "source": [
    "# boolean indexing"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "id": "e1c0b9bb-f86c-4ec0-b4c2-4d1b19505081",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "                               CountryName CountryCode  Year  \\\n",
      "0                               Arab World         ARB  1960   \n",
      "1                   Caribbean small states         CSS  1960   \n",
      "2           Central Europe and the Baltics         CEB  1960   \n",
      "3  East Asia & Pacific (all income levels)         EAS  1960   \n",
      "4    East Asia & Pacific (developing only)         EAP  1960   \n",
      "\n",
      "   Total Population  Urban population (% of total)  \n",
      "0      9.249590e+07                      31.285384  \n",
      "1      4.190810e+06                      31.597490  \n",
      "2      9.140158e+07                      44.507921  \n",
      "3      1.042475e+09                      22.471132  \n",
      "4      8.964930e+08                      16.917679  \n",
      "[(91401583.0, 44.5079211390026), (92237118.0, 45.206665319194), (93014890.0, 45.866564696018), (93845749.0, 46.5340927663649), (94722599.0, 47.2087429803526)]\n"
     ]
    }
   ],
   "source": [
    "# Initialize reader object: urb_pop_reader\n",
    "urb_pop_reader = pd.read_csv('ind_pop_data.csv', chunksize=1000)\n",
    "\n",
    "# Get the first DataFrame chunk: df_urb_pop\n",
    "df_urb_pop = next(urb_pop_reader)\n",
    "\n",
    "# Check out the head of the DataFrame\n",
    "print(df_urb_pop.head())\n",
    "\n",
    "\n",
    "# Check out specific country: df_pop_ceb\n",
    "df_pop_ceb = df_urb_pop[df_urb_pop['CountryCode']=='CEB']   # boolean indexing\n",
    "\n",
    "# Zip DataFrame columns of interest: pops\n",
    "pops = zip(df_pop_ceb['Total Population'], df_pop_ceb['Urban population (% of total)'])\n",
    "\n",
    "# Turn zip object into list: pops_list\n",
    "pops_list = list(pops)\n",
    "\n",
    "# Print pops_list\n",
    "print(pops_list)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "56382e7f-c3ae-4c5a-a971-a2c797a7fd2d",
   "metadata": {},
   "source": [
    "## Writing an iterator to load data in chunks (3)\n",
    "\n",
    "You're getting used to reading and processing data in chunks by now. Let's push your skills a little further by adding a column to a DataFrame.\n",
    "\n",
    "Starting from the code of the previous exercise, you will be using a list comprehension to create the values for a new column 'Total Urban Population' from the list of tuples that you generated earlier. Recall from the previous exercise that the first and second elements of each tuple consist of, respectively, values from the columns 'Total Population' and 'Urban population (% of total)'. The values in this new column 'Total Urban Population', therefore, are the product of the first and second element in each tuple. Furthermore, because the 2nd element is a percentage, you need to divide the entire result by 100, or alternatively, multiply it by 0.01.\n",
    "\n",
    "You will also plot the data from this new column to create a visualization of the urban population data.\n",
    "\n",
    "The packages pandas and matplotlib.pyplot have been imported as pd and plt respectively for your use.\n",
    "Instructions\n",
    "100 XP\n",
    "\n",
    "    Write a list comprehension to generate a list of values from pops_list for the new column 'Total Urban Population'. The output expression should be the product of the first and second element in each tuple in pops_list. Because the 2nd element is a percentage, you also need to either multiply the result by 0.01 or divide it by 100. In addition, note that the column 'Total Urban Population' should only be able to take on integer values. To ensure this, make sure you cast the output expression to an integer with int().\n",
    "    Create a scatter plot where the x-axis are values from the 'Year' column and the y-axis are values from the 'Total Urban Population' column.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "id": "a08c7335-1a18-433b-899d-6926f8e8484d",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/tmp/ipykernel_2171/3742347169.py:16: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame.\n",
      "Try using .loc[row_indexer,col_indexer] = value instead\n",
      "\n",
      "See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n",
      "  df_pop_ceb['Total Urban Population'] = [int(i)*int(j)*0.01 for (i,j) in pops_list]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYQAAAERCAYAAAB7FtAjAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAXoUlEQVR4nO3df5RfdX3n8ed7kmESSNCYpF1khODvVTYEOnSx7GJEd7WC42lTW6xgK1JqtyoeakM5dVGxe043W10Uf0E97VFZcFezFcRfpyuk6lZLE0hGU0SFgoSyJcbwI5iMk3zf+8f3Tvlmzsx37jeZe+/8eD7O+Z7v937u/d77yp185z2f+7n3fiMzkSSpr+kAkqTZwYIgSQIsCJKkggVBkgRYECRJBQuCJAmYowUhIv4iIh6JiO+WWPa/R8T24vH9iHi0hoiSNOfEXLwOISLOAfYBn8rMU3t439uA0zPz4srCSdIcNSd7CJn5deAnnW0R8ZyI+EpEbIuIb0TECyd56+uBm2oJKUlzzOKmA8yg64G3ZOYPIuLfAh8Fzh2fGREnA6cAtzWUT5JmtXlRECJiGfBLwGcjYrx5YMJiFwCfy8xDdWaTpLliXhQE2oe+Hs3MdV2WuQD4/XriSNLcMyfHECbKzMeBf4yI1wFE22nj84vxhBXAtxqKKEmz3pwsCBFxE+1f7i+IiF0R8WbgDcCbI2IHsBN4bcdbLgA+k3PxlCpJqsmcPO1UkjTz5mQPQZI08+bcoPKqVatyzZo1TceQpDll27ZtP87M1d2WmXMFYc2aNWzdurXpGJI0p0TEA9Mt4yEjSRJgQZAkFSwIkiTAgiBJKlgQJEmABUGS5oQ9+0bZ8eCj7Nk3Wtk25txpp5K00Ny8/SGu2DxCf18fY60WmzasZXjdiTO+HXsIkjSL7dk3yhWbRzgw1uKJ0YMcGGuxcfNIJT0FC4IkzWK79u6nv+/wX9X9fX3s2rt/xrdlQZCkWWxwxVLGWq3D2sZaLQZXLJ3xbVkQJGkWW7lsgE0b1rKkv4/lA4tZ0t/Hpg1rWbls4pdCHj0HlSVplhtedyJnP3cVu/buZ3DF0kqKAVgQJGlOWLlsoLJCMM5DRpIkwIIgSSpYECRJgAVBklSwIEiSAAuCJKlgQZAkARYESVLBgiBJAiwIkqSCBUGSBFgQJEkFC4IkCbAgSJIKFgRJEmBBkCQVKi8IEbEoIu6KiFu7LLMhIjIihqrOI0maXB09hMuAu6eaGRHLi2X+roYskqQpVFoQImIQOA/4RJfF3gf8V+BAlVkkSd1V3UO4BtgItCabGRFnAM/KzC92W0lEXBoRWyNi6+7du2c+pSSpuoIQEecDj2Tmtinm9wEfAP5gunVl5vWZOZSZQ6tXr57hpJIkqLaHcDYwHBH3A58Bzo2IGzrmLwdOBbYUy5wF3OLAsiQ1o7KCkJlXZuZgZq4BLgBuy8wLO+Y/lpmrMnNNscy3geHM3FpVJknS1Gq/DiEiro6I4bq3K0nqbnEdG8nMLcCW4vVVUyyzvo4skqTJeaWyJAmwIEiSChYESRJgQZAkFSwIkiTAgiBJKlgQJEmABUGSVLAgSJIAC4IkqWBBkCQBFgRJUsGCIEkCLAiSpIIFQZIElPw+hIhYBPx85/KZ+aOqQkmS6jdtQYiItwHvBv4ZaBXNCaytMJckqWZlegiXAS/IzD1Vh5EkNafMGMKDwGNVB5EkNatMD+E+YEtEfBEYHW/MzA9UlkqSVLsyBeFHxeOY4iFJmoemLQiZ+V6AiFhWTO+rOpQkqX7TjiFExKkRcRewE9gZEdsi4sXVR5Mk1anMoPL1wOWZeXJmngz8AfDn1caSJNWtTEE4LjNvH5/IzC3AcZUlkiQ1otRZRhHxn4FPF9MX0j7zSJI0j5TpIVwMrAb+d/FYXbRJkuaRMmcZ7QXeXkMWSVKDpiwIEXFNZr4jIr5A+95Fh8nM4UqTSZJq1a2HMD5m8Gd1BJG0sOzZN8quvfsZXLGUlcsGmo4juhSEzNxWvFyXmR/snBcRlwF/U2UwSfPXzdsf4orNI/T39THWarFpw1qG153YdKwFr8yg8m9N0vbbM5xD0gKxZ98oV2we4cBYiydGD3JgrMXGzSPs2Tc6/ZtVqW5jCK8HfhM4JSJu6Zi1HPhJ1cEkzU+79u6nv6+PA//y9SrQ39fHrr37PXTUsG5jCH8LPAysAt7f0f4EMFJlKEnz1+CKpYy1Woe1jbVaDK5Y2lAijes2hvAA8ADwkvriSJrvVi4bYNOGtWycMIZg76B5Zb5C8yzgWuBf07799SLgycw8vuJskuap4XUncvZzV3mW0SxT5tYVHwYuAD4LDAFvBJ5fZShJ89/KZQMWglmmzFlGZOYPgUWZeSgz/xJ4VdkNRMSiiLgrIm6dZN5bIuI7EbE9Ir4ZES8qH12SNJPK9BB+GhHHANsjYhPtgeZShaRwGXA3MNkhphsz8+MAETEMfIAeio0kaeaU+cV+Ee1xg7cCTwLPAjaUWXlEDALnAZ+YbH5mPt4xeRyT3CJDklSPMje3e6B4uR94b4/rvwbYSPvahUlFxO8Dl9MesD53imUuBS4FOOmkk3qMIEkqo9uFad+hy1/smbm224oj4nzgkczcFhHru6znI8BHIuI3gXcxyZXRmXk97W9uY2hoyF6EJFWgWw/h/KNc99nAcES8GlgCHB8RN2TmhVMs/xngY0e5TUnSEZruwrQjlplXAlcCFD2Ed04sBhHxvMz8QTF5HvADJEmNKHNh2hM8dejoGKCfo7gwLSKuBrZm5i3AWyPiFcAYsJfJb6QnSapBmUHlfxkQjogAXguc1ctGMnMLsKV4fVVH+2W9rEeSVJ1ericg2z4PvLKaOJKkppQ5ZPSrHZN9tG9fcaCyRJKkRpS5Uvk1Ha8PAvfTPmwkSZpHyowhvKmOIJKkZk07hhARz46IL0TE7oh4JCJujohn1xFOklSfMoPKNwL/CzgBeCbt22DfVGUoSVL9yhSEYzPz05l5sHjcQPvKY0nSPFJmUPnLEfFHtG8tkcBvAF+KiGcAZOZPKswnSapJmYLw68Xz705ov4B2gXA8QZLmgTJnGZ1SRxBJUrPKXJjWD/wecE7RtAW4LjPHKswlSapZmUNGH6N9Q7uPFtMXFW2XVBVKklS/MgXhzMw8rWP6tojYUVUgSVIzypx2eiginjM+UVyUdqi6SJKkJpTpIfwhcHtE3AcEcDLg7SwkaZ7pWhAiYjXwGPCLwM8Vzfdk5mjVwSRJ9ZrykFFEXALsBK4FtgNrMnPEYiBJ81O3HsI7gBdn5u5i3OB/ALfUkkqSVLtug8o/y8zdAJl5HzBQTyRJUhO69RAGI+JDU01n5turiyVJqlu3gvCHE6a3VRlEktSsKQtCZn6yziCSpGaVuTBNkrQAWBAkSYAFQZJUKHP769XA7wBrOpfPzIuriyVJqluZexndDHwD+D94UztJmrfKFIRjM/OKypNIkhpVZgzh1oh4deVJJEmNKlMQLqNdFPZHxOMR8UREPF51MElSvaY9ZJSZy+sIIklqVpkxBCJiBfA8YMl4W2Z+vapQkqT6lTnt9BLah40GaX8vwlnAt4BzK00mSapV2TGEM4EHMvNlwOnAo1WGkiTVr0xBOJCZBwAiYiAzvwe8oNpYkqS6lRlD2BURTwc+D/x1ROwFHqgylCSpfmXOMvqV4uV7IuJ24GnAVypNJUmqXamb20XEGRHxdmAtsCszf1Z2AxGxKCLuiohbJ5l3eUT8Q0SMRMTXIuLk8tElSTNp2oIQEVcBnwRWAquAv4yId/WwjcuAu6eYdxcwlJlrgc8Bm3pYryRpBpXpIbwBODMz352Z76Z92ulFZVYeEYPAecAnJpufmbdn5k+LyW/TPrVVktSAMgXhn+i4IA0YAB4quf5rgI1Aq8Sybwa+PNmMiLg0IrZGxNbdu3eX3LQkqRdTDipHxLVAAo8BOyPir4vp/wDcMd2KI+J84JHM3BYR66dZ9kJgCHjpZPMz83rgeoChoaGcbtuSpN51O8toa/G8DfirjvYtJdd9NjBc3Cl1CXB8RNyQmRd2LhQRrwD+GHhpZo6WXLckaYZNWRAy85MRsQj4VGa+odcVZ+aVwJUARQ/hnZMUg9OB64BXZeYjvW5DkjRzuo4hZOYh4OSIOGamNhgRV0fEcDH534BlwGcjYntE3DJT25HqtGffKDsefJQ9++zkau4qc6XyfcD/LX5ZPznemJkfKLuRzNxCcagpM6/qaH9F2XVIs9XN2x/iis0j9Pf1MdZqsWnDWobXndh0LKlnZc4yuhe4tVh2ecdDWvD27Bvlis0jHBhr8cToQQ6Mtdi4ecSeguakMreueG8dQaS5aNfe/fT39XGg48zq/r4+du3dz8plAw0mk3rX7bTTL9A+zXRcAj8Gbs/MG6oOJs0FgyuWMtY6/DKbsVaLwRVLG0okHbluPYQ/m6TtGcCFEXFqZv5RRZmkOWPlsgE2bVjLxgljCPYONBdFZm/XeRWnom7LzHWVJJrG0NBQbt26dfoFpRrt2TfKrr37GVyx1GKgWSkitmXmULdlSn2ncqfMPBQRR55KmodWLhuwEGjO6zaG8IxJmlcAbwR2VpZIktSIbj2EbbQHkse7A+ODyluA36s2liSpbt1uXXFKnUEkSc0q9Y1pkqT5z4IgSQIsCJKkQrezjM7o9sbMvHPm40iSmtLtLKP3d5mXwLkznEWS1KBuZxm9rM4gml288lZaeEpdqRwRpwIvov1VmABk5qeqCqVmeX9/aWGadlA5It4NXFs8XgZsAoa7vklzlvf3lxauMmcZ/RrwcuD/ZeabgNOAp1WaSo0Zv79/p/H7+0ua38oUhP2Z2QIORsTxwCPAs6qNpaZ4f39p4SpTELZGxNOBP6d9f6M7gW9VGUrNGb+//5L+PpYPLGZJf5/395cWiJ6+DyEi1gDHZ+ZIZYmm4fch1MOzjKT5ZUa+DyEivpaZLwfIzPsntml+8v7+0sLT7UrlJcCxwKqIWMFTt8E+HvAcREmaZ7r1EH4XeAfwTNrjBuMeBz5cYSZJUgO6Xan8QeCDEfG2zLy2xkySpAaUuVL5uoh4O3BOMb0FuC4zxypLJUmqXZmC8FGgv3gGuAj4GHBJVaEkSfXrNqi8ODMPAmdm5mkds26LiB3VR5Mk1anbhWl3FM+HIuI5440R8WzgUKWpJEm163bIaPw003cCt0fEfcX0GuBNVYaSJNWvW0FYHRGXF6+vAxYVrw8BpwO3VxlMklSvbgVhEbCMp3oKne9ZXlkiSVIjuhWEhzPz6tqSSJIa1W1QeWLPQJI0j3UrCN68TpIWkCkLQmb+pM4gkqRmlfmCHEnSAlB5QYiIRRFxV0TcOsm8cyLizog4GBG/VnUWSdLU6ughXAbcPcW8HwG/DdxYQw5JUheVFoSIGATOAz4x2fzMvL/4Os7WZPMlSfWpuodwDbCRo/yFHxGXRsTWiNi6e/fuGQkmSTpcZQUhIs4HHsnMbUe7rsy8PjOHMnNo9erVM5BOkjRRlT2Es4HhiLgf+AxwbkTcUOH2JElHobKCkJlXZuZgZq4BLgBuy8wLq9qeJOno1H4dQkRcHRHDxeszI2IX8DraX9W5s+48kqS2Ml+hedQycwvt72ImM6/qaP97YLCODJKk7rxSWZIEWBAkSQULgiQJsCBIkgoWBEkSYEGQJBUsCJIkwIIgSSpYECRJgAVBklSwIEiSAAuCJKlgQZAkARYESVLBgiBJAiwIkqSCBUGSBFgQJEkFC4IkCbAgSJIKFgRJEmBBkCQVLAiSJMCCIEkqWBAkSYAFQZJUsCBIkoAFVBD27Btlx4OPsmffaNNRJGlWWtx0gDrcvP0hrtg8Qn9fH2OtFps2rGV43YlNx5KkWWXe9xD27Bvlis0jHBhr8cToQQ6Mtdi4ecSegiRNMO8Lwq69++nvO/yf2d/Xx669+xtKJEmz07wvCIMrljLWah3WNtZqMbhiaUOJJGl2mvcFYeWyATZtWMuS/j6WDyxmSX8fmzasZeWygaajSdKssiAGlYfXncjZz13Frr37GVyx1GIgSZNYEAUB2j0FC4EkTW3eHzKSJJVjQZAkARYESVLBgiBJAiwIkqRCZGbTGXoSEbuBB47w7auAH89gnJlirt6Yq3ezNZu5enM0uU7OzNXdFphzBeFoRMTWzBxqOsdE5uqNuXo3W7OZqzdV5/KQkSQJsCBIkgoLrSBc33SAKZirN+bq3WzNZq7eVJprQY0hSJKmttB6CJKkKVgQJEltmTlnHsBfAI8A3+1oOw34FvAd4AvA8R3z1hbzdhbzlxTtv1BM/xD4EMWhswnbimLeD4ER4IxZkms98BiwvXhcVUOu/wI8COyb5udzZZH9HuCVsyEXsAbY37G/Pl5lLuBY4IvA94r2P50N+6tsrrr3V9H+FWBH0f5xYNEs+TyWybWemj+PHfNv6VzXke6vw95XZqHZ8gDOAc6YsEP/Hnhp8fpi4H3F68XFjjitmF45/gMF7gDOKnbal4FfnmRbry7mRbHs382SXOuBW2veX2cBJ9D9F++Lig/PAHAKcO9kH6AGcq2Z6kNTRS7av3hfVrQdA3xjip9jrfurh1y17q/i9fHFcwCbgQtmyeexTK711Px5LKZ/Fbhxqp9VL/vrsPeVWWg2PSb+h6VdnccHx58F/EPHDrlhkvefAHyvY/r1wHWTLHcd8PqO6XuAE2ZBrtL/AWci14R1dfvFeyVwZcf0V4GXzIJch22nzv1VLPdB4Hdm0/6aJldj+wvop/1X8m9MMq/Wz2MPuWr/PALLgG/S/qNiqoLQ0/4af8yHMYSdwGuL16+jvVMBng9kRHw1Iu6MiI1F+4nAro737yraJjqR9uGI6ZarOxfASyJiR0R8OSJe3EOmI8lVVt37qxenRMRdEfE3EfHve3zvEeeKiKcDrwG+Nsl6G9tf0+SCBvZXRHyV9uGUJ4DPTbLeRvZXiVxQ/+fxfcD7gZ92We8R7a/5UBAuBv5TRGwDlgM/K9oXA/8OeEPx/CsR8fJ5kOtO2vckOQ24Fvj8LMl1tKrK9TBwUmaeDlwO3BgRx1edKyIWAzcBH8rM+3rYXtO5GtlfmflK2r3kAeDcHrbXdK5aP48RsQ54Tmb+VY/bKWXOF4TM/F5m/sfM/AXa/9HvLWbtAr6emT/OzJ8CX6J9/O4hYLBjFYNF20QP8VS17rZcrbky8/HM3Fe8/hLQHxGrKsxVVt37q+x6RzNzT/F6W7He59eQ63rgB5l5zRSrbmp/dc3V4P4iMw8AN/PUX8ydGvv/1S1XA5/HlwBDEXE/7cNGz4+ILZOs+oj215wvCBHxc8VzH/Au2mcDQPuY7L+JiGOLv4peSvv43MPA4xFxVkQE8EbaP+yJbgHeGG1nAY8V7200V0T8q2I+EfGLtH+Ge6rKVXa9tPfXBRExEBGnAM+jPUjeaK6IWB0Ri4rXzy5ylf6L/UhyRcSfAE8D3tFl1bXvrzK56t5fEbEsIk4o3rMYOI/2mVAT1fp5LJur7s9jZn4sM5+ZmWto9xy+n5nrJ1n1ke2vsoMhs+FBu4I+DIzRrqBvBi4Dvl88/pSOUzWBC2kfo/susKmjfahouxf48Ph7gLcAbyleB/CRYpnvAEOzJNdbi/fuAL4N/FINuTYV728Vz+8p2oeBqzuW++Mi+z1McgZLE7mADcV7t9Pu3r+myly0/xJL4G6eOhXxkqb3V9lcDeyvn6d9ps1I0X4tsLjpz2MPuWr/PHbMX8PhA9RHtL86H966QpIEzINDRpKkmWFBkCQBFgRJUsGCIEkCLAiSpIIFQZpGcS73NyPilzvaXhcRX2kylzTTPO1UKiEiTgU+C5xO+7YCdwGvysx7u75x8nUtzsyDMxxROmoWBKmkiNgEPAkcVzyfDJxK+26Y78nMmyNiDfDpYhmAt2bm30bEeto3JdsLvDAzS98OQqqLBUEqKSKOo3317s+AW4GdmXlDtO8eegft3kMCrcw8EBHPA27KzKGiIHwRODUz/7GJ/NJ0FjcdQJorMvPJiPifwD7g14HXRMQ7i9lLgJOAfwI+XNyV8hCH3xjuDouBZjMLgtSbVvEIYENm3tM5MyLeA/wz7a9G7AMOdMx+sqaM0hHxLCPpyHwVeFvHnS5PL9qfBjycmS3gItpfXynNCRYE6ci8j/Zg8khE7CymAT4K/FZE7ABeiL0CzSEOKkuSAHsIkqSCBUGSBFgQJEkFC4IkCbAgSJIKFgRJEmBBkCQV/j9AgVvH2PYiGQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "\n",
    "\n",
    "# Code from previous exercise\n",
    "urb_pop_reader = pd.read_csv('ind_pop_data.csv', chunksize=1000)\n",
    "df_urb_pop = next(urb_pop_reader)\n",
    "df_pop_ceb = df_urb_pop[df_urb_pop['CountryCode'] == 'CEB']\n",
    "pops = zip(df_pop_ceb['Total Population'], \n",
    "           df_pop_ceb['Urban population (% of total)'])\n",
    "pops_list = list(pops)\n",
    "\n",
    "\n",
    "# Use list comprehension to create new DataFrame column 'Total Urban Population'\n",
    "df_pop_ceb['Total Urban Population'] = [int(i)*int(j)*0.01 for (i,j) in pops_list]\n",
    "\n",
    "# Plot urban population data\n",
    "#df_pop_ceb.plot(kind='scatter', x='df_pop_ceb['Year']', y='df_pop_ceb['Total Urban Population']')\n",
    "df_pop_ceb.plot(kind='scatter', x='Year', y='Total Urban Population')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4132367f-a55b-46b6-8c6d-14c167dd94e5",
   "metadata": {},
   "source": [
    "# df_pop_ceb.plot(kind='scatter', x='df_pop_ceb['Year']', y='df_pop_ceb['Total Urban Population']')\n",
    "\n",
    "# *****************************************************************************************************\n",
    "\n",
    "df_pop_ceb.plot(kind='scatter', x='Year', y='Total Urban Population')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "id": "a2b824c2-e955-4a10-88fb-d6e0c345f566",
   "metadata": {},
   "outputs": [],
   "source": [
    "life_exp = [43.828, 76.423, 72.301, 42.731, 75.32, 81.235, 79.829, 75.635, 64.062, 79.441, 56.728, \n",
    " 65.554, 74.852, 50.728, 72.39, 73.005, 52.295, 49.58, 59.723, 50.43, 80.653, 44.74100000000001,\n",
    " 50.651, 78.553, 72.961, 72.889, 65.152, 46.462, 55.322, 78.782, 48.328, 75.748, 78.273, 76.486, \n",
    " 78.332, 54.791, 72.235, 74.994, 71.33800000000002, 71.878, 51.57899999999999, 58.04, 52.947, 79.313,\n",
    " 80.657, 56.735, 59.448, 79.406, 60.022, 79.483, 70.259, 56.007, 46.38800000000001, 60.916,\n",
    " 70.19800000000001, 82.208, 73.33800000000002, 81.757, 64.69800000000001, 70.65, 70.964, 59.545,\n",
    " 78.885, 80.745, 80.546, 72.567, 82.603, 72.535, 54.11, 67.297, 78.623, 77.58800000000002, 71.993,\n",
    " 42.592, 45.678, 73.952, 59.44300000000001, 48.303, 74.241, 54.467, 64.164, 72.801, 76.195, 66.803,\n",
    " 74.543, 71.164, 42.082, 62.069, 52.90600000000001, 63.785, 79.762, 80.204, 72.899, 56.867, 46.859,\n",
    " 80.196, 75.64, 65.483, 75.53699999999998, 71.752, 71.421, 71.688, 75.563, 78.098, 78.74600000000002,\n",
    " 76.442, 72.476, 46.242, 65.528, 72.777, 63.062, 74.002, 42.56800000000001, 79.972, 74.663, 77.926,\n",
    " 48.159, 49.339, 80.941, 72.396, 58.556, 39.613, 80.884, 81.70100000000002, 74.143, 78.4, 52.517,\n",
    " 70.616, 58.42, 69.819, 73.923, 71.777, 51.542, 79.425, 78.242, 76.384, 73.747, 74.249, 73.422, 62.698,\n",
    " 42.38399999999999, 43.487]\n",
    "\n",
    "pop = [31.889923, 3.600523, 33.333216, 12.420476, 40.301927, 20.434176, 8.199783, 0.708573, 150.448339, \n",
    " 10.392226, 8.078314, 9.119152, 4.552198, 1.639131, 190.010647, 7.322858, 14.326203, 8.390505, 14.131858,\n",
    " 17.696293, 33.390141, 4.369038, 10.238807, 16.284741, 1318.683096, 44.22755, 0.71096, 64.606759, 3.80061,\n",
    " 4.133884, 18.013409, 4.493312, 11.416987, 10.228744, 5.46812, 0.496374, 9.319622, 13.75568, 80.264543,\n",
    " 6.939688, 0.551201, 4.906585, 76.511887, 5.23846, 61.083916, 1.454867, 1.688359, 82.400996, 22.873338,\n",
    " 10.70629, 12.572928, 9.947814, 1.472041, 8.502814, 7.483763, 6.980412, 9.956108, 0.301931, 1110.396331,\n",
    " 223.547, 69.45357, 27.499638, 4.109086, 6.426679, 58.147733, 2.780132, 127.467972, 6.053193, 35.610177,\n",
    " 23.301725, 49.04479, 2.505559, 3.921278, 2.012649, 3.193942, 6.036914, 19.167654, 13.327079, 24.821286,\n",
    " 12.031795, 3.270065, 1.250882, 108.700891, 2.874127, 0.684736, 33.757175, 19.951656, 47.76198, 2.05508,\n",
    " 28.90179, 16.570613, 4.115771, 5.675356, 12.894865, 135.031164, 4.627926, 3.204897, 169.270617, 3.242173,\n",
    " 6.667147, 28.674757, 91.077287, 38.518241, 10.642836, 3.942491, 0.798094, 22.276056, 8.860588, 0.199579, \n",
    " 27.601038, 12.267493, 10.150265, 6.144562, 4.553009, 5.447502, 2.009245, 9.118773, 43.997828, 40.448191, \n",
    " 20.378239, 42.292929, 1.133066, 9.031088, 7.554661, 19.314747, 23.174294, 38.13964, 65.068149, 5.701579,\n",
    " 1.056608, 10.276158, 71.158647, 29.170398, 60.776238, 301.139947, 3.447496, 26.084662, 85.262356, \n",
    " 4.018332, 22.211743, 11.746035, 12.311143]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "id": "017b6735-1812-4dea-bdc3-d351326c14ef",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXAAAAD8CAYAAABuHP8oAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAdHklEQVR4nO3df4wd1XUH8O/xeg1rmrIkuAgWtnaUyG4SBC4rEslSFZskJKWBFRACpZVLaV0qleaXXBYpKiBFykaulB9SlcRJKly1IgZDFhM3MRUGVbVa2t0sFiFglQIGXhxwUm8U2Ru8Xp/+8d7bfft2ftyZd2fm3rnfj4Twe/u8b9688Zkz5557R1QVRETknxVVbwAREeXDAE5E5CkGcCIiTzGAExF5igGciMhTDOBERJ5aWeabnX/++bp27doy35KIyHtTU1M/V9U13c+XGsDXrl2LycnJMt+SiMh7InIk6nmWUIiIPMUATkTkKQZwIiJPMYATEXmq1EFMIvLLxHQDO/Yfxk9nZnHR4AC2X70eoxuHqt4samEAJ6JIE9MN3P3Is5idmwcANGZmcfcjzwIAg7gjGMDJGcz23LJj/+GF4N02OzePHfsP83txBAM4OYHZnnt+OjOb6XkqX20HMSemG9g0fgDrxvZh0/gBTEw3qt4kSpCU7VE1LhocyPQ8la+WAbydzTVmZqFYzOYYxN0Vl9U1ZmZ5Aq7I9qvXY6C/b8lzA/192H71+oq2iLrVMoAzm/NPUlbHE3A1RjcO4YvXX4qhwQEIgKHBAXzx+ktZ0nJILWvgrN35Z/vV65fUwLtVNXgW+sDq6MahoD6vb2oZwC8aHEAjIlizdueudpDYsf9w5HcHNE/AZQbUMgZWQz9BUG9qGcCjsjnW7twUFcDigvjg6v5SO1VstdHFBeleThAM/ATUNIB3ZnM8wN0VF8BuuGIID081lp2AVVFqX7KNUlxSkM5zgpiYbuC+x57D8ZNzC8+x5TJctQzgAGt3ZcqbDcYFsCdfOIYvXn/pst/5md3PRP6eosY2bJTikoJ01hNE98kg6nfymA+LUQAXkc8A+DMACuBZALcBuBDAdwG8A8AUgD9W1VMFbSc5KirD3P7QIdz32HOYOTmXGNCTAljUCTiutFLU2IaNUlzSZ8x6gog6GZi8F9VXahuhiAwB+GsAI6r6PgB9AG4G8CUAX1bVdwE4DuD2IjeU3BQVVObOKI6fnEvtwTedKNKelNWYmYV0vbbIsQ0bbXRJnzFrn3VagOYgfXhMSygrAQyIyByA1QCOAtgC4A9bP98F4F4AX7e9geQ2k6wv7vLeJMPtzvAVgLT+P2RpbCOpBNRrKS7pM2Ydq4nL2Dt/J4UlNYCrakNE/g7AqwBmATyOZslkRlVPt172OoDIo05EtgHYBgDDw8M2tpkckhRUOkUFepMAFpXht4P3wbEtAJoB+N69z2Fmtjmwd97qftzz8fcaBd6iWwXTPmOWE0Rcr7wIcMMV0b+H3Sr1Jqqa/AKR8wA8DOCTAGYAPARgD4B7W+UTiMglAH7QKrHEGhkZUd7UuF6SBtY69YngjGrmILJubB+ijlAB8PL4NZiYbmD7Q4cwd2bpq/r7BDtuvCz1fdqlmW6dJwiXdJ+s2gb6+5aVd6K+m6jXkftEZEpVR7qfN5lK/yEAL6vqMVWdA/AIgE0ABkWkncFfDIDznAPUWSdOMq+aa12atDr5jv2HlwVvAJibV6OlE3ybtTu6cQjnnLX8wjlqqQguKVF/JgH8VQAfEJHVIiIArgLwEwBPArix9ZqtAB4tZhPJVe3BxU/vfsaojNKWJYikDfQlBVqTIOzjinumJx3fTk6UXWoAV9Wn0SyZ/AjNFsIVAHYCuAvAZ0XkRTRbCb9T4HaSYzpXfIyTlJWbBpG0TpCkQGsShDdvWFNqZ0sWcUsim550fDw5UTZGXSiqeg+Ae7qefgnAlda3iLyQ1pPcJ4KDY1tia8wrRDAx3TCqxSYN9G2/en1kDRwATrx1OvE9JqYbeHiqsaTGLogfEOz8e0UPDCYNrpr2p/u6pAQHXs3VcjlZKl5aBj3fGhyPKoG0f25jidjRjUPY8YnLMNC//FCemZ1LfI+4DpcnXzgW+35lrTWfNs3epD/dx+VguZZ/NrWdSk9NRWUzae2DfdIsTLTf63MPHloI6m12p393F0LS3yNPjbis+0SmbZtp+6FvS0rwPpzZMAOvsSKzmbjMuu2W91+y8OfRjUM4E9OuamNALe8U8zw14rIGBkOtX3PgNRsG8BozaSPLe+/QuPbBPhH80QeG8YXRS5c8X2RAyjvFPM8tw8oKrKHezizUE1deLKHUWFo20+ssxF5nEdoKSHmnmOdZdrisgcFQl0T2deC1KgzgNZa22l2Z9cYiA1LcFPPBgX7ce23ylPqsNeIyA6tv9WsbQj1x5ZU6ld4mTqUvV9pU6rRp6j5h6xnVWdxUembgNZaWzdTp3qEhZqtEDOAFqzozTJsEk6Xe2H07L5MSRdGq3r9EVWIAL1AZdzXvRZZ648R0A9v3HMLc/GLRZWZ2DtsfOrTkd5XJ9f3rE54I/cQaeIF8W6o0SdxnAar7PHXav1XisrPuYw28JJ2ZTNyp0adJCe3PkzTrsvPzlJnJcdKHHZz96C8GcItMb27gyyBh1s9TdknD50FYl0oWPBH6izMxLUqb0g34NSnB5PP0r5CFz1P2DQR8na0YtcTBp3c/g8vve7ySRZs4+9FfDOAWJWUsvqwG1yktAxsc6MeOTyzetqzsTC7rant5lw2wLe7EmLZ6YlF8PRESSyhWxV3S+zqoljRFXQA8c89HjF5fZCZn2v/tUsdK0gmtitozZz/6ixm4RXXLZLZfvT5mkdbooLx5w5rI127esKby7Nel+0OmndCqqD2PbhzCwbEteHn8Ghwc28Lg7QkGcMvO7rixwOBAv1clk26jG4dw6weGjW85FncjhO8fOlr5Iv0uDdSlLcXL2jOZYgnFkqiOjbdOn6lwi+z4wuilGPnttxtdXscFw5nZuWXPlV0qyFreMekSydtJ0n5N56zWtqKu2FzqeiF7OJHHEk4qSZ7sE0eAngLK5yeexQNPv4Z5VfSJ4Jb3X7JsLXIg22QVk9famvxSxf01824rVYcTeQrm0iV6VeLWVjm7f8WyTLOts6QCZBtQ/PzEs/in/3x14fG86sLj7iCeZaDOZGJL3Gs+92C2pQXKWISLE3XqiwHcEp8nldgSFyQBpE4IyhNQHnj6tdjno7Jw02BpcjKOe037Zs3t93MBk4v6YgC3JNQ7iUSVAOJKRraXGOi+SXLa86ZMTsZJLZauZbdMLuqLXSiWZJ1UUgdZbprc2abWfR/NtiwBJamDpU/imh/NmLSDpnWSuJTd1q29lRYxA7cotJsK5KmtTkw3cPLU6WXPdwYUk4G9pP7tW95/SdaPsoRJvbz95889eCgy48+T3RY1oMmJOtUpepCaAdxxLrd/Za2txi2O1XljCNMZk0kZblT9OyuTk3H751Gf6cRbpzEx3TD+roqeKRpacuGCMmb/MoA7rOzp31lPFllrq3FrgPzq16cTXxOV1SctW2BT5z4ZXN0PVeCXs3PL9k93T3d7XRPA7LuqU6eIy0lHmcr4ThnAHVbmP+o8J4usA7cmnRsmWf3EdAMn3kouw9jQvU86A3T3/tmx//CyVsnOqfppAa0unSIurTlTtTK+Uw5iOqzMf9R51wo5a+XiIXTe6uSlA5Lqwu33SlvatB0gumd3pr13HmnL6Xbun7jvpB3A0gZ667Kkq0trzlStjO+UAdxhZf6jzlvP7gykv55LXjogrXOjMTOLo79c/n6dmXVcUF29aqX1DM/kRNl+Tdx30idiFNBsdIpUvWAYUJ8rCRvK6P5hAHdYme1fWU8WeTKtdqtlUpvfma6Gju4FwcoMECYnyvZr4r6ruJ707u3ttQ3VtKWz6CBflysJG8poLWYN3GFltn/ZqmenBdKkzo0o55y10mjwsogAEbVPOnXun6jBzLNWrsBZK1dELuYVtb29dIrEnVDve+y5hePn3IF+nDh1GnPzzZNKEfXpUCe0xSm6+4cB3HFltX9lPVn0Ekg73ytt8avuE0KZAaJ7nyR1obR1lpFmZufQ3yfoXyGY67i0EMSvnZ5X3Inz+Mm5hRNKGatCsue8XFyNkHKxtcJd2gqGUas5utqmFvdZzlnVh5On5pcsIWB7NcA8K0G2CYCXx6+xsh1UDK5GSFbZyrSSyhRxmbWrk1LisuATp5Z/NtuZ7+YNa5aszJhFWfVpV0+8PmMAp9xsBNLuckqfCOZVMeThP/CkBa6i2Bh4bQfFvNl3WfVp9ocXgyUUco6tTK3sjC+urBQ3kNnrzT7iliZI0r9C8Btnr8TMyfg6fhF4w5Pe5C6hiMh6ALs7nnongL8F8I+t59cCeAXATap63MbGUrhsZWpVZHxZ1kO3kfmmTTQCmhOcVq9aWXnZgv3hxUgN4Kp6GMDlACAifQAaAL4HYAzAE6o6LiJjrcd3Fbep9cb6YJOt5QOqWlskqaxk+/tNC34D/X245+PvdeI44prkxchaA78KwP+q6hERuQ7AB1vP7wLwFBjAc2F9cJGtTM21jK+IgdekmrtrYwjsDy9G1pmYNwN4oPXnC1T1aOvPPwNwQdRfEJFtIjIpIpPHjh3LuZn1xvUjFtmayRfCjMC42Z9f+eTlODi2xZngDYR5w5MyGGfgIrIKwLUA7u7+maqqiESOhqrqTgA7geYgZs7trDXXssUq2crUQsj4fJs042r7p8+ylFA+BuBHqvpG6/EbInKhqh4VkQsBvGl/88LA+uAiW0HJt+CWF4Ni2LIE8FuwWD4BgL0AtgIYb/3/UYvbFZQQssUsbAUlBjeqO6MALiLnAPgwgL/oeHocwIMicjuAIwBusr95YQglWyQiuziRh4jIcXETebgeOBGRpxjAiYg8xQBOROQpBnAiIk9xOVmijLhuDbmCAZwoA65bQy5hAKclmF0mq2qVQ6IoDOC0gNllOq5bQy7hICYt4KqI6UJY5ZD8wQBOC5hdpotbwjXUdWuoWgzgtIDZZTqua00uYQ2cFnBVRDNc5ZBc4XwAZ1dEeXxYFZHHg3v4nVTH6QDOrojyuZxd8nhwD7+TajldA/exK2JiuoFN4wewbmwfNo0fwMR0o+pNqg0fj4e643dSLaczcN+6IpiNFMu34yEE/E6q5XQG7ltXBLORYvl2PISA30lTVVfeTgdw33pumY0Uy7fjIQT8ThavvBszs1AsXnmXEcSdLqH40BXRqa53l3ely8C34yEE/E6qXR/H6QAOuN0V0a2OfdSu1fV9Oh7qKuqEfnBsS9WbVZkqr7ydD+A+qWM2wtX3snHlaqUorp3QXVDllTcDuGV1yxBZ1zcXQnDjCX25Kq+8nR7EpOqxy8BcCF1IPKEvV+X6OMzAKZHPdf2yyxkhBLe6DtT3qqorb2bglMjX1feqaO0K4WqFbYNuYQZOqXys6xdRq03L6H2+WjFVx4F6nzGAUy3ZLmeYDFCGEtx8PKHXFQM41ZLtWq1pRs/gRmViDZxqyXattuwBSq5qSSaYgVMt2S5nlNl9EUI/OdnBAE61ZbOcUeYAJSfLkCkGcCIDZQ5QhtBPTnYwgBMZKmuAkpNlyBQHMckLIQ3qcbIMmWIGTs4LbVAvlH5y6h0DODkvxEE99pOTCZZQyHkc1COKZpSBi8gggG8DeB8ABfCnAA4D2A1gLYBXANykqseL2EgKm0uDenW/YQP5xTQD/yqAH6rqBgCXAXgewBiAJ1T13QCeaD0mss6VQb0qb15LFCU1AxeRcwH8HoA/AQBVPQXglIhcB+CDrZftAvAUgLuK2EgKm61BvV6z57QbNjAzp7KJqia/QORyADsB/ATN7HsKwKcANFR1sPUaAXC8/bjr728DsA0AhoeHrzhy5Ii9racFvLRP1t3JAjSz+Li1zaP252d2P4O4fy0D/X3Gv5soKxGZUtWR7udNSigrAfwugK+r6kYAJ9BVLtHmWSDy2FbVnao6oqoja9asyb7llIqX9umy3O4sbn+eO9Af+bv7RGp/KzVyk0kAfx3A66r6dOvxHjQD+hsiciEAtP7/ZjGbSGlCuBdjr7J0ssTtTxFE1uLnY65i2SVDRUsN4Kr6MwCviUh7xOgqNMspewFsbT23FcCjhWwhpWKbXbostzuL228zJ+ciby83FMCt1MhNphN57gTwzyKyCsBLAG5DM/g/KCK3AzgC4KZiNpHSuNRm56osqwkm7c+4CTa9rFTI8QvKyyiAq+ozAJYV0NHMxqliPt2LsapglaWTJev+7KVLJrRlAsiu1C4Um0ZGRnRycrK09wuJi1lc9zZt3rAGD081vOjWKGt/bho/EJntDw0O4ODYFuvvR36K60JhAKdCRLXtCaJblUIOVuvG9kXuEwHw8vg1ZW8OOaqXNkKizKI6OeJShZAHW7MMrhJ1YwCnQmQJyiEHK1eWCSA/MYBTIeKCsnQ9Dj1YjW4cimxNdG1MgNzE9cCpEHGdHDdcMYQnXzjm1GBr1bj2N+XFAO4hFztOurmyAJUP+4ooLwZwz/jUN9xrZtnrZ/VpXxHlwRq4Z0Ja96TXzxrSvqIwMYB7JqR1T3r9rCHtKwoTSyieCWndk7TPmlbfDmlfUZiYgXsmpL7hpM9qsgZ6SPuKwsQA7pmQ+oaTPqtJfTukfUVh4loolEvV7XlcQ4RCwrVQyBoXbuHGNUSIGMBTTUw3sGn8ANaN7cOm8QO8zyTcaM9jfZuIXSiJOBEkmgvtebZmehL5jAE8QVKmGXKgcKU9L2mmZ9U1eqIysISSwIVM00Wuly9cqNHHbRfLcWQTA3gCDpRFc709z4UafTdXTyrkN5ZQEvh0s+CyubwEqotXTizHURGYgSdwPdOkaC5eObl4UiH/MQNP4XKmSdFcvHJyZeCX6oUZONWOi1dOrg/8kp+YgQcgxJY6166cyu5bD/E7DxEDeM1xMpI7yjqp8DsPB0soNediSx0V2xPO7zwczMBrjt0PS7lQWig6Q+Z3Hg5m4DXnYktdVVyZTFN0hszvPBwM4DXH7odFrpQWis6Q+Z2HgyWUDFy4/M6Kq/YtKqu0UPW9Ovmdh4MB3JDPI/uutdRVpYzJNCbHSRkTjfidh4ElFEOuXH5TfmWUFnivTioTM3BDHNn3XxmlBdPjhBky2cAAbsjltSx8rM3bkvWzFx04XT5OqH5YQjHk6si+K61xVbD92W1MrnH1OKF6YgA35GrdMuTavM3Pbutk4OpxQvVkVEIRkVcA/ArAPIDTqjoiIm8HsBvAWgCvALhJVY8Xs5lucLFuGXJt3uZnt3nDBRePE6qnLBn4ZlW9XFVHWo/HADyhqu8G8ETrMZUs5Fl3Nj97yCdC8lcvJZTrAOxq/XkXgNGet4YyC6HmGlebtvnZQz4Rkr9Mu1AUwOMiogC+qao7AVygqkdbP/8ZgAui/qKIbAOwDQCGh4d73FzqVvdZdyYTY2x8dhfu4hNyNxHlI6qa/iKRIVVtiMhvAfhXAHcC2Kuqgx2vOa6q5yX9npGREZ2cnOxxkykkm8YPRLblDQ0O4ODYFqvvVWUA7T5RAc0TCAdACQBEZKqjfL3AKANX1Ubr/2+KyPcAXAngDRG5UFWPisiFAN60usVEKLc2XeXgI+9aT3mk1sBF5BwReVv7zwA+AuDHAPYC2Np62VYAjxa1kRSuUGrTHESlPEwGMS8A8O8icgjAfwHYp6o/BDAO4MMi8j8APtR6TGRVCIO0QDgnKrIrtYSiqi8BuCzi+V8AuKqIjSJqszFQmaW2XVUd3IVBVPKP0SCmLRzEpF5lDbBZBgerHkhkFwrFiRvEZAAnb+QJsFm6WMrseCHKIi6Acy0U8kaetU+yDA5yIJF8wwBO3sgTYLMMDnIgkXzDAE7eyBNgs3SxhNLxQvXBAE7eyBNgsyzvyqVgyTccxCSvsFOjGtzv1eppKj2RK7jWdvlMFhSjajCAe4jZELWVcSxwnRZ3MYB7JuRsiCeupco6Fthe6S4OYnom1Htghnzz5jhlHQtsr3QXA7hnys6GbNyp3YZQT1xJyjoW2F7pLgZwz5SZDbmU9fIyfrmyjgW2V7qLNXDPlLlqnUuDVxcNDkSuUxLyZXyZxwK7f9zEDNwzZWZDLmW9vIxfjpkxMQP3UFnZkEtZb91v3pwXM+OwMYBTrKyX6EW3+TFYES3FAE6xsmS9IfenE1WFAZwSmWa9Lg14EoWCg5hkhUsDnkShYAZOVrg04OkSTv+nIjEDJytcbPOrehapSxOhqJ4YwMkK13qSXQienP5PRWMJhaxxqc3PhUFVjgtQ0ZiBUy25EDy5ih8VjQG85qquA1fFheDp4rgA1QtLKDXm+uSaojo0JqYbOH7ircifbd6wpuffb4rT/6loDOA15kIdOE5RJ5fF33sm8udPvnAs9+/Ow6VxAaofBvAac6EOHKeok0vU7+2U5bOzh5tcxxp4jblQB45T1Mkl7e+bfnYX2hCJ0jCA15jLg2hFnVyS/n6Wz84ebvIBA3iNuTa5plNRJ5eo3wsAgwP9mT67y+UnojbWwGvO1UG0ojo0bP1eru1CPhBVLe3NRkZGdHJysrT3I8qru0sGaF4huHIFQ2ERkSlVHel+nhk4UQT2cJMPGMCJYrhafiJq4yAmEZGnjAO4iPSJyLSIfL/1eJ2IPC0iL4rIbhFZVdxmEhFRtywZ+KcAPN/x+EsAvqyq7wJwHMDtNjeMiIiSGQVwEbkYwDUAvt16LAC2ANjTeskuAKMFbF9wQl09kIiyMx3E/AqAvwHwttbjdwCYUdXTrcevA4gc7RGRbQC2AcDw8HDuDQ2B66sHEpFbUjNwEfkDAG+q6lSeN1DVnao6oqoja9aUt5Snjzh9m4iyMMnANwG4VkR+H8DZAH4TwFcBDIrIylYWfjEAXuv3iNO3iSiL1AxcVe9W1YtVdS2AmwEcUNVbATwJ4MbWy7YCeLSwrQyEy6sHEpF7eukDvwvAZ0XkRTRr4t+xs0nhcnn1QCJyT6aZmKr6FICnWn9+CcCV9jcpXJy+TURZcCq9Yzh9m4hMcSo9EZGnGMCJiDzFAE5E5CkGcCIiTzGAExF5qtRbqonIMQBHcv71cwH8ssC/l/a6uJ9HPW/yXPfj8wH83GA786py/2X9Wdb9V+d9l/Rz0+fT9if3X7bnqvi3O6iqy9ciUVUv/gOws8i/l/a6uJ9HPW/yXMTjybruv6w/y7r/6rzvkn5u+nza/uT+y3/sVb3/fCqhPFbw30t7XdzPo543eS7v58mryv2X9Weu7T+fjr2o5033cVF82n+uHXuJ71dqCYXiicikRtx1mtJx3/WG+683Ve4/nzLwuttZ9QZ4jPuuN9x/vals/zEDJyLyFDNwIiJPMYATEXmKAZyIyFMM4A4SkXNEZJeIfEtEbq16e3wjIu8Uke+IyJ6qt8VHIjLaOvZ2i8hHqt4en4jI74jIN0Rkj4j8ZdHvxwBeEhH5BxF5U0R+3PX8R0XksIi8KCJjraevB7BHVf8cwLWlb6yDsuw/VX1JVW+vZkvdlHH/TbSOvTsAfLKK7XVJxn33vKreAeAmNO8nXCgG8PLcD+CjnU+ISB+AvwfwMQDvAXCLiLwHzZtEv9Z62dLb1IfrfpjvP1rufmTff59v/Tx09yPDvhORawHsA/AvRW8YA3hJVPXfAPxf19NXAnixlTGeAvBdANcBeB3NIA7wOwKQef9Rlyz7T5q+BOAHqvqjsrfVNVmPPVXdq6ofA1B4+ZPBoVpDWMy0gWbgHgLwCIAbROTrKH/ark8i95+IvENEvgFgo4jcXc2meSHu+LsTwIcA3Cgid1SxYR6IO/Y+KCJfE5FvooQMnPfEdJCqngBwW9Xb4StV/QWa9VvKQVW/BuBrVW+Hj7Tjxu9lYAZerQaASzoeX9x6jsxw//WG+y8/J/YdA3i1/hvAu0VknYisAnAzgL0Vb5NPuP96w/2XnxP7jgG8JCLyAID/ALBeRF4XkdtV9TSAvwKwH8DzAB5U1eeq3E5Xcf/1hvsvP5f3HRezIiLyFDNwIiJPMYATEXmKAZyIyFMM4EREnmIAJyLyFAM4EZGnGMCJiDzFAE5E5CkGcCIiT/0/bPIh89pOtSAAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "\n",
    "# where pop is mapped on the horizontal axis, and life_exp is mapped on the vertical axis. \n",
    "plt.scatter(pop, life_exp)\n",
    "plt.xscale('log')\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "df6254de-f5ef-4a98-878e-dede662008b2",
   "metadata": {},
   "source": [
    "## Writing an iterator to load data in chunks (4)\n",
    "\n",
    "In the previous exercises, you've only processed the data from the first DataFrame chunk. This time, you will aggregate the results over all the DataFrame chunks in the dataset. This basically means you will be processing the entire dataset now. This is neat because you're going to be able to process the entire large dataset by just working on smaller pieces of it!\n",
    "\n",
    "You're going to use the data from 'ind_pop_data.csv', available in your current directory. The packages pandas and matplotlib.pyplot have been imported as pd and plt respectively for your use.\n",
    "Instructions\n",
    "100 XP\n",
    "\n",
    "    Initialize an empty DataFrame data using pd.DataFrame().\n",
    "    In the for loop, iterate over urb_pop_reader to be able to process all the DataFrame chunks in the dataset.\n",
    "    Using the method append() of the DataFrame data, append df_pop_ceb to data.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 91,
   "id": "e80dfb7e-e12a-4cb3-9160-e351d94f71c5",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/tmp/ipykernel_2171/4167670917.py:21: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame.\n",
      "Try using .loc[row_indexer,col_indexer] = value instead\n",
      "\n",
      "See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n",
      "  df_pop_ceb['Total Urban Population'] = [int(tup[0] * tup[1] * 0.01) for tup in pops_list]\n",
      "/tmp/ipykernel_2171/4167670917.py:28: UserWarning: Matplotlib is currently using agg, which is a non-GUI backend, so cannot show the figure.\n",
      "  plt.show()\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAgFUlEQVR4nO3de5RcZZnv8e+vcxWSkEACIiEEBPUwTMKlRT1xOYBnGEUH9ICKwsjFORkcuXiNuI7OIJ6ZNbDUI14AGUYOCgOOgENARJkJiDdwOphEgqIhoiSiIeGSi+Taz/lj7yI7larqXd2967Z/n7VqddXeu3Y9b7rTT+/9Pu/7KiIwM7Py6mt3AGZm1l5OBGZmJedEYGZWck4EZmYl50RgZlZyTgRmZiXnRGBmVnJOBGY5SHqXpAFJGyU9Kenbkl4r6RJJ29LtlcezmfeFpE3p9rWSbpI0tX0tMdudE4HZECR9EPgc8I/AfsAs4ErglPSQr0fEpMxjatUp5kbEJOAQYBpwSSviNsvLicCsAUl7AZcC74uI2yJiU0Rsi4g7IuIjzZwrItYDC4HDi4jVbLicCMwaew0wEfjmSE8kaRrwFuCBkZ7LbDR1ZSKQ9BVJayQ9nOPY/ytpSfr4Zfb+rVkO+wBrI2J7g2PeLunZzOPeqv0PpT93a0luK325oFjNhqUrEwHw/4A35DkwIj4QEUdGxJHAF4DbCozLes86YLqksQ2O+beImJp5HF+1/+i032AicBXwfUkTC4rXrGldmQgi4n7g6ew2SS+VdLekxZK+L+kVNd76TuCmlgRpveLHwBaSWzojEhHbgGuBg4EjRno+s9HS6K+cbnMNcF5E/ErSq0iqOk6o7JR0EMl/wEVtis+6UEQ8J+nvgC9J2g58F9gG/A/geOCPec8laQxwDvA8sLKAcM2GpScSgaRJwH8HviGpsnlC1WGnA7dExI5WxmbdLyI+I+n3wMeBG4ENwGLgH4ATgXdIekvV2w6JiDXp86WSAhgEHgXeGhFPY9Yh1K0L00iaDdwZEUdImgI8GhH7Nzj+pyQlgD9qVYxmZt2gK/sIqqX12b+W9DYAJeZW9qf9BdNI7veamVlGVyYCSTeR/FJ/uaRVkt4DnAG8R9JSYDk7R31Cclvo5ujWyx8zswJ17a0hMzMbHV15RWBmZqOn66qGpk+fHrNnz253GGZmXWXx4sVrI2JGrX1dlwhmz57NwMBAu8MwM+sqkn5Tb59vDZmZlZwTgZlZyTkRmJmVnBOBmVnJORGYmZWcE4FZB1m3cQtLn3iWdRu3tDsUK5GuKx8161W3L1nNR29dxri+PrYNDnL5qXM4+cgDWLdxC6ueeZ6Z017EPpN2Tqpba3u9Y80acSIwa4PqX9jrNm7ho7cuY/O2QTYzCMCCW5exYfN2PvWtR3ZLDrWSRkDNRFLr8+rFYeXkRGDWYrV+iR+0z56M6+t7IQkAjJH45J2PsHX7rsnh8P2n7JY0PnLLUkBsqTp23qHT+cGKtTUTRL0rECsf9xGYFaj6nn/2L/8NW7azedsgC25dxp7jx7BtcHCX927bMcj4Mdpl27i+PpY88Szj+nb9rztGfYzp2/3Y5b9bX/PzVvxhQ83t6zZucT9FCfmKwKwgef/yH9fXx6atO7j81DksyBz/iTcdzqe+9cgu59w2OMiRB07dLWnsiEEI7XYsRM3PqyST6u03PvhbrrxvRVO3l6z7ORGYFaDePf87z3/t7n/5Dw4yc9qLmHvgVOYdOn2XX7aTJ47dJTlcfuocDt1v8m5J4/JT57zwGdltf/KSvWp+Xq1ksnXHDr5074qmbi9V2uoE0d2cCMwKsOqZ53P/5X/5qXNe+AW6z6QJu/wyPfnIA3ZLDo2219pW6/NqJZP3HXco19y/ki3bd405e3spb4JwcuguTgRmBZg57UVN/eXfSHVyaLS91ra8yQTgS/et2C3mereX6iWIepVO1rncWWw2SrKdrPtMmsDlp85h4rg+Jk8Yy8Rxfbv95T/3wKkt+2u53udlt9eLud7tpUqCyKpUOtXqhLbO5SsCs1FQrxSzmb/8O0G9mGvdXqqZIHYMMn5sH1u379w2rq+PVc88/8J4iW769ygLJwKzEarXMTzv0Ol1b+t0smZuL+WtdJo57UUet9DBnAjMRqhex3Dlr+BekTdB1Kp0AhomS2svJwKzEWrUMVwGeSqdltYZt+BbRp3BicCsSdW/tCqdrPVKQsuoOjk0Spa+ZdR+TgRmTaj3S6sbO4ZbqV6yhPq3jAD/e7aIE4FZTnk6hf0Lq75mbhk1murCRp/HEZjlVOkUzqrc57Z8qscz1LplVJnqot5YhFqT4nmivJHxFYFZTmXvFC5CrVtG9aa6WPXM8zWntBjOOgy2KycCszrcKdwaeae62HP8mFFZhwGcIKo5EZjV4E7h1qruX6mVcDdt3VFj8Z4+2HX27WFPlFdmTgRmVdwp3H61Eu66jVtGvA5DowQB5a1SciIwq1KWkcKdrjrhNipBzbsOQ70EMZwFeXrp9lKhiUDSVOBa4AgggHMj4seZ/ccBtwO/TjfdFhGXFhmT2VDcKdy5RroOQ60EMZwFeXptEFzRVwRXAHdHxGmSxgN71Djm+xHx5oLjMMvNncKdbaTrMIx0QZ7D95/Sc7eXCksEkvYCXgecDRARW4GtRX2e2UhUX+a7U7g35EkQ0NyCPL243nORVwQHA08B10maCywGLoqITVXHvUbSUuB3wIcjYnn1iSTNB+YDzJo1q8CQrYzqXea7U7h35alSatd6z+1IGoqIYk4s9QMPAPMi4kFJVwDrI+ITmWOmAIMRsVHSScAVEXFYo/P29/fHwMBAITFb+azbuIV5ly1i87ad/7Enjuvjhx89wUmgZGr9Al64ZPVuCeLkIw/YbXvl9tKGLTtX5Jk8YSxfOuNo5n9toObPV6sHx0laHBH9tfYVeUWwClgVEQ+mr28BLs4eEBHrM8/vknSlpOkRsbbAuMxe4Aohq2j3es/DHRw3Ggqbaygifg88Ienl6abXA7ssXSTpxZKUPj82jWddUTGZVXOFkA2ldes99zGmb9cxEdVJo6h1oIuuGroAuDGtGFoJnCPpPICIuBo4DXivpO3A88DpUdS9KjM8bYQVZ6TrPTc7OG40r1oL6yMoivsIbLga1X53elWHdbe8/Q+w++C4eYdOH5V+rEZ9BE4EVgruFLZOlLdqqF6ndTPa1Vls1jHcKWydaKSD40aLE4GVgjuFrdsVOa7FK5RZT6pesapeVYevBsx8RWA9yGsJmDXHicB6itcSMGuebw1ZT/EC82bNcyKwnuJOYbPmORFYT3GnsFnz3EdgXc9rCZiNjBOBdTWvJWA2cr41ZF0rWyFU1KyMZmXgRGBdyxVCZqPDicC6liuEzEaHE4F1LVcImY0OdxZb16g1Pa8rhMxGLlcikDQG2C97fET8tqigzKo1WlTGFUJmIzNkIpB0AfD3wB/ghcncA5hTYFxmLxhq/iAzG5k8VwQXAS+PCC8qb23hRWXMipWns/gJ4LmiAzGrx9VBZsXKc0WwErhP0reAF0bqRMRnC4vKLKNSHVS9ZquvBsxGR55E8Nv0MT59mLWcq4PMijNkIoiITwJImpS+3lh0UGa1SkVdHWRWjDxVQ0cAXwP2Tl+vBd4dEcsLjs1KqlGpqJmNvjydxdcAH4yIgyLiIOBDwD8XG5aVlSeSM2u9PIlgz4i4t/IiIu4D9iwsIis1TyRn1nq5qoYkfYLk9hDAmSSVRGajzqWiZq2X54rgXGAGcFv6mJFuG5KkqZJukfQLST+X9Jqq/ZL0eUkrJC2TdHSzDbDe4onkzFovT9XQM8CFwzz/FcDdEXGapPHAHlX73wgclj5eBVyVfrUSc6moWWvVTQSSPhcR75d0B8ncQruIiJMbnVjSXsDrgLPT47cCW6sOOwX4akQE8EB6BbF/RDzZXDOs17hU1Kx1Gl0RVPoEPj3Mcx8MPAVcJ2kusBi4KCI2ZY45gGQKi4pV6bZdEoGk+cB8gFmzZg0zHOtEtcYLmFlr1e0jiIjF6dMjI+J72QdwZI5zjwWOBq6KiKOATcDFwwkyIq6JiP6I6J8xY8ZwTmEd6PYlq5l32SLOvPZB5l22iIVLVrc7JLNSytNZfFaNbWfneN8qYFVEPJi+voUkMWStBg7MvJ6ZbrMe5/ECZp2jUR/BO4F3AQdLWpjZNRl4eqgTR8TvJT0h6eUR8SjweuCRqsMWAudLupmkk/g59w+Ug6eWNuscjfoIfkRyr3468JnM9g3AspznvwC4Ma0YWgmcI+k8gIi4GrgLOAlYAfwROKep6K1rebyAWedQUrDTPfr7+2NgYKDdYdgoWLhk9W5TS3tOIbNiSFocEf219uWZdO7VwBeA/0YyDfUYYFNETBnVKK10PF7ArDPkmWLii8DpwDeAfuDdwMuKDMrKw+MFzNovT9UQEbECGBMROyLiOuANxYZlZmatkueK4I9pZ+8SSZeTdCDnSiBmFR44Zta58iSCvyLpFzgf+ABJ3f+pRQZlvcULzZh1tjyTzv0mffo88Mliw7Fekx04VhkzsODWZcw7dLqvDMw6RKMBZT+jxmRzFRExp5CIrKd44JhZ52t0RfDmlkVhPcsDx8w6X91EkLklZDZslYVmqgeO+WrArHPkGVC2gZ23iMYD4/CAMmuCB46ZdbY8ncWTK88liWQxmVcXGZT1Hg8cM+tcTY0HiMS/A39RTDjWC9Zt3MLSJ571lNJmXSLPraH/mXnZRzLNxObCIrKu5jEDZt0nz4Cyv8w83w48TnJ7yGwXHjNg1p3y9BF4jQDLxWMGzLrTkH0Ekg6RdIekpyStkXS7pENaEZx1F48ZMOtOeTqL/xX4N2B/4CUk01HfVGRQ1p0qYwYmjutj8oSxTBzX5zEDZl0gTx/BHhHxtczrGyR9pKiArLt5zIBZ98mTCL4t6WLgZpKBZe8A7pK0N0BEDLmQvZWLxwyYdZc8ieDt6de/qdp+OklicH+BmVkXy1M1dHArArHu48VmzHpDngFl44D3Aq9LN90HfDkithUYl3U4Dxwz6x15qoauAo4Brkwfx6TbrKSyA8c2bNnO5m2DLLh1maeUMOtSefoIXhkRczOvF0laWlRA1vk8cMyst+S5Itgh6aWVF+lgsh3FhWSdzgPHzHpLnkTwEeBeSfdJ+h6wCPhQsWFZJ/PAMbPe0vDWkKQZwHPAscC+6eZHI8I3g0vOA8fMekejxev/GvhH4DHgYGB+RCxs5uSSHgc2kNxK2h4R/VX7jwNuB36dbrotIi5t5jOsfTxwzKw3NLoieD/wJxHxVNovcCPQVCJIHR8Raxvs/35EvHkY5zUzs1HQqI9ga0Q8BRARKwH/6VdSXnHMrLc1uiKYKenz9V5HxIU5zh/AdyUFySC0a2oc85q0HPV3wIcjYnn1AZLmA/MBZs2aleNjbbR44JhZ71NE1N4hndXojRFx/ZAnlw6IiNWS9gXuAS6IiPsz+6cAgxGxUdJJwBURcVijc/b398fAwMBQH22jYN3GLcy7bBGbt+0sFZ04ro8ffvQE9w2YdRlJi6v7aSvqXhHk+UU/lIhYnX5dI+mbJNVH92f2r888v0vSlZKmD9GnYC3igWNm5ZBnHMGwSNpT0uTKc+BE4OGqY14sSenzY9N41hUVkzXHA8fMyqGwRADsB/wgvf//E+BbEXG3pPMknZcecxrwcHrM54HTo969Kms5DxwzK4e6fQSdyn0Erefpps2637D6CDJvngH8L2B29viIOHe0ArTO5oFjZr0tz+yjtwPfB/4DTzZnZtZz8i5e/9HCI7GO4NtAZuWTJxHcKemkiLir8GisrTx4zKyc8lQNXUSSDJ6XtF7SBknrh3yXdRWvOmZWXnkWr5/cikCsvTx4zKy88twaQtI04DBgYmVbdqoI634ePGZWXkPeGkrXJbgf+A7wyfTrJcWGZa3mwWNm5ZXniuAi4JXAAxFxvKRXkCxYYz3Gq46ZlVOeRLA5IjZLQtKEiPiFpJcXHpm1hQePmZVPnkSwStJU4N+BeyQ9A/ymyKCseB4vYGYVeaqG3po+vUTSvcBewN2FRmWF8ngBM8vKNfuopKMlXQjMAVZFxNZiw7KieLyAmVXLUzX0d8D1wD7AdOA6SR8vOjArRmW8QFZlvICZlVOePoIzgLkRsRlA0j8BS4D/U2BcVhCPFzCzanluDf2OzEAyYAKwuphwrGgeL2Bm1epeEUj6AhDAc8BySfekr/+cZMUx61IeL2BmWY1uDVWWAVsMfDOz/b7CorGW8XgBM6uomwgi4npJY4CvRsQZLYzJRpHHC5jZUBp2FkfEDkkHSRrvktHu4/ECZpZHnqqhlcAPJS0ENlU2RsRnC4vKRiw7XqAytfSCW5cx79DpvjIws13kSQSPpY8+wGsTdAmvL2BmeeWZYuKTrQjERpfHC5hZXo3KR+8gKRetCGAtcG9E3FB0YDYylfECC6r6CHw1YGbVGl0RfLrGtr2BMyUdEREXFxSTDVN1hZDHC5hZHo3KR79Xa3vaabwYcCLoIPUqhDxewMyGkmv20ayI2FFEIDZ8nlHUzEaiUR/B3jU2TwPeDSzPc3JJjwMbgB3A9ojor9ov4ArgJOCPwNkR8VCuyO0FrhAys5Fo1EewmKSDWOnrSmfxfcB7m/iM4yNibZ19bwQOSx+vAq5Kv1oTXCFkZiPRqI/g4BZ8/ikkU1gE8ICkqZL2j4gnW/DZPcMVQmY2EnkGlI1EAN+VFMCXI+Kaqv0HAE9kXq9Kt+2SCCTNB+YDzJo1q7hou0St+YNcIWRmw1V0InhtRKyWtC/Jwve/iIj7mz1JmkCuAejv748hDu9pjeYPcoWQmQ1H01VDzYiI1enXNSRTWR9bdchq4MDM65l40Zu6XB1kZkVoVDV0dKM3DlXdI2lPoC8iNqTPTwQurTpsIXC+pJtJOomfc/9Afa4OMrMiNLo19JkG+wI4YYhz7wd8M6kQZSzwrxFxt6TzACLiauAuktLRFSTlo+fkjLuUXB1kZkVoVDV0/EhOHBErgbk1tl+deR7A+0byOWXi6iAzK0KuzmJJRwCHk1nEPiK+WlRQVp+rg8xstA2ZCCT9PXAcSSK4i2QQ2A8AJ4IWqFUq6uogMxtNea4ITiO5xfPTiDhH0n6Ap6FuAS81aWatkKd89PmIGAS2S5oCrGHXkk8rgEtFzaxV8iSCAUlTgX8mmX/oIeDHRQZlO0tFsyqlomZmoynPUpV/mz69WtLdwJSIWFZsWOZSUTNrlSGvCCT9Z+V5RDweEcuy26wYlVLRieP6mDxhLBPH9blU1MwK0Whk8URgD2C6pGnsnI56CsnEcFYwl4qaWSs0ujX0N8D7gZeQ9AtUrAe+WGBMpVSrTBRcKmpmxWs0svgK4ApJF0TEF1oYU+m4TNTM2ilP1dCXJV0o6Zb0cb6kcYVHVhIuEzWzdsuTCK4Ejkm/Vp5fVWRQZeIyUTNrt0adxWMjYjvwyojITh63SNLS4kMrB5eJmlm7Nboi+En6dYekl1Y2SjoE2FFoVCXiMlEza7dGVUOVctEPA/dKWpm+no3XDRhVLhM1s3ZqlAhmSPpg+vzLwJj0+Q7gKODeIgPrVS4TNbNO0ygRjAEmsfPKIPueyYVF1MNcJmpmnahRIngyIqrXGLZhypaJVtYcXnDrMuYdOt1XAmbWVo06i6uvBGwEXCZqZp2qUSJ4fcuiKAGXiZpZp6qbCCLi6VYG0utcJmpmnSrX4vU2OlwmamadyImgQF543sy6gRNBQVwqambdIs+kc9YkzyhqZt3EiaAALhU1s27iRFAAl4qaWTcpPBFIGiPpp5LurLHvbElPSVqSPv666HhawaWiZtZNWtFZfBHwc5JF72v5ekSc34I4WsqlombWLQq9IpA0E3gTcG2Rn9Nu6zZuYekTz+7WGbzPpAnMPXCqk4CZdbSirwg+Byyg8Wylp0p6HfBL4AMR8UT1AZLmA/MBZs2aVUCYw+cyUTPrdoVdEUh6M7AmIhY3OOwOYHZEzAHuAa6vdVBEXBMR/RHRP2PGjAKiHR6XiZpZLyjy1tA84GRJjwM3AydIuiF7QESsi4jKb81rgWMKjGfUuUzUzHpBYYkgIj4WETMjYjZwOrAoIs7MHiNp/8zLk0k6lbuGy0TNrBe0fByBpEslnZy+vFDScklLgQuBs1sdz0i4TNTMeoEiot0xNKW/vz8GBgbaHcYu6q1DbGbWKSQtjoj+Wvs86VwTvPC8mfUiJ4KcXCZqZr3Kcw3l4DJRM+tlTgQ5uEzUzHqZE0EOLhM1s17mRJCDy0TNrJe5szgnzyZqZr3KiaAJLhM1s17kW0N11Jta2sys1/iKoAaPGTCzMvEVQRWPGTCzsnEiqOIxA2ZWNk4EVTxmwMzKxomgiscMmFnZuLO4Bo8ZMLMyKX0i8NTSZlZ2pU4ELhM1MytxH4HLRM3MEqVNBC4TNTNLlDYRuEzUzCxR2kTgMlEzs0SpO4tdJmpmVqJE4DJRM7PaSpEIXCZqZlZfz/cRuEzUzKyxnk8ELhM1M2us5xOBy0TNzBorPBFIGiPpp5LurLFvgqSvS1oh6UFJs0f7810mambWWCs6iy8Cfg5MqbHvPcAzEXGopNOBy4B3jHYALhM1M6uv0CsCSTOBNwHX1jnkFOD69PktwOslqYhY9pk0gbkHTnUSMDOrUvStoc8BC4DBOvsPAJ4AiIjtwHPAPtUHSZovaUDSwFNPPVVQqGZm5VRYIpD0ZmBNRCwe6bki4pqI6I+I/hkzZoxCdGZmVlHkFcE84GRJjwM3AydIuqHqmNXAgQCSxgJ7AesKjMnMzKoUlggi4mMRMTMiZgOnA4si4syqwxYCZ6XPT0uPiaJiMjOz3bV8iglJlwIDEbEQ+Bfga5JWAE+TJAwzM2shddsf4JKeAn4zzLdPB9aOYjidqNfb2Ovtg95vo9vXHgdFRM1O1q5LBCMhaSAi+tsdR5F6vY293j7o/Ta6fZ2n56eYMDOzxpwIzMxKrmyJ4Jp2B9ACvd7GXm8f9H4b3b4OU6o+AjMz213ZrgjMzKyKE4GZWcl1fSKQ9BVJayQ9nNk2V9KPJf1M0h2SpmT2zUn3LU/3T0y3H5O+XiHp80XNgtqsZton6QxJSzKPQUlHpvs6sn3QdBvHSbo+3f5zSR/LvOcNkh5N23hxO9pSS5PtGy/punT7UknHZd7Tkd9DSQdKulfSI+n/q4vS7XtLukfSr9Kv09LtSuNfIWmZpKMz5zorPf5Xks6q95mtNIz2vSL93m6R9OGqc3XkzygR0dUP4HXA0cDDmW3/BfxZ+vxc4FPp87HAMmBu+nofYEz6/CfAqwEB3wbe2O62Ndu+qvf9KfBY5nVHtm8Y38N3ATenz/cAHgdmA2OAx4BDgPHAUuDwdrdtGO17H3Bd+nxfYDHQ18nfQ2B/4Oj0+WTgl8DhwOXAxen2i4HL0ucnpfErbc+D6fa9gZXp12np82ld2L59gVcC/wB8OHOejv0Z7forgoi4n2R6iqyXAfenz+8BTk2fnwgsi4il6XvXRcQOSfsDUyLigUi+Y18F3lJ48Dk02b6sd5JM9kcntw+abmMAeyqZpPBFwFZgPXAssCIiVkbEVpK2n1J07Hk02b7DgUXp+9YAzwL9nfw9jIgnI+Kh9PkGkoWoDmDX9UauZ2e8pwBfjcQDwNS0fX8B3BMRT0fEMyT/Lm9oXUtqa7Z9EbEmIv4L2FZ1qo79Ge36RFDHcnb+A7+NdIZTkv98Iek7kh6StCDdfgCwKvP+Vem2TlWvfVnvAG5Kn3db+6B+G28BNgFPAr8FPh0RT5NZ2yLV6W2s176lJLP2jpV0MHBMuq8rvodKlps9CngQ2C8inkx3/R7YL31e73vV8d/DnO2rp2Pb16uJ4FzgbyUtJrmU25puHwu8Fjgj/fpWSa9vT4gjUq99AEh6FfDHiHi41pu7RL02HgvsAF4CHAx8SNIh7QlxROq17yskvyAGSBZ2+hFJezuepEnArcD7I2J9dl96FdPVteq93L6Wzz7aChHxC5LbQEh6GclymZD8B7s/Itam++4iuXd7AzAzc4qZJGsldKQG7as4nZ1XA5C0pWvaBw3b+C7g7ojYBqyR9EOgn+QvreyVUUe3sV77Ilmp7wOV4yT9iOSe9DN08PdQ0jiSX5I3RsRt6eY/SNo/Ip5Mb/2sSbe/sA5JqtKW1cBxVdvvKzLuvJpsXz312t12PXlFIGnf9Gsf8HHg6nTXd4A/lbRHeo/5z4BH0su79ZJenVZivBu4vQ2h59KgfZVtbyftH4DkHidd1D5o2MbfAiek+/Yk6Wz8BUnn62GSDpY0niQZLmx13HnVa1/6s7ln+vzPge0R0dE/o2k8/wL8PCI+m9mVXW/kLHbGuxB4d1o99GrgubR93wFOlDQtrcA5Md3WVsNoXz2d+zPa7t7qkT5I/vJ9kqRjZhXwHuAikr+ifgn8E+kI6vT4M0nuzz4MXJ7Z3p9uewz4YvY9Xda+44AHapynI9vXbBuBScA30u/hI8BHMuc5KT3+MeB/t7tdw2zfbOBRkg7J/yCZOrijv4ckt1mDpCJvSfo4iaQq7z+BX6Vt2Ts9XsCX0nb8DOjPnOtcYEX6OKfdbRtm+16cfp/Xk3T2ryLp6O/Yn1FPMWFmVnI9eWvIzMzycyIwMys5JwIzs5JzIjAzKzknAjOzknMiMBtCWu/+A0lvzGx7m6S72xmX2Whx+ahZDpKOIBm/cBTJiPyfAm+IiMeGca6xkYwgNusITgRmOUm6nGTCuz3TrwcBRwDjgEsi4vZ0UrKvpccAnB8RP1KyrsCnSKaKeEVEvKy10ZvV50RgllM69cNDJBPE3Qksj4gbJE0lWSvgKJIRqIMRsVnSYcBNEdGfJoJvAUdExK/bEb9ZPT056ZxZESJik6SvAxtJ5nP6y8wKVBOBWcDvgC8qWRluB8nU5xU/cRKwTuREYNacwfQh4NSIeDS7U9IlwB+AuSTFGJszuze1KEazprhqyGx4vgNckM5MiaSj0u17AU9GxCDwVyTLE5p1NCcCs+H5FEkn8TJJy9PXAFcCZ0laCrwCXwVYF3BnsZlZyfmKwMys5JwIzMxKzonAzKzknAjMzErOicDMrOScCMzMSs6JwMys5P4/rXMQ81nupxQAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Initialize reader object: urb_pop_reader\n",
    "urb_pop_reader = pd.read_csv('ind_pop_data.csv', chunksize=1000)\n",
    "\n",
    "# Initialize empty DataFrame: data\n",
    "data = pd.DataFrame()\n",
    "\n",
    "# Iterate over each DataFrame chunk   ################################################################\n",
    "for df_urb_pop in urb_pop_reader:     # Cause its a iterator, thus we can loop over it, chunk by chunk\n",
    "\n",
    "    # Check out specific country: df_pop_ceb\n",
    "    df_pop_ceb = df_urb_pop[df_urb_pop['CountryCode'] == 'CEB']\n",
    "\n",
    "    # Zip DataFrame columns of interest: pops\n",
    "    pops = zip(df_pop_ceb['Total Population'],\n",
    "                df_pop_ceb['Urban population (% of total)'])\n",
    "\n",
    "    # Turn zip object into list: pops_list\n",
    "    pops_list = list(pops)\n",
    "\n",
    "    # Use list comprehension to create new DataFrame column 'Total Urban Population'\n",
    "    df_pop_ceb['Total Urban Population'] = [int(tup[0] * tup[1] * 0.01) for tup in pops_list]\n",
    "    \n",
    "    # Append DataFrame chunk to data: data\n",
    "    data = data.append(df_pop_ceb)\n",
    "\n",
    "# Plot urban population data\n",
    "data.plot(kind='scatter', x='Year', y='Total Urban Population', title='CEB')\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "22e29d45-265f-4978-80a7-438d460f5862",
   "metadata": {},
   "source": [
    "## Writing an iterator to load data in chunks (5)\n",
    "\n",
    "This is the last leg. You've learned a lot about processing a large dataset in chunks. In this last exercise, you will put all the code for processing the data into a single function so that you can reuse the code without having to rewrite the same things all over again.\n",
    "\n",
    "You're going to define the function plot_pop() which takes two arguments: the filename of the file to be processed, and the country code of the rows you want to process in the dataset.\n",
    "\n",
    "Because all of the previous code you've written in the previous exercises will be housed in plot_pop(), calling the function already does the following:\n",
    "\n",
    "    Loading of the file chunk by chunk,\n",
    "    Creating the new column of urban population values, and\n",
    "    Plotting the urban population data.\n",
    "\n",
    "That's a lot of work, but the function now makes it convenient to repeat the same process for whatever file and country code you want to process and visualize!\n",
    "\n",
    "You're going to use the data from 'ind_pop_data.csv', available in your current directory. The packages pandas and matplotlib.pyplot has been imported as pd and plt respectively for your use.\n",
    "\n",
    "After you are done, take a moment to look at the plots and reflect on the new skills you have acquired. The journey doesn't end here! If you have enjoyed working with this data, you can continue exploring it using the pre-processed version available on Kaggle.\n",
    "Instructions\n",
    "100 XP\n",
    "\n",
    "    Define the function plot_pop() that has two arguments: first is filename for the file to process and second is country_code for the country to be processed in the dataset.\n",
    "    Call plot_pop() to process the data for country code 'CEB' in the file 'ind_pop_data.csv'.\n",
    "    Call plot_pop() to process the data for country code 'ARB' in the file 'ind_pop_data.csv'.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "43ebe6a8-d3ac-4fde-9792-5d7735ccbb9f",
   "metadata": {},
   "source": [
    "# *****************************************************************************************************\n",
    "# *****************************************************************************************************\n",
    "# *****************************************************************************************************\n",
    "# *****************************************************************************************************\n",
    "# *****************************************************************************************************\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 92,
   "id": "24f27d11-c0ad-4053-9347-9dca82f83509",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/tmp/ipykernel_2171/2864785283.py:23: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame.\n",
      "Try using .loc[row_indexer,col_indexer] = value instead\n",
      "\n",
      "See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n",
      "  df_pop_ceb['Total Urban Population'] = [int(tup[0] * tup[1] * 0.01) for tup in pops_list]\n",
      "/tmp/ipykernel_2171/2864785283.py:30: UserWarning: Matplotlib is currently using agg, which is a non-GUI backend, so cannot show the figure.\n",
      "  plt.show()\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAgFUlEQVR4nO3de5RcZZnv8e+vcxWSkEACIiEEBPUwTMKlRT1xOYBnGEUH9ICKwsjFORkcuXiNuI7OIJ6ZNbDUI14AGUYOCgOOgENARJkJiDdwOphEgqIhoiSiIeGSi+Taz/lj7yI7larqXd2967Z/n7VqddXeu3Y9b7rTT+/9Pu/7KiIwM7Py6mt3AGZm1l5OBGZmJedEYGZWck4EZmYl50RgZlZyTgRmZiXnRGBmVnJOBGY5SHqXpAFJGyU9Kenbkl4r6RJJ29LtlcezmfeFpE3p9rWSbpI0tX0tMdudE4HZECR9EPgc8I/AfsAs4ErglPSQr0fEpMxjatUp5kbEJOAQYBpwSSviNsvLicCsAUl7AZcC74uI2yJiU0Rsi4g7IuIjzZwrItYDC4HDi4jVbLicCMwaew0wEfjmSE8kaRrwFuCBkZ7LbDR1ZSKQ9BVJayQ9nOPY/ytpSfr4Zfb+rVkO+wBrI2J7g2PeLunZzOPeqv0PpT93a0luK325oFjNhqUrEwHw/4A35DkwIj4QEUdGxJHAF4DbCozLes86YLqksQ2O+beImJp5HF+1/+i032AicBXwfUkTC4rXrGldmQgi4n7g6ew2SS+VdLekxZK+L+kVNd76TuCmlgRpveLHwBaSWzojEhHbgGuBg4EjRno+s9HS6K+cbnMNcF5E/ErSq0iqOk6o7JR0EMl/wEVtis+6UEQ8J+nvgC9J2g58F9gG/A/geOCPec8laQxwDvA8sLKAcM2GpScSgaRJwH8HviGpsnlC1WGnA7dExI5WxmbdLyI+I+n3wMeBG4ENwGLgH4ATgXdIekvV2w6JiDXp86WSAhgEHgXeGhFPY9Yh1K0L00iaDdwZEUdImgI8GhH7Nzj+pyQlgD9qVYxmZt2gK/sIqqX12b+W9DYAJeZW9qf9BdNI7veamVlGVyYCSTeR/FJ/uaRVkt4DnAG8R9JSYDk7R31Cclvo5ujWyx8zswJ17a0hMzMbHV15RWBmZqOn66qGpk+fHrNnz253GGZmXWXx4sVrI2JGrX1dlwhmz57NwMBAu8MwM+sqkn5Tb59vDZmZlZwTgZlZyTkRmJmVnBOBmVnJORGYmZWcE4FZB1m3cQtLn3iWdRu3tDsUK5GuKx8161W3L1nNR29dxri+PrYNDnL5qXM4+cgDWLdxC6ueeZ6Z017EPpN2Tqpba3u9Y80acSIwa4PqX9jrNm7ho7cuY/O2QTYzCMCCW5exYfN2PvWtR3ZLDrWSRkDNRFLr8+rFYeXkRGDWYrV+iR+0z56M6+t7IQkAjJH45J2PsHX7rsnh8P2n7JY0PnLLUkBsqTp23qHT+cGKtTUTRL0rECsf9xGYFaj6nn/2L/8NW7azedsgC25dxp7jx7BtcHCX927bMcj4Mdpl27i+PpY88Szj+nb9rztGfYzp2/3Y5b9bX/PzVvxhQ83t6zZucT9FCfmKwKwgef/yH9fXx6atO7j81DksyBz/iTcdzqe+9cgu59w2OMiRB07dLWnsiEEI7XYsRM3PqyST6u03PvhbrrxvRVO3l6z7ORGYFaDePf87z3/t7n/5Dw4yc9qLmHvgVOYdOn2XX7aTJ47dJTlcfuocDt1v8m5J4/JT57zwGdltf/KSvWp+Xq1ksnXHDr5074qmbi9V2uoE0d2cCMwKsOqZ53P/5X/5qXNe+AW6z6QJu/wyPfnIA3ZLDo2219pW6/NqJZP3HXco19y/ki3bd405e3spb4JwcuguTgRmBZg57UVN/eXfSHVyaLS91ra8yQTgS/et2C3mereX6iWIepVO1rncWWw2SrKdrPtMmsDlp85h4rg+Jk8Yy8Rxfbv95T/3wKkt+2u53udlt9eLud7tpUqCyKpUOtXqhLbO5SsCs1FQrxSzmb/8O0G9mGvdXqqZIHYMMn5sH1u379w2rq+PVc88/8J4iW769ygLJwKzEarXMTzv0Ol1b+t0smZuL+WtdJo57UUet9DBnAjMRqhex3Dlr+BekTdB1Kp0AhomS2svJwKzEWrUMVwGeSqdltYZt+BbRp3BicCsSdW/tCqdrPVKQsuoOjk0Spa+ZdR+TgRmTaj3S6sbO4ZbqV6yhPq3jAD/e7aIE4FZTnk6hf0Lq75mbhk1murCRp/HEZjlVOkUzqrc57Z8qscz1LplVJnqot5YhFqT4nmivJHxFYFZTmXvFC5CrVtG9aa6WPXM8zWntBjOOgy2KycCszrcKdwaeae62HP8mFFZhwGcIKo5EZjV4E7h1qruX6mVcDdt3VFj8Z4+2HX27WFPlFdmTgRmVdwp3H61Eu66jVtGvA5DowQB5a1SciIwq1KWkcKdrjrhNipBzbsOQ70EMZwFeXrp9lKhiUDSVOBa4AgggHMj4seZ/ccBtwO/TjfdFhGXFhmT2VDcKdy5RroOQ60EMZwFeXptEFzRVwRXAHdHxGmSxgN71Djm+xHx5oLjMMvNncKdbaTrMIx0QZ7D95/Sc7eXCksEkvYCXgecDRARW4GtRX2e2UhUX+a7U7g35EkQ0NyCPL243nORVwQHA08B10maCywGLoqITVXHvUbSUuB3wIcjYnn1iSTNB+YDzJo1q8CQrYzqXea7U7h35alSatd6z+1IGoqIYk4s9QMPAPMi4kFJVwDrI+ITmWOmAIMRsVHSScAVEXFYo/P29/fHwMBAITFb+azbuIV5ly1i87ad/7Enjuvjhx89wUmgZGr9Al64ZPVuCeLkIw/YbXvl9tKGLTtX5Jk8YSxfOuNo5n9toObPV6sHx0laHBH9tfYVeUWwClgVEQ+mr28BLs4eEBHrM8/vknSlpOkRsbbAuMxe4Aohq2j3es/DHRw3Ggqbaygifg88Ienl6abXA7ssXSTpxZKUPj82jWddUTGZVXOFkA2ldes99zGmb9cxEdVJo6h1oIuuGroAuDGtGFoJnCPpPICIuBo4DXivpO3A88DpUdS9KjM8bYQVZ6TrPTc7OG40r1oL6yMoivsIbLga1X53elWHdbe8/Q+w++C4eYdOH5V+rEZ9BE4EVgruFLZOlLdqqF6ndTPa1Vls1jHcKWydaKSD40aLE4GVgjuFrdsVOa7FK5RZT6pesapeVYevBsx8RWA9yGsJmDXHicB6itcSMGuebw1ZT/EC82bNcyKwnuJOYbPmORFYT3GnsFnz3EdgXc9rCZiNjBOBdTWvJWA2cr41ZF0rWyFU1KyMZmXgRGBdyxVCZqPDicC6liuEzEaHE4F1LVcImY0OdxZb16g1Pa8rhMxGLlcikDQG2C97fET8tqigzKo1WlTGFUJmIzNkIpB0AfD3wB/ghcncA5hTYFxmLxhq/iAzG5k8VwQXAS+PCC8qb23hRWXMipWns/gJ4LmiAzGrx9VBZsXKc0WwErhP0reAF0bqRMRnC4vKLKNSHVS9ZquvBsxGR55E8Nv0MT59mLWcq4PMijNkIoiITwJImpS+3lh0UGa1SkVdHWRWjDxVQ0cAXwP2Tl+vBd4dEcsLjs1KqlGpqJmNvjydxdcAH4yIgyLiIOBDwD8XG5aVlSeSM2u9PIlgz4i4t/IiIu4D9iwsIis1TyRn1nq5qoYkfYLk9hDAmSSVRGajzqWiZq2X54rgXGAGcFv6mJFuG5KkqZJukfQLST+X9Jqq/ZL0eUkrJC2TdHSzDbDe4onkzFovT9XQM8CFwzz/FcDdEXGapPHAHlX73wgclj5eBVyVfrUSc6moWWvVTQSSPhcR75d0B8ncQruIiJMbnVjSXsDrgLPT47cCW6sOOwX4akQE8EB6BbF/RDzZXDOs17hU1Kx1Gl0RVPoEPj3Mcx8MPAVcJ2kusBi4KCI2ZY45gGQKi4pV6bZdEoGk+cB8gFmzZg0zHOtEtcYLmFlr1e0jiIjF6dMjI+J72QdwZI5zjwWOBq6KiKOATcDFwwkyIq6JiP6I6J8xY8ZwTmEd6PYlq5l32SLOvPZB5l22iIVLVrc7JLNSytNZfFaNbWfneN8qYFVEPJi+voUkMWStBg7MvJ6ZbrMe5/ECZp2jUR/BO4F3AQdLWpjZNRl4eqgTR8TvJT0h6eUR8SjweuCRqsMWAudLupmkk/g59w+Ug6eWNuscjfoIfkRyr3468JnM9g3AspznvwC4Ma0YWgmcI+k8gIi4GrgLOAlYAfwROKep6K1rebyAWedQUrDTPfr7+2NgYKDdYdgoWLhk9W5TS3tOIbNiSFocEf219uWZdO7VwBeA/0YyDfUYYFNETBnVKK10PF7ArDPkmWLii8DpwDeAfuDdwMuKDMrKw+MFzNovT9UQEbECGBMROyLiOuANxYZlZmatkueK4I9pZ+8SSZeTdCDnSiBmFR44Zta58iSCvyLpFzgf+ABJ3f+pRQZlvcULzZh1tjyTzv0mffo88Mliw7Fekx04VhkzsODWZcw7dLqvDMw6RKMBZT+jxmRzFRExp5CIrKd44JhZ52t0RfDmlkVhPcsDx8w6X91EkLklZDZslYVmqgeO+WrArHPkGVC2gZ23iMYD4/CAMmuCB46ZdbY8ncWTK88liWQxmVcXGZT1Hg8cM+tcTY0HiMS/A39RTDjWC9Zt3MLSJ571lNJmXSLPraH/mXnZRzLNxObCIrKu5jEDZt0nz4Cyv8w83w48TnJ7yGwXHjNg1p3y9BF4jQDLxWMGzLrTkH0Ekg6RdIekpyStkXS7pENaEZx1F48ZMOtOeTqL/xX4N2B/4CUk01HfVGRQ1p0qYwYmjutj8oSxTBzX5zEDZl0gTx/BHhHxtczrGyR9pKiArLt5zIBZ98mTCL4t6WLgZpKBZe8A7pK0N0BEDLmQvZWLxwyYdZc8ieDt6de/qdp+OklicH+BmVkXy1M1dHArArHu48VmzHpDngFl44D3Aq9LN90HfDkithUYl3U4Dxwz6x15qoauAo4Brkwfx6TbrKSyA8c2bNnO5m2DLLh1maeUMOtSefoIXhkRczOvF0laWlRA1vk8cMyst+S5Itgh6aWVF+lgsh3FhWSdzgPHzHpLnkTwEeBeSfdJ+h6wCPhQsWFZJ/PAMbPe0vDWkKQZwHPAscC+6eZHI8I3g0vOA8fMekejxev/GvhH4DHgYGB+RCxs5uSSHgc2kNxK2h4R/VX7jwNuB36dbrotIi5t5jOsfTxwzKw3NLoieD/wJxHxVNovcCPQVCJIHR8Raxvs/35EvHkY5zUzs1HQqI9ga0Q8BRARKwH/6VdSXnHMrLc1uiKYKenz9V5HxIU5zh/AdyUFySC0a2oc85q0HPV3wIcjYnn1AZLmA/MBZs2aleNjbbR44JhZ71NE1N4hndXojRFx/ZAnlw6IiNWS9gXuAS6IiPsz+6cAgxGxUdJJwBURcVijc/b398fAwMBQH22jYN3GLcy7bBGbt+0sFZ04ro8ffvQE9w2YdRlJi6v7aSvqXhHk+UU/lIhYnX5dI+mbJNVH92f2r888v0vSlZKmD9GnYC3igWNm5ZBnHMGwSNpT0uTKc+BE4OGqY14sSenzY9N41hUVkzXHA8fMyqGwRADsB/wgvf//E+BbEXG3pPMknZcecxrwcHrM54HTo969Kms5DxwzK4e6fQSdyn0Erefpps2637D6CDJvngH8L2B29viIOHe0ArTO5oFjZr0tz+yjtwPfB/4DTzZnZtZz8i5e/9HCI7GO4NtAZuWTJxHcKemkiLir8GisrTx4zKyc8lQNXUSSDJ6XtF7SBknrh3yXdRWvOmZWXnkWr5/cikCsvTx4zKy88twaQtI04DBgYmVbdqoI634ePGZWXkPeGkrXJbgf+A7wyfTrJcWGZa3mwWNm5ZXniuAi4JXAAxFxvKRXkCxYYz3Gq46ZlVOeRLA5IjZLQtKEiPiFpJcXHpm1hQePmZVPnkSwStJU4N+BeyQ9A/ymyKCseB4vYGYVeaqG3po+vUTSvcBewN2FRmWF8ngBM8vKNfuopKMlXQjMAVZFxNZiw7KieLyAmVXLUzX0d8D1wD7AdOA6SR8vOjArRmW8QFZlvICZlVOePoIzgLkRsRlA0j8BS4D/U2BcVhCPFzCzanluDf2OzEAyYAKwuphwrGgeL2Bm1epeEUj6AhDAc8BySfekr/+cZMUx61IeL2BmWY1uDVWWAVsMfDOz/b7CorGW8XgBM6uomwgi4npJY4CvRsQZLYzJRpHHC5jZUBp2FkfEDkkHSRrvktHu4/ECZpZHnqqhlcAPJS0ENlU2RsRnC4vKRiw7XqAytfSCW5cx79DpvjIws13kSQSPpY8+wGsTdAmvL2BmeeWZYuKTrQjERpfHC5hZXo3KR+8gKRetCGAtcG9E3FB0YDYylfECC6r6CHw1YGbVGl0RfLrGtr2BMyUdEREXFxSTDVN1hZDHC5hZHo3KR79Xa3vaabwYcCLoIPUqhDxewMyGkmv20ayI2FFEIDZ8nlHUzEaiUR/B3jU2TwPeDSzPc3JJjwMbgB3A9ojor9ov4ArgJOCPwNkR8VCuyO0FrhAys5Fo1EewmKSDWOnrSmfxfcB7m/iM4yNibZ19bwQOSx+vAq5Kv1oTXCFkZiPRqI/g4BZ8/ikkU1gE8ICkqZL2j4gnW/DZPcMVQmY2EnkGlI1EAN+VFMCXI+Kaqv0HAE9kXq9Kt+2SCCTNB+YDzJo1q7hou0St+YNcIWRmw1V0InhtRKyWtC/Jwve/iIj7mz1JmkCuAejv748hDu9pjeYPcoWQmQ1H01VDzYiI1enXNSRTWR9bdchq4MDM65l40Zu6XB1kZkVoVDV0dKM3DlXdI2lPoC8iNqTPTwQurTpsIXC+pJtJOomfc/9Afa4OMrMiNLo19JkG+wI4YYhz7wd8M6kQZSzwrxFxt6TzACLiauAuktLRFSTlo+fkjLuUXB1kZkVoVDV0/EhOHBErgbk1tl+deR7A+0byOWXi6iAzK0KuzmJJRwCHk1nEPiK+WlRQVp+rg8xstA2ZCCT9PXAcSSK4i2QQ2A8AJ4IWqFUq6uogMxtNea4ITiO5xfPTiDhH0n6Ap6FuAS81aWatkKd89PmIGAS2S5oCrGHXkk8rgEtFzaxV8iSCAUlTgX8mmX/oIeDHRQZlO0tFsyqlomZmoynPUpV/mz69WtLdwJSIWFZsWOZSUTNrlSGvCCT9Z+V5RDweEcuy26wYlVLRieP6mDxhLBPH9blU1MwK0Whk8URgD2C6pGnsnI56CsnEcFYwl4qaWSs0ujX0N8D7gZeQ9AtUrAe+WGBMpVSrTBRcKmpmxWs0svgK4ApJF0TEF1oYU+m4TNTM2ilP1dCXJV0o6Zb0cb6kcYVHVhIuEzWzdsuTCK4Ejkm/Vp5fVWRQZeIyUTNrt0adxWMjYjvwyojITh63SNLS4kMrB5eJmlm7Nboi+En6dYekl1Y2SjoE2FFoVCXiMlEza7dGVUOVctEPA/dKWpm+no3XDRhVLhM1s3ZqlAhmSPpg+vzLwJj0+Q7gKODeIgPrVS4TNbNO0ygRjAEmsfPKIPueyYVF1MNcJmpmnahRIngyIqrXGLZhypaJVtYcXnDrMuYdOt1XAmbWVo06i6uvBGwEXCZqZp2qUSJ4fcuiKAGXiZpZp6qbCCLi6VYG0utcJmpmnSrX4vU2OlwmamadyImgQF543sy6gRNBQVwqambdIs+kc9YkzyhqZt3EiaAALhU1s27iRFAAl4qaWTcpPBFIGiPpp5LurLHvbElPSVqSPv666HhawaWiZtZNWtFZfBHwc5JF72v5ekSc34I4WsqlombWLQq9IpA0E3gTcG2Rn9Nu6zZuYekTz+7WGbzPpAnMPXCqk4CZdbSirwg+Byyg8Wylp0p6HfBL4AMR8UT1AZLmA/MBZs2aVUCYw+cyUTPrdoVdEUh6M7AmIhY3OOwOYHZEzAHuAa6vdVBEXBMR/RHRP2PGjAKiHR6XiZpZLyjy1tA84GRJjwM3AydIuiF7QESsi4jKb81rgWMKjGfUuUzUzHpBYYkgIj4WETMjYjZwOrAoIs7MHiNp/8zLk0k6lbuGy0TNrBe0fByBpEslnZy+vFDScklLgQuBs1sdz0i4TNTMeoEiot0xNKW/vz8GBgbaHcYu6q1DbGbWKSQtjoj+Wvs86VwTvPC8mfUiJ4KcXCZqZr3Kcw3l4DJRM+tlTgQ5uEzUzHqZE0EOLhM1s17mRJCDy0TNrJe5szgnzyZqZr3KiaAJLhM1s17kW0N11Jta2sys1/iKoAaPGTCzMvEVQRWPGTCzsnEiqOIxA2ZWNk4EVTxmwMzKxomgiscMmFnZuLO4Bo8ZMLMyKX0i8NTSZlZ2pU4ELhM1MytxH4HLRM3MEqVNBC4TNTNLlDYRuEzUzCxR2kTgMlEzs0SpO4tdJmpmVqJE4DJRM7PaSpEIXCZqZlZfz/cRuEzUzKyxnk8ELhM1M2us5xOBy0TNzBorPBFIGiPpp5LurLFvgqSvS1oh6UFJs0f7810mambWWCs6iy8Cfg5MqbHvPcAzEXGopNOBy4B3jHYALhM1M6uv0CsCSTOBNwHX1jnkFOD69PktwOslqYhY9pk0gbkHTnUSMDOrUvStoc8BC4DBOvsPAJ4AiIjtwHPAPtUHSZovaUDSwFNPPVVQqGZm5VRYIpD0ZmBNRCwe6bki4pqI6I+I/hkzZoxCdGZmVlHkFcE84GRJjwM3AydIuqHqmNXAgQCSxgJ7AesKjMnMzKoUlggi4mMRMTMiZgOnA4si4syqwxYCZ6XPT0uPiaJiMjOz3bV8iglJlwIDEbEQ+Bfga5JWAE+TJAwzM2shddsf4JKeAn4zzLdPB9aOYjidqNfb2Ovtg95vo9vXHgdFRM1O1q5LBCMhaSAi+tsdR5F6vY293j7o/Ta6fZ2n56eYMDOzxpwIzMxKrmyJ4Jp2B9ACvd7GXm8f9H4b3b4OU6o+AjMz213ZrgjMzKyKE4GZWcl1fSKQ9BVJayQ9nNk2V9KPJf1M0h2SpmT2zUn3LU/3T0y3H5O+XiHp80XNgtqsZton6QxJSzKPQUlHpvs6sn3QdBvHSbo+3f5zSR/LvOcNkh5N23hxO9pSS5PtGy/punT7UknHZd7Tkd9DSQdKulfSI+n/q4vS7XtLukfSr9Kv09LtSuNfIWmZpKMz5zorPf5Xks6q95mtNIz2vSL93m6R9OGqc3XkzygR0dUP4HXA0cDDmW3/BfxZ+vxc4FPp87HAMmBu+nofYEz6/CfAqwEB3wbe2O62Ndu+qvf9KfBY5nVHtm8Y38N3ATenz/cAHgdmA2OAx4BDgPHAUuDwdrdtGO17H3Bd+nxfYDHQ18nfQ2B/4Oj0+WTgl8DhwOXAxen2i4HL0ucnpfErbc+D6fa9gZXp12np82ld2L59gVcC/wB8OHOejv0Z7forgoi4n2R6iqyXAfenz+8BTk2fnwgsi4il6XvXRcQOSfsDUyLigUi+Y18F3lJ48Dk02b6sd5JM9kcntw+abmMAeyqZpPBFwFZgPXAssCIiVkbEVpK2n1J07Hk02b7DgUXp+9YAzwL9nfw9jIgnI+Kh9PkGkoWoDmDX9UauZ2e8pwBfjcQDwNS0fX8B3BMRT0fEMyT/Lm9oXUtqa7Z9EbEmIv4L2FZ1qo79Ge36RFDHcnb+A7+NdIZTkv98Iek7kh6StCDdfgCwKvP+Vem2TlWvfVnvAG5Kn3db+6B+G28BNgFPAr8FPh0RT5NZ2yLV6W2s176lJLP2jpV0MHBMuq8rvodKlps9CngQ2C8inkx3/R7YL31e73vV8d/DnO2rp2Pb16uJ4FzgbyUtJrmU25puHwu8Fjgj/fpWSa9vT4gjUq99AEh6FfDHiHi41pu7RL02HgvsAF4CHAx8SNIh7QlxROq17yskvyAGSBZ2+hFJezuepEnArcD7I2J9dl96FdPVteq93L6Wzz7aChHxC5LbQEh6GclymZD8B7s/Itam++4iuXd7AzAzc4qZJGsldKQG7as4nZ1XA5C0pWvaBw3b+C7g7ojYBqyR9EOgn+QvreyVUUe3sV77Ilmp7wOV4yT9iOSe9DN08PdQ0jiSX5I3RsRt6eY/SNo/Ip5Mb/2sSbe/sA5JqtKW1cBxVdvvKzLuvJpsXz312t12PXlFIGnf9Gsf8HHg6nTXd4A/lbRHeo/5z4BH0su79ZJenVZivBu4vQ2h59KgfZVtbyftH4DkHidd1D5o2MbfAiek+/Yk6Wz8BUnn62GSDpY0niQZLmx13HnVa1/6s7ln+vzPge0R0dE/o2k8/wL8PCI+m9mVXW/kLHbGuxB4d1o99GrgubR93wFOlDQtrcA5Md3WVsNoXz2d+zPa7t7qkT5I/vJ9kqRjZhXwHuAikr+ifgn8E+kI6vT4M0nuzz4MXJ7Z3p9uewz4YvY9Xda+44AHapynI9vXbBuBScA30u/hI8BHMuc5KT3+MeB/t7tdw2zfbOBRkg7J/yCZOrijv4ckt1mDpCJvSfo4iaQq7z+BX6Vt2Ts9XsCX0nb8DOjPnOtcYEX6OKfdbRtm+16cfp/Xk3T2ryLp6O/Yn1FPMWFmVnI9eWvIzMzycyIwMys5JwIzs5JzIjAzKzknAjOzknMiMBtCWu/+A0lvzGx7m6S72xmX2Whx+ahZDpKOIBm/cBTJiPyfAm+IiMeGca6xkYwgNusITgRmOUm6nGTCuz3TrwcBRwDjgEsi4vZ0UrKvpccAnB8RP1KyrsCnSKaKeEVEvKy10ZvV50RgllM69cNDJBPE3Qksj4gbJE0lWSvgKJIRqIMRsVnSYcBNEdGfJoJvAUdExK/bEb9ZPT056ZxZESJik6SvAxtJ5nP6y8wKVBOBWcDvgC8qWRluB8nU5xU/cRKwTuREYNacwfQh4NSIeDS7U9IlwB+AuSTFGJszuze1KEazprhqyGx4vgNckM5MiaSj0u17AU9GxCDwVyTLE5p1NCcCs+H5FEkn8TJJy9PXAFcCZ0laCrwCXwVYF3BnsZlZyfmKwMys5JwIzMxKzonAzKzknAjMzErOicDMrOScCMzMSs6JwMys5P4/rXMQ81nupxQAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEWCAYAAAB8LwAVAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAkNElEQVR4nO3dfbyVZZ3v8c8XRCjwAUUdExFLm2JMtNmpM3hGtMnQKa3j1MFHevAw02RZTamdqSw9p1dR0xkbMyMjtSmdmcyRzDRm1Gx0KDeFKJiJlApRIJCCCQr85o/72nKzWWvte+297rXXw/f9eq3Xvu/rfljX5UZ+XM+KCMzMzPobMdwZMDOz1uQAYWZmFTlAmJlZRQ4QZmZWkQOEmZlV5ABhZmYVOUCYmVlFDhBmQyTpbkkbJI3OpV0r6XlJmyStl7RA0qty198haVu6vknSCknvGZ4SmFXmAGE2BJImA/8DCOC0fpfnRMQ44CBgFfC1ftf/KyLGpXvOAOZIOrrkLJsV5gBhNjTnAQuBa4FZlW6IiOeAfwGOqvaSiPgZ8DDw6obn0GyQdhvuDJi1ufOALwA/BhZKOiAifpu/QdJY4ExgebWXSHod8Eqgt8S8mtWl42oQkuZJWiPpoQL3TpJ0l6SfSVoi6dRm5NE6g6TjgUOAf4mIRcBjwFm5Wz4s6XfARuB44Nx+rzhO0u8kbQR+AnwDeLT0jJsV1HEBgqyqP6PgvR8j+5/7aGAmcFVZmbKONAv4QUQ8lc6/xc7NTJ+PiL2BycBzwB/2e35hROwdEXsAfwD8EfDpUnNsVoeOCxARcQ+wPp8m6RWSbpe0SNKPcqNJAtgzHe8F/LqJWbU2JuklwNuBEyT9RtJvgA8CUyVNzd8bEU8AFwJXpOd2kZqlbgLeXG7OzYrrlj6IucBfR8Sjko4lqymcBHwS+IGk9wFjgT8fvixam3kLsA14DfB8Lv1fyPoldhIRCyT9GpgNXNH/uqR9gbcCS8vIrNlgdFwNoj9J44A/Bf5V0mLgK8CB6fKZwLURMRE4FfiGpI7/b2INMQv4ekQ8ERG/6fsAVwJnU/kfX58DLsrNl/iTvnkQZCOY1gLva0bmzYpQJ24YlMam3xoRR0jaE3gkIg6scN9SYEZEPJnOVwDHRcSapmbYzKwFdfy/liPiGeCXkt4GoExfG/ETwOtT+quBMWT/ijMz63odV4OQdAMwHZgA/Ba4FLgT+DJZ09Io4MaIuEzSFOCrwDiyDuuLIuIHw5FvM7NW03EBwszMGqPjm5jMzGxwOmqY64QJE2Ly5MnDnQ0zs7axaNGipyJiv0rXOipATJ48md5eL2VjZlaUpMerXXMTk5mZVVRagJB0cFoIb5mkpZIurHDP2WmRvAcl3ZdfokDSr1L6YkmuFpiZNVmZTUxbgb+NiJ9K2gNYJGlBRCzL3fNL4ISI2CDpFLIlMY7NXT8xtxCamZk1UWkBIiJWA6vT8UZJD5PtrLUsd899uUcWAhPLyo+ZmdWnKX0QaemLo8k2Vanm3cD3c+dBtpDeIkmza7x7tqReSb1r13oStJlZo5Q+iiktlncT8IG07EWle04kCxDH55KPj4hVkvYHFkj6eVrKeycRMZesaYqenh7P+jOzrrJu0xZWbniOieNfwr7jRg/8QB1KDRCSRpEFh29GxHeq3HMkcA1wSkSs60uPiFXp5xpJNwPHALsECDOzbnXL4lVcfNMSRo0YwQvbtzPnjCM57aiDGvb+MkcxCfga8HBEfKHKPZOA7wDnRsQvculjU8d2336+JwMDbiFqZtYt1m3awsU3LWHzC9vZuGUrm1/YzkU3LWHdpi0N+44yaxDTyPbgfTDtwwDwf4BJABFxNfAJYF/gqiyesDUieoADgJtT2m7AtyLi9hLzambWVlZueI5RI0awme0vpo0aMYKVG55rWFNTmaOY/hPQAPecD5xfIX0FMHXXJ8zMDGDi+JfwwvbtO6W9sH07E8dX3NV2UDyT2sysDe07bjRzzjiSMaNGsMfo3RgzagRzzjiyoR3VHbUWk5lZJ+s/Yum0ow5i2mET2nMUk5mZNUa1EUv7jhvd8MDQx01MZmYtrhkjlipxgDAza3F9I5by+kYslckBwsysxTVjxFIlDhBmZi2uGSOWKnEntZlZi6m0vlLZI5YqcYAwM2shtdZXKnPEUiVuYjIzaxHDNVqpGgcIM7MWMVyjlapxgDAzaxHDNVqpGgcIM7MWMVyjlapxJ7WZ2TBpldFK1ThAmJkNg1YarVSNm5jMzJqs1UYrVeMAYWbWZK02WqmaMvekPljSXZKWSVoq6cIK90jSFyUtl7RE0mtz12ZJejR9ZpWVTzOzZmu10UrVlFmD2Ar8bURMAY4D3itpSr97TgEOT5/ZwJcBJO0DXAocCxwDXCppfIl5NTNrmlYbrVRNmXtSrwZWp+ONkh4GDgKW5W47Hbg+IgJYKGlvSQcC04EFEbEeQNICYAZwQ1n5NTMrU7N3g2uEpoxikjQZOBr4cb9LBwFP5s5XprRq6ZXePZus9sGkSZMak2EzswYajt3gGqH0TmpJ44CbgA9ExDONfn9EzI2Inojo2W+//Rr9ejOzIWmXEUuVlBogJI0iCw7fjIjvVLhlFXBw7nxiSquWbmbWVtplxFIlZY5iEvA14OGI+EKV2+YD56XRTMcBT6e+izuAkyWNT53TJ6c0M7O20i4jliopswYxDTgXOEnS4vQ5VdJfS/rrdM9twApgOfBV4G8AUuf05cD96XNZX4e1mVkrW7dpCw88+bsXm5DaZcRSJcoGEHWGnp6e6O3tHe5smFmXqrV8RqV1l1qBpEUR0VPpmtdiMjNrgHxn9GayJqWLblrCtMMmvDhaqZUCQxFeasPMrAHauTO6GgcIM7MGaOfO6GocIMzMGqCdO6OrcR+EmdkgtPpmP43gAGFmVqd22OynEdzEZGZWh3ZeOqNeDhBmZnXoxNFK1ThAmJnVoRNHK1XjAGFmVodOHK1UjTupzcwG0I6b/TSCA4SZWQ3tutlPI7iJycysim4asVSJA4SZWRXdNGKpEgcIM7MqumnEUiUOEGZmSSdt9tMI7qQ2M6N6Z3S3jFiqpFCAkDQSOCB/f0Q8McAz84A3AWsi4ogK1z8CnJ3Lx6uB/SJivaRfARuBbcDWarsdmZk1Qidu9tMIAzYxSXof8FtgAfC99Lm1wLuvBWZUuxgRn4uIoyLiKOCjwA/77Tt9Yrru4GBmper2zuhqitQgLgT+MCLW1fPiiLhH0uSCt58J3FDP+83MGqXbO6OrKdJJ/STwdFkZkPRSsprGTbnkAH4gaZGk2QM8P1tSr6TetWvXlpVNM+tg3d4ZXU2RGsQK4G5J3wNenB0SEV9oUB7eDNzbr3np+IhYJWl/YIGkn0fEPZUejoi5wFyAnp6eaFCezKyDdcNmP41QJEA8kT67p0+jzaRf81JErEo/10i6GTgGqBggzMzq0S2b/TTCgAEiIj4FIGlcOt/UqC+XtBdwAnBOLm0sMCIiNqbjk4HLGvWdZta9BhqtZDsbMEBIOgL4BrBPOn8KOC8ilg7w3A3AdGCCpJXApcAogIi4Ot32VuAHEfFs7tEDgJsl9eXvWxFxex1lMjOrqG+0Ul9wgB2jlRwgdlWkiWku8KGIuAtA0nTgq8Cf1nooIs4c6MURcS3ZcNh82gpgaoF8mZnVxaOV6lNkFNPYvuAAEBF3A2NLy5GZWQPll8/waKX6FBrFJOnjZM1MkPUXrCgvS2ZmjVGtQ9qjlYopUoN4F7Af8J302S+lmZm1rFp7Oew7bjRTD97bwWEARUYxbQDe34S8mJk1jDukh65qgJD0DxHxAUnfJZvZvJOIOK3UnJmZ1aH/5Dd3SA9drRpEX5/D55uRETOzwarW1zDnjCO5qF+6aw/FVQ0QEbEoHR4VEVfkr0m6EPhhmRkzMyui1uQ3d0gPTZFO6lkV0t7R4HyYmQ3KQEt1u0N68Gr1QZwJnAUcKml+7tIewPrKT5mZNZf7GspTqw/iPmA1MAH4+1z6RmBJmZkyM6ulf4e0+xrKUasP4nHgceBPmpcdM7PaPPmteYpsOXqcpPslbZL0vKRtkp5pRubMzPI8+a25inRSX0m2JeijwEuA84EvlZkpM7NKvHd0cxUJEETEcmBkRGyLiK+TbRFqZlaq/EJ74A7pZiuyWN/vJe0OLJY0h6zjulBgMTMbLE9+G35FAsS5wEjgAuCDwMHAGWVmysy6mye/tYYBawIR8XhEPBcRz0TEpyLiQ6nJqSZJ8yStkfRQlevTJT0taXH6fCJ3bYakRyQtl3RJfUUys3bnyW+todZEuQepsEhfn4g4coB3X0vWwX19jXt+FBFv6ve9I8k6wd8ArATulzQ/IpYN8H1m1qa80F5rqtXE9KYa1wYUEfdImjyIR48BlqetR5F0I3A64ABh1oHc19C6BpooV7Y/kfQA8GvgwxGxFDgIeDJ3z0rg2GovkDQbmA0wadKkErNqZo3mvobWNmAntaSN7Ghq2h0YBTwbEXsO8bt/ChwSEZsknQr8G3B4vS+JiLnAXICenp6qTWJm1noG2tSn72PDo8iOcnv0HUsSWXPPcUP94oh4Jnd8m6SrJE0AVpGNlOozMaWZWQfI9ze4r6G11TWfITL/BrxxqF8s6Q9SwEHSMSkv64D7gcMlHZrmX8wE5ld/k5m1i1sWr2LaZ+/knGt+zLTP3sm9y59izhlHMmbUCPYYvRtjRo1wX0MLKdLE9D9zpyOAHmBzgeduAKYDEyStBC4la54iIq4G/hJ4j6StwHPAzIgIYKukC4A7yOZfzEt9E2bWxqr1N9x78Unce/FJ7mtoQUUmyr05d7wV+BVZM1NNEXHmANevJBsGW+nabcBtBfJmZm2iVn+D5zS0piJ9EO9sRkbMrLN4bkP7K7Lc98slfVfS2jQz+hZJL29G5sysPfXva5i/eNWLG/u4v6F9FGli+hbZzOa3pvOZwA3UmJtgZt3Lcxs6R5FRTC+NiG9ExNb0+SdgTNkZM7P25HWUOkeRAPF9SZdImizpEEkXAbdJ2kfSPmVn0Mxam/ds6FxFmpjenn7+Vb/0mWQzrN0fYdalvI5SZysyiunQZmTEzNqL+xo6X5GJcqOA9wB/lpLuBr4SES+UmC8za0H5oateR6nzFWli+jLZDOir0vm5Ke38sjJlZq2nf3PSx/9iivsaOlyRAPG6iJiaO78zLdFtZl2iUnPS5d9bxsffNIXLb13mvoYOVSRAbJP0ioh4DLKJc8C2crNlZq2kWnPSES/by+sodbAiAeIjwF2SVgACDgG8/IZZB6tnmQz3NXSumgFC0n7A02TbgO6fkh+JiC1lZ8zMhoeHrlqfqgFC0vnAp4HHgEOB2RHhfRnMOpiHrlperRrEB4A/ioi1qd/hm3jjHrOO5qGrlldrqY3nI2ItQESsAPynwqzDeZkMy6tVg5go6YvVziPi/eVly8yaoX9ndN+S3O5rMKgdID7S73xRPS+WNA94E7AmIo6ocP1s4GKykVEbgfdExAPp2q9S2jZga0T01PPdZjawap3R7muwPlUDRERcN8R3X0u2pej1Va7/EjghIjZIOgWYy857TJwYEU8NMQ9mVkGtzmj3NVifIst9D0pE3AOsr3H9vojYkE4XAhPLyouZ7WygPRvMoMQAUad3A9/PnQfwA0mLJM2u9aCk2ZJ6JfWuXbu21EyatbP8vg3ujLYiisykLpWkE8kCxPG55OMjYpWk/YEFkn6eaiS7iIi5ZM1T9PT0ROkZNmtDlfob3BltAymy3Pd+wP8GJufvj4h3DfXLJR0JXAOcEhHrcu9elX6ukXQz2UzuigHCzGqr1t9w78UneR0lq6lIDeIW4EfAv9PARfokTQK+A5wbEb/IpY8FRkTExnR8MnBZo77XrNvUmvzmvaGtliIB4qURcXG9L5Z0AzAdmCBpJXAp2b4SRMTVwCeAfYGrJMGO4awHADentN2Ab0XE7fV+v1m3qmehPbNaigSIWyWdGhG31fPiiDhzgOvnU2HToTRre+quT5jZQLzQnjWSImr360raCIwFtgAvkE1si4jYs/zs1aenpyd6e3uHOxtmTZOvLQBM++ydbH5hR21hzKgR3HvxSew7bvQuNQszAEmLqk1GHrAGERF7ND5LZjZU/WsL751+mBfas4YqNMxV0njgcGBMX1q1YadmVr5KI5OuvOtRsgr+Du5rsKEYcKJc2hfiHuAO4FPp5yfLzZaZ1VJpJvTuI0dywYmHMWbUCPYYvRtjRo1wX4MNSZEaxIXA64CFEXGipFeRbSRkZsOk2siks46dxFnHTnJfgzVEkaU2NkfEZgBJoyPi58AflpstM8vLL5MBvLgsd6Xawr7jRnt+gzVEkRrESkl7A/9GtuzFBuDxMjNlZjt4WW4bLkVGMb01HX5S0l3AXoAnrpk1gZfltuFUdBTTa8kW0wvg3oh4vtRcmRkw8B7RZmUqMorpE8B1ZMtiTAC+LuljZWfMzLxHtA2vIp3UZwOvi4hLI+JS4Djg3HKzZdad6umMNitbkSamX5NNkNuczkcDq0rLkVmXcme0tZqqAULSP5L1OTwNLJW0IJ2/AfhJc7Jn1h3cGW2tqFYNom/Vu0XAzbn0u0vLjVmXcme0taKqASIirpM0Erg+Is5uYp7MukJ+dVV3RlsrqtkHERHbJB0iaXcPbTVrHO8Rbe2gyH4Q1wOvBuYDz/alR8QXys1a/bwfhLWDdZu2VN23AXBntDVVrf0gigxzfQy4Nd27R+5T5IvnSVoj6aEq1yXpi5KWS1qSJuT1XZsl6dH0mVXk+8xaUf+hq5VWYs33N3gdJWsVRZba+NQQ3n8tcCVwfZXrp5DtM3E4cCzwZeBYSfuQ7WHdQzZyapGk+RGxYQh5MWu6Sk1J0w6b4P4GawtVaxCSvitpfu5zi6SvSTqn6MvTpkLra9xyOlkneETEQmBvSQcCbwQWRMT6FBQWADOKfq9ZK8gPXd24ZSubX9jORTctAfDkN2sLtWoQn6+Qtg9wjqQjIuKSBnz/QcCTufOVKa1a+i4kzQZmA0yaNKkBWTJrjFpDVz35zdpBrWGuP6yULmk+2dyIRgSIIYuIucBcyDqphzk71sXyw1b3HTd6wKGrnvxmra5IJ/VOImJbA79/FXBw7nxiSquWbtaSblm8immfvZNzrvkx0z57J/MXr/I6Stb2ai21sU+F5PHAecDSBn3/fOACSTeSdVI/HRGrJd0BfFrS+HTfycBHG/SdZg1Va5kMNyVZO6vVB7GIbASR0nkAT5EttfGeIi+XdAMwHZggaSXZyKRRABFxNXAbcCqwHPg98M50bb2ky4H706sui4hand1mw2agZTLclGTtqlYfxKFDfXlEnDnA9QDeW+XaPGDeUPNgVgYvk2HdoNCOcma2g5fJsG4x4FIb7cRLbVjZvEyGdZpaS224BmFWh1r9DV4iwzpNrVFMr612DSAiftr47Ji1lnrnNph1klo1iL+vcS2AkxqcF7OWUm0LUPc3WLeoNYrpxGZmxKyVeG6DWcE+CElHAFOAMX1pEVFthVaztue5DWYFAoSkS8kmu00hm9h2CvCfVF/C26ztuK/BbFdF1mL6S+D1wG8i4p3AVGCvUnNl1kReR8mssiJNTM9FxHZJWyXtCaxh54X0zNqW+xrMqisSIHol7Q18lWx9pk3Af5WZKbNmcV+DWXVFthz9m3R4taTbgT0jYkm52TIrj9dRMiumSCf1f0TE6wEi4lf908zaiddRMiuu1kzqMcBLyZbqHs+OZb/3pMr2n2atrFp/w70Xn8S9F5/kvgazfmrVIP4K+ADwMiC/rMYzwJUl5smsIfoPXfU6Smb1qTWT+grgCknvi4h/bGKezIasUlPStMMmuL/BrA5F5kF8RdL7JX07fS6QNKrIyyXNkPSIpOWSLqlw/f9LWpw+v5D0u9y1bblr84sXybpdvilp45atbH5hOxfdlI2r8NwGs+KKDHO9imyb0KvS+bnAl4Hzaz0kaSTwJeANwErgfknzI2JZ3z0R8cHc/e8Djs694rmIOKpA/sx2UqspyXMbzIqr1Um9W0RsBV4XEVNzl+6U9ECBdx8DLI+IFel9NwKnA8uq3H8m2Z7VZnWrZ+iq5zaYFVOriekn6ec2Sa/oS5T0cmBbgXcfBDyZO19JldFPkg4BDgXuzCWPkdQraaGktxT4PutS/ZfKuHf5U25KMmuAWk1MfcNaPwzcJWlFOp8MvLPB+ZgJfDsi8oHnkIhYlQLSnZIejIjHdsmkNBuYDTBp0qQGZ8tanYeumpWnVoDYT9KH0vFXgJHpeBtZX8FdA7x7FTuv2TQxpVUyE3hvPiEiVqWfKyTdnb5zlwAREXOBuZDtST1AnqzNeeiqWfPUChAjgXHsqEnkn9mjwLvvBw6XdChZYJgJnNX/JkmvAsaTW98pTcz7fURskTQBmAbMKfCd1sE8dNWsuWoFiNURcdlgXxwRWyVdANxBFmzmRcRSSZcBvRHRN3R1JnBjROT/9f9qsuG128n6ST6TH/1k3adWU5KXyjArR5E+iEGLiNvINhnKp32i3/knKzx3H/CaoX6/ta96mpI8dNWsHLUChBfjs2ExmKYkD101a7yqw1wjYn0zM2IGngVt1kqKzKQ2K1W+OclNSWatwwHChlX/5qSP/8UUNyWZtYgii/WZlaJSc9Ll31vGx980xU1JZi3ANQhrmqIjk4542V6eBW3WAhwgrCnqHZnkpiSz4ecmJiudRyaZtSfXIKzhPMnNrDM4QFhDeZKbWedwE5M1jJuSzDqLaxA2JJ7kZta5HCBs0DzJzayzuYnJClm3aQsPPPk71m3a8uK5J7mZdTbXIGxAlTqeD9l3rCe5mXU41yBsJ0VqChfdtISxu4+sOcnN232atT/XIOxF9dQUnn1+m3dyM+twpQYISTOAK8i2HL0mIj7T7/o7gM+R7VkNcGVEXJOuzQI+ltL/b0RcV2Zeu121LT1vveD4qjWFqQfv7ZFJZh2stAAhaSTwJeANwErgfknzK+wt/c8RcUG/Z/cBLgV6gAAWpWc3lJXfblRkiOpANQWPTDLrXGXWII4BlkfECgBJNwKnA/0DRCVvBBb07WonaQEwA7ihpLx2nXqGqLqmYNadyuykPgh4Mne+MqX1d4akJZK+LengOp9F0mxJvZJ6165d24h8d5xGDFF1x7NZ9xnuTurvAjdExBZJfwVcB5xUzwsiYi4wF6Cnpycan8X25iGqZjZYZdYgVgEH584nsqMzGoCIWBcRW9LpNcAfF33WKsvXFjxE1cyGoswaxP3A4ZIOJfvLfSZwVv4GSQdGxOp0ehrwcDq+A/i0pPHp/GTgoyXmtSP0ry28d/phHqJqZoNWWoCIiK2SLiD7y34kMC8ilkq6DOiNiPnA+yWdBmwF1gPvSM+ul3Q5WZABuKyvw9oy/fdcqDRM9cq7HgW003PueDazokrtg4iI24Db+qV9Inf8UarUDCJiHjCvzPy1q6L9CruPHMnsP3s5X7p7uYeomlndhruT2gZQpKZQa0LbWcdO4qxjJ7mmYGZ1c4BoYY1c+sKBwczq5QDRQvK1BcBLX5jZsHKAaBGNGoHkfgUzaxQHiGHgEUhm1g4cIJrMI5DMrF04QJTII5DMrJ05QJTEI5DMrN05QDSIRyCZWadxgGgAj0Ays07kAFEnj0Ays27hAFFF/0AAHoFkZt3FAaKCSoFg2mETPALJzLpKmRsGtY0im+ws/fXTjBqx83+ufL9Cpa06vfmOmbWzrq9BFO1gBnkEkpl1la6uQVSqLVx516M8v23XQPBHL9uzak0BcG3BzDpOV9cgVm54rq4O5tOOOsg1BTPrGqUGCEkzgCvIthy9JiI+0+/6h4DzybYcXQu8KyIeT9e2AQ+mW5+IiNManb+J419SdwezRyCZWbcorYlJ0kjgS8ApwBTgTElT+t32M6AnIo4Evg3MyV17LiKOSp+GBwfI/rJ3B7OZWWVl1iCOAZZHxAoASTcCpwPL+m6IiLty9y8EzikxPxW52cjMrLIyO6kPAp7Mna9MadW8G/h+7nyMpF5JCyW9pdpDkman+3rXrl07qIy6tmBmtquW6KSWdA7QA5yQSz4kIlZJejlwp6QHI+Kx/s9GxFxgLkBPT080JcNmZl2gzBrEKuDg3PnElLYTSX8O/B1wWkRs6UuPiFXp5wrgbuDoEvNqZmb9lBkg7gcOl3SopN2BmcD8/A2Sjga+QhYc1uTSx0sanY4nANPI9V2YmVn5Smtiioitki4A7iAb5jovIpZKugzojYj5wOeAccC/SoIdw1lfDXxF0nayIPaZiHCAMDNrIkV0TrN9T09P9Pb2Dnc2zMzahqRFEdFT8VonBQhJa4HHB/n4BOCpBman1XR6+aDzy+jytb9WLOMhEbFfpQsdFSCGQlJvtSjaCTq9fND5ZXT52l+7lbGrF+szM7PqHCDMzKwiB4gd5g53BkrW6eWDzi+jy9f+2qqM7oMwM7OKXIMwM7OKHCDMzKyijg0QkuZJWiPpoVzaVEn/JelBSd+VtGfu2pHp2tJ0fUxK/+N0vlzSF5WmfLeCesoo6WxJi3Of7ZKOStdasox1lm+UpOtS+sOSPpp7ZoakR1L5LhmOslRSZ/l2l/T1lP6ApOm5Z1r193ewpLskLUv/X12Y0veRtEDSo+nn+JSulP/lkpZIem3uXbPS/Y9KmjVcZepvEGV8Vfr9bpH04X7var0/pxHRkR/gz4DXAg/l0u4HTkjH7wIuT8e7AUuAqel8X2BkOv4JcBwgsuXITxnusg2mjP2eew3wWO68JctY5+/wLODGdPxS4FfAZLJlXh4DXg7sDjwATBnusg2ifO8Fvp6O9wcWASNa/Pd3IPDadLwH8AuyzcPmAJek9EuAz6bjU1P+lcrz45S+D7Ai/RyfjscPd/kGWcb9gdcB/w/4cO49LfnntGNrEBFxD7C+X/IrgXvS8QLgjHR8MrAkIh5Iz66LiG2SDgT2jIiFkf0WrwfeUnrmC6qzjHlnAjcCtHIZ6yxfAGMl7Qa8BHgeeIbcxlUR8TxZuU8vO+9F1Fm+KcCd6bk1wO+Anhb//a2OiJ+m443Aw2R7wpwOXJduu44d+T0duD4yC4G9U/neCCyIiPURsYHsv8uM5pWkunrLGBFrIuJ+4IV+r2rJP6cdGyCqWMqO/+hvY8dy5K8EQtIdkn4q6aKUfhDZRkd9Btr0qBVUK2Pe/wJuSMftVsZq5fs28CywGngC+HxErKf+jauGW7XyPQCcJmk3SYcCf5yutcXvT9JksiX7fwwcEBGr06XfAAek42q/q7b4HRYsYzUtWcZuCxDvAv5G0iKy6uDzKX034Hjg7PTzrZJePzxZHLJqZQRA0rHA7yPioUoPt4Fq5TsG2Aa8DDgU+Ftlm021m2rlm0f2l0Yv8A/AfWTlbXmSxgE3AR+IiGfy11Ktp+3H2ndqGVtiR7lmiYifkzUnIemVwF+kSyuBeyLiqXTtNrK24X8i2+ioT8VNj1pJjTL2mcmO2gNk5WmbMtYo31nA7RHxArBG0r1kuxQ+SYGNq1pFtfJFxFbgg333SbqPrL17Ay38+5M0iuwvzm9GxHdS8m8lHRgRq1MTUt9eMNU2GVsFTO+XfneZ+a5HnWWsptAGa83WVTUISfunnyOAjwFXp0t3AK+R9NLUhn0CsCxVEZ+RdFwaGXIecMswZL2wGmXsS3s7qf8BsjZU2qiMNcr3BHBSujaWrJPz5xTYuKqVVCtf+rM5Nh2/AdgaES39ZzTl52vAwxHxhdyl+UDfSKRZ7MjvfOC8NJrpOODpVL47gJOVbSQ2niyA3tGUQgxgEGWspjX/nA53L3lZH7J/Ja8m6wxaCbwbuJDsX12/AD5Dmkme7j+HrP33IWBOLr0npT0GXJl/Zrg/gyjjdGBhhfe0ZBnrKR9p46n0O1wGfCT3nlPT/Y8Bfzfc5Rpk+SYDj5B1gv472RLNrf77O56saWUJsDh9TiUbJfgfwKOpLPuk+wV8KZXjQaAn9653AcvT553DXbYhlPEP0u/6GbKBBivJBhm05J9TL7VhZmYVdVUTk5mZFecAYWZmFTlAmJlZRQ4QZmZWkQOEmZlV5ABhNkhpvP5/Sjoll/Y2SbcPZ77MGsXDXM2GQNIRZPMvjiZbmeBnwIyIeGwQ79otshnTZi3BAcJsiCTNIVsocGz6eQhwBDAK+GRE3JIWcvtGugfggoi4T9m+DpeTLZnxqoh4ZXNzb1adA4TZEKUlMH5KtrDercDSiPgnSXuT7dVwNNls2+0RsVnS4cANEdGTAsT3gCMi4pfDkX+zarpqsT6zMkTEs5L+GdhEttbVm3O7hY0BJgG/Bq5UtovfNrIl5vv8xMHBWpEDhFljbE8fAWdExCP5i5I+CfwWmEo2OGRz7vKzTcqjWV08ismsse4A3pdW+UTS0Sl9L2B1RGwHziXbYtKspTlAmDXW5WSd00skLU3nAFcBsyQ9ALwK1xqsDbiT2szMKnINwszMKnKAMDOzihwgzMysIgcIMzOryAHCzMwqcoAwM7OKHCDMzKyi/waPC0+cSHEZiQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Define plot_pop()\n",
    "def plot_pop(filename, country_code):\n",
    "\n",
    "    # Initialize reader object: urb_pop_reader\n",
    "    urb_pop_reader = pd.read_csv(filename, chunksize=1000)\n",
    "\n",
    "    # Initialize empty DataFrame: data\n",
    "    data = pd.DataFrame()\n",
    "    \n",
    "    # Iterate over each DataFrame chunk\n",
    "    for df_urb_pop in urb_pop_reader:\n",
    "        # Check out specific country: df_pop_ceb\n",
    "        df_pop_ceb = df_urb_pop[df_urb_pop['CountryCode'] == country_code]\n",
    "\n",
    "        # Zip DataFrame columns of interest: pops\n",
    "        pops = zip(df_pop_ceb['Total Population'],\n",
    "                    df_pop_ceb['Urban population (% of total)'])\n",
    "\n",
    "        # Turn zip object into list: pops_list\n",
    "        pops_list = list(pops)\n",
    "\n",
    "        # Use list comprehension to create new DataFrame column 'Total Urban Population'\n",
    "        df_pop_ceb['Total Urban Population'] = [int(tup[0] * tup[1] * 0.01) for tup in pops_list]\n",
    "    \n",
    "        # Append DataFrame chunk to data: data\n",
    "        data = data.append(df_pop_ceb)\n",
    "\n",
    "    # Plot urban population data     ############################################################\n",
    "    data.plot(kind='scatter', x='Year', y='Total Urban Population',  title=country_code)\n",
    "    plt.show()\n",
    "\n",
    "\n",
    "\n",
    "# Set the filename: fn\n",
    "fn = 'ind_pop_data.csv'\n",
    "\n",
    "# Call plot_pop for country code 'CEB'\n",
    "plot_pop(fn, 'CEB')\n",
    "\n",
    "# Call plot_pop for country code 'ARB'\n",
    "plot_pop(fn, 'ARB')\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 86,
   "id": "dde1f6e9-686c-4ea4-b1d4-e99b0ff37b08",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "\n",
    "\n",
    "#print(help(plt))   # Fuck you wrote docstring so complicated"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0e21b5b6-8364-418e-8da8-83438459e8e1",
   "metadata": {},
   "source": [
    "## Final thoughts\n",
    "\n",
    "\n",
    "\n",
    "Congratulations on making it through the case study, in which you had to apply skills from both of these Data Science Toolbox courses, You have just used you new skills of writing user defined functions, iterators, list comprehensions and generators to explore and extract results from the dataset.  These are all tools that you utilize time and time again in your work as a Data Scientist and you have worked hard to earn your Pythonista strips.  \n",
    "\n",
    "\n",
    "Good work, and keep on coding in PythonLand.  \n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b3eda14a-d9fe-497d-a5fc-4895fe0ecf08",
   "metadata": {},
   "source": [
    "# Now what if I learned everything in this Data Science Toolbox courses, but have to write the functions and everything just from the scratch?  Can I remember and know what to use and how to use those skills?  Need to go through these course after next week in order to really master it.  \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "947594a0-dadb-4915-9118-084a428da1d4",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7c8afe70-fe8f-4e6f-9b3d-eab595a0654c",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "12c4bf5a-7439-4888-860f-057006eeb6e5",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2662500c-e7c8-4167-b7f9-f9ecbb6b761c",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}