tutorial_environmental_insights.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "440a2f27-ed66-45ec-953f-c802c31b43cb",
   "metadata": {},
   "source": [
    "<h1 style=\"text-align: center; font-size:1vw\">Environmental Insights Tutorial</h1>\n",
    "<h3 style=\"text-align: center; font-size:1vw\">Before starting the tutorial, please ensure that you read the README.md file for this python package. </h3>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "06ce13d1-463b-44c0-9673-18087e43706c",
   "metadata": {},
   "source": [
    "<center><img src=\"_static/environmental_insights.png\" width=\"400\">"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "96ffc37a-6462-42f9-9250-5fa878be22c0",
   "metadata": {},
   "source": [
    "# Installing prerequisites\n",
    "### Please ensure that you have got all the relevant packages installed. If not, then code to do this for you is avaliable in the file \"package_installation.ipynb\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fdeb5258-e06c-4d05-aaff-121d4c796484",
   "metadata": {},
   "source": [
    "# Import relevant modules for the tutorial, both from the Environmental Insight package (air_pollution_functions, data, models) and auxiliary modules (numpy and matplotlib)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1a396667-b8ae-43f3-a770-c45a8297b7e6",
   "metadata": {},
   "outputs": [],
   "source": [
    "from environmental_insights import air_pollution_functions as ei_air_pollution_functions\n",
    "from environmental_insights import data as ei_data\n",
    "from environmental_insights import models as ei_models\n",
    "\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import matplotlib"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "74bdec1b-8529-42a5-bc0a-1c954d32c95d",
   "metadata": {},
   "source": [
    "# Loading of the example dataset for the tutorial"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "578459e0-5919-462c-9787-6f72971dc61a",
   "metadata": {},
   "source": [
    "## Load in the data that represents the gridded system used for both the global and the UK Model. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "dc43abcb-4111-4f37-aa91-4d3a51e0875d",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Load in the grids that represent the UK Model\n",
    "uk_grids = ei_data.get_uk_grids()\n",
    "display(uk_grids)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "258ad121-b0da-48b3-b8a2-fd7b265f78a8",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Load in the grids that represent the Global Model\n",
    "global_grids = ei_data.get_global_grids()\n",
    "display(global_grids)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "96f5fb63-6709-4615-8890-22b1ced1304e",
   "metadata": {},
   "source": [
    "## Load in data for a particular timestamp for the global dataset for all of the grids."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "846dfc92-8b2a-4448-8637-255b9b5d3e1a",
   "metadata": {},
   "source": [
    "For the global model the outputs produced are at the hourly level across all of 2022. As such the possible timestamps that can be used are 01-01-2022 000000 to 12-31-2022 230000."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cb717e28-3265-4428-bf38-39f2ae78e063",
   "metadata": {},
   "outputs": [],
   "source": [
    "# The format for the Global data is month-day-year HourMinuteSecond\n",
    "\n",
    "global_complete_dataset = (\n",
    "    ei_data.air_pollution_concentration_complete_set_real_time_global(\n",
    "        \"07-02-2022 080000\"\n",
    "    )\n",
    ")\n",
    "display(global_complete_dataset)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b78fae68-e949-4349-8651-151a9022e346",
   "metadata": {},
   "source": [
    "## Load in data for a particular timestamp for the England dataset for all of the grids, and for a single point (latitude and longitude) for a single timestamp. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "79f5c906-0c45-4294-9523-18399344e0ca",
   "metadata": {},
   "source": [
    "For the England model the outputs produced are at the hourly level across all of 2018. As such the possible timestamps that can be used are 2018-01-01 000000 2018-12-31 230000"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "808f627f-765e-43c2-a576-09bae13aa36e",
   "metadata": {},
   "outputs": [],
   "source": [
    "# The format for the UK dataset is year-month-day HourMinuteSecond\n",
    "\n",
    "uk_complete_dataset = (\n",
    "    ei_data.air_pollution_concentration_complete_set_real_time_united_kingdom(\n",
    "        \"2018-01-01 080000\"\n",
    "    )\n",
    ")\n",
    "display(uk_complete_dataset)\n",
    "\n",
    "uk_single_datapoint = (\n",
    "    ei_data.air_pollution_concentration_nearest_point_real_time_united_kingdom(\n",
    "        51.5, 0.12, \"2018-01-01 080000\", uk_grids\n",
    "    )\n",
    ")\n",
    "display(uk_single_datapoint)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e8ac4ecb-06f9-4a21-9d66-a442d6a30f18",
   "metadata": {},
   "source": [
    "# Visualise the different datapoints that have been loaded in. In both the [UK Daily Air Quality Index](https://uk-air.defra.gov.uk/air-pollution/daqi), and the higher level Daily Air Quality Bands"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a7bc8b76-ac4b-4b86-ad29-5395c68fa752",
   "metadata": {},
   "source": [
    "### UK Model Visualisation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c736dcd3-251d-4ef5-904c-3b1941d923d4",
   "metadata": {},
   "outputs": [],
   "source": [
    "air_pollution_DF_daily_air_quality_index_uk = ei_air_pollution_functions.air_pollution_concentrations_to_UK_daily_air_quality_index(\n",
    "    uk_complete_dataset, \"no2\", \"no2 Prediction mean\"\n",
    ")\n",
    "air_pollution_DF_daily_air_quality_index_uk = uk_grids.merge(\n",
    "    air_pollution_DF_daily_air_quality_index_uk, on=\"UK Model Grid ID\"\n",
    ")\n",
    "\n",
    "ei_air_pollution_functions.visualise_air_pollution_daily_air_quality_index(\n",
    "    air_pollution_DF_daily_air_quality_index_uk,\n",
    "    \"no2 AQI\",\n",
    "    \"uk_2018_01_01_080000_air_quality_index\",\n",
    ")\n",
    "ei_air_pollution_functions.visualise_air_pollution_daily_air_quality_bands(\n",
    "    air_pollution_DF_daily_air_quality_index_uk,\n",
    "    \"no2 Air Quality Index AQI Band\",\n",
    "    \"uk_2018_01_01_080000_air_quality_bands\",\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fffd530a-30e3-46a0-8b6b-3892ec4a43a0",
   "metadata": {},
   "source": [
    "### Global Model Visualisation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "43a240bb-c180-42c2-a6c5-54f6eeb2939b",
   "metadata": {},
   "outputs": [],
   "source": [
    "air_pollution_DF_daily_air_quality_index_global = ei_air_pollution_functions.air_pollution_concentrations_to_UK_daily_air_quality_index(\n",
    "    global_complete_dataset, \"no2\", \"no2\"\n",
    ")\n",
    "air_pollution_DF_daily_air_quality_index_global = global_grids.merge(\n",
    "    air_pollution_DF_daily_air_quality_index_global, on=\"Global Model Grid ID\"\n",
    ")\n",
    "\n",
    "ei_air_pollution_functions.visualise_air_pollution_daily_air_quality_index(\n",
    "    air_pollution_DF_daily_air_quality_index_global,\n",
    "    \"no2 AQI\",\n",
    "    \"global_2018_01_01_080000_air_quality_index\",\n",
    ")\n",
    "ei_air_pollution_functions.visualise_air_pollution_daily_air_quality_bands(\n",
    "    air_pollution_DF_daily_air_quality_index_global,\n",
    "    \"no2 Air Quality Index AQI Band\",\n",
    "    \"global_2018_01_01_080000_air_quality_bands\",\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b2dad3a0-3ed1-40cf-99c5-ab22596599b9",
   "metadata": {},
   "source": [
    "\n",
    "# The code from this point onwards required you to download the data from google drive while a more permanent solution for online downloading of the data is sought. \n",
    "# Load the typical day data for the UK\n",
    "A core issue with the use of the data within this package is the amount of data that is avaliable (TBs of data). As such the use of the typical day, e.g. a typical monday in January at 8AM is provided to make conducting analysis more manageable. \n",
    "The dataset that is used in this tutorial is for Friday in January at midnight. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "60d93300-8f6c-4bad-9ecd-e93a4251f3ff",
   "metadata": {},
   "outputs": [],
   "source": [
    "uk_complete_typical_day_january_friday_midnight = (\n",
    "    ei_data.air_pollution_concentration_typical_day_real_time_united_kingdom(\n",
    "        1, \"Friday\", 0\n",
    "    )\n",
    ")\n",
    "uk_single_datapoint_typical_day_january_friday_midnight = (\n",
    "    ei_data.air_pollution_concentration_nearest_point_typical_day_united_kingdom(\n",
    "        1, \"Friday\", 0, 51.5, 0.12, uk_grids\n",
    "    )\n",
    ")\n",
    "display(uk_complete_typical_day_january_friday_midnight)\n",
    "display(uk_single_datapoint_typical_day_january_friday_midnight)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8100f77c-6adb-4eaf-94aa-197ccb3611ce",
   "metadata": {},
   "outputs": [],
   "source": [
    "air_pollution_DF_8am = (\n",
    "    ei_data.air_pollution_concentration_complete_set_real_time_united_kingdom(\n",
    "        \"2018-01-01 080000\"\n",
    "    )\n",
    ")\n",
    "air_pollution_DF_9am = (\n",
    "    ei_data.air_pollution_concentration_complete_set_real_time_united_kingdom(\n",
    "        \"2018-01-01 090000\"\n",
    "    )\n",
    ")\n",
    "air_pollution_DF_8am = ei_air_pollution_functions.air_pollution_concentrations_to_UK_daily_air_quality_index(\n",
    "    air_pollution_DF_8am, \"no2\", \"no2 Prediction mean\"\n",
    ")\n",
    "air_pollution_DF_9am = ei_air_pollution_functions.air_pollution_concentrations_to_UK_daily_air_quality_index(\n",
    "    air_pollution_DF_9am, \"no2\", \"no2 Prediction mean\"\n",
    ")\n",
    "\n",
    "air_pollution_DF_8am = uk_grids.merge(air_pollution_DF_8am, on=\"UK Model Grid ID\")\n",
    "air_pollution_DF_9am = uk_grids.merge(air_pollution_DF_9am, on=\"UK Model Grid ID\")\n",
    "display(air_pollution_DF_8am)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "76e792ea-a6d4-4c2c-ba06-d75fa07132dc",
   "metadata": {},
   "source": [
    "# Visualise the change in the air pollution concentration and air quality index for NO2 between 8am and 9am on 1st January 2018."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9a763c3e-e4d9-4119-8b15-c97a72afd4b6",
   "metadata": {},
   "outputs": [],
   "source": [
    "ei_air_pollution_functions.change_in_concentrations_visulisation(\n",
    "    air_pollution_DF_8am,\n",
    "    air_pollution_DF_9am,\n",
    "    \"no2 Prediction mean\",\n",
    "    \"uk_concentration_change_between_8_9_am\",\n",
    ")\n",
    "ei_air_pollution_functions.change_in_aqi_visulisation(\n",
    "    air_pollution_DF_8am,\n",
    "    air_pollution_DF_9am,\n",
    "    \"no2 AQI\",\n",
    "    \"uk_aqi_change_between_8_9_am\",\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f17c1246-a4c6-42b3-a091-5243034fb76e",
   "metadata": {},
   "source": [
    "# Visualising the changes in the air pollution concentrations across a number of timestamps. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "58682916-b4c1-49b9-9e33-4e918095519c",
   "metadata": {},
   "source": [
    "Alongside being able to visualise the changes in air pollution spatially, there is the ability to visualise them temporally, with an aggregate across all of the desired locations.\n",
    "The example below gives the simple hypothetical scenario of changing the values based on simply doubling, or halving the concerntations. However a model could be plugged into this process as will be seen later. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5a5e1031-89e0-4f16-ab6e-7f8abee71371",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Show the change in concentration line example\n",
    "\n",
    "# A single month should be used in the example code, with the list days being populated with the days to be analysed, out of [\"Monday\", \"Tuesday\", \"Wednesday\", \"Thursday\", \"Friday\", \"Saturday\", \"Sunday\"]\n",
    "month = 1\n",
    "days = [\"Friday\"]\n",
    "\n",
    "\n",
    "# The baseline_DFs represent the DFs that will create the black link in the graph, with change_* being the DFs that contain the concentrations with some change, in this case the change_positive_DFs being the doubling of the concentrations and\n",
    "# change_negative_DFs being the halving of the concentrations.\n",
    "baseline_DFs = dict()\n",
    "change_postive_DFs = dict()\n",
    "change_negative_DFs = dict()\n",
    "for day_of_week in days:\n",
    "\n",
    "    # Create a nested list for each day\n",
    "    baseline_DFs_single_day = dict()\n",
    "    change_postive_DFs_single_day = dict()\n",
    "    change_negative_DFs_single_day = dict()\n",
    "    for hour in np.arange(0, 24):\n",
    "\n",
    "        # load in the typical day data of interest\n",
    "        air_pollution_DF = (\n",
    "            ei_data.air_pollution_concentration_typical_day_real_time_united_kingdom(\n",
    "                month, day_of_week, hour\n",
    "            )\n",
    "        )\n",
    "\n",
    "        # Standardise the column names\n",
    "        air_pollution_DF = air_pollution_DF.rename(\n",
    "            columns={\"nox Prediction mean\": \"Model Predicition\"}\n",
    "        )\n",
    "        baseline_DFs_single_day[hour] = air_pollution_DF\n",
    "        air_pollution_DF_change = air_pollution_DF.copy(deep=True)\n",
    "\n",
    "        # Double all of the concentrations and add the DF to the corresponding list.\n",
    "        air_pollution_DF_change[\"Model Predicition\"] = (\n",
    "            air_pollution_DF_change[\"Model Predicition\"] * 2\n",
    "        )\n",
    "        change_postive_DFs_single_day[hour] = air_pollution_DF_change\n",
    "\n",
    "        # Repeat the process but for the halving of the concentrations\n",
    "        air_pollution_DF_change = air_pollution_DF.copy(deep=True)\n",
    "        air_pollution_DF_change[\"Model Predicition\"] = (\n",
    "            air_pollution_DF_change[\"Model Predicition\"] * 0.5\n",
    "        )\n",
    "        change_negative_DFs_single_day[hour] = air_pollution_DF_change\n",
    "\n",
    "    baseline_DFs[day_of_week] = baseline_DFs_single_day\n",
    "    change_postive_DFs[day_of_week] = change_postive_DFs_single_day\n",
    "    change_negative_DFs[day_of_week] = change_negative_DFs_single_day\n",
    "\n",
    "\n",
    "display(change_postive_DFs[\"Friday\"][0])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7fbbee98-9e67-4c3a-9dc7-023785e0ce97",
   "metadata": {},
   "source": [
    "### Visualise the changes based on the list of dataframe."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "342ce1b3-fab1-4abc-bb01-00e74eaa0b03",
   "metadata": {},
   "outputs": [],
   "source": [
    "ei_air_pollution_functions.change_in_concentration_line(\n",
    "    \"nox\",\n",
    "    baseline_DFs,\n",
    "    change_postive_DFs,\n",
    "    [\"Friday\"],\n",
    "    list(np.arange(0, 24)),\n",
    "    \"nox_change_line_positive\",\n",
    ")\n",
    "ei_air_pollution_functions.change_in_concentration_line(\n",
    "    \"nox\",\n",
    "    baseline_DFs,\n",
    "    change_negative_DFs,\n",
    "    [\"Friday\"],\n",
    "    list(np.arange(0, 24)),\n",
    "    \"nox_change_line_negative\",\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "75ebe22b-f255-4321-8822-4d3dbf38ffbc",
   "metadata": {},
   "source": [
    "# Example of using the model to create new predictions based on a changing feature vector. \n",
    "Exploring the air pollution change when the wind gust doubles across all locations within the feature vector. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6b6b8553-ebe4-4a2f-bd34-3bdbb9e668b0",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Read in a particular model of interest, and the typical day feature vector.\n",
    "climate_projection_model = ei_models.load_model_united_kingdom(\n",
    "    \"0.5\", \"Climate Projections\", \"no2\"\n",
    ")\n",
    "typical_day_feature_vector = ei_models.load_feature_vector_typical_day_united_kingdom(\n",
    "    1, \"Friday\", 8, uk_grids\n",
    ")\n",
    "display(typical_day_feature_vector)\n",
    "\n",
    "typical_day_feature_vector_climate_change = typical_day_feature_vector.copy(deep=True)\n",
    "\n",
    "# Double the wind gusts within the feature vector DF.\n",
    "typical_day_feature_vector_climate_change[\"instantaneous_10m_wind_gust\"] = (\n",
    "    typical_day_feature_vector_climate_change[\"instantaneous_10m_wind_gust\"] * 2\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f70132a1-bfb3-411c-ba71-90a659215c18",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Calculate the air pollution predicitons for the old and the new feature vector and describe the data, highlighting the changes between the scenarios.\n",
    "display(\n",
    "    ei_models.make_concentration_predicitions_united_kingdom(\n",
    "        climate_projection_model,\n",
    "        typical_day_feature_vector,\n",
    "        ei_models.get_model_feature_vector(\"Climate Projections\"),\n",
    "    ).describe()\n",
    ")\n",
    "display(\n",
    "    ei_models.make_concentration_predicitions_united_kingdom(\n",
    "        climate_projection_model,\n",
    "        typical_day_feature_vector_climate_change,\n",
    "        ei_models.get_model_feature_vector(\"Climate Projections\"),\n",
    "    ).describe()\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3058085f-5c01-4916-99a4-a7ef0a7ca806",
   "metadata": {},
   "source": [
    "# Access Up to Date OpenStreetMaps data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "97cda1fd-34c4-4ddf-9d22-bca1a9d803a3",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Access the amenities of interest, in this case hospitals.\n",
    "bbox = [51.29, -0.51, 51.69, 0.33]  # Example bounding box around Berlin\n",
    "amenities_gdf = ei_data.get_amenities_as_geodataframe(\"hospital\", *bbox)\n",
    "display(amenities_gdf)\n",
    "\n",
    "# Access the highways of interest, in this case motorways.\n",
    "bbox = [49.8, -10.5, 60.9, 2.2]\n",
    "highways_gdf = ei_data.get_highways_as_geodataframe(\"motorway\", *bbox)\n",
    "highways_gdf.crs = 4326\n",
    "highways_gdf = highways_gdf.to_crs(3395)\n",
    "display(highways_gdf)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "658250b4-512a-49fa-b8fa-07f37f5c1151",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Add into the feature vector a new distance feature vector element for a new moroway onto the exists motorway network.\n",
    "start_point = [0.071113, 52.231664]\n",
    "end_point = [1.3, 52.6]\n",
    "uk_grids_centroid = uk_grids.copy(deep=True)\n",
    "uk_grids_centroid[\"geometry\"] = uk_grids_centroid[\"geometry\"].centroid\n",
    "new_data, highways_user_added = ei_data.calculate_new_metrics_distance_total(\n",
    "    highways_gdf, \"motorway\", start_point, end_point, uk_grids_centroid, uk_grids\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e72b587b-2e07-4830-b2c7-0efe4e3ca724",
   "metadata": {},
   "outputs": [],
   "source": [
    "# visualise the new motorway segment (red) alongside the currently existing network (blue)\n",
    "color_map = {\"osm\": \"blue\", \"User Added\": \"red\"}\n",
    "\n",
    "fig, axes = plt.subplots(1, figsize=(15, 15))\n",
    "highways_gdf.plot(ax=axes, color=highways_user_added[\"source\"].map(color_map))\n",
    "axes.axis(\"off\")\n",
    "# Create custom legend handles\n",
    "legend_elements = [\n",
    "    matplotlib.lines.Line2D([0], [0], color=\"blue\", lw=2, label=\"Current\\nMotorway\"),\n",
    "]\n",
    "\n",
    "# Add the custom legend to the axis\n",
    "axes.legend(\n",
    "    handles=legend_elements, fontsize=20, bbox_to_anchor=(1.2, 0.95), markerscale=2\n",
    ")\n",
    "\n",
    "\n",
    "fig, axes = plt.subplots(1, figsize=(15, 15))\n",
    "highways_user_added.plot(ax=axes, color=highways_user_added[\"source\"].map(color_map))\n",
    "axes.axis(\"off\")\n",
    "# Create custom legend handles\n",
    "legend_elements = [\n",
    "    matplotlib.lines.Line2D([0], [0], color=\"red\", lw=2, label=\"Proposed\\nMotorway\"),\n",
    "    matplotlib.lines.Line2D([0], [0], color=\"blue\", lw=2, label=\"Current\\nMotorway\"),\n",
    "]\n",
    "\n",
    "# Add the custom legend to the axis\n",
    "axes.legend(\n",
    "    handles=legend_elements, fontsize=20, bbox_to_anchor=(1.2, 0.95), markerscale=2\n",
    ")\n",
    "\n",
    "# Load in the different models.\n",
    "air_pollutants = [\"no2\", \"o3\", \"pm10\", \"pm2.5\", \"so2\"]\n",
    "complete_models = dict()\n",
    "for air_pollutant in air_pollutants:\n",
    "    complete_models[air_pollutant] = ei_models.load_model_united_kingdom(\n",
    "        \"0.5\", \"Transport Infrastructure Policy\", air_pollutant\n",
    "    )\n",
    "\n",
    "typical_day_feature_vector = ei_models.load_feature_vector_typical_day_united_kingdom(\n",
    "    1, \"Friday\", 8, uk_grids\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "43b0fb44-0de5-4c36-9114-5910003d058c",
   "metadata": {},
   "outputs": [],
   "source": [
    "# The same process as above, is conducted with a real model, and the example of changing the motorway network analysed in the feature vector.\n",
    "baseline_DFs_air_pollutant = dict()\n",
    "change_DFs_air_pollutant = dict()\n",
    "for air_pollutant in air_pollutants:\n",
    "    month = 1\n",
    "    days = [\"Friday\"]\n",
    "    baseline_DFs = dict()\n",
    "    changeDFs = dict()\n",
    "    for day_of_week in days:\n",
    "        display(day_of_week)\n",
    "        baseline_DFs_single_day = dict()\n",
    "        change_DFs_single_day = dict()\n",
    "        for hour in np.arange(0, 24):\n",
    "\n",
    "            # Read in the relevant feature vector for the desired timestamp.\n",
    "            feature_vector = ei_models.load_feature_vector_typical_day_united_kingdom(\n",
    "                month, day_of_week, hour, uk_grids\n",
    "            )\n",
    "\n",
    "            # Create the baseline based on the current data\n",
    "            air_pollution_estimation_baseline = (\n",
    "                ei_models.make_concentration_predicitions_united_kingdom(\n",
    "                    complete_models[air_pollutant],\n",
    "                    feature_vector,\n",
    "                    ei_models.get_model_feature_vector(\n",
    "                        \"Transport Infrastructure Policy\"\n",
    "                    ),\n",
    "                )\n",
    "            )\n",
    "            air_pollution_estimation_baseline = (\n",
    "                air_pollution_estimation_baseline.rename(\n",
    "                    columns={\"Model Predicition\": \"Model Predicition Baseline\"}\n",
    "                )\n",
    "            )\n",
    "\n",
    "            # Modify the feature vector to include details of the new motorway segment.\n",
    "            feature_vector_modified = ei_data.replace_feature_vector_column(\n",
    "                feature_vector, new_data, \"Road Infrastructure Distance motorway\"\n",
    "            )\n",
    "            feature_vector_modified = ei_data.replace_feature_vector_column(\n",
    "                feature_vector_modified, new_data, \"Total Length motorway\"\n",
    "            )\n",
    "\n",
    "            # Calculate the new air pollution concentrations based on the modified feature vector.\n",
    "            air_pollution_estimation_modified = (\n",
    "                ei_models.make_concentration_predicitions_united_kingdom(\n",
    "                    complete_models[air_pollutant],\n",
    "                    feature_vector_modified,\n",
    "                    ei_models.get_model_feature_vector(\n",
    "                        \"Transport Infrastructure Policy\"\n",
    "                    ),\n",
    "                )\n",
    "            )\n",
    "            air_pollution_estimation_modified = (\n",
    "                air_pollution_estimation_modified.rename(\n",
    "                    columns={\"Model Predicition\": \"Model Predicition Modified\"}\n",
    "                )\n",
    "            )\n",
    "\n",
    "            air_pollution_estimation = air_pollution_estimation_modified.merge(\n",
    "                air_pollution_estimation_baseline, on=\"UK Model Grid ID\"\n",
    "            )\n",
    "\n",
    "            air_pollution_estimation_difference = air_pollution_estimation[\n",
    "                air_pollution_estimation[\"Model Predicition Baseline\"]\n",
    "                != air_pollution_estimation[\"Model Predicition Modified\"]\n",
    "            ]\n",
    "\n",
    "            baseline_DFs_single_day[hour] = air_pollution_estimation_difference[\n",
    "                [\"UK Model Grid ID\", \"Model Predicition Baseline\"]\n",
    "            ].rename(columns={\"Model Predicition Baseline\": \"Model Predicition\"})\n",
    "            change_DFs_single_day[hour] = air_pollution_estimation_difference[\n",
    "                [\"UK Model Grid ID\", \"Model Predicition Modified\"]\n",
    "            ].rename(columns={\"Model Predicition Modified\": \"Model Predicition\"})\n",
    "\n",
    "        baseline_DFs[day_of_week] = baseline_DFs_single_day\n",
    "        changeDFs[day_of_week] = change_DFs_single_day\n",
    "    baseline_DFs_air_pollutant[air_pollutant] = baseline_DFs\n",
    "    change_DFs_air_pollutant[air_pollutant] = changeDFs"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f7a29641-c479-458c-aea0-91afdc4cf7b0",
   "metadata": {},
   "source": [
    "# Visualise the changes in air pollution across a typical friday due to the placement of the new motorway segment "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "bf1ee50b-136e-44cf-9165-45ae575fff5a",
   "metadata": {},
   "outputs": [],
   "source": [
    "chage_concentration_lines_figs = dict()\n",
    "for air_pollutant in air_pollutants:\n",
    "    chage_concentration_lines_figs[air_pollutant] = (\n",
    "        ei_air_pollution_functions.change_in_concentration_line(\n",
    "            air_pollutant,\n",
    "            baseline_DFs_air_pollutant[air_pollutant],\n",
    "            change_DFs_air_pollutant[air_pollutant],\n",
    "            [\"Friday\"],\n",
    "            list(np.arange(0, 24)),\n",
    "            \"motorway_addition_\" + air_pollutant,\n",
    "        )\n",
    "    )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2215029b-582b-4a63-ae23-8bc929b841b4",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2b656e69-4f56-4a3c-83c2-b31b03199d3e",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}