Skip to content

Commit def7092

Browse files
alexisbcookalexisbcook
authored
Merge pull request #266 from Kaggle/ml-extra-credit
Intro to ML: Add Titanic Extra Credit
2 parents 4d7ab1d + e4001e6 commit def7092

File tree

4 files changed

+376
-9
lines changed

4 files changed

+376
-9
lines changed

‎notebooks/machine_learning/raw/tut_titanic.ipynb

+346
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,346 @@
1+
{
2+
"cells": [
3+
{
4+
"cell_type": "markdown",
5+
"metadata": {},
6+
"source": [
7+
"In the final exercise of the **[Intro to Machine Learning](https://www.kaggle.com/learn/intro-to-machine-learning)** course, you learned how to make a submission to a Kaggle competition. But some of the work was already completed for you, since you were provided a notebook with partially completed code. \n",
8+
"\n",
9+
"In this tutorial, you'll explore a **full workflow** that you can use to get started (from the very beginning!) with creating a submission to any Kaggle competition. We'll use the **[Titanic competition](https://www.kaggle.com/c/titanic)** as an example.\n",
10+
"\n",
11+
"# Part 1: Get started\n",
12+
"\n",
13+
"In this section, you'll learn more about the competition and make your first submission. \n",
14+
"\n",
15+
"## Join the competition!\n",
16+
"\n",
17+
"The first thing to do is to join the competition! Open a new window with **[the competition page](https://www.kaggle.com/c/titanic)**, and click on the **\"Join Competition\"** button, if you haven't already. (_If you see a \"Submit Predictions\" button instead of a \"Join Competition\" button, you have already joined the competition, and don't need to do so again._)\n",
18+
"\n",
19+
"![](https://i.imgur.com/rRFchA8.png)\n",
20+
"\n",
21+
"This takes you to the rules acceptance page. You must accept the competition rules in order to participate. These rules govern how many submissions you can make per day, the maximum team size, and other competition-specific details. Then, click on **\"I Understand and Accept\"** to indicate that you will abide by the competition rules.\n",
22+
"\n",
23+
"## The challenge\n",
24+
"\n",
25+
"The competition is simple: we want you to use the Titanic passenger data (name, age, price of ticket, etc) to try to predict who will survive and who will die.\n",
26+
"\n",
27+
"## The data\n",
28+
"\n",
29+
"To take a look at the competition data, click on the **<a href=\"https://www.kaggle.com/c/titanic/data\" target=\"_blank\" rel=\"noopener noreferrer\"><b>Data tab</b></a>** at the top of the competition page. Then, scroll down to find the list of files. \n",
30+
"\n",
31+
"![](https://i.imgur.com/LiM3JA7.png)\n",
32+
"\n",
33+
"There are three files in the data: (1) **train.csv**, (2) **test.csv**, and (3) **gender_submission.csv**.\n",
34+
"\n",
35+
"### (1) train.csv\n",
36+
"\n",
37+
"**train.csv** contains the details of a subset of the passengers on board (891 passengers, to be exact -- where each passenger gets a different row in the table). To investigate this data, click on the name of the file under the **\"Data Sources\"** column (on the left of the screen). Once you've done this, all of the column names (along with a brief description of what they contain) are listed to the right of the screen, under the **\"Columns\"** heading. \n",
38+
"\n",
39+
"![](https://i.imgur.com/w5HFxp8.png)\n",
40+
"\n",
41+
"You can view all of the data in the same window. \n",
42+
"\n",
43+
"![](https://i.imgur.com/CEPZi6z.png)\n",
44+
"\n",
45+
"The values in the second column (**\"Survived\"**) can be used to determine whether each passenger survived or not: \n",
46+
"- if it's a \"1\", the passenger survived.\n",
47+
"- if it's a \"0\", the passenger died.\n",
48+
"\n",
49+
"For instance, the first passenger listed in **train.csv** is Mr. Owen Harris Braund. He was 22 years old when he died on the Titanic.\n",
50+
"\n",
51+
"### (2) test.csv\n",
52+
"\n",
53+
"Using the patterns you find in **train.csv**, you have to predict whether the other 418 passengers on board (in **test.csv**) survived. \n",
54+
"\n",
55+
"Click on **test.csv** (under the **\"Data Sources\"** column) to examine its contents. Note that **test.csv** does not have a **\"Survived\"** column - this information is hidden from you, and how well you do at predicting these hidden values will determine how highly you score in the competition! \n",
56+
"\n",
57+
"### (3) gender_submission.csv\n",
58+
"\n",
59+
"The **gender_submission.csv** file is provided as an example that shows how you should structure your predictions. It predicts that all female passengers survived, and all male passengers died. Your hypotheses regarding survival will probably be different, which will lead to a different submission file. But, just like this file, your submission should have:\n",
60+
"- a **\"PassengerId\"** column containing the IDs of each passenger from **test.csv**.\n",
61+
"- a **\"Survived\"** column (that you will create!) with a \"1\" for the rows where you think the passenger survived, and a \"0\" where you predict that the passenger died.\n",
62+
"\n",
63+
"## Your first submission\n",
64+
"\n",
65+
"As a benchmark, you'll download the **gender_submission.csv** file and submit it to the competition. Begin by clicking on the download link to the right of the name of the file. \n",
66+
"\n",
67+
"![](https://i.imgur.com/Pl1DIA8.png)\n",
68+
"\n",
69+
"This downloads the file to your computer. Then:\n",
70+
"- Click on the blue **\"Submit Predictions\"** button in the top right corner of the competition page. (_This button now appears where the **\"Join Competition\"** button was._)\n",
71+
"- Scroll down to **\"Step 1: Upload submission file\"**. Upload the file you just downloaded. Then, click on the blue **\"Make Submission\"** button. \n",
72+
"\n",
73+
"In a few seconds, your submission will be scored, and you'll receive a spot on the leaderboard. Next, we'll walk you through how to outperform this initial submission!"
74+
]
75+
},
76+
{
77+
"cell_type": "markdown",
78+
"metadata": {},
79+
"source": [
80+
"# Part 2: Your coding environment\n",
81+
"\n",
82+
"In this section, you'll train your own machine learning model to improve your predictions. \n",
83+
"\n",
84+
"## The Notebook\n",
85+
"\n",
86+
"The first thing to do is to create a Kaggle Notebook where you'll store all of your code. You can use Kaggle Notebooks to getting up and running with writing code quickly, and without having to install anything on your computer. (_If you are interested in deep learning, we also offer free GPU and TPU access!_) \n",
87+
"\n",
88+
"Begin by clicking on the **<a href=\"https://www.kaggle.com/c/titanic/kernels\" target=\"_blank\">Notebooks tab</a>** on the competition page. Then, click on **\"New Notebook\"**.\n",
89+
"\n",
90+
"![](https://i.imgur.com/DHPyh7s.png)\n",
91+
"\n",
92+
"Next, click on **\"Create\"**. (_Don't change the default settings: so, **\"Python\"** should appear under \"Select language\", and you should have **\"Notebook\"** selected under \"Select type\"._)\n",
93+
"\n",
94+
"![](https://i.imgur.com/qUVvr8k.png)\n",
95+
"\n",
96+
"Your notebook will take a few seconds to load. In the top left corner, you can see the name of your notebook -- something like **\"kernel2daed3cd79\"**.\n",
97+
"\n",
98+
"![](https://i.imgur.com/64ZFT1L.png)\n",
99+
"\n",
100+
"You can edit the name by clicking on it. Change it to something more descriptive, like **\"Getting Started with Titanic\"**. \n",
101+
"\n",
102+
"![](https://i.imgur.com/uwyvzXq.png)\n",
103+
"\n",
104+
"## Your first lines of code\n",
105+
"\n",
106+
"When you start a new notebook, it has two gray boxes for storing code. We refer to these gray boxes as \"code cells\".\n",
107+
"\n",
108+
"![](https://i.imgur.com/q9mwkZM.png)\n",
109+
"\n",
110+
"The first code cell already has some code in it. To run this code, put your cursor in the code cell. (_If your cursor is in the right place, you'll notice a blue vertical line to the left of the gray box._) Then, either hit the play button (which appears to the left of the blue line), or hit **[Shift] + [Enter]** on your keyboard.\n",
111+
"\n",
112+
"If the code runs successfully, three lines of output are returned. Below, you can see the same code that you just ran, along with the output that you should see in your notebook."
113+
]
114+
},
115+
{
116+
"cell_type": "code",
117+
"execution_count": null,
118+
"metadata": {
119+
"_kg_hide-input": false
120+
},
121+
"outputs": [],
122+
"source": [
123+
"# This Python 3 environment comes with many helpful analytics libraries installed\n",
124+
"# It is defined by the kaggle/python docker image: https://github.com/kaggle/docker-python\n",
125+
"# For example, here's several helpful packages to load in \n",
126+
"\n",
127+
"import numpy as np # linear algebra\n",
128+
"import pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n",
129+
"\n",
130+
"# Input data files are available in the \"../input/\" directory.\n",
131+
"# For example, running this (by clicking run or pressing Shift+Enter) will list all files under the input directory\n",
132+
"\n",
133+
"import os\n",
134+
"for dirname, _, filenames in os.walk('/kaggle/input'):\n",
135+
" for filename in filenames:\n",
136+
" print(os.path.join(dirname, filename))\n",
137+
"\n",
138+
"# Any results you write to the current directory are saved as output."
139+
]
140+
},
141+
{
142+
"cell_type": "markdown",
143+
"metadata": {},
144+
"source": [
145+
"This shows us where the competition data is stored, so that we can load the files into the notebook. We'll do that next.\n",
146+
"\n",
147+
"## Load the data\n",
148+
"\n",
149+
"The second code cell in your notebook now appears below the three lines of output with the file locations.\n",
150+
"\n",
151+
"![](https://i.imgur.com/OQBax9n.png)\n",
152+
"\n",
153+
"Type the two lines of code below into your second code cell. Then, once you're done, either click on the blue play button, or hit **[Shift] + [Enter]**. "
154+
]
155+
},
156+
{
157+
"cell_type": "code",
158+
"execution_count": null,
159+
"metadata": {},
160+
"outputs": [],
161+
"source": [
162+
"train_data = pd.read_csv(\"../input/titanic/train.csv\")\n",
163+
"train_data.head()"
164+
]
165+
},
166+
{
167+
"cell_type": "markdown",
168+
"metadata": {},
169+
"source": [
170+
"Your code should return the output above, which corresponds to the first five rows of the table in **train.csv**. It's very important that you see this output **in your notebook** before proceeding with the tutorial!\n",
171+
"> _If your code does not produce this output_, double-check that your code is identical to the two lines above. And, make sure your cursor is in the code cell before hitting **[Shift] + [Enter]**.\n",
172+
"\n",
173+
"The code that you've just written is in the Python programming language. It uses a Python \"module\" called **pandas** (abbreviated as `pd`) to load the table from the **train.csv** file into the notebook. To do this, we needed to plug in the location of the file (which we saw was `/kaggle/input/titanic/train.csv`). \n",
174+
"> If you're not already familiar with Python (and pandas), the code shouldn't make sense to you -- but don't worry! The point of this tutorial is to (quickly!) make your first submission to the competition. At the end of the tutorial, we suggest resources to continue your learning.\n",
175+
"\n",
176+
"At this point, you should have at least three code cells in your notebook. \n",
177+
"![](https://i.imgur.com/ReLhYca.png)\n",
178+
"\n",
179+
"Copy the code below into the third code cell of your notebook to load the contents of the **test.csv** file. Don't forget to click on the play button (or hit **[Shift] + [Enter]**)!"
180+
]
181+
},
182+
{
183+
"cell_type": "code",
184+
"execution_count": null,
185+
"metadata": {},
186+
"outputs": [],
187+
"source": [
188+
"test_data = pd.read_csv(\"../input/titanic/test.csv\")\n",
189+
"test_data.head()"
190+
]
191+
},
192+
{
193+
"cell_type": "markdown",
194+
"metadata": {},
195+
"source": [
196+
"As before, make sure that you see the output above in your notebook before continuing. \n",
197+
"\n",
198+
"Once all of the code runs successfully, all of the data (in **train.csv** and **test.csv**) is loaded in the notebook. (_The code above shows only the first 5 rows of each table, but all of the data is there -- all 891 rows of **train.csv** and all 418 rows of **test.csv**!_)\n",
199+
"\n",
200+
"# Part 3: Improve your score\n",
201+
"\n",
202+
"Remember our goal: we want to find patterns in **train.csv** that help us predict whether the passengers in **test.csv** survived.\n",
203+
"\n",
204+
"It might initially feel overwhelming to look for patterns, when there's so much data to sort through. So, we'll start simple.\n",
205+
"\n",
206+
"## Explore a pattern\n",
207+
"\n",
208+
"Remember that the sample submission file in **gender_submission.csv** assumes that all female passengers survived (and all male passengers died). \n",
209+
"\n",
210+
"Is this a reasonable first guess? We'll check if this pattern holds true in the data (in **train.csv**).\n",
211+
"\n",
212+
"Copy the code below into a new code cell. Then, run the cell."
213+
]
214+
},
215+
{
216+
"cell_type": "code",
217+
"execution_count": null,
218+
"metadata": {
219+
"scrolled": true
220+
},
221+
"outputs": [],
222+
"source": [
223+
"women = train_data.loc[train_data.Sex == 'female'][\"Survived\"]\n",
224+
"rate_women = sum(women)/len(women)\n",
225+
"\n",
226+
"print(\"% of women who survived:\", rate_women)"
227+
]
228+
},
229+
{
230+
"cell_type": "markdown",
231+
"metadata": {},
232+
"source": [
233+
"Before moving on, make sure that your code returns the output above. The code above calculates the percentage of female passengers (in **train.csv**) who survived.\n",
234+
"\n",
235+
"Then, run the code below in another code cell:"
236+
]
237+
},
238+
{
239+
"cell_type": "code",
240+
"execution_count": null,
241+
"metadata": {},
242+
"outputs": [],
243+
"source": [
244+
"men = train_data.loc[train_data.Sex == 'male'][\"Survived\"]\n",
245+
"rate_men = sum(men)/len(men)\n",
246+
"\n",
247+
"print(\"% of men who survived:\", rate_men)"
248+
]
249+
},
250+
{
251+
"cell_type": "markdown",
252+
"metadata": {},
253+
"source": [
254+
"The code above calculates the percentage of male passengers (in **train.csv**) who survived.\n",
255+
"\n",
256+
"From this you can see that almost 75% of the women on board survived, whereas only 19% of the men lived to tell about it. Since gender seems to be such a strong indicator of survival, the submission file in **gender_submission.csv** is not a bad first guess, and it makes sense that it performed reasonably well!\n",
257+
"\n",
258+
"But at the end of the day, this gender-based submission bases its predictions on only a single column. As you can imagine, by considering multiple columns, we can discover more complex patterns that can potentially yield better-informed predictions. Since it is quite difficult to consider several columns at once (or, it would take a long time to consider all possible patterns in many different columns simultaneously), we'll use machine learning to automate this for us.\n",
259+
"\n",
260+
"## Your first machine learning model\n",
261+
"\n",
262+
"We'll build a [**random forest model**](https://www.kaggle.com/dansbecker/random-forests). This model is constructed of several \"trees\" (there are three trees in the picture below, but we'll construct 100!) that will individually consider each passenger's data and vote on whether the individual survived. Then, the random forest model makes a democratic decision: the outcome with the most votes wins!\n",
263+
"\n",
264+
"![](https://i.imgur.com/AC9Bq63.png)\n",
265+
"\n",
266+
"The code cell below looks for patterns in four different columns (**\"Pclass\"**, **\"Sex\"**, **\"SibSp\"**, and **\"Parch\"**) of the data. It constructs the trees in the random forest model based on patterns in the **train.csv** file, before generating predictions for the passengers in **test.csv**. The code also saves these new predictions in a CSV file **my_submission.csv**.\n",
267+
"\n",
268+
"Copy this code into your notebook, and run it in a new code cell."
269+
]
270+
},
271+
{
272+
"cell_type": "code",
273+
"execution_count": null,
274+
"metadata": {
275+
"_kg_hide-output": false
276+
},
277+
"outputs": [],
278+
"source": [
279+
"from sklearn.ensemble import RandomForestClassifier\n",
280+
"\n",
281+
"y = train_data[\"Survived\"]\n",
282+
"\n",
283+
"features = [\"Pclass\", \"Sex\", \"SibSp\", \"Parch\"]\n",
284+
"X = pd.get_dummies(train_data[features])\n",
285+
"X_test = pd.get_dummies(test_data[features])\n",
286+
"\n",
287+
"model = RandomForestClassifier(n_estimators=100, max_depth=5, random_state=1)\n",
288+
"model.fit(X, y)\n",
289+
"predictions = model.predict(X_test)\n",
290+
"\n",
291+
"output = pd.DataFrame({'PassengerId': test_data.PassengerId, 'Survived': predictions})\n",
292+
"output.to_csv('my_submission.csv', index=False)\n",
293+
"print(\"Your submission was successfully saved!\")"
294+
]
295+
},
296+
{
297+
"cell_type": "markdown",
298+
"metadata": {},
299+
"source": [
300+
"Make sure that your notebook outputs the same message above (`Your submission was successfully saved!`) before moving on.\n",
301+
"> Again, don't worry if this code doesn't make sense to you! For now, we'll focus on how to generate and submit predictions.\n",
302+
"\n",
303+
"Once you're ready, click on the blue **\"Save Version\"** button in the top right corner of your notebook. This will generate a pop-up window. \n",
304+
"- Ensure that the **\"Save and Run All\"** option is selected, and then click on the blue **\"Save\"** button.\n",
305+
"- This generates a window in the bottom left corner of the notebook. After it has finished running, click on the number to the right of the **\"Save Version\"** button. This pulls up a list of versions on the right of the screen. Click on the ellipsis **(...)** to the right of the most recent version, and select **Open in Viewer**. \n",
306+
"- Click on the **Output** tab on the right of the screen. Then, click on the **\"Submit to Competition\"** button to submit your results.\n",
307+
"\n",
308+
"![](https://i.imgur.com/kKKnHpx.png)\n",
309+
"\n",
310+
"Once your file is successfully submitted, you should receive a message saying that you've moved up the leaderboard. Great work!"
311+
]
312+
},
313+
{
314+
"cell_type": "markdown",
315+
"metadata": {},
316+
"source": [
317+
"# Part 4: Keep learning!\n",
318+
"\n",
319+
"Can you use what you learned about random forests in the **[Intro to Machine Learning](https://www.kaggle.com/learn/intro-to-machine-learning)** course to generate even better predictions? \n",
320+
"\n",
321+
"Check out the **[Intermediate Machine Learning](https://www.kaggle.com/learn/intermediate-machine-learning)** course to learn about more advanced techniques!"
322+
]
323+
}
324+
],
325+
"metadata": {
326+
"kernelspec": {
327+
"display_name": "Python 3",
328+
"language": "python",
329+
"name": "python3"
330+
},
331+
"language_info": {
332+
"codemirror_mode": {
333+
"name": "ipython",
334+
"version": 3
335+
},
336+
"file_extension": ".py",
337+
"mimetype": "text/x-python",
338+
"name": "python",
339+
"nbconvert_exporter": "python",
340+
"pygments_lexer": "ipython3",
341+
"version": "3.6.5"
342+
}
343+
},
344+
"nbformat": 4,
345+
"nbformat_minor": 1
346+
}

‎notebooks/machine_learning/setup_data.sh

+13
Original file line numberDiff line numberDiff line change
@@ -31,5 +31,18 @@ do
3131
unzip ${comp}.zip
3232
chmod 700 *.csv
3333
cp *.csv ..
34+
cd ../..
35+
done
36+
37+
COMPDATASET="titanic"
38+
39+
for comp in $COMPDATASET
40+
do
41+
dest="input/$comp"
42+
mkdir -p $dest
43+
kaggle competitions download $comp -p $dest
44+
cd $dest
45+
unzip ${comp}.zip
46+
chmod 700 *.csv
3447
cd ..
3548
done

‎notebooks/machine_learning/testing.yaml

+2-2
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# Rendered ipynb files and kernel metadata json files for this config will be saved at notebooks/<track>/<tag>/
22
tag: testing
3-
public: true
3+
public: false
44
# If true, then exercise kernels synced to kaggle will have internet enabled and will begin
55
# with a cell that pip installs the current learntools branch. (Useful for testing
66
# notebooks on Kernels without requiring image docker deploy for every learntools change)
@@ -20,4 +20,4 @@ testing: true
2020
# should keep their original slugs under author B's namespace. But if author A wants to push
2121
# new testing versions of the notebook, they can set author: author_A in testing.yaml to generate
2222
# new versions of the kernels under their own username.
23-
#author: jane_doe
23+
author: alexisbcook

0 commit comments

Comments
 (0)