You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardexpand all lines: working_with_databases_from_r.qmd
+20-18
Original file line number
Diff line number
Diff line change
@@ -2,16 +2,17 @@
2
2
3
3
{width="250"}
4
4
5
-
*Artwork by \@allison_horst*
5
+
*Artwork by [\@allison_horst](https://x.com/allison_horst)*
6
6
7
-
Before we start thinking about working with health care data spread across a database using the OMOP common data model, let's first do a quick data analysis from R using a simpler dataset held in a database to quickly understand the general approach. For this we'll use data from [palmerpenguins package](https://allisonhorst.github.io/palmerpenguins/), which contains data on penguins collected from the [Palmer Station](https://en.wikipedia.org/wiki/Palmer_Station) in Antarctica.
7
+
Before we start thinking about working with healthcare data spread across a database using the OMOP common data model, let's first do a quick data analysis with R using a simpler dataset held in a database to quickly understand the general approach. For this we'll use data from [palmerpenguins package](https://allisonhorst.github.io/palmerpenguins/), which contains data on penguins collected from the [Palmer Station](https://en.wikipedia.org/wiki/Palmer_Station) in Antarctica.
8
8
9
9
## Getting set up
10
10
11
-
Assuming that you have R and RStudio already set up, first we need to install a few packages not included in base R if we don´t already have them.
11
+
Assuming that you have R and RStudio already set up, first we need to install a few packages not included in base R if we don't already have them.
12
12
13
13
```{r, eval=FALSE}
14
14
install.packages("dplyr")
15
+
install.packages("dbplyr")
15
16
install.packages("ggplot2")
16
17
install.packages("DBI")
17
18
install.packages("duckdb")
@@ -22,6 +23,7 @@ Once installed, we can load them like so.
22
23
23
24
```{r, message=FALSE, warning=FALSE}
24
25
library(dplyr)
26
+
library(dbplyr)
25
27
library(ggplot2)
26
28
library(DBI)
27
29
library(duckdb)
@@ -30,34 +32,34 @@ library(palmerpenguins)
30
32
31
33
## Taking a peek at the data
32
34
33
-
We can get an overview of the data using the `glimpse()` command.
35
+
The package `palmerpenguins` contains two datasets, one of them called `penguins`, which we will use in this chapter. We can get an overview of the data using the `glimpse()` command.
34
36
35
37
```{r}
36
38
glimpse(penguins)
37
39
```
38
40
39
-
Or we could take a look at the first rows of the data using `head()`
41
+
Or we could take a look at the first rows of the data using `head()` :
40
42
41
43
```{r}
42
44
head(penguins, 5)
43
45
```
44
46
45
47
## Inserting data into a database
46
48
47
-
Let's put our penguins data into a [duckdb database](https://duckdb.org/). We create the database, add the penguins data, and then create a reference to the table containing the data.
49
+
Let's put our penguins data into a [duckdb database](https://duckdb.org/). We need to first create the database and then add the penguins data to it.
48
50
49
51
```{r}
50
52
db <- dbConnect(duckdb::duckdb(), dbdir = ":memory:")
51
53
dbWriteTable(db, "penguins", penguins)
52
54
```
53
55
54
-
We can see that our database now has one table
56
+
We can see that our database now has one table:
55
57
56
58
```{r}
57
59
DBI::dbListTables(db)
58
60
```
59
61
60
-
And now that the data is in a database we could use SQL to get the first rows that we saw before
62
+
And now that the data is in a database we could use SQL to get the first rows that we saw before.
Database connections from R can be made using the [DBI package](https://dbi.r-dbi.org/). The back-end for `DBI` is facilitated by database specific driver packages. Above we we created a new, empty, in-process [duckdb](https://duckdb.org/) database which we then added database. But we could have instead connected to an existing duckdb database. This could, for example, look like
71
+
Database connections from R can be made using the [DBI package](https://dbi.r-dbi.org/). The back-end for `DBI` is facilitated by database specific driver packages. In the code snipets above we created a new, empty, in-process [duckdb](https://duckdb.org/) database to which we then added our dataset. But we could have instead connected to an existing duckdb database. This could, for example, look like
70
72
71
73
```{r, eval = FALSE}
72
74
db <- dbConnect(duckdb::duckdb(),
73
75
dbdir = here("my_duckdb_database.ducdkb"))
74
76
```
75
77
76
-
In this book for simplicity we will mostly be working with in-process duckdb databases with synthetic data. However, when analysing real patient data we will be more often working with client-server databases, where we are connecting from our computer to a central server with the database or working with data held in the cloud. The approaches shown throughout this book will work in the same way for these other types of database management systems, but the way to connect to the database will be different (although still using DBI). In general, creating connections are supported by associated back-end packages. For example a connection to a Postgres database would use the RPostgres R package and look something like:
78
+
In this book for simplicity we will mostly be working with in-process duckdb databases with synthetic data. However, when analysing real patient data we will be more often working with client-server databases, where we are connecting from our computer to a central server with the database or working with data held in the cloud. The approaches shown throughout this book will work in the same way for these other types of database management systems, but the way to connect to the database will be different (although still using DBI). In general, creating connections is supported by associated back-end packages. For example a connection to a Postgres database would use the RPostgres R package and look something like:
77
79
78
80
```{r, eval=FALSE}
79
81
db <- DBI::dbConnect(RPostgres::Postgres(),
@@ -86,7 +88,7 @@ db <- DBI::dbConnect(RPostgres::Postgres(),
86
88
87
89
## Translation from R to SQL
88
90
89
-
Instead of using SQL, we could instead use the same R code as before. Now it will query the data held in a database. To do this, first we create a reference to the table in the database.
91
+
Instead of using SQL to query our database, we might instead want to use the same R code as before. However, instead of working with the local dataset, now we will need it to query the data held in the database. To do this, first we can create a reference to the table in the database as such:
90
92
91
93
```{r}
92
94
penguins_db <- tbl(db, "penguins")
@@ -99,7 +101,7 @@ Once we have this reference, we can then use it with familiar looking R code.
99
101
head(penguins_db, 5)
100
102
```
101
103
102
-
The magic here is provided by the `dbplyr` which takes the R code and converts it into SQL, which in this case looks like the SQL we could have written instead.
104
+
The magic here is provided by the `dbplyr`package, which takes the R code and converts it into SQL. In this case the query looks like the SQL we wrote directly before.
103
105
104
106
```{r}
105
107
head(penguins_db, 5) |>
@@ -108,7 +110,7 @@ head(penguins_db, 5) |>
108
110
109
111
## Example analysis
110
112
111
-
More complicated SQL can also be generated by using familiar `dplyr` code. For example, we could get a summary of bill length by species like so
113
+
More complicated SQL can also be generated by using familiar `dplyr` code. For example, we could get a summary of bill length by species like so:
112
114
113
115
```{r, warning=FALSE}
114
116
penguins_db |>
@@ -131,7 +133,7 @@ penguins_db |>
131
133
)
132
134
```
133
135
134
-
The benefit of using dbplyr now becomes quite clear if we take a look at the corresponding SQL that is generated for us.
136
+
The benefit of using `dbplyr` now becomes quite clear if we take a look at the corresponding SQL that is generated for us:
135
137
136
138
```{r, warning=FALSE}
137
139
penguins_db |>
@@ -151,9 +153,9 @@ penguins_db |>
151
153
show_query()
152
154
```
153
155
154
-
Instead of having to write this somewhat complex SQL specific to duckdb we can use the friendlier dplyr syntax that may well be more familiar if coming from an R programming background.
156
+
Instead of having to write this somewhat complex SQL specific to `duckdb` we can use the friendlier `dplyr` syntax that may well be more familiar if coming from an R programming background.
155
157
156
-
Now suppose we are particularly interested in the body mass variable. We can first notice that there are a couple of missing records for this.
158
+
Not having to worry about the SQL translation behind our queries allows us to interrogate the database in a simple way even for more complex questions. For instance, suppose now that we are particularly interested in the body mass variable. We can first notice that there are a couple of missing records for this.
157
159
158
160
```{r}
159
161
penguins_db |>
@@ -217,11 +219,11 @@ penguins |>
217
219
theme(legend.position = "none")
218
220
```
219
221
220
-
As well as having an example of working with data in database from R, you also have an example of [Simpson´s paradox](https://en.wikipedia.org/wiki/Simpson%27s_paradox)!
222
+
As well as having an example of working with data in database from R, you also have an example of [Simpson's paradox](https://en.wikipedia.org/wiki/Simpson%27s_paradox)!
221
223
222
224
## Disconnecting from the database
223
225
224
-
And now we've reached the end of this example, we can close our connection to the database.
226
+
Now that we've reached the end of this example, we can close our connection to the database using the `DBI` package.
0 commit comments