You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardexpand all lines: dbplyr_packages.qmd
+13-14
Original file line number
Diff line number
Diff line change
@@ -1,6 +1,6 @@
1
1
# Building analytic pipelines for a data model {#sec-dbplyr_packages}
2
2
3
-
In the previous chapters we've seen that after connecting to a database we can create references to the various table we're interested in and write bespoke analytic code. However, if we are working with the same database over and over again we are likely to want to build some tooling for tasks we are often performing.
3
+
In the previous chapters we've seen that after connecting to a database we can create references to the various tables we've interested in it and write bespoke analytic code to query them. However, if we are working with the same database over and over again we are likely to want to build some tooling for tasks we are often performing.
4
4
5
5
To see how we can develop a data model with associated methods and functions we'll use the Lahman baseball data. We can see below how the data is stored across various related tables.
6
6
@@ -51,9 +51,9 @@ lahman$People |>
51
51
::: {.callout-note collapse="true"}
52
52
## The dm package
53
53
54
-
In this chapter we will be creating a bespoke data model for our database. This approach can be further extended using the `dm`, which also provides various helpful functions for creating a data model and working with it.
54
+
In this chapter we will be creating a bespoke data model for our database. This approach can be further extended using the `dm` package, which also provides various helpful functions for creating a data model and working with it.
55
55
56
-
Similar to above, we can use the `dm` to create a single object to our database tables.
56
+
Similar to above, we can use `dm` to create a single object to access our database tables.
57
57
58
58
```{r, message=FALSE, warning=FALSE}
59
59
library(dm)
@@ -79,7 +79,7 @@ For more information on the dm package see <https://dm.cynkra.com/index.html>
79
79
80
80
## Creating functions for the data model
81
81
82
-
We can also now make various functions specific to our Lahman data model to facilitate data analyses. Given we know the structure of the data, we build a set of functions that abstract away some of the complexities of working with data in a database.
82
+
We can also now make various functions specific to our Lahman data model to facilitate data analyses. Given we know the structure of the data, we can build a set of functions that abstract away some of the complexities of working with data in a database.
83
83
84
84
Let's start by making a small function to get the teams players have played for. We can see that the code we use follows on from the last couple of chapters.
`collect()` brings data out of the database and into R. When working with large data sets, as is often the case when interacting with a database, we typically want to keep as much computation as possible on the database side. In the case of our `getTeams()` function, for example, it does everything on the database side and so collecting will just bring out the result of the teams the person played for. In this case we could also use `pull()` get our result out as a vector rather that a data frame.
140
+
The function `collect()` brings data out of the database and into R. When working with large datasets, as is often the case when interacting with a database, we typically want to keep as much computation as possible on the database side. In the case of our `getTeams()` function, for example, it does everything on the database side and so collecting will just bring out the result of the teams the person played for. In this case we could also use `pull()` to get our result out as a vector rather that a data frame.
141
141
142
142
```{r, message=FALSE, warning=FALSE}
143
143
getTeams(lahman, "Barry Bonds") |>
@@ -191,7 +191,7 @@ lahman$Pitching |>
191
191
```
192
192
:::
193
193
194
-
We could then use our `addBirthCountry()` function as part of a larger query to summarise the proportion of players from each country over the time (based on their presence in the batting table).
194
+
We could then use our `addBirthCountry()` function as part of a larger query to summarise the proportion of players from each country over time (based on their presence in the batting table).
As part of our `lahmanFromCon()` function our data model object has the class a "lahman_ref". Therefore as well as creating user-facing functions to work with our lahman data model, we can also define methods for the this object.
260
+
As part of our `lahmanFromCon()` function our data model object has the class "lahman_ref". Therefore as well as creating user-facing functions to work with our lahman data model, we can also define methods for this object.
261
261
262
262
```{r, message=FALSE, warning=FALSE}
263
263
class(lahman)
264
264
```
265
265
266
-
With this we can make some specific methods for "lahman_ref". For example we can define a print method.
266
+
With this we can make some specific methods for a "lahman_ref" object. For example, we can define a print method like so:
267
267
268
268
```{r, message=FALSE, warning=FALSE}
269
269
print.lahman_ref <- function(x, ...) {
@@ -312,7 +312,7 @@ lahman$Salaries |>
312
312
.by = c("birthCountry", "birthYear"))
313
313
```
314
314
315
-
Although the R code on the face of it looks fine, when we look at the SQL we can see that our query has two joins to the People table. One join gets birth country and the other birth year.
315
+
Although the R code on the face of it looks fine, when we look at the SQL we can see that our query has two joins to the People table. One join gets information on the birth country and the other on the birth year.
316
316
317
317
::: {.callout-note collapse="true"}
318
318
## Show query
@@ -327,7 +327,7 @@ lahman$Salaries |>
327
327
```
328
328
:::
329
329
330
-
To improve performance, we could instead have a single function to get both of these both birth country and birth year at the same time.
330
+
To improve performance, we could instead have a single function to get both of these, birth country and birth year, at the same time.
All this is to show that when working with databases we should keep in mind what is going on behind the scenes in terms of SQL actually being executed.
358
+
Now this query outputs the same result but is simpler than the previous one, thus lowering the computational cost of the analysis. All this is to show that when working with databases we should keep in mind what is going on behind the scenes in terms of the SQL code actually being executed.
359
359
360
360
### Piping and SQL
361
361
@@ -391,7 +391,7 @@ lahman$People |>
391
391
392
392
### Computing intermediate queries
393
393
394
-
Let's say we want to summarise home runs in the batting table and stike outs in the pitching table by college players attended and their birth year. We could do this like so:
394
+
Let's say we want to summarise home runs in the batting table and stike outs in the pitching table by the college players attended and their birth year. We could do this like so:
395
395
396
396
```{r, echo = TRUE}
397
397
players_with_college <- lahman$People |>
@@ -417,7 +417,7 @@ lahman$Pitching |>
417
417
collect()
418
418
```
419
419
420
-
Looking at the SQL we can see, however, that there is some duplication. As part of each full query we have run our players_with_college query.
420
+
Looking at the SQL we can see, however, that there is some duplication, because as part of each full query we have run our players_with_college query.
0 commit comments