Skip to content

Commit

Permalink
Bump version and add the copy materilized pipe
Browse files Browse the repository at this point in the history
  • Loading branch information
Alberto Juan committed Jan 8, 2024
1 parent 6077fd0 commit 26023a0
Show file tree
Hide file tree
Showing 3 changed files with 41 additions and 4 deletions.
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
TB_VERSION_WARNING=0
VERSION=0.0.0
VERSION=0.0.1
36 changes: 33 additions & 3 deletions v3/delete_column_from_landing_data_source_releases/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,10 @@

> Remember to follow the [instructions](../README.md) to setup your Tinybird Data Project before jumping into the use-case steps
- Just delete the desired column from the Data Source

[Pull Request](https://github.com/tinybirdco/use-case-examples/pull/162/files)

- Delete the desired column from the Data Source
`datasources/analytics_events.datasource`
```diff
SCHEMA >
`timestamp` DateTime `json:$.timestamp`,
Expand All @@ -14,4 +14,34 @@ SCHEMA >
`version` LowCardinality(String) `json:$.version`,
- `environment` Nullable(String) `json:$.environment`,
`payload` String `json:$.payload`
```
```
- Add a tag to the same Data Source file to specify the field to use to auto-populate the data
`datasources/analytics_events.datasource`
```diff
ENGINE "MergeTree"
ENGINE_PARTITION_KEY "toYYYYMM(timestamp)"
ENGINE_SORTING_KEY "timestamp"
ENGINE_TTL "timestamp + toIntervalDay(60)"

+ TAGS "backfill_column=timestamp"
```

- Bump the Version to re-create the Data Source
`.tinyenv`
```diff
- VERSION=0.0.0
+ VERSION=0.0.1
```

- Add a new Materialized Pipe to define how to move the data (copy all the fields except the field being removed)
`datasources/live_to_new.datasource`
```sql
NODE live_to_new
SQL >
SELECT timestamp, session_id, action, version, payload
FROM live.analytics_events
TYPE MATERIALIZED
DATASOURCE analytics_events
```


Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
NODE live_to_new
SQL >
SELECT timestamp, session_id, action, version, payload
FROM live.analytics_events

TYPE MATERIALIZED
DATASOURCE analytics_events

0 comments on commit 26023a0

Please sign in to comment.