Skip to content

Commit

Permalink
deploy: 37fb6fa
Browse files Browse the repository at this point in the history
  • Loading branch information
@romaninsh
romaninsh committed Nov 27, 2024
1 parent 8d50df9 commit d6d8a3d
Show file tree
Hide file tree
Showing 11 changed files with 1,069 additions and 279 deletions.
174 changes: 130 additions & 44 deletions 1-dataset.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -153,21 +153,61 @@ <h1 class="menu-title">DORM - The Dry ORM</h1>
<div id="content" class="content">
<main>
<h1 id="data-sets"><a class="header" href="#data-sets">Data Sets</a></h1>
<p>Most of Rust apps operate with data which is readily available in memory. Business apps
are stateless, loading too much data is an anti-pattern.</p>
<p>Depending of the capabilities of the underlying data storage layer, you should look to
create as many requests and retrieve as little data as possible.</p>
<p><code>DataSet</code> is is a representation of collection of records - stored remotely.</p>
<p>Traditional ORMs operate with records. If you have used SeaORM or Diesel - you need
to temporarily "un-learn" those. DORM syntax may look similar, but it's very different to
the classic ORM pattern:</p>
<ol>
<li>
<p><strong>DORM operates with Data Sets</strong>. A set can contain either a single record, no records or
a huge number of records. Set represents records in remote database (or API) and does
not store anything in memory.</p>
</li>
<li>
<p><strong>DORM executes operations remotely</strong>. Changing multiple records in ORM involves
fetching all the records, modifying them and saving them back. DORM prefers to
execute changes remotely, if underlying DataSource supports it.</p>
</li>
</ol>
<p>As a developer, you will always know when DORM interacts with the database, because
those operations are async. Majority of DORM operations - like traversing relationship,
calculating sub-queries - those are sync.</p>
<h2 id="sqltable-and-sqlquery"><a class="header" href="#sqltable-and-sqlquery"><code>sql::Table</code> and <code>sql::Query</code></a></h2>
<p>DORM implements sql::Table and sql::Query - because we all love SQL. However you can
define other data sources - such as NoSQL, REST API, GraphQL, etc. Those extensions
do not need to be in same crate as DORM. For the rest of this book I will only
focus on SQL as a predominant use-case.</p>
<p>DORM is quite fluid in the way how you use <code>table</code> and <code>query</code>, you can use one to
compliment or create another, but they serve different purpose:</p>
<ol>
<li><code>sql::Table</code> has a structure - fields, joins and relations are defined dynamically.</li>
<li><code>sql::Query</code> on other hand is transient. It consists of smaller pieces which we call <code>sql::Chunk</code>.</li>
<li><code>sql::Table</code> can create and return <code>sql::Query</code> object from various methods</li>
</ol>
<p><code>sql::Chunk</code> trait that is implemented by <code>sql::Query</code> (or other arbitrary expressions)
acts as a <strong>glue</strong> betwene tables. For instance when you run traverse relations:</p>
<pre><pre class="playground"><code class="language-rust"><span class="boring">#![allow(unused)]
</span><span class="boring">fn main() {
</span>let client = Client::table().with_id(1);

let order_lines = client.ref_orders().ref_order_lines();
<span class="boring">}</span></code></pre></pre>
<ol>
<li>DORM executes <code>field query</code> operation on a client set for field <code>id</code>.</li>
<li>DORM creates <code>orders</code> table and adds <code>condition</code> on <code>client_id</code> field.</li>
<li>DORM executes <code>field query</code> operation on <code>orders</code> table for field <code>id</code>.</li>
<li>DORM creates <code>order_lines</code> table and adds <code>condition</code> on <code>order_id</code> field.</li>
</ol>
<p>DORM weaves between <code>sql::Table</code> and <code>sql::Query</code> without reaching out to the
database behind a simple and intuitive code.</p>
<h2 id="readabledataset-and-writabledataset"><a class="header" href="#readabledataset-and-writabledataset">ReadableDataSet and WritableDataSet</a></h2>
<p>DORM provides two traits: <code>ReadableDataSet</code> and <code>WritableDataSet</code>. DORM also provides
several implementations of these traits:</p>
<p>DORM provides two traits: <code>ReadableDataSet</code> and <code>WritableDataSet</code>. As name
suggests - you can fetch records from Readable set. You can add, modify or delete
records in Writable set.</p>
<p>DORM implements those traits:</p>
<ul>
<li><code>sql::Table</code> - implements both <code>ReadableDataSet</code> and <code>WritableDataSet</code>.</li>
<li><code>sql::Query</code> - implements <code>ReadableDataSet</code> only.</li>
</ul>
<p>Design of DORM allow you to extend those into NoSQL (like MongoDB or GraphQL sources)
as well as custom RestAPI sources. Those extensions do not need to be part of DORM,
they can be implemented as separate crates.</p>
<h2 id="operating-with-data-sets"><a class="header" href="#operating-with-data-sets">Operating with Data sets</a></h2>
<p>At the very basic level - you can iterate through a readable data set.</p>
<pre><pre class="playground"><code class="language-rust"><span class="boring">#![allow(unused)]
Expand All @@ -178,43 +218,89 @@ <h2 id="operating-with-data-sets"><a class="header" href="#operating-with-data-s
println!("{}", client.name);
}
<span class="boring">}</span></code></pre></pre>
<p>But quite often the advanced persistence layer could allow us to do much more. DORM approach is
to provide ways how one Data Set can yield a different Data Set. Here are some examples:</p>
<p>There are more ways to fetch data from <code>ReadableDataSet</code>:</p>
<ul>
<li><code>get</code> - returns all records in a Vec using default entity type</li>
<li><code>get_as</code> - return all records using a custom type</li>
<li><code>get_all_untyped</code> - return all records as a raw JSON object</li>
<li><code>get_some</code> and <code>get-some_as</code> - return only one record (or none)</li>
<li><code>get_row_untyped</code> - return single record as a raw JSON object</li>
<li><code>get_col_untyped</code> - return only a single column as a raw JSON values</li>
<li><code>get_one_untyped</code> - return first column of a first row as a raw JSON value</li>
</ul>
<p>In most cases you would use <code>get</code> and <code>get_some</code>:</p>
<pre><pre class="playground"><code class="language-rust"><span class="boring">#![allow(unused)]
</span><span class="boring">fn main() {
</span>let client = Client::table().with_id(1);

let Some(client_data) = client.get_some().await? else { // fetch single record
// handle error
};

for client_order in client.orders().get().await? { // fetch multiple records
println!("{}", client_order.id);
}
<span class="boring">}</span></code></pre></pre>
<h2 id="creating-queries-from-tables"><a class="header" href="#creating-queries-from-tables">Creating Queries from Tables</a></h2>
<p>Sometimes you do not want result, but would prefer a query object instead. This gives you
a chance to tweak a query or use it elsewhere.</p>
<p>DORM provides trait TableWithQueries that generates Query objects for you:</p>
<ul>
<li>Get a set containing subset of records - by filtering.</li>
<li>Converting <code>sql::table</code> into <code>sql::query</code> for further manipulation.</li>
<li>Execute operation over set, such as calculate sum of a field from all records, or create comma-separated list of values.</li>
<li>Modify or delete multiple records.</li>
<li><code>get_empty_query</code> - returns a query with conditions and joins, but no fields</li>
<li><code>get_select_query</code> - like <code>get_empty_query</code> but adds all physical fields</li>
<li><code>get_select_query_for_field_names</code> - Provided with a slice of field names and expressions, only includes those into a query.</li>
<li><code>get_select_query_for_field</code> - Provided a query for individual field or
expression, which you have to pass through an argument.</li>
<li><code>get_select_query_for_fields</code> - Provided a query for multiple fields</li>
</ul>
<p>DORM prefers to off-load operation execution to the persistence layer, but because this may increase
complexity, DORM also provides a way to abstract this complexity away.</p>
<h3 id="example---lazy-expression-fields"><a class="header" href="#example---lazy-expression-fields">Example - Lazy expression fields</a></h3>
<p>In our introduction example, we came across an aggregate field: <code>total</code>:</p>
<p>There are generally two things you can do with a query:</p>
<ol>
<li>Tweak and execute it</li>
<li>Use it as a <code>Chunk</code> elsewhere</li>
</ol>
<h3 id="tweaking-and-executing-a-query"><a class="header" href="#tweaking-and-executing-a-query">Tweaking and Executing a query</a></h3>
<pre><pre class="playground"><code class="language-rust"><span class="boring">#![allow(unused)]
</span><span class="boring">fn main() {
</span>table
.has_many("line_items", "order_id", || Box::new(LineItem::table()))
.with_expression("total", |t| {
let item = t.sub_line_items();
item.sum(item.total()).render_chunk()
})
</span>let vip_orders = Client::table().add_condition(Client::table().is_vip().eq(true)).ref_orders();

let query = vip_orders
.get_select_query_for_field_names(&amp;["id", "client_id", "client"]) // excluding `total` here
.with_column("total".to_string(), expr!("sum({})", vip_orders.total())) // add as aggregate
.with_column("order_count".to_string(), expr!("count(*)"))
.with_group_by(vip_orders.client_id());

let result = postgres().query_raw(&amp;query).await?;
<span class="boring">}</span></code></pre></pre>
<h3 id="using-a-query-as-a-chunk-elsewhere"><a class="header" href="#using-a-query-as-a-chunk-elsewhere">Using a query as a <code>Chunk</code> elsewhere</a></h3>
<pre><pre class="playground"><code class="language-rust"><span class="boring">#![allow(unused)]
</span><span class="boring">fn main() {
</span>// TODO - hypothetical example, not implemented in bakery_model

let product_123 = Product::table().with_code("PRD-123");
let john = Client::table().with_email("john@example.com");

let new_order = Order::table()
.insert(Order {
product: product_123,
client: john,
quantity: 1,
}).await?;
<span class="boring">}</span></code></pre></pre>
<pre><pre class="playground"><code class="language-rust"><span class="boring">#![allow(unused)]
</span><span class="boring">fn main() {
</span>// TODO: test
let john = Client::table().with_email("john@example.com");

let order = Order::table()
.with_condition(Order::table().client_id().in(john.get_select_query_for_field(john.id())))
<span class="boring">}</span></code></pre></pre>
<p>Lets examine more generally what is happening here. Use of <code>has_many</code> creates</p>
<p>This is a very simple example of a lazy expression field. It is a field that is calculated
by a closure. The closure is passed a reference to the table. The closure can then use
the table to create a new field.</p>
<p>The above example is equivalent to this SQL:</p>
<pre><code class="language-sql">SELECT id,
(SELECT SUM((SELECT price FROM product WHERE id = product_id) * quantity)
FROM order_line WHERE order_line.order_id = ord.id) AS total
FROM ord
</code></pre>
<p>In this example, we are using a sub-query to calculate the total. The sub-query is
created by calling <code>sub_line_items()</code> on the table. This method returns a new table
that is a subset of the original table. The sub-query is then used to create a new
field called <code>total</code> that is a sum of the price and quantity.</p>
<p>The implementation
of <code>sql::Table</code> however provides ability to create new Data Sets from existing ones.</p>
<p>Method <code>get_ref()</code> does exactly that, when you traverse relationships.</p>
<h2 id="conclusion"><a class="header" href="#conclusion">Conclusion</a></h2>
<p>DataSet is a powerful concept that sets aside DORM from the usual ORM pattern.
<code>sql::Table</code> and <code>sql::Query</code> are the building blocks you interract with most
often in DORM.</p>
<p>Understanding this would allow you to implement missing features (such as table grouping)
and eventually devleop extensions for your data model.</p>

</main>

Expand All @@ -224,7 +310,7 @@ <h3 id="example---lazy-expression-fields"><a class="header" href="#example---laz
<i class="fa fa-angle-left"></i>
</a>

<a rel="next prefetch" href="1-table-and-fields.html" class="mobile-nav-chapters next" title="Next chapter" aria-label="Next chapter" aria-keyshortcuts="Right">
<a rel="next prefetch" href="1a-table.html" class="mobile-nav-chapters next" title="Next chapter" aria-label="Next chapter" aria-keyshortcuts="Right">
<i class="fa fa-angle-right"></i>
</a>

Expand All @@ -238,7 +324,7 @@ <h3 id="example---lazy-expression-fields"><a class="header" href="#example---laz
<i class="fa fa-angle-left"></i>
</a>

<a rel="next prefetch" href="1-table-and-fields.html" class="nav-chapters next" title="Next chapter" aria-label="Next chapter" aria-keyshortcuts="Right">
<a rel="next prefetch" href="1a-table.html" class="nav-chapters next" title="Next chapter" aria-label="Next chapter" aria-keyshortcuts="Right">
<i class="fa fa-angle-right"></i>
</a>
</nav>
Expand Down
Loading

0 comments on commit d6d8a3d

Please sign in to comment.