|
| 1 | +--- |
| 2 | +title: "Maps" |
| 3 | +--- |
| 4 | + |
| 5 | +# Overview |
| 6 | + |
| 7 | +There are multiple different implementations of key-value maps inside the framework, suited for different usecases. |
| 8 | +We will go over their differences and similarities, and how to choose which one to use. |
| 9 | + |
| 10 | +## Aptos Blockchain performance and gas cost considerations |
| 11 | + |
| 12 | +State on the Aptos Blockchain is managed as a set of resources. Transactions |
| 13 | +performance heavily depends on how reads and writes to resources. |
| 14 | +Storage gas costs are paid based on number of resources that exist, and their sizes. |
| 15 | +IO gas costs are paid based on number of resources read and modified, and their sizes, |
| 16 | +but are generally significantly smaller than storage gas costs. |
| 17 | +That means that writing to a new resource has the highest (storage) gas cost, and deleting |
| 18 | +an existing resource gives the largest refund. |
| 19 | +Additionally, transactions modifying the same resource conflict with one another, and cannot be |
| 20 | +executed in parallel. |
| 21 | + |
| 22 | +One useful analogy is thinking about each resource being a file on a disk, |
| 23 | +then performance of smart contract would correlate well to a program that |
| 24 | +operates on files in the same way. |
| 25 | + |
| 26 | +## Different Map implementations |
| 27 | + |
| 28 | +- `OrderedMap` is a struct, and is, similar to `vector`, fully contained within the resource that stores it. |
| 29 | + With it, it is bounded in size to the size of a single resource. |
| 30 | + It provides regular map functions, as well as accessing elements in order, like front/back or prev/next. |
| 31 | + When you need an inline mapping, that will fit in a resource, this is the option to choose. |
| 32 | + It's implementation is SortedVectorMap, but because of limited size and efficiency of memcpy, all main operations are practically O(log(n)). |
| 33 | +- `Table` is unbounded in size, puts each (key, value) pair in the separate resource. You can `add` or `remove` elements, |
| 34 | + or check if it `contains` some key, but cannot be iterated on. When keys or values are large / unbounded, we can use the `Table`. |
| 35 | + Also if we want to parallelize transactions and we have a few elements that are modified extremely often, `Table` can provide that. |
| 36 | + Note that `Table` cannot be destroyed, because it doesn't know if it is empty. |
| 37 | + - `TableWithLength` is wrapper around the `Table`, that adds tracking of it's `length`, allowing `length`, `empty` and `destroy_empty` |
| 38 | + operations on top of the `Table`. Adding or removing elements to `TableWithLength` cannot be done in parallel. |
| 39 | +- `BigOrderedMap` groups multiple (key, value) pairs in a single resource, but is unbounded in size - and uses more resources as needed. |
| 40 | + It's implementation is a BPlusTreeMap, where each node is a resource containing OrderedMap, with inner nodes only containing keys, while leaves contain values as well. |
| 41 | + It is opportunistically parallel - if map has large enough elements to be using multiple resources, modifying the map for keys that are not close |
| 42 | + to each other should generally be parallel operation. |
| 43 | + It is configured so that each resource containing internal node has the same capacity in number of keys, |
| 44 | + and each resource containing leaf node has the same capacity in the number of (key, value) pairs. |
| 45 | + Capacity of nodes (both leaf and inner degree) are configurable - to allow the tradeoff between storage gas cost on one end, |
| 46 | + and other gas costs and parallelism on the other. |
| 47 | + It provides regular map functions, as well as accessing elements in order, like front/back or prev/next. |
| 48 | + |
| 49 | +Note: |
| 50 | +- `SimpleMap` has been deprecated, and replaced with `OrderedMap`. |
| 51 | +- `SmartTable` has been deprecated, and replaced with `BigOrderedMap`. |
| 52 | + |
| 53 | +## Common map operations: |
| 54 | + |
| 55 | +Most maps above support the same set of functions (for actual signatures and restrictions, check out the corresponding implementations): |
| 56 | + |
| 57 | +#### Creating Tables |
| 58 | + |
| 59 | +- `new<K, V>(): Self`: creates an empty map |
| 60 | + |
| 61 | +#### Destroying Tables |
| 62 | + |
| 63 | +All except `Table` support: |
| 64 | +- `destroy_empty<K, V>(table: Self<K, V>)`: Destroys an empty map. (not supported by `Table`) |
| 65 | +- `destroy<K, V>(self: Self<K, V>, dk: |K|, dv: |V|)`: Destroys a map with given functions that destroy correponding elements. (not supported by `Table` and `TableWithLength`) |
| 66 | + |
| 67 | +#### Managing Entries |
| 68 | + |
| 69 | +- `add<K, V>(table: &mut Self<K, V>, key: K, value: V)`: Adds a key-value pair to the map. |
| 70 | +- `remove<K, V>(table: &mut Self<K, V>, key: K): V`: Removes and returns the value associated with a key. |
| 71 | +- `upsert<K, V>(table: &mut Self<K, V>, key: K, value: V): Option<V>`: Inserts or updates a key-value pair. |
| 72 | +- `add_all<K, V>(table: &mut Self<K, V>, keys: vector<K>, values: vector<V>)`: Adds multiple key-value pairs to the map. (not supported by `Table` and `TableWithLength`) |
| 73 | + |
| 74 | +#### Retrieving Entries |
| 75 | + |
| 76 | +- `contains<K, V>(self: &Self<K, V>, key: &K): bool`: Checks whether key exists in the map. |
| 77 | +- `borrow<K, V>(table: &Self<K, V>, key: &K): &V`: Returns an immutable reference to the value associated with a key. |
| 78 | +- `borrow_mut<K: drop, V>(table: &mut Self<K, V>, key: K): &mut V`: Returns a mutable reference to the value associated with a key. |
| 79 | + (`BigOrderedMap` only allows `borrow_mut` when value type has a static constant size, due to modification being able to break it's invariants otherwise. Use `remove()` and `add()` combination instead) |
| 80 | + |
| 81 | +#### Order-dependant functions |
| 82 | + |
| 83 | +These set of functions are only implemented by `OrderedMap` and `BigOrderedMap`. |
| 84 | + |
| 85 | +- `borrow_front<K, V>(self: &Self<K, V>): (&K, &V)` |
| 86 | +- `borrow_back<K, V>(self: &Self<K, V>): (&K, &V)` |
| 87 | +- `pop_front<K, V>(self: &mut Self<K, V>): (K, V)` |
| 88 | +- `pop_back<K, V>(self: &mut Self<K, V>): (K, V)` |
| 89 | +- `prev_key<K: copy, V>(self: &Self<K, V>, key: &K): Option<K>` |
| 90 | +- `next_key<K: copy, V>(self: &Self<K, V>, key: &K): Option<K>` |
| 91 | + |
| 92 | +#### Utility Functions |
| 93 | + |
| 94 | +- `length<K, V>(table: &Self<K, V>): u64`: Returns the number of entries in the table. (not supported by `Table`) |
| 95 | + |
| 96 | +#### Traversal Functions |
| 97 | + |
| 98 | +These set of functions are not implemented by `Table` and `TableWithLength`. |
| 99 | + |
| 100 | +- `keys<K: copy, V>(self: &Self<K, V>): vector<K>` |
| 101 | +- `values<K, V: copy>(self: &Self<K, V>): vector<V>` |
| 102 | +- `to_vec_pair<K, V>(self: Self<K, V>): (vector<K>, vector<V>)` |
| 103 | +- `for_each_ref<K, V>(self: &Self<K, V>, f: |&K, &V|)` |
| 104 | + |
| 105 | +- `to_ordered_map<K, V>(self: &BigOrderedMap<K, V>): OrderedMap<K, V>`: Converts `BigOrderedMap` into `OrderedMap` |
| 106 | + |
| 107 | +## Example Usage |
| 108 | + |
| 109 | +### Creating and Using a OrderedMap |
| 110 | + |
| 111 | +```move filename="map_usage.move" |
| 112 | +module 0x42::map_usage { |
| 113 | + use aptos_framework::ordered_map; |
| 114 | +
|
| 115 | + public entry fun main() { |
| 116 | + let map = ordeded_map::new<u64, u64>(); |
| 117 | + map.add(1, 100); |
| 118 | + map.add(2, 200); |
| 119 | +
|
| 120 | + let length = map.length(); |
| 121 | + assert!(length == 2, 0); |
| 122 | +
|
| 123 | + let value1 = map.borrow(&1); |
| 124 | + assert!(*value1 == 100, 0); |
| 125 | +
|
| 126 | + let value2 = map.borrow(&2); |
| 127 | + assert!(*value2 == 200, 0); |
| 128 | +
|
| 129 | + let removed_value = map.remove(&1); |
| 130 | + assert!(removed_value == 100, 0); |
| 131 | +
|
| 132 | + map.destroy_empty(); |
| 133 | + } |
| 134 | +} |
| 135 | +``` |
| 136 | + |
| 137 | +## Additional details for BigOrderedMap |
| 138 | + |
| 139 | +It’s current implementation is BPlusTreeMap, which is chosen as it is best suited for the onchain storage layout - where majority |
| 140 | +of cost comes from loading and writing to storage items, and there is no partial read/write of them. It also rebalances in a way that reduces amount of writes needed. |
| 141 | +- It keeps root node directly inside the struct. This allows performance optimization when map has few elements, and only grows to multiple resources when needed. |
| 142 | +- It allows for parallelism - writing to keys to a large enough map, that are not close enough is generally parallel. More elements are in the map, the more often will operations be parallel. |
| 143 | +- All operations have predictable and reasonable upper-bound on performance (how long they take, or how much execution and io gas they consume), allowing for safe usage across variety of usecases. |
| 144 | + - Each key/value has the same size restrictions, and map maintains it’s resources to never exceed the resource limits, and so one insert cannot affect whether a future inserts will succeed or fail. |
| 145 | +- By default, operation gets hit with storage gas charging when it needs to grow. But, it has an option to pre-pay and pre-allocate storage slots, and to reuse them as elements are added/removed, allowing applications to achieving predictable storage gas charges. (and with that overall gas charges) |
| 146 | + |
| 147 | +Because it's layout affects what can be inserted and performance, there are a few ways to create and configure it: |
| 148 | + |
| 149 | +- `new<K, V>(): Self<K, V>`: Returns a new `BigOrderedMap` with the default configuration. Only allowed to be called with constant size types. For variable sized types, another constructor is needed, to explicitly select automatic or specific degree selection. |
| 150 | +- `new_with_type_size_hints<K, V>(avg_key_bytes: u64, max_key_bytes: u64, avg_value_bytes: u64, max_value_bytes: u64): Self<K, V>`: Returns a map that is configured to perform best when keys and values are of given `avg` sizes, and guarantees to fit elements up to given `max` sizes. |
| 151 | +- `new_with_config<K, V>(inner_max_degree: u16, leaf_max_degree: u16, reuse_slots: bool): Self<K, V>`: Returns a new `BigOrderedMap` with the provided max degree consts (the maximum # of children a node can have, both inner and leaf). If 0 is passed for either, then it is dynamically computed based on size of first key and value, and keys and values up to 100x times larger will be accepted. |
| 152 | + If non-0 is passed, sizes of all elements must respect (or their additions will be rejected): |
| 153 | + - `key_size * inner_max_degree <= MAX_NODE_BYTES` |
| 154 | + - `entry_size * leaf_max_degree <= MAX_NODE_BYTES` |
| 155 | + |
| 156 | + `reuse_slots` means that removing elements from the map doesn't free the storage slots and returns the refund. |
| 157 | + Together with `allocate_spare_slots`, it allows to preallocate slots and have inserts have predictable gas costs. |
| 158 | + (otherwise, inserts that require map to add new nodes, cost significantly more, compared to the rest) |
| 159 | + |
| 160 | +## Source Code |
| 161 | + |
| 162 | +- [ordered_map.move](https://github.com/aptos-labs/aptos-core/blob/main/aptos-move/framework/aptos-framework/sources/datastructures/ordered_map.move) |
| 163 | +- [table.move](https://github.com/aptos-labs/aptos-core/blob/6f5872b567075fe3615e1363d35f89dc5eb45b0d/aptos-move/framework/aptos-stdlib/sources/table.move) |
| 164 | +- [table_with_length.move](https://github.com/aptos-labs/aptos-core/blob/6f5872b567075fe3615e1363d35f89dc5eb45b0d/aptos-move/framework/aptos-stdlib/sources/table.move) |
| 165 | +- [big_ordered_map.move](https://github.com/aptos-labs/aptos-core/blob/main/aptos-move/framework/aptos-framework/sources/datastructures/big_ordered_map.move) |
0 commit comments