Skip to content

Commit

Permalink
State Storage - State
Browse files Browse the repository at this point in the history
  • Loading branch information
staffik committed Nov 5, 2024
1 parent 1ee5a74 commit c832bc4
Showing 1 changed file with 29 additions and 0 deletions.
29 changes: 29 additions & 0 deletions neps/nep-0568.md
Original file line number Diff line number Diff line change
Expand Up @@ -108,6 +108,35 @@ Splitting a shard's Flat State is performed in multiple steps:

### State Storage - State

Each shard’s Trie is stored in the `State` column of the database, with keys prefixed by `ShardUId`, followed by a node's hash.
This structure uniquely identifies each shard’s data. To avoid copying all entries under a new `ShardUId` during resharding,
we use a mapping strategy that allows child shards to access ancestor shard data without directly creating new entries.

#### Mapping strategy and the database key structure

The `DBCol::ShardUIdMapping` column facilitates this approach by linking each child shard’s `ShardUId`
to the `ShardUId` of the closest ancestor shard that holds the relevant data.
Initially, this column is empty, as existing shards map to themselves.
During a resharding event, a mapping entry is added to `ShardUIdMapping`, pointing each child shard’s `ShardUId` to the appropriate ancestor’s `ShardUId`.
This column remains compact and is cached by RocksDB, making lookups efficient on each access.

This mapping logic is implemented within `TrieStoreAdapter` and `TrieStoreUpdateAdapter`, which are layers over the `Store` interface.
These adapters specifically handle interactions with the `State` column, applying the shard mapping logic during Trie-related database operations.
Although child shards are accessed with their own `ShardUId` at a high level,
these adapters check `ShardUIdMapping` to access data under the relevant ancestor’s `ShardUId` where necessary,
applying this behavior to both read and write operations.

#### Mapping retention and cleanup

Mappings in `ShardUIdMapping` persist as long as any descendant shard still references the ancestor shard’s data.
When no descendants rely on a particular ancestor shard, its mapping entry can be removed, allowing the ancestor’s data to be garbage collected.
If a full state sync is performed for a child shard (downloading and storing all its data independently),
the mapping to the ancestor shard can also be removed, as the child’s data is now directly stored under its own `ShardUId`.

This approach allows efficient management of shard state during resharding events,
enabling seamless transitions without altering storage structures directly.


### Stateless Validation

### State Sync
Expand Down

0 comments on commit c832bc4

Please sign in to comment.