Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: replace dependency on Block with BlockHeader and Chunks on apply chunk #12746

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

Longarithm
Copy link
Member

@Longarithm Longarithm commented Jan 16, 2025

Context

We want to improve chunk processing efficiency by applying chunks optimistically (#10584), when all partial chunks for the next height and block metadata (from an OptimisticBlock) are already available. This allows the results of chunk application to be reused when the actual block is received, enabling the next chunk to be produced immediately.

Currently, this work is a step towards supporting OptimisticBlock. While OptimisticBlock is not introduced yet, this refactor prepares the codebase for its implementation by reducing data dependency on the current block.

Change

I replaced the dependency on Block with block_metadata: &BlockHeader and chunk_headers: &Chunks. The block_metadata currently is just a block header information and later must be aligned with OptimisticBlock. Chunk headers are just taken from block, and for optimistic block, they must be supplied by ShardsManager.

Some APIs are refactored to reflect that change.

Next steps

  • block_metadata must become the OptimisticBlock, or BlockMetadata, etc - we need to think about it.
  • Then, we call get_update_shard_job for OptimisticBlock and reuse result if it is called for the actual Block.

@Longarithm Longarithm marked this pull request as ready for review January 16, 2025 10:31
@Longarithm Longarithm requested a review from a team as a code owner January 16, 2025 10:31
@Longarithm Longarithm requested a review from miloserdow January 16, 2025 10:36
Copy link

codecov bot commented Jan 16, 2025

Codecov Report

Attention: Patch coverage is 87.71930% with 14 lines in your changes missing coverage. Please review.

Project coverage is 70.73%. Comparing base (8806e63) to head (b21eb2c).

Files with missing lines Patch % Lines
chain/chain/src/chain.rs 83.13% 4 Missing and 10 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master   #12746      +/-   ##
==========================================
- Coverage   70.74%   70.73%   -0.01%     
==========================================
  Files         848      848              
  Lines      174301   174335      +34     
  Branches   174301   174335      +34     
==========================================
+ Hits       123303   123312       +9     
- Misses      45856    45871      +15     
- Partials     5142     5152      +10     
Flag Coverage Δ
backward-compatibility 0.16% <0.00%> (-0.01%) ⬇️
db-migration 0.16% <0.00%> (-0.01%) ⬇️
genesis-check 1.35% <0.00%> (-0.01%) ⬇️
linux 69.23% <85.08%> (-0.01%) ⬇️
linux-nightly 70.30% <86.84%> (-0.02%) ⬇️
pytests 1.65% <0.00%> (-0.01%) ⬇️
sanity-checks 1.46% <0.00%> (-0.01%) ⬇️
unittests 70.56% <87.71%> (-0.01%) ⬇️
upgradability 0.20% <0.00%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Contributor

@VanBarbascu VanBarbascu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Kudos!
This is a huge step in plugging optimistic block in the chunk processing flow!

// Before `FixApplyChunks` feature, gas price was taken from current
// block by mistake. Preserve it for backwards compatibility.
let gas_price = if !is_new_chunk
&& protocol_version < ProtocolFeature::FixApplyChunks.protocol_version()
let gas_price = if protocol_version >= ProtocolFeature::FixApplyChunks.protocol_version()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
let gas_price = if protocol_version >= ProtocolFeature::FixApplyChunks.protocol_version()
let gas_price = if ProtocolFeature::FixApplyChunks.enabled(protocol_version)

@@ -3722,30 +3742,38 @@ impl Chain {
fn get_update_shard_job(
&self,
me: &Option<AccountId>,
block: &Block,
// TODO(#10584): introduce separate structure which can be derived from
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can be an enum that either has Block or OptimisticBlock.

let epoch_id = self.epoch_manager.get_epoch_id_from_prev_block(prev_hash)?;
let shard_layout = self.epoch_manager.get_shard_layout(&epoch_id)?;
let shard_id = shard_layout.get_shard_id(shard_index)?;
let chunk_header = &chunk_headers[shard_index];
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In Chunks struct, they say that Index trait is deprecated and recommends using .get() instead. Just to align with the new resharding implementattion.

Suggested change
let chunk_header = &chunk_headers[shard_index];
let chunk_header = &chunk_headers.get(shard_index);

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants