From f688d0d7d0da6663f0fd2e1dd2f12572a18586fc Mon Sep 17 00:00:00 2001 From: Alexander Hentschel Date: Thu, 31 Oct 2024 15:16:05 -0700 Subject: [PATCH 01/16] added markdown [.md] extension to filenames that were previously missing it --- .../{20210916-network-layer-api => 20210916-network-layer-api.md} | 0 protocol/{20211129-sync-engine => 20211129-sync-engine.md} | 0 ...211609-component-interface => 20211609-component-interface.md} | 0 ....md => 20231003-service-account-specified-epoch-switchover.md} | 0 4 files changed, 0 insertions(+), 0 deletions(-) rename protocol/{20210916-network-layer-api => 20210916-network-layer-api.md} (100%) rename protocol/{20211129-sync-engine => 20211129-sync-engine.md} (100%) rename protocol/{20211609-component-interface => 20211609-component-interface.md} (100%) rename protocol/{20231003-service-account-specified-epoch-switchover.md.md => 20231003-service-account-specified-epoch-switchover.md} (100%) diff --git a/protocol/20210916-network-layer-api b/protocol/20210916-network-layer-api.md similarity index 100% rename from protocol/20210916-network-layer-api rename to protocol/20210916-network-layer-api.md diff --git a/protocol/20211129-sync-engine b/protocol/20211129-sync-engine.md similarity index 100% rename from protocol/20211129-sync-engine rename to protocol/20211129-sync-engine.md diff --git a/protocol/20211609-component-interface b/protocol/20211609-component-interface.md similarity index 100% rename from protocol/20211609-component-interface rename to protocol/20211609-component-interface.md diff --git a/protocol/20231003-service-account-specified-epoch-switchover.md.md b/protocol/20231003-service-account-specified-epoch-switchover.md similarity index 100% rename from protocol/20231003-service-account-specified-epoch-switchover.md.md rename to protocol/20231003-service-account-specified-epoch-switchover.md From c609359b889eef2765873d500de413f8f883d8cb Mon Sep 17 00:00:00 2001 From: Alexander Hentschel Date: Thu, 31 Oct 2024 15:19:15 -0700 Subject: [PATCH 02/16] initial draft of version beacon flip --- ...1215-accessnode-streaming-api-expansion.md | 2 + .../20241031-execution-stack-versioning.md | 172 ++++++++++++++++++ .../Execution_Stack_Versioning_(2).png | Bin 0 -> 908140 bytes .../Execution_Stack_Versioning_(3).png | Bin 0 -> 964876 bytes .../Execution_Stack_Versioning_(5).png | Bin 0 -> 905211 bytes .../Execution_Stack_Versioning_goal.png | Bin 0 -> 339202 bytes .../VersioningConcepts.png | Bin 0 -> 342823 bytes 7 files changed, 174 insertions(+) create mode 100644 protocol/20241031-execution-stack-versioning.md create mode 100644 protocol/20241031-execution-stack-versioning/Execution_Stack_Versioning_(2).png create mode 100644 protocol/20241031-execution-stack-versioning/Execution_Stack_Versioning_(3).png create mode 100644 protocol/20241031-execution-stack-versioning/Execution_Stack_Versioning_(5).png create mode 100644 protocol/20241031-execution-stack-versioning/Execution_Stack_Versioning_goal.png create mode 100644 protocol/20241031-execution-stack-versioning/VersioningConcepts.png diff --git a/protocol/20231215-accessnode-streaming-api-expansion.md b/protocol/20231215-accessnode-streaming-api-expansion.md index 473c93461..7126049bc 100644 --- a/protocol/20231215-accessnode-streaming-api-expansion.md +++ b/protocol/20231215-accessnode-streaming-api-expansion.md @@ -6,6 +6,8 @@ sponsor: Jerome Pimmel (jerome.pimmel@flowfoundation.org), Peter Argue (peter.ar updated: 2023-12-15 --- + + # [FLIP 229] Access Streaming API Expansion ## Objective diff --git a/protocol/20241031-execution-stack-versioning.md b/protocol/20241031-execution-stack-versioning.md new file mode 100644 index 000000000..3f476514b --- /dev/null +++ b/protocol/20241031-execution-stack-versioning.md @@ -0,0 +1,172 @@ +--- +status: draft +flip: ??? +authors: Alex Hentschel (alex.hentschel@flowfoundation.org) +sponsor: Jordan Schalm (jordan.schalm@flowfoundation.org), Yurii Oleksyshyn (yurii.oleksyshyn@flowfoundation.org) +updated: 2024-10-31 +--- + + +# [FLIP ???] Utilize Dynamic Protocol State for Version Beacon (coordinating upgrades of the Execution Stack) + +## Objective +![Overview](20241031-execution-stack-versioning/Execution_Stack_Versioning_goal.png) + +- Versioning the **Execution Stack** used by ENs, VNs, ANs + + Outlook: ANs can decide on whose blocks’ execution states they can run scripts for across Execution HCUs + + +## Terminology + +See blog post [[1](https://forum.flow.com/t/protocol-version-upgrade-mechanisms-discussion/5717)] for further details + +**Software Version** - The version identifier of a binary distribution of Flow Node software. +By convention, we use semver-ish tag in Git and Docker releases. + +Software has bugs. and is frequently incomplete (e.g. API returning ‘not yet implemented’). +The software version is a meaningful reference to describe what the software does in the real world. + +However, we also desire a compact identifier [which we will call the ‘Component Version’] of how a Flow node *should* behave. + +![VersioningConcepts.png](20241031-execution-stack-versioning/VersioningConcepts.png) + +**Component Version:** version identifier for a component of the flow protocol. It references one specific behaviour of a sub-system (e.g. Execution Stack or HotStuff) of Flow, as prescribed by the protocol. + +In the nutshell, for every block there is one and only one correct way of how to process that block, and how to evolve the execution state. For distributed BFT systems, we need this notion of ‘correct behaviour’, which is inherently implementation agnostic. We want to explicitly express that up to a certain view $v$, we want the protocol to behave in one way and for higher views differently. + +### Relationships between **Software and Component Version** + +- Conceptually, for every block, each component of Flow has one and only one component version. + +- A software version can implement multiple Component Versions. +E.g. AN supporting script execution across HCU boundaries + + ❗Don’t couple the software version to the component version! + + +### Reasons we want to move away from existing Version Beacon: + +Current Version Beacon: + +1. requires that nodes have (potentially long) history (have seen version beacon service event, which is not guaranteed for nodes joining at epoch boundaries) + + Better: each block specifies which component version is to be used for processing it + +2. based on height and hence not usable for upgrading most consensus-related aspects (any many other protocol aspects). + + Better: using View instead of height for triggering behaviour changes is generally applicable and more robust (view monotonously increases over time, while height might also decrease). + + +## Dynamic Protocol State already implements better mechanism + +💡 In a nutshell, the Protocol State tracks information about each block, including a mechanism to transfer information from the Execution state to the Protocol State in a BFT manner. + +- Flow’s Protocol State to tracks and and provides simple access to information about each blocks (such as epoch number, staking phase, staked nodes allowed to participate as of this block, nodes public keys, etc) 👉 [code](https://github.com/onflow/flow-go/blob/3496c0f02d51602994d4fe60b32fcb00aab084f4/state/protocol/protocol_state.go#L91) +- The Protocol State now also tracks the Component Versions of the most critical consensus component (at the moment: its own version) 👉 [code](https://github.com/onflow/flow-go/blob/3496c0f02d51602994d4fe60b32fcb00aab084f4/state/protocol/protocol_state.go#L100) + +☑️ The Protocol State already tracks its own Component Version. You can take a look at these places in the code + - Protocol State reports its own [version](https://github.com/onflow/flow-go/blob/3496c0f02d51602994d4fe60b32fcb00aab084f4/state/protocol/kvstore.go#L30-L43) as part of every block + - mechanism for [scheduling version upgrades (at future view)](https://github.com/onflow/flow-go/blob/a6b157ce2770be9356e1cf35d1b0fff63f5e4a76/state/protocol/protocol_state/kvstore/upgrade_statemachine.go#L78-L142) exists + - mechanism [enforcing that node supports and uses correct](https://github.com/onflow/flow-go/blob/a6b157ce2770be9356e1cf35d1b0fff63f5e4a76/state/protocol/protocol_state/state/protocol_state.go#L235-L248) version as specified by the protocol + +