From 9fbdaafb7e08e24523f951e48b6b8985c4722ea0 Mon Sep 17 00:00:00 2001 From: Bowen Wang Date: Tue, 12 Mar 2024 21:43:07 -0700 Subject: [PATCH 1/6] Reduce the number of refund receipts --- neps/nep-9999.md | 129 +++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 129 insertions(+) create mode 100644 neps/nep-9999.md diff --git a/neps/nep-9999.md b/neps/nep-9999.md new file mode 100644 index 000000000..201227784 --- /dev/null +++ b/neps/nep-9999.md @@ -0,0 +1,129 @@ +--- +NEP: 0 +Title: Reduce the number of gas refunds +Authors: Evgeny Kuzyakov , Bowen Wang +Status: New +DiscussionsTo: https://github.com/nearprotocol/neps/pull/0000 +Type: Protocol +Version: 1.0.0 +Created: 2024-03-12 +LastUpdated: 2023-03-12 +--- + +[This is a NEP (NEAR Enhancement Proposal) template, as described in [NEP-0001](https://github.com/near/NEPs/blob/master/neps/nep-0001.md). Use this when creating a new NEP. The author should delete or replace all the comments or commented brackets when merging their NEP.] + + + +## Summary + +[Gas refund](https://docs.near.org/concepts/basics/transactions/gas#attach-extra-gas-get-refunded) is a mechanism that allows users to get refunded for gas that is not used during the execution of a smart contract. Due to [pessimistic gas pricing](https://docs.near.org/concepts/basics/transactions/gas-advanced#pessimistic-gas-price-inflation), however, even transactions that do not involve function calls generate refunds because users need to pay at a high price and get refunded the difference. Gas refunds lead to nontrivial overhead in runtime and other places, which hurts the performance of the protocol. This proposal aims to reduce the number of gas refunds and prepare for future changes that completely remove gas refunds. + +## Motivation + +Refund receipts create nontrivial overhead: they need to be merklized and sent across shards. In addition, the processing of refund receipts requires additional storage reads and writes, which is not optimal for the performance of the protocol. In addition, when there is congestion, refund receipts may be delayed during execution. Whenever this happens, it requires two additional storage writes to store a gas refund receipt and two additional reads and writes when they are later processed, which incurs a significant overhead. To optimize the performance of the protocol under congestion, it is imperative that we reduce the number of refund receipts. + +## Specification + +Pessimistic gas pricing is removed as a part of this change. This means that transactions that do not involve function calls will not generate gas refund receipts as a result. For function calls, this proposal changes the gas refund to be +``` +refund = max(0, gas_refund - 5Tgas); +``` +per receipt. In other words, for each gas refund, there is a 5Tgas penalty that is applied. The penalty is added to gas burnt. This means that for receipts that involve gas refund less than 5Tgas, there will be no gas refunds. + +## Reference Implementation + +The protocol changes are as follows: +* When a transaction is converted to a receipt, there is no longer a `pessmistic_gas_price` multiplier when the signer balance is deducted. Instead, the signer is charged `transaction_gas_cost * gas_price`. There is no gas refund for actions other than function calls. +* For function calls, if X gas is attached during the execution of a receipt and Y gas is burnt, then max(0, X-Y-5Tgas) is refunded at the original gas price. + +## Security Implications + +This change may lead to less correct charging for transactions when there is congestion. However, the entire gas price mechanism needs to be rethought any ways and when only one or two shards are congested, the gas price wouldn't change so there is no difference. + +## Alternatives + +One altnerative is to completely remove gas refunds by burning all prepaid gas. This idea was [discussed](https://github.com/near/NEPs/issues/107) before. However, it would be a very drastic change and may very negatively damage the developer experience. +The approach outlined in this proposal has less impact on developer and user experience and may be extended to burning all prepaid gas in the future. + +## Future possibilities + +Burning all prepaid gas is a natural extension to this proposal, which would completely get rid of gas refunds. This, however, would be a major change to the developer experience of NEAR and should be treated cautiously. +At the very least, developers should be able to easily estimate how much gas a function within a smart contract is going to consume during execution. + +## Consequences + +[This section describes the consequences, after applying the decision. All consequences should be summarized here, not just the "positive" ones. Record any concerns raised throughout the NEP discussion.] + +### Positive + +* p1 + +### Neutral + +* n1 + +### Negative + +* n1 + +### Backwards Compatibility + +This proposal changes how gas refund works today and as a result, users may need to pay slightly more when they send a function call transaction. However, the difference is quite small (0.0005N at most). + + +## Changelog + +[The changelog section provides historical context for how the NEP developed over time. Initial NEP submission should start with version 1.0.0, and all subsequent NEP extensions must follow [Semantic Versioning](https://semver.org/). Every version should have the benefits and concerns raised during the review. The author does not need to fill out this section for the initial draft. Instead, the assigned reviewers (Subject Matter Experts) should create the first version during the first technical review. After the final public call, the author should then finalize the last version of the decision context.] + +### 1.0.0 - Initial Version + +> Placeholder for the context about when and who approved this NEP version. + +#### Benefits + +> List of benefits filled by the Subject Matter Experts while reviewing this version: + +* Benefit 1 +* Benefit 2 + +#### Concerns + +> Template for Subject Matter Experts review for this version: +> Status: New | Ongoing | Resolved + +| # | Concern | Resolution | Status | +| --: | :------ | :--------- | -----: | +| 1 | | | | +| 2 | | | | + +## Copyright + +Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/). From bdedc83ddcd415c12d23e2d3e556e47072c90a69 Mon Sep 17 00:00:00 2001 From: Bowen Wang Date: Tue, 12 Mar 2024 21:45:15 -0700 Subject: [PATCH 2/6] update NEP --- neps/{nep-9999.md => nep-0536.md} | 36 ++----------------------------- 1 file changed, 2 insertions(+), 34 deletions(-) rename neps/{nep-9999.md => nep-0536.md} (74%) diff --git a/neps/nep-9999.md b/neps/nep-0536.md similarity index 74% rename from neps/nep-9999.md rename to neps/nep-0536.md index 201227784..bd53261d6 100644 --- a/neps/nep-9999.md +++ b/neps/nep-0536.md @@ -1,47 +1,15 @@ --- -NEP: 0 +NEP: 536 Title: Reduce the number of gas refunds Authors: Evgeny Kuzyakov , Bowen Wang Status: New -DiscussionsTo: https://github.com/nearprotocol/neps/pull/0000 +DiscussionsTo: https://github.com/near/NEPs/pull/536 Type: Protocol Version: 1.0.0 Created: 2024-03-12 LastUpdated: 2023-03-12 --- -[This is a NEP (NEAR Enhancement Proposal) template, as described in [NEP-0001](https://github.com/near/NEPs/blob/master/neps/nep-0001.md). Use this when creating a new NEP. The author should delete or replace all the comments or commented brackets when merging their NEP.] - - - ## Summary [Gas refund](https://docs.near.org/concepts/basics/transactions/gas#attach-extra-gas-get-refunded) is a mechanism that allows users to get refunded for gas that is not used during the execution of a smart contract. Due to [pessimistic gas pricing](https://docs.near.org/concepts/basics/transactions/gas-advanced#pessimistic-gas-price-inflation), however, even transactions that do not involve function calls generate refunds because users need to pay at a high price and get refunded the difference. Gas refunds lead to nontrivial overhead in runtime and other places, which hurts the performance of the protocol. This proposal aims to reduce the number of gas refunds and prepare for future changes that completely remove gas refunds. From 8b39ab5eeb5ad646a2a0b4c972e77e177b70bc70 Mon Sep 17 00:00:00 2001 From: Bowen Wang Date: Mon, 18 Mar 2024 12:36:45 -0700 Subject: [PATCH 3/6] update --- neps/nep-0536.md | 21 ++++++++++++++------- 1 file changed, 14 insertions(+), 7 deletions(-) diff --git a/neps/nep-0536.md b/neps/nep-0536.md index bd53261d6..3dcc73441 100644 --- a/neps/nep-0536.md +++ b/neps/nep-0536.md @@ -20,17 +20,25 @@ Refund receipts create nontrivial overhead: they need to be merklized and sent a ## Specification -Pessimistic gas pricing is removed as a part of this change. This means that transactions that do not involve function calls will not generate gas refund receipts as a result. For function calls, this proposal changes the gas refund to be +Pessimistic gas pricing is removed as a part of this change. This means that transactions that do not involve function calls will not generate gas refund receipts as a result. For function calls, this proposal introduces cost of refund to be ``` -refund = max(0, gas_refund - 5Tgas); +REFUND_FIXED_COST = action_receipt_creation + action_transfer + action_add_function_call_key +refund_cost = max(REFUND_FIXED_COST, 0.05 * gas_refund); ``` -per receipt. In other words, for each gas refund, there is a 5Tgas penalty that is applied. The penalty is added to gas burnt. This means that for receipts that involve gas refund less than 5Tgas, there will be no gas refunds. +per receipt. The refund fixed cost includes consideration for implicit accounts (created on transfer) and refund for access key allowance, which requires an access key update. The design of refund cost is supposed to penalize developers from attaching too much gas +and creating unnecessary refunds. Some examples: +* If the contract wants to refund 280Tgas, burning 5% of it would be about 14Tgas, which is a significant cost and developers would be encouraged to optimize it on the frontend. +* If refund is 100Tgas, then 5% is 5Tgas, which is still significant and discourages developers from doing so. +* If the refund is <10Tgas (very common case for cross-contract call self-callbacks), the penalty should be just 500Ggas, which is less than the gas refund cost. So only the fixed refund cost will be charged from gas to spawn the gas refund receipt. No UX will be broken for legacy cross-contract call contracts, so long as frontend correctly estimates the required gas in worst case scenario. + ## Reference Implementation The protocol changes are as follows: * When a transaction is converted to a receipt, there is no longer a `pessmistic_gas_price` multiplier when the signer balance is deducted. Instead, the signer is charged `transaction_gas_cost * gas_price`. There is no gas refund for actions other than function calls. -* For function calls, if X gas is attached during the execution of a receipt and Y gas is burnt, then max(0, X-Y-5Tgas) is refunded at the original gas price. +* For function calls, if X gas is attached during the execution of a receipt and Y gas is burnt, then `max(0, X-Y-refund_cost)` is refunded at the original gas price where `refund_cost = max(REFUND_FIXED_COST, 0.05 * X-Y)`. +* Tokens burnt on refund cost is counted towards tx_balance_burnt and the part over `REFUND_FIXED_COST` is not counted towards gas limit to avoid artificially limiting throughput. +* Because refund cost is now separate, action costs do not need to account for refund and therefore should be recalculated and reduced. ## Security Implications @@ -43,7 +51,7 @@ The approach outlined in this proposal has less impact on developer and user exp ## Future possibilities -Burning all prepaid gas is a natural extension to this proposal, which would completely get rid of gas refunds. This, however, would be a major change to the developer experience of NEAR and should be treated cautiously. +* Burning all prepaid gas is a natural extension to this proposal, which would completely get rid of gas refunds. This, however, would be a major change to the developer experience of NEAR and should be treated cautiously. At the very least, developers should be able to easily estimate how much gas a function within a smart contract is going to consume during execution. ## Consequences @@ -64,8 +72,7 @@ At the very least, developers should be able to easily estimate how much gas a f ### Backwards Compatibility -This proposal changes how gas refund works today and as a result, users may need to pay slightly more when they send a function call transaction. However, the difference is quite small (0.0005N at most). - +Developers may need to change the amount of gas they attach when they write client side code that interacts with smart contracts to avoid getting penalized. However, this is not too difficult to do. ## Changelog From 55f192fb346062d25dd9322813d122ce318327ae Mon Sep 17 00:00:00 2001 From: Bowen Wang Date: Thu, 28 Mar 2024 10:34:26 -0700 Subject: [PATCH 4/6] address comment --- neps/nep-0536.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/neps/nep-0536.md b/neps/nep-0536.md index 3dcc73441..043888550 100644 --- a/neps/nep-0536.md +++ b/neps/nep-0536.md @@ -35,7 +35,7 @@ and creating unnecessary refunds. Some examples: ## Reference Implementation The protocol changes are as follows: -* When a transaction is converted to a receipt, there is no longer a `pessmistic_gas_price` multiplier when the signer balance is deducted. Instead, the signer is charged `transaction_gas_cost * gas_price`. There is no gas refund for actions other than function calls. +* When a transaction is converted to a receipt, there is no longer a `pessmistic_gas_price` multiplier when the signer balance is deducted. Instead, the signer is charged `transaction_gas_cost * gas_price`. If the transaction succeeds, then unless the transaction contains a function call action, it will not generate any refund. On the other hand, when a transaction with multiple action fails, there is gas refund for the rest of unexecuted actions, same as how the protocol works today. * For function calls, if X gas is attached during the execution of a receipt and Y gas is burnt, then `max(0, X-Y-refund_cost)` is refunded at the original gas price where `refund_cost = max(REFUND_FIXED_COST, 0.05 * X-Y)`. * Tokens burnt on refund cost is counted towards tx_balance_burnt and the part over `REFUND_FIXED_COST` is not counted towards gas limit to avoid artificially limiting throughput. * Because refund cost is now separate, action costs do not need to account for refund and therefore should be recalculated and reduced. From 758104eca48176ae8eb0ec3bb70f9a171f1f3762 Mon Sep 17 00:00:00 2001 From: Bowen Wang Date: Thu, 28 Mar 2024 10:35:02 -0700 Subject: [PATCH 5/6] Update neps/nep-0536.md Co-authored-by: Michael Birch --- neps/nep-0536.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/neps/nep-0536.md b/neps/nep-0536.md index 043888550..8baa92026 100644 --- a/neps/nep-0536.md +++ b/neps/nep-0536.md @@ -36,7 +36,7 @@ and creating unnecessary refunds. Some examples: The protocol changes are as follows: * When a transaction is converted to a receipt, there is no longer a `pessmistic_gas_price` multiplier when the signer balance is deducted. Instead, the signer is charged `transaction_gas_cost * gas_price`. If the transaction succeeds, then unless the transaction contains a function call action, it will not generate any refund. On the other hand, when a transaction with multiple action fails, there is gas refund for the rest of unexecuted actions, same as how the protocol works today. -* For function calls, if X gas is attached during the execution of a receipt and Y gas is burnt, then `max(0, X-Y-refund_cost)` is refunded at the original gas price where `refund_cost = max(REFUND_FIXED_COST, 0.05 * X-Y)`. +* For function calls, if X gas is attached during the execution of a receipt and Y gas is used+burnt, then `max(0, X-Y-refund_cost)` is refunded at the original gas price where `refund_cost = max(REFUND_FIXED_COST, 0.05 * X-Y)`. In the case the refund is 0 then no refund receipt is generated. * Tokens burnt on refund cost is counted towards tx_balance_burnt and the part over `REFUND_FIXED_COST` is not counted towards gas limit to avoid artificially limiting throughput. * Because refund cost is now separate, action costs do not need to account for refund and therefore should be recalculated and reduced. From 6acad903b2e861536cadcfb4648a7a278a0cb334 Mon Sep 17 00:00:00 2001 From: Lyudmil Ivanov <55487633+flmel@users.noreply.github.com> Date: Fri, 27 Sep 2024 13:52:32 +0300 Subject: [PATCH 6/6] chore: md lint --- neps/nep-0536.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/neps/nep-0536.md b/neps/nep-0536.md index 8baa92026..6f94d751a 100644 --- a/neps/nep-0536.md +++ b/neps/nep-0536.md @@ -21,12 +21,15 @@ Refund receipts create nontrivial overhead: they need to be merklized and sent a ## Specification Pessimistic gas pricing is removed as a part of this change. This means that transactions that do not involve function calls will not generate gas refund receipts as a result. For function calls, this proposal introduces cost of refund to be -``` + +```rust REFUND_FIXED_COST = action_receipt_creation + action_transfer + action_add_function_call_key refund_cost = max(REFUND_FIXED_COST, 0.05 * gas_refund); ``` + per receipt. The refund fixed cost includes consideration for implicit accounts (created on transfer) and refund for access key allowance, which requires an access key update. The design of refund cost is supposed to penalize developers from attaching too much gas and creating unnecessary refunds. Some examples: + * If the contract wants to refund 280Tgas, burning 5% of it would be about 14Tgas, which is a significant cost and developers would be encouraged to optimize it on the frontend. * If refund is 100Tgas, then 5% is 5Tgas, which is still significant and discourages developers from doing so. * If the refund is <10Tgas (very common case for cross-contract call self-callbacks), the penalty should be just 500Ggas, which is less than the gas refund cost. So only the fixed refund cost will be charged from gas to spawn the gas refund receipt. No UX will be broken for legacy cross-contract call contracts, so long as frontend correctly estimates the required gas in worst case scenario. @@ -35,6 +38,7 @@ and creating unnecessary refunds. Some examples: ## Reference Implementation The protocol changes are as follows: + * When a transaction is converted to a receipt, there is no longer a `pessmistic_gas_price` multiplier when the signer balance is deducted. Instead, the signer is charged `transaction_gas_cost * gas_price`. If the transaction succeeds, then unless the transaction contains a function call action, it will not generate any refund. On the other hand, when a transaction with multiple action fails, there is gas refund for the rest of unexecuted actions, same as how the protocol works today. * For function calls, if X gas is attached during the execution of a receipt and Y gas is used+burnt, then `max(0, X-Y-refund_cost)` is refunded at the original gas price where `refund_cost = max(REFUND_FIXED_COST, 0.05 * X-Y)`. In the case the refund is 0 then no refund receipt is generated. * Tokens burnt on refund cost is counted towards tx_balance_burnt and the part over `REFUND_FIXED_COST` is not counted towards gas limit to avoid artificially limiting throughput.