Wallet Decoupling

Wallet Decoupling

Team Members

Name	GitHub Handle	Focus
Johannes Lund	@anviking	100%
Matt Parsons	@parsonsmatt	100%
Matthias Benkort	@KtorZ	50%
Pawel Jakubas	@paweljakubas	100%

Current State

Cardano-SL Wallet

BListener

The node implements the BListener interface which boils down to two actions: apply a block and switch to a fork.

Node State Adaptor

Various pieces required by the node were historically not derived from blocks and from what it receives via the BListener. Most of them are gathered under the NodeStateAdaptor.

Recovery

During recovery, the wallet currently peeks into the node's database to resolve the current UTxO, the initial one and the historical data by applying blocks since the genesis one.

Blund

The wallet currently receives Blunds from the BListener instead of mere Blocks. However, Undos from Blunds are only used to retrieve some transaction Metadata which are otherwise available in the UTxO maintained by the wallet.

Update

Being the only communication channel between Daedalus and the node itself, the wallet is currently used to convey some specific process supervision commands (e.g. apply-update). This is currently done via the "Internal" HTTP API.

Transaction Submission

The wallet needs to submit transactions to the network. Right now, the wallet is hooked up to the node's inner diffusion layer from which it can push new transactions.

Cardano-SL Haskell Backend (Byron Reboot / current)

There's an existing working node-to-node communication protocol which is used by nodes and the Rust CLI. This protocol enables us (amongst other things) to:

• Submit new transactions to the network
• Receive blocks from the network, from a given hash

Cardano-SL Haskell Backend (Shelley)

? unknown ?

Cardano-SL Rust Backend

Node-to-Node protocol interface implemented; Rest being worked on, mostly following the implementation of the existing Haskell backend.

Coupling Areas, First Pointers

We assume the Wallet fully trusts the node it is connected to such that, validation and verification work is done by the node and acknowledged by the Wallet.

Receiving Blocks

We want the Wallet to be able to receive blocks as they get validated by the underlying node. The Wallet is interested in the blocks for several reasons:

• It maintains its own UTxO via a set of specified ledger rules (see Formal specification for a Cardano Wallet)

• It tracks a bunch of Protocol Parameters that are specified in the genesis block and may change via updates. For the Wallet, the relevant parameters are:

  • slot length
  • epoch length (NOTE: in current version, can't change via update, but will in future versions of Ouroboros).
  • fee policy
  • transaction max size

The parameters are currently not extracted from the block but queried directly via some IO using the fact that Wallet runs as a node. Knowing how to transition from an initial state via applying a block, it would be possible for the Wallet to keep track of these parameters as it applies blocks (see A Simplified Formal Specification of a UTxO Ledger). There's a major question about rollbacks and how to "un-apply" rules.

Ideally, we want the ability for the Wallet to ask for two things:

• An initial state (derived from the genesis block by the node) where state can be many things (e.g. protocol parameters, system start, stake pools distribution, utxo)

• The current state (for fast syncing)

Then, a consumer can apply ledger rules they're interested in as they receive blocks, assuming node has validated pre-conditions for applying those rules. Having the initial state and the transitions is all we need to keep state in sync.

Node Monitoring

The wallet currently offers some basic monitoring of the node via the /api/v1/node-info and /api/v1/node-settings. Underneath, this piggy-backs once more on the Node State Adaptor and the fact that the Wallet is ultimately a node. This includes:

• syncProgress: Syncing progression, in percentage.
• blockchainHeight: If known, the current blockchain height, in number of blocks.
• localBlockchainHeight: Local blockchain height, in number of blocks.
• localTimeInformation: Information about the clock on this node.
• subscriptionStatus: Is the node connected to the network and to what peers?
• slotDuration: Duration of a slot.
• softwareInfo: Various pieces of information about the current software.
• projectVersion: Current project's version.
• gitRevision: Git revision of this deployment.

Strictly considering the wallet, these pieces of information are quite out-of-scope. They should instead be provided by a node itself and queried by clients directly from the underlying node. At the moment, the Wallet is used as a kind of proxy to relay these pieces of information, but in order to properly decouple the Wallet from the node, we should look into moving this API onto the node, with a possible redesign as we do it.

In order to avoid breaking the API versioning contract, we will be required to support the Node API in the V1 of cardano-wallet. The V2 of the wallet API will not need to support it.

Additionally, the Wallet also may require the systemStart to be available in such API in order to compute a few things from it.

Node Management

Falls into ths category, actions that are triggered by clients to act upon the node:

• apply-update: Make the node shutdown with a special exit code (20) which gets picked up by the launcher.

As for the node monitoring, this is a bit out-of-scope for the Wallet. This responsibility should be moved outside of the Wallet and, during a transition / compatibility period, may still be supported by the Wallet and forwarded to the node.

Restoration

With the ability to query any block from a node (as described in Receiving Blocks), a Wallet can avoid reads to a node's database. Note that, in order to perform restoration while the Wallet is still processing blocks for existing wallets, we need to be able to listen to blocks on multiple (possibly different) channels.

Slot Start

Historically, the Wallet attaches a timestamp to transations' metadata. This information isn't actually present in the transaction as they're represented in the block and can't really be derived from it. Our best approximation during restoration is to consider the beginning of a slot as the timestamp of a transaction. To retrieve this slot time, we could still leverage the functions defined by the node, and make sure they don't require any access to the database.

Knowing the slot length(s), epoch length(s), current time, current slot and system start should be enough information to recover any slot start.

Diffusion / Transaction Submission

Being a node, the Wallet "shortcuts" the transaction submission by directly pushing transactions to the node's queue. In order to fully decouple the wallet, we would need the wallet to use the Node <-> Node protocol in order to submit transactions.

Milestones

Total Estimate: 100 + ? points

A. We Can Build, Test & Work on Cardano-Wallet in a Dedicated Repository (14 points)

Transfer Existing Code From cardano-sl to a New Repository (2 points)

1. • We should extract wallet-new from cardano-sl into its own repository cardano-wallet.
2. • We should define a stack.yaml file allowing developers to easily build the project locally.
3. • We may restructure the .cabal file to accomodate the new structure.

Experiment With Travis CI (5 points)

4. • We should understand how to setup Travis-CI for cardano-wallet.
5. • We should have 'regular' CI times under an hour.
6. • We should build cardano-wallet sources, and run tests in CI.
7. • We should run hlint in CI.
8. • We may run weeder in CI.

Experiment With Coveralls (2 points)

9. • We may integrate the CI with coveralls.io and have coverage reports available
10. • We may display code coverage on the README and monitor it

Finalize CI & cardano-wallet Repository Setup (5 points)

11. • We should have a complete pipeline setup and ready for development
12. • We may document the CI steps and setup in the Wiki
13. • We may cache builds of stack LTS
14. • We may cache builds of cardano-sl
15. • We may build the Swagger / Redoc API documentation in CI & publish it to gh-pages.

B. cardano-sl Nodes Can Start an HTTP Server & Provide a Monitoring API (17 points)

Define The Node Monitoring & Node Management API(s) (1 point)

1. There must be a Servant API representing the Node Monitoring & Management API
2. The type from B.1. must be convertible to Swagger
3. The type from B.1. should be defined outside of cardano-wallet, on cardano-sl.

Implement the API(s) on The Node's End (8 points)

4. Handlers corresponding to the API defined in B.1. must be implemented
5. There mustn't be any dependencies from cardano-sl -> cardano-wallet introduced in B.4.

Start HTTP Server & Serve Implement API on The Node's End (5 points)

6. Nodes must host an HTTP(s) server to serve the API defined in B.1.
7. The server started in B.6. must be as powerful as the current wallet server (i.e. TLS, client cert validation, HTTP/2.0 support).
8. Nodes' CLI should be extended to support passing options to the server from B.6..
9. CLI Options should define a --tlsca and --tlscert to pass TLS certificates to the server.
10. CLI Options may define a --no-client-auth to disable client certificate verification.
11. CLI Options may define a --no-tls to completely disable TLS, in which case, certs aren't required.

Integrate the Wallet to Use the Node's API (3 points)

12. cardano-wallet backward-compatible node monitoring API should solely rely on a Node's API and relay requests through HTTP(s) to the node
13. We may define a Haskell client to easily talk to the Node API from cardano-wallet

C. We Can Submit Transactions to a Node Over The Network Using The Current Protocol (27 points)

Comprehend & Integrate With (Part of) The Node-To-Node Protocol to Submit Transactions (8 points)

• 1. We must understand and document relevant bits of the Node-To-Node Protocol.
• 2. We must distinguish clearly what is transmitted "over-the-wire" from what is just plain shared Haskell structures.

Clearly Identify Current Areas Where Transactions Are Submitted / Resubmitted (3 points)

• 3. We must clearly identify and document how transactions are currently submitted to the node
• 4. We must identify the remaining steps for the wallet to achieve full decoupling regarding submission.

Abstract Away Submission of Transactions (5 points)

• 5. Bits related to submission of transaction must be abstracted in a protocol-neutral interface.
• 6. The interface from C.5 mustn't expose any protocol related parameters.
• 7. It must be possible to emulate the interface from C.5 using the existing code.
• 8. An implementation of C.7 may be provided as an illustration.

Provide an Implementation Using The Node-to-Node Protocol (8 points)

• 9. C.5 must be implemented using the Node-to-Node protocol.
• 10. The implementation from C.9 may include integration tests.

Update Existing Code & Submit Transaction Over The Network (3 points)

• 11. Areas identified in C.3 must make use of the interface in C.5 using the implementation from C.9.

D. We Can Consume Blocks From a Node Via The Network Using The Current Protocol (42)

Comprehend & Integrate With (Part of) The Node-To-Node Protocol to Receive Blocks (8 points)

• 1. We must understand and document relevant bits of the Node-To-Node Protocol.
• 2. We must distinguish clearly what is transmitted "over-the-wire" from what is just plain shared Haskell structures.

Abstract Away Receiving Blocks, Fork Switching, Getting Initial & Current State (8 points)

• 3. Bits related to receiving blocks must be abstracted in a protocol-neutral interface.
• 4. The interface from D.3 mustn't expose any protocol related parameters.
• 5. It must be possible to emulate the interface from D.3 using the existing BListener.
• 6. An implementation of D.5 may be provided as an illustration.

Provide an Implementation Using The Node-to-Node Protocol (13 points)

• 7. D.3 must be implemented using the Node-to-Node protocol.
• 8. The implementation from D.7 may include integration tests.

Replace Current BListener and NodeStateAdaptor Usage And Query Information From The Network (13 points)

• 9. Areas identified in D.1 must make use of the interface in D.3 using the implementation from D.7.

E. We Can Start The Wallet as a Dedicated Process (? points)

Clean-up Node-Specific Arguments And Options From The CLI (? points)

Review CLI, Build & Last Coupling Areas (? points)