How accurate is this picture of how transactions are processed on the NEAR platform?

Question

After reading more about how transactions are processed by NEAR I came up with this picture of how a few key parts are related.

I am seeking some pointers on how to correct this.

First a few key points I'm currently aware of, only some of which are illustrated below, are:

an Action must be one of 7 supported operations on the network
- CreateAccount to make a new account (for a person, company, contract, car, refrigerator, etc)
- DeployContract to deploy a new contract (with its own account)
- FunctionCall to invoke a method on a contract (with budget for compute and storage)
- Transfer to transfer tokens from one account to another
- Stake to express interest in becoming a proof-of-stake validator at the next available opportunity
- AddKey to add a key to an existing account (either FullAccess or FunctionCall access)
- DeleteKey to delete an existing key from an account
- DeleteAccount to delete an account (and transfer balance to a beneficiary account)
a Transaction is a collection of Actions augmented with critical information about their
- origin (ie. cryptographically signed by signer)
- destination or intention (ie. sent or applied to receiver)
- recency (ie. block_hash distance from most recent block is within acceptable limits)
- uniqueness (ie. nonce must be unique for a given signer)
a SignedTransaction is a Transaction cryptographically signed by the signer account mentioned above
Receipts are basically what NEAR calls Actions after they pass from outside (untrusted) to inside (trusted) the "boundary of trust" of our network. Having been cryptographically verified as valid, recent and unique, a Receipt is an Action ready for processing on the blockchain.
since, by design, each Account lives on one and only one shard in the system, Receipts are either applied to the shard on which they first appear or are routed across the network to the proper "home shard" for their respective sender and receiver accounts. DeleteKey is an Action that would never need to be routed to more than 1 shard while Transfer would always be routed to more than 1 shard unless both signer and receiver happen to have the same "home shard"
a "finality gadget" is a collection of rules that balances the urgency of maximizing blockchain "liveness" (ie. responsiveness / performance) with the safety needed to minimize the risk of accepting invalid transactions onto the blockchain. One of these rules includes "waiting for a while" before finalizing (or sometimes reversing) transactions -- this amounts to waiting a few minutes for 120 blocks to be processed before confirming that a transaction has been "finalized".

          ---.
  o--------o |     o------------------------o     o-------------------o
  | Action | |     |      Transaction       |     | SignedTransaction |
  o--------o |     |                        |     |                   |
             |     | o--------o             |     |  o-------------o  |
  o--------o |     | | Action |  signer     |     |  | Transaction |  |
  | Action | | --> | o--------o  receiver   | --> |  |             |  |  ---.
  o--------o |     | | Action |  block_hash |     |  |             |  |     |
             |     | o--------o  nonce      |     |  |             |  |     |
  o--------o |     | | Action |             |     |  |             |  |     |
  | Action | |     | o--------o             |     |  o-------------o  |     |
  o--------o |     o------------------------o     o-------------------o     |
          ---'                                                              |
                                                                            |
                              sent to network                               |
.---------------------------------------------------------------------------'
|                               <----------
|
|                                                                   ---.
|                                       XXX o--------o     o---------o |
|                                      XX   | Action | --> | Receipt | |
|    o--------------------------------o     o--------o     o---------o |
|    |                                |                                |
|    |  1. Validation (block_hash)    |     o--------o     o---------o |
'--> |  2. Verification (signer keys) |     | Action | --> | Receipt | |  --.
     |  3. Routing (receiver)         |     o--------o     o---------o |    |
     |                                |                                |    |
     o--------------------------------o     o--------o     o---------o |    |
            transaction arrives        XX   | Action | --> | Receipt | |    |
                                        XXX o--------o     o---------o |    |
                                                                    ---'    |
                                                                            |
                applied locally OR propagated to other shards               |
.---------------------------------------------------------------------------'
|                               <----------
|
|
|              --.         .-------.  .--.  .--.         .--.  o-----------o
|    o---------o |         |       |  |  |  |  |         |  |  |           |
'--> | Receipt | |  Shard  |       |  |  |  |  |         |  |  |           |
     o---------o |    A    |       |  |  |  |  |         |  |  |           |
|              --'         |       |  |  |  |  |         |  |  |           |
|                          |       |  |  |  |  |         |  |  |           |
|              --.         |       |  |  |  |  |         |  |  |   Block   |
|    o---------o |         | Block |  |  |  |  |  o o o  |  |  |    (i)    |
'--> | Receipt | |         |  (i)  |  |  |  |  |         |  |  | finalized |
     o---------o |         |       |  |  |  |  |         |  |  |           |
|                |  Shard  |       |  |  |  |  |         |  |  |           |
|    o---------o |    B    |       |  |  |  |  |         |  |  |           |
'--> | Receipt | |         |       |  |  |  |  |         |  |  |           |
     o---------o |         |       |  |  |  |  |         |  |  |           |
               --'         '-------'  '--'  '--'         '--'  o-----------o

                          |                                                |
                          '------------------------------------------------'
                                     about 3 blocks to finality

I used http://asciiflow.com and made some tweaks by hand. glad you like it. accuracy of the diagram is poor though, based on most recent answers. — amgando, Dec 26 '20 at 12:54

score 2 · Answer 1 · answered Dec 13 '19 at 20:57

2

It's unclear to me what you mean by "routed to more than one shard". A receipt can only be routed to one shard. Also I don't understand your description of finality gadget, and I don't know where you get "120 blocks" from. Normally you just need to wait for 3 blocks for a block to be finalized.

answered Dec 13 '19 at 20:57

berryguy

1,144
3
4

This is just a property `sender=receiver`. Some actions are required to be `sender=receiver`. https://nomicon.io/Runtime/Actions.html – fckt Dec 16 '19 at 11:37
1

In that case I would still consider the receipt to be routed to the same shard, even though there is no network message involved. – berryguy Dec 16 '19 at 16:29

Lucio M. Tato · Answer 2 · 2020-11-02T11:12:25.587

Great explanation! Core protocol devs should complete that picture and include in the low-level documentation!

There's some corrections. A Transaction with all its actions gets converted to a single Receipt. Receipts can have several actions too. Every receipt goes to a single specific shard/receiver account. In the case of a "Transfer" action inside a Transaction/Receipt, it can generate new receipts to complete the transfer:

e.g. Alice sends 100N to Bob

Receipt 1, action Transfer: acting on Alice's account. Alice's account gets 100N deducted. If that succeeds a 2nd Receipt is created:
Receipt 2- single action: act on Bob's account to "increase balance by 100N". This second receipt gets "published" to be routed to Bob's shard.
if the 2nd receipt fails (no Bob account) a 3rd Receipt is created to refund 100N to Alice. This 3rd Receipt is again published to be routed back to Alice's shard.

So every receipt (can have more than one action) but is directed to a single specific account and then a single shard.

.- At least this is what I understand 'til now -.

I'm reading the code Sherif, more details:

Even if a Transaction has more than one action, each transaction is converted to a single receipt. A Receipt can have more than one action, but a single ´receiver´.
All Receipts are validated. When routed to other shards (if the ´receiver´ account is not in the current shard) the receiving node will re-validate the receipt before processing. So there's no trusted/untrusted boundary. Everything gets re-validated in the nodes before processing.
All local receipts are processed first, then delayed receipts are checked (waiting for data), and then receipts received from other nodes are processed.
Some Recepits can be "Data Receipts", containing chunks of data required to execute other receipts. It's like sending input data for actions in chunks to other nodes. When all the data chunks are received the related "Action Receipt" is executed.
When an "Action Receipts" has all it's data, every action inside the receipt is executed: code and code There's a loop for every action in the receipt, and the action is applied to the receiver account.

.-to be continued-.

Thanks Lucio, appreciate the detailed follow up. Here's the latest from Evgeny Kuzyakov which is captured in this doc and a recording. It may help add some context to your exploration: https://docs.google.com/document/d/1eSo2Hh6PJfQOmEhw4M2TxLqGjCVbBzqFF58jNRHgwWc/edit — amgando, Dec 07 '20 at 15:37

score 1 · Answer 3 · edited Nov 02 '20 at 11:10

"Receipts are either applied to the shard on which they first appear or are routed across the network to the proper "home shard" for their respective sender and receiver accounts."

So here is my understanding; AccountID sends a transaction to the shard they are on e.g. assigned to for the given epoch since every epoch there is a reshuffling of accounts across shards. The shard (set of AccountIDs of validators etc.) verifies the transaction. If the receiver is on another shard, a receipt is created and routed to the other shard. While the transaction from the sender can be included in the next block, it will take up to three blocks to validate it and finalize the routing to the receiver shard.

How accurate is this picture of how transactions are processed on the NEAR platform?

3 Answers3