docs(dip27): version 2 Asset Unlock transactions with stable txids

dip-0002/special-transactions.md

@@ -4,8 +4,8 @@
 Here is a table of current proposed types and their associated DIP. Future DIPs
 may introduce more types.
 
 *Note:* This table refers to the _payload_ version which relates only to the special transaction
 payload and is distinct from the _transaction_ version.
 
 | Type | Transaction Type | DIP Number and Name | Payload Version | State |
 | ---- | ---------------- | ------------------- | --------------- | ----- |
@@ -13,8 +13,8 @@
 | 2 | Provider Update Service Transaction (ProUpServTx) | [DIP 003: Deterministic Masternode List](https://github.com/dashpay/dips/blob/master/dip-0003.md) | 1 | Active |
 | 3 | Provider Update Registrar Transaction (ProUpRegTx) | [DIP 003: Deterministic Masternode List](https://github.com/dashpay/dips/blob/master/dip-0003.md) | 1 | Active |
 | 4 | Provider Update Revocation Transaction (ProUpRevTx) | [DIP 003: Deterministic Masternode List](https://github.com/dashpay/dips/blob/master/dip-0003.md) | 1 | Active |
-| 5 | Coinbase Transaction (CbTx) | [DIP 004: Simplified Verification of Deterministic Masternode Lists](https://github.com/dashpay/dips/blob/master/dip-0004.md) | 3 | Active |
+| 5 | Coinbase Transaction (CbTx) | [DIP 004: Simplified Verification of Deterministic Masternode Lists](https://github.com/dashpay/dips/blob/master/dip-0004.md) | 4 | Active |
 | 6 | Quorum Commitment | [DIP 006: Long Living Masternode Quorums](https://github.com/dashpay/dips/blob/master/dip-0006.md) | 1 | Active |
 | 7 | Masternode Hard Fork Signal | [DIP 023: Enhanced Hard Fork Mechanism](https://github.com/dashpay/dips/blob/master/dip-0023.md) | 1 | Active |
 | 8 | Asset Lock | [DIP 027: Dash Core Credit Pool](https://github.com/dashpay/dips/blob/master/dip-0027.md) | 1 | Active |
-| 9 | Asset Unlock | [DIP 027: Dash Core Credit Pool](https://github.com/dashpay/dips/blob/master/dip-0027.md) | 1 | Active |
+| 9 | Asset Unlock | [DIP 027: Dash Core Credit Pool](https://github.com/dashpay/dips/blob/master/dip-0027.md) | 2 | Active |

dip-0004.md

@@ -73,6 +73,14 @@
 More information on the `bestCLHeightDiff` and `bestCLSignature` fields can be found in [DIP0029 - Randomness Beacon For LLMQ Selection](https://github.com/dashpay/dips/blob/master/dip-0029.md#change-to-the-coinbase-transaction).
 More information on the `creditPoolBalance` field will be described in a future document.
 
+Starting with version >= 4, the following fields are added:
+
+| Field | Type | Size | Description |
+| ----- | ---- | ---- | ----------- |
+| merkleRootAssetUnlocks | uint256 | 32 | Merkle root of the instance hashes of the block's version 2 Asset Unlock transactions |
+
+The transaction hash of a version 2 Asset Unlock transaction excludes its quorum signing fields (see [DIP0027 - Credit Asset Locks](https://github.com/dashpay/dips/blob/master/dip-0027.md#transaction-identity-for-version-2)), so the block's merkle root does not commit to those bytes. This field restores the commitment: it is the merkle root of the instance hashes (the hashes of the full serialization) of the block's version 2 Asset Unlock transactions in block order, or all zeros when the block contains none. A block whose commitment does not match its transactions must be rejected without marking the block hash invalid, like a block with mutated transaction data, since a relaying peer could have altered the signing fields without affecting the merkle root.
+
 ### Height in CbTx and deprecation of BIP34
 
 The CbTx contains the “height” field. It acts as a guaranteed variance in the CbTx so that each block’s CbTx gets a different hash. This is meant as a replacement for the height value currently found in the coinbase input (BIP34). With the deployment of this DIP, BIP34 becomes obsolete for new blocks and nodes should not enforce the presence of the block height in the coinbase input's “scriptSig” anymore.
@@ -175,8 +183,8 @@
 | Field | Type | Size | Description |
 |--|--|--|--|
 | signature | BLSSig | 96 | ChainLock Signature |
 | indexSetCount | compactSize uint | 1-9 | Number of quorum indexes using the same signature for their member calculation
 | indexSet | uint16_t[] | variable | Quorum indexes indicating which newQuorums entries use this signature for their member calculation
 
 ## Tracking/Updating and verifying masternode lists based on MNLISTDIFF
 

dip-0027.md

@@ -23,6 +23,8 @@
   * [Proof of Finality](#proof-of-finality)
 * [Asset Unlocking](#asset-unlocking)
   * [Asset Unlock Transaction](#asset-unlock-transaction)
+    * [Signing Asset Unlock Transactions](#signing-asset-unlock-transactions)
+    * [Transaction Identity for Version 2](#transaction-identity-for-version-2)
   * [Client verification](#client-verification)
   * [Withdrawal Completion](#withdrawal-completion)
 * [Withdrawal Safety](#withdrawal-safety)
@@ -104,7 +106,7 @@ The format of the Asset Unlock special transaction payload:
 
 | **Field** | **Type** | **Size** | **Description** |
 | - | - | - | - |
-| version | uint8_t | 1 | Asset Unlock version number. Currently set to 1 |
+| version | uint8_t | 1 | Asset Unlock version number. 1 or 2. Set to 2 once the v24 hard fork activates |
 | index | uint64 | 8 | The index of the transaction |
 | fee | uint32 | 4 | The miner's fee in duffs |
 | signHeight | uint32 | 4 | The height of the Core chain known by Platform at the moment of the Asset Unlock signing (Core ChainLock height) |
@@ -118,7 +120,7 @@ An active Platform validator quorum quorum must sign Asset Unlock transactions p
 | DIP-7 parameter | Value |
 |-|-|
 | Request ID | `SHA256(SHA256("plwdtx", index))` |
-| Message hash | The asset lock transaction, with the `quorumSig` field set to zeros, hashed as described in [DIP-2](./dip-0002.md#serialization-hashing-and-signing) |
+| Message hash | The Asset Unlock transaction, with the `quorumSig` field set to zeros, hashed as described in [DIP-2](./dip-0002.md#serialization-hashing-and-signing) |
 
 For reference, the following table shows some example index values and their request IDs. See the [provided script](dip-0027/dip-0027-request-id-calc.py) for example code:
 
@@ -127,15 +129,41 @@ For reference, the following table shows some example index values and their req
 | 101       | fcc76a643c5c668244fdcef09833955d6f4b803fa6c459f7732983c2332389fd |
 | 123456789 | ebd9d75ad72184bec8e8f25a499eb5386ba564eb7ce70a4a3b9db652599d6d72 |
 
+#### Transaction Identity for Version 2
+
+Version 2 Asset Unlock transactions are serialized identically to version 1 transactions. The version field instead changes how the transaction hash is computed.
+
+When an Asset Unlock transaction is not mined before it expires, Platform re-signs the withdrawal and broadcasts a replacement transaction. The replacement differs from the original only in the `signHeight`, `quorumHash`, and `quorumSig` fields; the index, fee, and outputs are unchanged. Under version 1 hashing each replacement has a new transaction hash, so any transaction spending an output of an unmined withdrawal becomes invalid when the withdrawal is re-signed.
+
+To resolve this, the transaction hash (txid) of a version 2 Asset Unlock transaction is computed over the transaction serialized with the `signHeight`, `quorumHash`, and `quorumSig` fields set to zeros, hashed as described in [DIP-2](./dip-0002.md#serialization-hashing-and-signing). Every re-signed instance of one withdrawal is therefore the same transaction: all instances share one txid, the transaction's outputs are tracked in the UTXO set and referenced by spending inputs under that txid as for any other transaction, and a transaction spending an output of an unmined version 2 Asset Unlock transaction remains valid regardless of which instance is eventually mined. Only one instance of a withdrawal can ever be mined since the index must be unique.
+
+Two related hashes remain distinct from the txid:
+
+* The _instance hash_: the hash of the full serialization (what version 1 hashing produces). It distinguishes the re-signed instances of one withdrawal and is used for relay and for the coinbase commitment described below.
+* The signing session's message hash, which sets only the `quorumSig` field to zeros and therefore still commits to `signHeight` and `quorumHash`.
+
+Because the txid excludes the quorum signing fields, a block's merkle root does not commit to them. Coinbase transactions must therefore commit to the instance hashes of the block's version 2 Asset Unlock transactions; see the version 4 coinbase transaction fields in [DIP-4](./dip-0004.md#coinbase-special-transaction). A block whose coinbase commitment does not match must be rejected without marking the block hash invalid, since a relaying peer could have altered the signing fields without affecting the merkle root.
+
+Relay also identifies these transactions by instance hash: a re-signed instance shares its txid with the instance peers already hold, so a txid-based announcement would never propagate it. Version 2 Asset Unlock transactions are announced with the inventory type `MSG_ASSET_UNLOCK` (34) whose hash is the instance hash, and a `getdata` for it is answered with a `tx` message. When a node receives an instance of a withdrawal already in its mempool, it should keep only the instance with the higher `signHeight`, provided its signature is valid. An expired instance should be retained awaiting a re-signed replacement rather than evicted, so that transactions spending its outputs are not evicted with it.
+
+For reference, the following table shows example version 2 txids. Both examples use a fee of 70000 duffs and a single output paying 100000000 duffs to the P2PKH script for the public key hash `1111111111111111111111111111111111111111`; the txid is independent of the `signHeight`, `quorumHash`, and `quorumSig` values. See the [provided script](dip-0027/dip-0027-txid-calc.py) for example code:
+
+| **Index** | **Transaction Hash (txid)**                                      |
+|-----------|------------------------------------------------------------------|
+| 101       | 3c4db73c8356407a5d7c78df5045bd280f2dc4fd644b06c4bfbdead3d5ae41cf |
+| 123456789 | a67e1107ae6e04b813bc8e81348266f5206d1ca93d305dc4323940e18cdbaf34 |
+
 ### Client verification
 
-All clients must verify the quorumSig field before accepting the transaction. Quorum signatures must be created by a recent quorum (one that is currently active or the most recently replaced one) to be considered valid. If the quorum is recent and the transaction is not yet mined, light clients should attempt to verify the quorumSig by retrieving the quorum public key as described in DIP-4 and verifying the signature. If the quorum is not recent and the transaction is not mined, the transaction should be ignored until it is mined to prevent attacks on the light client.
+All clients must verify the quorumSig field before accepting the transaction. Quorum signatures must be created by a recent quorum (one that is currently active or the most recently replaced one) to be considered valid. If the quorum is recent and the transaction is not yet mined, light clients should attempt to verify the quorumSig by retrieving the quorum public key as described in DIP-4 and verifying the signature. If the quorum is not recent and the transaction is not mined, the transaction should be ignored until it is mined to prevent attacks on the light client. Light clients must compute the transaction hash of version 2 Asset Unlock transactions as described above: merkle proofs of their inclusion prove this txid, not the hash of the full serialization.
 
 ### Withdrawal Completion
 
 Since Asset Unlock transactions do not have inputs, they are not eligible for InstantSend. Once mined into a block and subsequently ChainLocked, they are final and identical to all other outputs. A withdrawal should be considered complete when the corresponding Asset Unlock transaction is finalized on the Core chain.
 
-Asset Unlock transactions might not be mined for multiple reasons. For example, the quorum who signed a transaction expired, Core fees were too low, or the withdrawal limit was reached. To handle this situation, Asset Unlock transactions have an expiration period based on the quorum lifetime. Transactions are considered invalid if not signed by one of the active quorums or the most recently replaced quorum. In this case, the issuer can retry the withdrawal. To ensure that expired transactions are not included in the Core chain, Asset Lock transactions are refused once the block height exceeds _signHeight_ by 48 or more (i.e., height > _signHeight_ + 48).
+The outputs of a version 1 Asset Unlock transaction should not be spent until the transaction is mined, since the spend becomes invalid if the withdrawal is re-signed. The outputs of a version 2 Asset Unlock transaction may be spent before the transaction is mined, and these spends are eligible for InstantSend. Masternodes must only sign an InstantSend lock for such a spend after observing a validly signed instance of the withdrawal. At that point the withdrawal is irreversible on Platform, and Platform will re-sign the withdrawal until it is mined. Because the txid is stable across re-signs, both the spend and its InstantSend lock remain valid when the withdrawal is re-signed. Confirmation of such a spend depends on an instance of the withdrawal eventually being mined.
+
+Asset Unlock transactions might not be mined for multiple reasons. For example, the quorum who signed a transaction expired, Core fees were too low, or the withdrawal limit was reached. To handle this situation, Asset Unlock transactions have an expiration period based on the quorum lifetime. Transactions are considered invalid if not signed by one of the active quorums or the most recently replaced quorum. In this case, the issuer can retry the withdrawal. To ensure that expired transactions are not included in the Core chain, Asset Unlock transactions are refused once the block height exceeds _signHeight_ by 48 or more (i.e., height > _signHeight_ + 48). When a version 2 Asset Unlock transaction expires and is re-signed, the replacement is the same transaction (same txid), so transactions spending its outputs remain valid.
 
 ## Withdrawal Safety
 

dip-0027/dip-0027-txid-calc.py

@@ -0,0 +1,49 @@
+#!/usr/bin/python3
+# # Example showing how to compute the transaction hash (txid) of a version 2
+# # Asset Unlock transaction. The txid is the double-SHA256 of the transaction
+# # serialized with the signHeight, quorumHash, and quorumSig fields set to
+# # zeros, so every re-signed instance of one withdrawal shares one txid.
+import hashlib
+import struct
+
+def sha256(s):
+    return hashlib.new('sha256', s).digest()
+
+def compact_size(n):
+    if n < 253:
+        return struct.pack("B", n)
+    if n < 0x10000:
+        return struct.pack("<BH", 253, n)
+    if n < 0x100000000:
+        return struct.pack("<BI", 254, n)
+    return struct.pack("<BQ", 255, n)
+
+def serialize_with_compact_size(s):
+    return compact_size(len(s)) + s
+
+def withdrawal_txid(index, fee, outputs):
+    # Transaction version 3, type 9 (Asset Unlock)
+    tx = struct.pack("<HH", 3, 9)
+    # No inputs
+    tx += compact_size(0)
+    # Outputs
+    tx += compact_size(len(outputs))
+    for value, script in outputs:
+        tx += struct.pack("<q", value) + serialize_with_compact_size(script)
+    # nLockTime
+    tx += struct.pack("<I", 0)
+    # Payload with signHeight, quorumHash, and quorumSig set to zeros:
+    # version (2), index, fee, signHeight (0), quorumHash (zeros), quorumSig (zeros)
+    payload = struct.pack("<BQI", 2, index, fee)
+    payload += struct.pack("<I", 0)   # signHeight
+    payload += b"\x00" * 32           # quorumHash
+    payload += b"\x00" * 96           # quorumSig
+    tx += serialize_with_compact_size(payload)
+    return sha256(sha256(tx))[::-1].hex()
+
+# P2PKH output paying 100000000 duffs to public key hash 0x1111...11
+script = bytes.fromhex("76a914" + "11" * 20 + "88ac")
+outputs = [(100000000, script)]
+
+for index in [101, 123456789]:
+    print(withdrawal_txid(index, 70000, outputs))

project-words.txt

@@ -103,6 +103,7 @@ topup
 topups
 tprv
 tpub
+unmined
 unretrievable
 UTXO
 varint