Discussion:
[Bitcoin-development] Version bits proposal
Pieter Wuille
2015-05-27 01:48:05 UTC
Permalink
Hello everyone,

here is a proposal for how to coordinate future soft-forking consensus
changes: https://gist.github.com/sipa/bf69659f43e763540550

It supports multiple parallel changes, as well as changes that get
permanently rejected without obstructing the rollout of others.

Feel free to comment. As the gist does not support notifying participants
of new comments, I would suggest using the mailing list instead.

This is joint work with Peter Todd and Greg Maxwell.
--
Pieter
Douglas Roark
2015-05-27 02:31:21 UTC
Permalink
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512
Post by Pieter Wuille
here is a proposal for how to coordinate future soft-forking
https://gist.github.com/sipa/bf69659f43e763540550
It supports multiple parallel changes, as well as changes that get
permanently rejected without obstructing the rollout of others.
Feel free to comment. As the gist does not support notifying
participants of new comments, I would suggest using the mailing
list instead.
Hi Pieter. Thanks for posting the proposal. I think the concept itself
is pretty solid. I know some people have been proposing alternate
methods too. I hope they'll share here, assuming they haven't already.
As is, my comments concern typos and general copy editing.

- - Just speaking in general, I found the BIP to be a bit hard to read.
AFAIK, the basic facts are accurate. I just found myself having to
re-read certain passages two or three times. A little polish wouldn't
hurt. For example, using the "it" pronoun can be confusing, such as
multiple uses in the abstract. Specifying what "it" is (e.g., "The
proposed change relies on...") would really help. In addition, the way
the "W" value is handled seems like it could be improved a bit. I know
the wording is accurate. Seeing 1000 change to 1001 is still a little
weird.
- - In "Multi-stage soft forks," I presume the second sentence should
end as follows: "[...] with additional validation rules that get
enabled one by _one_." Depending on semantics, I'd consider changing
"one by one" to "incremental steps," but that's your call.
- - I found the "High bits" section to be confusing at first. It looks
like you chose to show everything as little endian data, matching
what's actually in the block. My personal preference would be to
represent the data, for readability purposes, as big endian. I doubt
I'm the only one who finds big endian to be much easier to process
mentally.
- - Some sort of legend showing A, I, W, etc. would really help, as
opposed to just running into them as one goes along. Otherwise, the
alphabet soup can be a bit confusing.

Thanks again.

- --
- ---
Douglas Roark
Senior Developer
Armory Technologies, Inc.
***@bitcoinarmory.com
PGP key ID: 92ADC0D7
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.22 (Darwin)
Comment: GPGTools - https://gpgtools.org

iQIcBAEBCgAGBQJVZSx5AAoJEGybVGGSrcDXrYEQAOIrsggoZv0LdJHZjPGpEkeb
7ULhO4krZtQmKXjWDP0KnHAsFiyo5EOh1fYFRZz11OCqO4QmteTLPbodZFz47tKp
tIYv5uc0qYhjfo5uLkzxuUky08VE4dUoELfqdbNciC45xHras7Wh/+KXc1a20Fib
TaisWx9aL6VfPf7urM8b6mQ9XMba4YB3e2syAY8AA+qAEEP4DK2V6tuOQJD3kxP2
tbHtJnDvkDoXEY6tnL7fePo9X/IrlXLi8vNWGqPIf/hoiHmdvU+ORwHta7z9YeIO
zi4LRs8n8sYmifY4nt6Wkkc1aoPsmpoXmI3tKgFM2h5bfdg0n3fN3K0nTMWtnR6z
HUq8JhrQkZUP8uunN/23bt94FZolvnHTdL9YuWoyrlJ0gQri5YxV1BAN4hM9oCZy
1SqlSmFRplIFWu45q8/I5duDSphmA4NP2qc59QRjftcGYpNxmzaeSViiCDWzAjI9
qTLZgLTa/nf3TFN8oU8RwquGpwD82/fFo9V+uKdNGj79kdV8WOv4sa9q63OTVimJ
w+r4l1gDZYyToe0heKtV2kL9Tt4HTn23bj7EvU+98uaKEpfWSP8a3BN9mPR7ork/
lNRGEGQ0tvkeDUzKy9IHuAjXo2XkKctbBRJwZJCGc5WW2sN0HdSu/GFPXrOOLf0J
JXqeKpfaS0UriFXkxVHO
=8uNL
-----END PGP SIGNATURE-----

------------------------------------------------------------------------------
Luke Dashjr
2015-05-27 03:46:15 UTC
Permalink
Post by Pieter Wuille
Feel free to comment. As the gist does not support notifying participants
of new comments, I would suggest using the mailing list instead.
I suggest adding a section describing how this interacts with and changes GBT.

Currently, the client tells the server what the highest block version it
supports is, and the server indicates a block version to use in its template,
as well as optional instructions for the client to forcefully use this version
despite its own maximum version number. Making the version a bitfield
contradicts the increment-only assumption of this design, and since GBT
clients are not aware of overall network consensus state, reused bits can
easily become confused. I suggest, therefore, that GBT clients should indicate
(instead of a maximum supported version number) a list of softforks by
identifier keyword, and the GBT server respond with a template indicating:
- An object of softfork keywords to bit values, that the server will accept.
- The version number, as presently conveyed, indicating the preferred softfork
flags.

Does this sound reasonable, and/or am I missing anything else?

Luke

------------------------------------------------------------------------------
Jorge Timón
2015-05-27 03:51:00 UTC
Permalink
It would also help to see the actual code changes required, which I'm sure
will be much shorter than the explanation itself.
Post by Luke Dashjr
Post by Pieter Wuille
Feel free to comment. As the gist does not support notifying participants
of new comments, I would suggest using the mailing list instead.
I suggest adding a section describing how this interacts with and changes GBT.
Currently, the client tells the server what the highest block version it
supports is, and the server indicates a block version to use in its template,
as well as optional instructions for the client to forcefully use this version
despite its own maximum version number. Making the version a bitfield
contradicts the increment-only assumption of this design, and since GBT
clients are not aware of overall network consensus state, reused bits can
easily become confused. I suggest, therefore, that GBT clients should indicate
(instead of a maximum supported version number) a list of softforks by
- An object of softfork keywords to bit values, that the server will accept.
- The version number, as presently conveyed, indicating the preferred softfork
flags.
Does this sound reasonable, and/or am I missing anything else?
Luke
------------------------------------------------------------------------------
_______________________________________________
Bitcoin-development mailing list
https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Tier Nolan
2015-05-27 09:35:03 UTC
Permalink
I think it would be better to have the deadlines set as block counts. That
eliminates the need to use the median time mechanism.

The deadline could be matched to a "start-line". The definition would then
be something like

BIP 105
Start block: 325000
End block: 350000
Activation: 750 of 1000
Implication: 950 of 1000
Bit: 9

This would allow creation of a simple table of known BIPs. It also keeps
multiple users of the bit as strictly separate.

The alternative to the start time is that it is set equal to the deadline
or implication time of the previous user of the bit.

Was the intention to change the 95% rule. You need 750 of the last 1000 to
activate and then must wait at least 1000 for implication?
Post by Jorge Timón
It would also help to see the actual code changes required, which I'm sure
will be much shorter than the explanation itself.
Post by Pieter Wuille
Post by Pieter Wuille
Feel free to comment. As the gist does not support notifying
participants
Post by Pieter Wuille
of new comments, I would suggest using the mailing list instead.
I suggest adding a section describing how this interacts with and changes GBT.
Currently, the client tells the server what the highest block version it
supports is, and the server indicates a block version to use in its template,
as well as optional instructions for the client to forcefully use this version
despite its own maximum version number. Making the version a bitfield
contradicts the increment-only assumption of this design, and since GBT
clients are not aware of overall network consensus state, reused bits can
easily become confused. I suggest, therefore, that GBT clients should indicate
(instead of a maximum supported version number) a list of softforks by
- An object of softfork keywords to bit values, that the server will accept.
- The version number, as presently conveyed, indicating the preferred softfork
flags.
Does this sound reasonable, and/or am I missing anything else?
Luke
------------------------------------------------------------------------------
_______________________________________________
Bitcoin-development mailing list
https://lists.sourceforge.net/lists/listinfo/bitcoin-development
------------------------------------------------------------------------------
_______________________________________________
Bitcoin-development mailing list
https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Peter Todd
2015-05-27 10:15:16 UTC
Permalink
Post by Tier Nolan
I think it would be better to have the deadlines set as block counts. That
eliminates the need to use the median time mechanism.
The median time mechanism is basically a way for hashing power to show
what time they think it is. Equally, the nVersion soft-fork mechanism is
a way for hashing power to show what features they want to support.

Block counts are inconvenient for planning, as there's no guarantee
they'll actually happen in any particular time frame, forward and back.
There's no particular incentive problems here - the median time clearly
shows support by a majority of hashing power - so I don't see any reason
to make planning more difficult.
Post by Tier Nolan
The deadline could be matched to a "start-line". The definition would then
be something like
BIP 105
Start block: 325000
End block: 350000
Activation: 750 of 1000
Implication: 950 of 1000
Bit: 9
This would allow creation of a simple table of known BIPs. It also keeps
multiple users of the bit as strictly separate.
If you assume no large reorganizations, your table of known BIPs can
just as easily be a list of block heights even if the median time
mechanism is used.

If you do assume there may be large reorganizations you can't have a
"simple table"
--
'peter'[:-1]@petertodd.org
000000000000000001643f7706f3dcbc3a386e4c1bfba852ff628d8024f875b6
Tier Nolan
2015-05-27 11:26:33 UTC
Permalink
Post by Peter Todd
The median time mechanism is basically a way for hashing power to show
what time they think it is. Equally, the nVersion soft-fork mechanism is
a way for hashing power to show what features they want to support.
Fair enough. It means slightly more processing, but the median time could
be cached in the header index, so no big deal.

Block counts are inconvenient for planning, as there's no guarantee
Post by Peter Todd
they'll actually happen in any particular time frame, forward and back.
I don't think the deadline needs to be set that accurately. A roughly 6
month deadline should be fine, but as you say a majority of miners is
needed to abuse the median time and it is already a miner poll.

Perhaps the number of blocks used in the median could be increased to
reduce "noise".

The median time could be median of the last 144 blocks plus 12 hours.
Post by Peter Todd
If you assume no large reorganizations, your table of known BIPs can
just as easily be a list of block heights even if the median time
mechanism is used.
I think it makes it easier to write the code. It reduced the state that
needs to be stored per BIP. You don't need to check if the previous bips
were all accepted.

Each bit is assigned to a particular BIP for a particular range of times
(or blocks).

If block numbers were used for the deadline, you just need to check the
block index for the deadline block.

enum {
BIP_INACTIVE = 0,
BIP_ACTIVE,
BIP_LOCKED
BIP_INVALID_BLOCK,
}

int GetBIPState(block, bip)
{
if (block.height == bip.deadline) // Bit must be set to match
locked/unlocked at deadline
{
int bipState = check_supermajority(...);
if (bipState == BIP_LOCKED && (block.nVersion & bip.bit)
return BIP_LOCKED;

if (bipState != BIP_LOCKED && (block.nVersion & (~bip.bit)))
return BIP_INACTIVE;

return BIP_INVALID_BLOCK;
}

if (block.height > deadline) // Look at the deadline block to determine
if the BIP is locked
return (block_index[deadline].nVersion & bip_bit) != 0 ? BIP_LOCKED
: BIP_INACTIVE;

if (block.height < startline + I) // BIP cannot activate/lock until
startline + implicit window size
return INACTIVE;

return check_supermajority(....) // Check supermajority of bit
}

The block at height deadline would indicate if the BIP was locked in.

Block time could still be used as long as the block height was set after
that. The deadline_time could be in six months. The startline height
could be the current block height and the deadline_height could be
startline + 35000.

The gives roughly

start time = now
deadline time = now + six months
deadline height = now + eight months

The deadline height is the block height when the bit is returned to the
pool but the deadline time is when the BIP has to be accepted.

It also helps with the warning system. For each block height, there is a
set of known BIP bits that are allowed. Once the final deadline is passed,
the expected mask is zeros.
Post by Peter Todd
Post by Tier Nolan
Was the intention to change the 95% rule. You need 750 of the last 1000
to activate and then must wait at least 1000 for implication?
You need 75% to start applying it, 95% to start rejecting blocks that
don't apply it.
I think the phrasing is ambiguous. I was just asking for clarification.

"Whenever I out of any W *subsequent* blocks (regardless of the block
itself) have bit B set,"

That suggests that the I of W blocks for the 95% rule must happen after
activation. This makes the rule checking harder. Easier to use the
current system, where blocks that were part of the 750 rule also count
towards the 95% rule.
Sergio Lerner
2015-05-27 22:52:18 UTC
Permalink
I like the idea but I think we should leave at least 16 bits of the
version fixed as an extra-nonce.
If we don't then miners may use them as a nonce anyway, and mess with
the soft-fork voting system.
My original proposal was this: https://github.com/bitcoin/bitcoin/pull/5102

Best regards


------------------------------------------------------------------------------
Patrick Strateman
2015-05-28 01:05:08 UTC
Permalink
There is absolutely no reason to do this.

Any reasonable micro-controller can build merkle tree roots
significantly faster than is necessary.

1 Th/s walks the nonce range once every 4.3ms.

The largest valid merkle trees are 14 nodes high.

That translates to 28 SHA256 ops per 4.3ms or 6511 SHA256 ops/second.

For reference an RPi 1 model B does 2451050 SHA256 ops/second.
Post by Sergio Lerner
I like the idea but I think we should leave at least 16 bits of the
version fixed as an extra-nonce.
If we don't then miners may use them as a nonce anyway, and mess with
the soft-fork voting system.
My original proposal was this: https://github.com/bitcoin/bitcoin/pull/5102
Best regards
------------------------------------------------------------------------------
_______________________________________________
Bitcoin-development mailing list
https://lists.sourceforge.net/lists/listinfo/bitcoin-development
------------------------------------------------------------------------------
Christian Decker
2015-05-28 07:51:39 UTC
Permalink
Agreed, there is no need to misuse the version field as well. There is more
than enough variability you could roll in the merkle tree including and
excluding transactions, and the scriptSig of the coinbase transaction,
which also influences the merkle root.

I have a fundamental dislike of retroactively changing semantics, and the
version field should be used just for that: a version. I don't even
particularly like flagging support for a fork in the version field, but
since I have no better solution, count me as supporting Sipa's proposal. We
definitely need a more comfortable way of rolling out new features.

Regards,
Chris

On Thu, May 28, 2015 at 3:08 AM Patrick Strateman <
Post by Patrick Strateman
There is absolutely no reason to do this.
Any reasonable micro-controller can build merkle tree roots
significantly faster than is necessary.
1 Th/s walks the nonce range once every 4.3ms.
The largest valid merkle trees are 14 nodes high.
That translates to 28 SHA256 ops per 4.3ms or 6511 SHA256 ops/second.
For reference an RPi 1 model B does 2451050 SHA256 ops/second.
Post by Sergio Lerner
I like the idea but I think we should leave at least 16 bits of the
version fixed as an extra-nonce.
If we don't then miners may use them as a nonce anyway, and mess with
the soft-fork voting system.
https://github.com/bitcoin/bitcoin/pull/5102
Post by Sergio Lerner
Best regards
------------------------------------------------------------------------------
Post by Sergio Lerner
_______________________________________________
Bitcoin-development mailing list
https://lists.sourceforge.net/lists/listinfo/bitcoin-development
------------------------------------------------------------------------------
_______________________________________________
Bitcoin-development mailing list
https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Adam Back
2015-05-28 08:11:21 UTC
Permalink
Or as far as that goes, permuting (the non-dependent) transactions in
the block by permuting the internal merkle tree nodes at increasing
depths. (Dependent because transactions that depend on each other
have to come in-order; but one could eg put the n-1 of each n sequence
of in-order transactions in the left-half and unordered in the right
half.)

That makes the tree manipulations maximum depth independent, and even
transaction independent possibly - just need to know enough depth in
the tree of hashes that are permutation safe.

Adam
Post by Christian Decker
Agreed, there is no need to misuse the version field as well. There is more
than enough variability you could roll in the merkle tree including and
excluding transactions, and the scriptSig of the coinbase transaction, which
also influences the merkle root.
I have a fundamental dislike of retroactively changing semantics, and the
version field should be used just for that: a version. I don't even
particularly like flagging support for a fork in the version field, but
since I have no better solution, count me as supporting Sipa's proposal. We
definitely need a more comfortable way of rolling out new features.
Regards,
Chris
On Thu, May 28, 2015 at 3:08 AM Patrick Strateman
Post by Patrick Strateman
There is absolutely no reason to do this.
Any reasonable micro-controller can build merkle tree roots
significantly faster than is necessary.
1 Th/s walks the nonce range once every 4.3ms.
The largest valid merkle trees are 14 nodes high.
That translates to 28 SHA256 ops per 4.3ms or 6511 SHA256 ops/second.
For reference an RPi 1 model B does 2451050 SHA256 ops/second.
Post by Sergio Lerner
I like the idea but I think we should leave at least 16 bits of the
version fixed as an extra-nonce.
If we don't then miners may use them as a nonce anyway, and mess with
the soft-fork voting system.
https://github.com/bitcoin/bitcoin/pull/5102
Best regards
------------------------------------------------------------------------------
_______________________________________________
Bitcoin-development mailing list
https://lists.sourceforge.net/lists/listinfo/bitcoin-development
------------------------------------------------------------------------------
_______________________________________________
Bitcoin-development mailing list
https://lists.sourceforge.net/lists/listinfo/bitcoin-development
------------------------------------------------------------------------------
_______________________________________________
Bitcoin-development mailing list
https://lists.sourceforge.net/lists/listinfo/bitcoin-development
------------------------------------------------------------------------------
Potter QQ
2015-06-01 14:50:28 UTC
Permalink
oh my God ...

·¢×ÔÎÒµÄ iPhone
Post by Peter Todd
The median time mechanism is basically a way for hashing power to show
what time they think it is. Equally, the nVersion soft-fork mechanism is
a way for hashing power to show what features they want to support.
Fair enough. It means slightly more processing, but the median time could be cached in the header index, so no big deal.
Post by Peter Todd
Block counts are inconvenient for planning, as there's no guarantee
they'll actually happen in any particular time frame, forward and back.
I don't think the deadline needs to be set that accurately. A roughly 6 month deadline should be fine, but as you say a majority of miners is needed to abuse the median time and it is already a miner poll.
Perhaps the number of blocks used in the median could be increased to reduce "noise".
The median time could be median of the last 144 blocks plus 12 hours.
Post by Peter Todd
If you assume no large reorganizations, your table of known BIPs can
just as easily be a list of block heights even if the median time
mechanism is used.
I think it makes it easier to write the code. It reduced the state that needs to be stored per BIP. You don't need to check if the previous bips were all accepted.
Each bit is assigned to a particular BIP for a particular range of times (or blocks).
If block numbers were used for the deadline, you just need to check the block index for the deadline block.
enum {
BIP_INACTIVE = 0,
BIP_ACTIVE,
BIP_LOCKED
BIP_INVALID_BLOCK,
}
int GetBIPState(block, bip)
{
if (block.height == bip.deadline) // Bit must be set to match locked/unlocked at deadline
{
int bipState = check_supermajority(...);
if (bipState == BIP_LOCKED && (block.nVersion & bip.bit)
return BIP_LOCKED;
if (bipState != BIP_LOCKED && (block.nVersion & (~bip.bit)))
return BIP_INACTIVE;
return BIP_INVALID_BLOCK;
}
if (block.height > deadline) // Look at the deadline block to determine if the BIP is locked
return (block_index[deadline].nVersion & bip_bit) != 0 ? BIP_LOCKED : BIP_INACTIVE;
if (block.height < startline + I) // BIP cannot activate/lock until startline + implicit window size
return INACTIVE;
return check_supermajority(....) // Check supermajority of bit
}
The block at height deadline would indicate if the BIP was locked in.
Block time could still be used as long as the block height was set after that. The deadline_time could be in six months. The startline height could be the current block height and the deadline_height could be startline + 35000.
The gives roughly
start time = now
deadline time = now + six months
deadline height = now + eight months
The deadline height is the block height when the bit is returned to the pool but the deadline time is when the BIP has to be accepted.
It also helps with the warning system. For each block height, there is a set of known BIP bits that are allowed. Once the final deadline is passed, the expected mask is zeros.
Post by Peter Todd
Was the intention to change the 95% rule. You need 750 of the last 1000 to activate and then must wait at least 1000 for implication?
You need 75% to start applying it, 95% to start rejecting blocks that don't apply it.
I think the phrasing is ambiguous. I was just asking for clarification.
"Whenever I out of any W subsequent blocks (regardless of the block itself) have bit B set,"
That suggests that the I of W blocks for the 95% rule must happen after activation. This makes the rule checking harder. Easier to use the current system, where blocks that were part of the 750 rule also count towards the 95% rule.
------------------------------------------------------------------------------
_______________________________________________
Bitcoin-development mailing list
https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Jorge Timón
2015-05-27 10:15:46 UTC
Permalink
Post by Tier Nolan
Was the intention to change the 95% rule. You need 750 of the last 1000
to activate and then must wait at least 1000 for implication?

You need 75% to start applying it, 95% to start rejecting blocks that don't
apply it.
Pieter Wuille
2015-06-03 20:42:44 UTC
Permalink
Thanks for all the comments.

I plan to address the feedback and work on an implementation next week.
Post by Pieter Wuille
Hello everyone,
here is a proposal for how to coordinate future soft-forking consensus
changes: https://gist.github.com/sipa/bf69659f43e763540550
It supports multiple parallel changes, as well as changes that get
permanently rejected without obstructing the rollout of others.
Feel free to comment. As the gist does not support notifying participants
of new comments, I would suggest using the mailing list instead.
This is joint work with Peter Todd and Greg Maxwell.
--
Pieter
Loading...