Discussion:
[bitcoin-dev] BIP draft: Hardfork bit
jl2012 via bitcoin-dev
2015-07-23 16:23:21 UTC
Permalink
Please feel free to comment, for technical issues and language

BIP: ??
Title: Hardfork bit
Author: jl2012 <***@xbt.hk>
Status: Draft
Type: Standard Track
Created: 2015-07-23

Abstract

This document specifies a proposed change to the semantics of the most
significant bit of the “version” field in Bitcoin block headers, as a
mechanism to indicate a hardfork is deployed. It alleviates certain
risks related to a hardfork by introducing an explicit “point of no
return” in the blockchain. This is a general mechanism which should be
employed by any planned hardfork in the future.

Motivation

Hardforks in Bitcoin are usually considered as difficult and risky, because:

1) Hardforks require not only support of miners, but also, most
importantly, supermajority support of the Bitcoin economy. As a
result, softfork deployment mechanisms described in BIP 34 or BIP XX
“Version bits” (https://gist.github.com/sipa/bf69659f43e763540550) are
not enough for introducing hardforks safely.

2) Full nodes and SPV nodes following original consensus rules may not
be aware of the deployment of a hardfork. They may stick to an
economic-minority fork and unknowingly accept devalued legacy tokens.

3) In the case which the original consensus rules are also valid under
the new consensus rules, users following the new chain may
unexpectedly reorg back to the original chain if it grows faster than
the new one. People may find their confirmed transactions becoming
unconfirmed and lose money.

The first issue involves soliciting support for a hardfork proposal,
which is more a political topic than a technical one. This proposal
aims at alleviating the risks related to the second and third issues.
It should be employed by any planned hardfork in the future.

Definitions

See BIP YY “Motivation and deployment of consensus rules changes”
https://github.com/jtimon/bips/blob/bip-forks/bip-forks.org

Specification

Hardfork bit: The most significant bit in nVersion is defined as the
hardfork bit. Currently, blocks with this header bit setting to 1 are
invalid, since BIP34 interprets nVersion as a signed number and
requires it to be >=2 (with BIP66, >=3). Among the 640 bits in the
block header, this is the only one which is fixed and serves no
purpose, and therefore the best way to indicate the deployment of a
hardfork.

Flag block: Any planned hardfork must have one and only one flag block
which is the “point of no return”. To ensure monotonicity, flag block
should be determined by block height, or as the first block with
GetMedianTimePast() greater than a threshold. Other mechanisms could
be difficult for SPV nodes to follow. The height/time threshold could
be a predetermined value or relative to other events (e.g. 1000 blocks
/ 10 days after 75% of miner support). The exact mechanism is out of
the scope of this BIP. No matter what mechanism is used, the threshold
is consensus critical. It must be publicly verifiable with only
blockchain data and the programme source code, and preferably
SPV-friendly.

Flag block is constructed in a way that nodes with the original
consensus rules must reject. On the other hand, nodes with the new
consensus rules must reject a block if it is not a flag block while it
is supposed to be. To achieve these goals, the flag block must 1) have
the hardfork bit setting to 1, 2) include a short predetermined unique
description of the hardfork anywhere in its coinbase, and 3) follow
any other rules required by the hardfork. If these conditions are not
fully satisfied, upgraded nodes shall reject the block.

The hardfork bit must be turned off in the decedents of the flag
block, until the deployment of the next hardfork. The requirement of
coinbase message is also limited to the flag block. In the rare case
that multiple hardforks share the same flag block, the coinbase shall
include all relevant messages and the order/position of the messages
shall not be consensus critical.

Although a hardfork is officially deployed after the flag block, the
exact behavioural change is out of the scope of this BIP. For example,
a hardfork may not be fully active until certain time after the flag
block.

Automatic warning system: When a flag block is found on the network,
full nodes and SPV nodes should look into its coinbase. They should
alert their users and/or stop accepting incoming transactions if it is
an unknown hardfork. It should be noted that the warning system could
become a DoS vector if the attacker is willing to give up the block
reward. Therefore, the warning may be issued only if a few blocks are
built on top of the flag block in a reasonable time frame. This will
in turn increase the risk in case of a real planned hardfork so it is
up to the wallet programmers to decide the optimal strategy. Human
warning system (e.g. the emergency alert system in Bitcoin Core) could
fill the gap.

Compatibility

As a mechanism to indicate hardfork deployment, this BIP breaks
backward compatibility intentionally. However, without further changes
in the block header format, full nodes and SPV nodes could still
verify the PoW of a flag block and its descendants.

This proposal is also compatible with the BIP XX “Version bits”. The
version bits mechanism could be employed to measure miner support
towards a hardfork proposal, and to determine the height or time
threshold of the flag block. Also, miners of the flag block may still
cast votes for other concurrent softfork or hardfork proposals as
normal.

After the flag block is generated, a miner may support either fork but
not both. It is not possible for miners in one fork to attack or
overtake the other fork because the forks are mutually exclusive.
Tier Nolan via bitcoin-dev
2015-07-23 17:59:40 UTC
Permalink
On Thu, Jul 23, 2015 at 5:23 PM, jl2012 via bitcoin-dev <
2) Full nodes and SPV nodes following original consensus rules may not be
aware of the deployment of a hardfork. They may stick to an
economic-minority fork and unknowingly accept devalued legacy tokens.
This change means that they are kicked off the main chain immediately when
the fork activates.

The change is itself a hard fork. Clients have be updated to get the
benefits.

3) In the case which the original consensus rules are also valid under the
new consensus rules, users following the new chain may unexpectedly reorg
back to the original chain if it grows faster than the new one. People may
find their confirmed transactions becoming unconfirmed and lose money.
I don't understand the situation here. Is the assumption of a group of
miners suddenly switching (for example, they realise that they didn't
intend to support the new rules)?
Flag block is constructed in a way that nodes with the original consensus
rules must reject. On the other hand, nodes with the new consensus rules
must reject a block if it is not a flag block while it is supposed to be.
To achieve these goals, the flag block must 1) have the hardfork bit
setting to 1, 2) include a short predetermined unique description of the
hardfork anywhere in its coinbase, and 3) follow any other rules required
by the hardfork. If these conditions are not fully satisfied, upgraded
nodes shall reject the block.
Ok, so set the bit and then include BIP-GIT-HASH of the canonical BIP on
github in the coinbase?

Since it is a hard fork, the version field could be completely
re-purposed. Set the bit and add the BIP number as the lower bits in the
version field. This lets SPV clients check if they know about the hard
fork.

There network protocol could be updated to add getdata support for asking
for a coinbase only merkleblock. This would allow SPV clients to obtain
the coinbase.

Automatic warning system: When a flag block is found on the network, full
nodes and SPV nodes should look into its coinbase. They should alert their
users and/or stop accepting incoming transactions if it is an unknown
hardfork. It should be noted that the warning system could become a DoS
vector if the attacker is willing to give up the block reward. Therefore,
the warning may be issued only if a few blocks are built on top of the flag
block in a reasonable time frame. This will in turn increase the risk in
case of a real planned hardfork so it is up to the wallet programmers to
decide the optimal strategy. Human warning system (e.g. the emergency alert
system in Bitcoin Core) could fill the gap.
If the rule was that hard forks only take effect 100 blocks after the flag
block, then this problem is eliminated.

Emergency hard forks may still have to take effect immediately though, so
it would have to be a custom not a rule.
jl2012 via bitcoin-dev
2015-07-23 19:26:33 UTC
Permalink
Post by Tier Nolan via bitcoin-dev
On Thu, Jul 23, 2015 at 5:23 PM, jl2012 via bitcoin-dev <
2) Full nodes and SPV nodes following original consensus rules may not be
aware of the deployment of a hardfork. They may stick to an
economic-minority fork and unknowingly accept devalued legacy tokens.
This change means that they are kicked off the main chain immediately when
the fork activates.
The change is itself a hard fork. Clients have be updated to get the
benefits.
I refrain from calling it the "main chain". I use "original chain" and
"new chain" instead as I make no assumption about the distribution of
mining power. This BIP still works when we have a 50/50 hardfork. The
main point is to protect all users on both chains, and allow them to
make an informed choice.
Post by Tier Nolan via bitcoin-dev
3) In the case which the original consensus rules are also valid under the
new consensus rules, users following the new chain may unexpectedly reorg
back to the original chain if it grows faster than the new one. People may
find their confirmed transactions becoming unconfirmed and lose money.
I don't understand the situation here. Is the assumption of a group of
miners suddenly switching (for example, they realise that they didn't
intend to support the new rules)?
Again, as I make no assumption about the mining power distribution,
the new chain may actually have less miner support. Without any
protection (AFAIK, for example, BIP100, 101, 102), the weaker new
chain will get 51%-attacked by the original chain constantly.
Post by Tier Nolan via bitcoin-dev
Flag block is constructed in a way that nodes with the original consensus
rules must reject. On the other hand, nodes with the new consensus rules
must reject a block if it is not a flag block while it is supposed to be.
To achieve these goals, the flag block must 1) have the hardfork bit
setting to 1, 2) include a short predetermined unique description of the
hardfork anywhere in its coinbase, and 3) follow any other rules required
by the hardfork. If these conditions are not fully satisfied, upgraded
nodes shall reject the block.
Ok, so set the bit and then include BIP-GIT-HASH of the canonical BIP on
github in the coinbase?
I guess the git hash is not known until the code is written? (correct
me if I'm wrong) As the coinbase message is consensus-critical, it
must be part of the source code and therefore you can't use any kind
of hash of the code itself (a chicken-and-egg problem)
Post by Tier Nolan via bitcoin-dev
Since it is a hard fork, the version field could be completely
re-purposed. Set the bit and add the BIP number as the lower bits in the
version field. This lets SPV clients check if they know about the hard
fork.
This may not be compatible with the other version bits voting mechanisms.
Post by Tier Nolan via bitcoin-dev
There network protocol could be updated to add getdata support for asking
for a coinbase only merkleblock. This would allow SPV clients to obtain
the coinbase.
Yes
Post by Tier Nolan via bitcoin-dev
Automatic warning system: When a flag block is found on the network, full
nodes and SPV nodes should look into its coinbase. They should alert their
users and/or stop accepting incoming transactions if it is an unknown
hardfork. It should be noted that the warning system could become a DoS
vector if the attacker is willing to give up the block reward. Therefore,
the warning may be issued only if a few blocks are built on top of the flag
block in a reasonable time frame. This will in turn increase the risk in
case of a real planned hardfork so it is up to the wallet programmers to
decide the optimal strategy. Human warning system (e.g. the emergency alert
system in Bitcoin Core) could fill the gap.
If the rule was that hard forks only take effect 100 blocks after the flag
block, then this problem is eliminated.
Emergency hard forks may still have to take effect immediately though, so
it would have to be a custom not a rule.
The flag block itself is a hardfork already and old miners will not
mine on top of the flag block. So your suggestion won't be helpful in
this situation.

To make it really meaningful, we need to consume one more bit of the
'version' field ("notice bit"). Supporting miners will turn on the
notice bit, and include a message in coinbase ("notice block"). When a
full node/SPV node find many notice blocks with the same coinbase
message, they could bet that the subsequent flag block is a legit one.
However, an attacker may still troll you by injecting an invalid flag
block after many legit notice blocks. So I'm not sure if it is worth
the added complexity.
Michael Ruddy via bitcoin-dev
2015-08-01 13:05:28 UTC
Permalink
Post by Tier Nolan via bitcoin-dev
Ok, so set the bit and then include BIP-GIT-HASH of the canonical BIP on
github in the coinbase?
I guess the git hash is not known until the code is written? (correct me if
I'm wrong) As the coinbase message is consensus-critical, it must be part of
the source code and therefore you can't use any kind of hash of the code
itself (a chicken-and-egg problem)
I took Tier's comment to mean that the commit hash (taken from git) of
the BIP for the particular hard fork being rolled out via your hard
fork bit mechanism could be used in the coinbase.
The commit for the BIP is separate from the commit for the code
changes implementing it.

By using the commit hash of the BIP, it means that the BIP cannot
specify itself what to put in the coinbase.
I'd suggest that your hard fork bit proposal allows BIP authors to
specify, within a 20 byte limit (to let them ripe160 a message or
something), the unique value to use.
This would be just as well and would allow the specific hard fork BIPs
to be updated without having to make code changes (e.g.- for simple
grammatical updates, post deployment clarification, etc...).

Perhaps more preferable, in order to keep the coinbases small, or, if
you're worried about BIP authors using duplicate values, then just
specify in your proposal that the coinbase message include
"BIP<NUMBER>" as BIP numbers are not going to be reused. All you are
trying to achieve is something that can be uniquely pattern matched.

Are the reasons for your use of the coinbase message 1) to guard
against one or more hard forks piggy-backing on another's flag block,
without prior collaboration, and 2) to have a nicer message in the
alerting system? If so, you might want to spell that out in your
proposal. At first, I was thinking that the coinbase thing was not
entirely necessary since the possibility of multiple hard forks taking
place concurrently is low. Multiple forking changes could be
coordinated to all use the same possible voting mechanisms, median
timestamp, and thus flag block. But if the concern is about competing
hard forks where two or more forks abandon the original chain and
attempt to use the same checkpoint block, then I can see why you'd
need it.

I was reading this proposal alongside another of your messages
(https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2015-July/009520.html)
proposing a version 0 or even a >1MB block in the specific case of a
block size limit hard fork. I can see the >1MB flag block creating DoS
banning problems. That's more of a practical issue than a consensus
issue due to the existing mono-culture of full nodes. So, currently a
1MB flag block is not as appealing as a version 0 or this hard fork
bit proposal. Besides the DoS ban, the >1MB proposal would mean that
older nodes do not have the chance to notice a fork is happening (for
alerting) as they would with a version 0 or hard fork bit.

A version 0 flag block would not be able to contain any transactions
unless the flag block version was assumed to be that of the most
recent version at the time. This is because we'd want to enforce the
rules of the previous soft forks that had been locked in. Other than
that, the version 0 idea seems pretty similar to the hard fork bit
proposal because you need more context than just the block itself to
determine if it's a valid flag block (and this extra context is
probably not great, but I don't have a better idea right now).

Were those reasons part of why you progressed towards this hard fork
bit proposal?
jl2012 via bitcoin-dev
2015-08-01 20:23:23 UTC
Permalink
Although the chance is very slim, it is possible to have multiple
hardforks sharing the same flag block. For example, different proposals
may decide the flag time based on voting result and 2 proposals may have
the same flag time just by chance. The coinbase message is to preclude
any potential ambiguity. It also provides additional info to warning
system of non-upgrading nodes.

If we are pretty sure that there won't be other hardfork proposal at the
same time, the coinbase message may not be necessary. With some prior
collaboration, this may also be avoided (e.g. no sharing flag block
allowed as consensus rules of the hardforks)

The "version 0" idea is not compatible with the version bits voting
system, so I have this hardfork bit BIP after thinking more carefully.
Otherwise they are technically similar.
Post by Michael Ruddy via bitcoin-dev
Post by Tier Nolan via bitcoin-dev
Ok, so set the bit and then include BIP-GIT-HASH of the canonical BIP on
github in the coinbase?
I guess the git hash is not known until the code is written? (correct me if
I'm wrong) As the coinbase message is consensus-critical, it must be part of
the source code and therefore you can't use any kind of hash of the code
itself (a chicken-and-egg problem)
I took Tier's comment to mean that the commit hash (taken from git) of
the BIP for the particular hard fork being rolled out via your hard
fork bit mechanism could be used in the coinbase.
The commit for the BIP is separate from the commit for the code
changes implementing it.
By using the commit hash of the BIP, it means that the BIP cannot
specify itself what to put in the coinbase.
I'd suggest that your hard fork bit proposal allows BIP authors to
specify, within a 20 byte limit (to let them ripe160 a message or
something), the unique value to use.
This would be just as well and would allow the specific hard fork BIPs
to be updated without having to make code changes (e.g.- for simple
grammatical updates, post deployment clarification, etc...).
Perhaps more preferable, in order to keep the coinbases small, or, if
you're worried about BIP authors using duplicate values, then just
specify in your proposal that the coinbase message include
"BIP<NUMBER>" as BIP numbers are not going to be reused. All you are
trying to achieve is something that can be uniquely pattern matched.
Are the reasons for your use of the coinbase message 1) to guard
against one or more hard forks piggy-backing on another's flag block,
without prior collaboration, and 2) to have a nicer message in the
alerting system? If so, you might want to spell that out in your
proposal. At first, I was thinking that the coinbase thing was not
entirely necessary since the possibility of multiple hard forks taking
place concurrently is low. Multiple forking changes could be
coordinated to all use the same possible voting mechanisms, median
timestamp, and thus flag block. But if the concern is about competing
hard forks where two or more forks abandon the original chain and
attempt to use the same checkpoint block, then I can see why you'd
need it.
I was reading this proposal alongside another of your messages
(https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2015-July/009520.html)
proposing a version 0 or even a >1MB block in the specific case of a
block size limit hard fork. I can see the >1MB flag block creating DoS
banning problems. That's more of a practical issue than a consensus
issue due to the existing mono-culture of full nodes. So, currently a
1MB flag block is not as appealing as a version 0 or this hard fork
bit proposal. Besides the DoS ban, the >1MB proposal would mean that
older nodes do not have the chance to notice a fork is happening (for
alerting) as they would with a version 0 or hard fork bit.
A version 0 flag block would not be able to contain any transactions
unless the flag block version was assumed to be that of the most
recent version at the time. This is because we'd want to enforce the
rules of the previous soft forks that had been locked in. Other than
that, the version 0 idea seems pretty similar to the hard fork bit
proposal because you need more context than just the block itself to
determine if it's a valid flag block (and this extra context is
probably not great, but I don't have a better idea right now).
Were those reasons part of why you progressed towards this hard fork
bit proposal?
Michael Ruddy via bitcoin-dev
2015-08-02 10:53:20 UTC
Permalink
I think your "hardfork bit" proposal is clever.
It addresses the particular valid concern of re-org facing users of a
fork that a small/near/fluctuating majority, or less, of mining power
supported.
While the "economic majority" argument may be enough on its own in
that case, it still has some aspect of being a hand wave.
This proposal adds support to those economic actors, which makes it
easier for them to switch if/when they choose. That is, it provides a
good fallback mechanism that allows them to make a decision and say,
"we're doing this".
Do you have the latest version up on github, or someplace where it
would be easier to collaborate on the specific text?
Although the chance is very slim, it is possible to have multiple hardforks
sharing the same flag block. For example, different proposals may decide the
flag time based on voting result and 2 proposals may have the same flag time
just by chance. The coinbase message is to preclude any potential ambiguity.
It also provides additional info to warning system of non-upgrading nodes.
If we are pretty sure that there won't be other hardfork proposal at the
same time, the coinbase message may not be necessary. With some prior
collaboration, this may also be avoided (e.g. no sharing flag block allowed
as consensus rules of the hardforks)
The "version 0" idea is not compatible with the version bits voting system,
so I have this hardfork bit BIP after thinking more carefully. Otherwise
they are technically similar.
jl2012 via bitcoin-dev
2015-08-03 08:54:27 UTC
Permalink
I have put it on the github:
https://github.com/jl2012/bips/blob/master/hardforkbit.mediawiki

I removed the specification of coinbase message to make it simpler.
Instead, it requires that a flag block must not be shared by multiple
hardfork proposals.

I'm not sure whether it is a Standard, Informational, or Process BIP

I'm also thinking whether we should call it "hardfork bit", "hardfork
flag", or with other name.
Post by Michael Ruddy via bitcoin-dev
I think your "hardfork bit" proposal is clever.
It addresses the particular valid concern of re-org facing users of a
fork that a small/near/fluctuating majority, or less, of mining power
supported.
While the "economic majority" argument may be enough on its own in
that case, it still has some aspect of being a hand wave.
This proposal adds support to those economic actors, which makes it
easier for them to switch if/when they choose. That is, it provides a
good fallback mechanism that allows them to make a decision and say,
"we're doing this".
Do you have the latest version up on github, or someplace where it
would be easier to collaborate on the specific text?
Gareth Williams via bitcoin-dev
2015-07-24 01:17:34 UTC
Permalink
Post by Tier Nolan via bitcoin-dev
I don't understand the situation here. Is the assumption of a group of
miners suddenly switching (for example, they realise that they didn't
intend to support the new rules)?
Or they're economically rational miners, and a large difficulty decrease on the original chain, without an equally large decrease in the value of original chain tokens, has made it profitable to switch?

It's dangerous to assume all miners will continue to support the side they have initially signaled. They're only invested in the chain they mine for a short time (until coinbase maturity.)

If both sides if a fork retain value you'd expect mining to redistribute itself WRT short term profitability at every difficulty adjustment, irrespective of initially signaled support for the fork.
Continue reading on narkive:
Loading...