Discussion:
New difficulty algorithm needed for SegWit2x fork? (reformatted text)
Add Reply
Scott Roberts via bitcoin-dev
2017-10-09 22:57:32 UTC
Reply
Permalink
Raw Message
Sorry, my previous email did not have the plain text I intended.

Background:

The bitcoin difficulty algorithm does not seem to be a good one. If there
is a fork due to miners seeking maximum profit without due regard to
security, users, and nodes, the "better" coin could end up being the
minority chain. If 90% of hashrate is really going to at least initially go
towards using SegWit2x, BTC would face 10x delays in confirmations
until the next difficulty adjustment, negatively affecting its price relative
to BTC1, causing further delays from even more miner abandonment
(until the next adjustment). The 10% miners remaining on BTC do not
inevitably lose by staying to endure 10x delays because they have 10x
less competition, and the same situation applies to BTC1 miners. If the
prices are the same and stable, all seems well for everyone, other things
aside. But if the BTC price does not fall to reflect the decreased hashrate,
he situation seems to be a big problem for both coins: BTC1 miners will
jump back to BTC when the difficulty adjustment occurs, initiating a
potentially never-ending oscillation between the two coins, potentially
worse than what BCH is experiencing. They will not issue coins too fast
like BCH because that is a side effect of the asymmetry in BCH's rise and
fall algorithm.

Solution:

Hard fork to implement a new difficulty algorithm that uses a simple rolling
average with a much smaller window. Many small coins have done this as
a way to stop big miners from coming on and then suddenly leaving, leaving
constant miners stuck with a high difficulty for the rest of a (long) averaging
window. Even better, adjust the reward based on recent solvetimes to
motivate more mining (or less) if the solvetimes are too slow (or too fast).
This will keep keep coin issuance rate perfectly on schedule with real time.

I recommend the following for Bitcoin, as fast, simple, and better than any
other difficulty algorithm I'm aware of. This is the result of a lot of work the
past year.

=== Begin difficulty algorithm ===
# Zawy v6 difficulty algorithm (modified for bitcoin)
# Unmodified Zawy v6 for alt coins:
# http://zawy1.blogspot.com/2017/07/best-difficulty-algorithm-zawy-v1b.html
# All my failed attempts at something better:
# https://github.com/seredat/karbowanec/commit/231db5270acb2e673a641a1800be910ce345668a
#
# Keep negative solvetimes to correct bad timestamps.
# Do not be tempted to use:
# next_D = sum(last N Ds) * T / [max(last N TSs) - min(last N TSs];
# ST= Solvetime, TS = timestamp

# set constants until next hard fork:

T=600; # coin's TargetSolvetime
N=30; # Averaging window. Smoother than N=15, faster response than N=60.
X=5;
limit = X^(2/N); # limit rise and fall in case of timestamp manipulation
adjust = 1/(1+0.67/N); # keeps avg solvetime on track

# begin difficulty algorithm

avg_ST=0; avg_D=0;
for ( i=height; i > height-N; i--) { # go through N most recent blocks
avg_ST += (TS[i] - TS[i-1]) / N;
avg_D += D[i]/N;
}
avg_ST = T*limit if avg_ST > T*limit;
avg_ST = T/limit if avg_ST < T/limit;

next_D = avg_D * T / avg_ST * adjust;

# Tim Olsen suggested changing reward to protect against hash attacks.
# Karbowanek coin suggested something similar.
# I could not find anything better than the simplest idea below.
# It was a great surprise that coin issuance rate came out perfect.
# BaseReward = coins per block

next_reward = BaseReward * avg_ST / T;

======= end algo ====

Due to the limit and keeping negative solvetimes in a true average,
timestamp errors resulting in negative solvetimes are corrected in the next
block. Otherwise, one would need to do like Zcash and cause a 5-block
delay in the response by resorting to the median of past 11 blocks (MPT)
as the most recent timestamp, offsetting the timestamps from their
corresponding difficulties by 5 blocks. (it does not cause an averaging
problem, but it does cause a 5-block delay in the response.)
Mark Friedenbach via bitcoin-dev
2017-10-10 02:19:11 UTC
Reply
Permalink
Raw Message
The problem of fast acting but non vulnerable difficulty adjustment algorithms is interesting. I would certainly like to see this space further explored, and even have some ideas myself.

However without commenting on the technical merits of this specific proposal, I think it must be said upfront that the stated goal is not good. The largest technical concern (ignoring governance) over B2X is that it is a rushed, poorly reviewed hard fork. Hard forks should not be rushed, and they should receive more than the usual level of expert and community review.

I¡¯m that light, doing an even more rushed hard fork on an even newer idea with even less review would be hypocritical at best. I would suggest reframing as a hardfork wishlist research problem for the next properly planned hard fork, if one occurs. You might also find the hardfork research group a more accommodating venue for this discussion:

https://bitcoinhardforkresearch.github.io/

> On Oct 9, 2017, at 3:57 PM, Scott Roberts via bitcoin-dev <bitcoin-***@lists.linuxfoundation.org> wrote:
>
> Sorry, my previous email did not have the plain text I intended.
>
> Background:
>
> The bitcoin difficulty algorithm does not seem to be a good one. If there
> is a fork due to miners seeking maximum profit without due regard to
> security, users, and nodes, the "better" coin could end up being the
> minority chain. If 90% of hashrate is really going to at least initially go
> towards using SegWit2x, BTC would face 10x delays in confirmations
> until the next difficulty adjustment, negatively affecting its price relative
> to BTC1, causing further delays from even more miner abandonment
> (until the next adjustment). The 10% miners remaining on BTC do not
> inevitably lose by staying to endure 10x delays because they have 10x
> less competition, and the same situation applies to BTC1 miners. If the
> prices are the same and stable, all seems well for everyone, other things
> aside. But if the BTC price does not fall to reflect the decreased hashrate,
> he situation seems to be a big problem for both coins: BTC1 miners will
> jump back to BTC when the difficulty adjustment occurs, initiating a
> potentially never-ending oscillation between the two coins, potentially
> worse than what BCH is experiencing. They will not issue coins too fast
> like BCH because that is a side effect of the asymmetry in BCH's rise and
> fall algorithm.
>
> Solution:
>
> Hard fork to implement a new difficulty algorithm that uses a simple rolling
> average with a much smaller window. Many small coins have done this as
> a way to stop big miners from coming on and then suddenly leaving, leaving
> constant miners stuck with a high difficulty for the rest of a (long) averaging
> window. Even better, adjust the reward based on recent solvetimes to
> motivate more mining (or less) if the solvetimes are too slow (or too fast).
> This will keep keep coin issuance rate perfectly on schedule with real time.
>
> I recommend the following for Bitcoin, as fast, simple, and better than any
> other difficulty algorithm I'm aware of. This is the result of a lot of work the
> past year.
>
> === Begin difficulty algorithm ===
> # Zawy v6 difficulty algorithm (modified for bitcoin)
> # Unmodified Zawy v6 for alt coins:
> # http://zawy1.blogspot.com/2017/07/best-difficulty-algorithm-zawy-v1b.html
> # All my failed attempts at something better:
> # https://github.com/seredat/karbowanec/commit/231db5270acb2e673a641a1800be910ce345668a
> #
> # Keep negative solvetimes to correct bad timestamps.
> # Do not be tempted to use:
> # next_D = sum(last N Ds) * T / [max(last N TSs) - min(last N TSs];
> # ST= Solvetime, TS = timestamp
>
> # set constants until next hard fork:
>
> T=600; # coin's TargetSolvetime
> N=30; # Averaging window. Smoother than N=15, faster response than N=60.
> X=5;
> limit = X^(2/N); # limit rise and fall in case of timestamp manipulation
> adjust = 1/(1+0.67/N); # keeps avg solvetime on track
>
> # begin difficulty algorithm
>
> avg_ST=0; avg_D=0;
> for ( i=height; i > height-N; i--) { # go through N most recent blocks
> avg_ST += (TS[i] - TS[i-1]) / N;
> avg_D += D[i]/N;
> }
> avg_ST = T*limit if avg_ST > T*limit;
> avg_ST = T/limit if avg_ST < T/limit;
>
> next_D = avg_D * T / avg_ST * adjust;
>
> # Tim Olsen suggested changing reward to protect against hash attacks.
> # Karbowanek coin suggested something similar.
> # I could not find anything better than the simplest idea below.
> # It was a great surprise that coin issuance rate came out perfect.
> # BaseReward = coins per block
>
> next_reward = BaseReward * avg_ST / T;
>
> ======= end algo ====
>
> Due to the limit and keeping negative solvetimes in a true average,
> timestamp errors resulting in negative solvetimes are corrected in the next
> block. Otherwise, one would need to do like Zcash and cause a 5-block
> delay in the response by resorting to the median of past 11 blocks (MPT)
> as the most recent timestamp, offsetting the timestamps from their
> corresponding difficulties by 5 blocks. (it does not cause an averaging
> problem, but it does cause a 5-block delay in the response.)
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-***@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
Ben Kloester via bitcoin-dev
2017-10-10 02:57:23 UTC
Reply
Permalink
Raw Message
Is there a contingency plan in the case that the incumbent chain following
the Bitcoin Core consensus rules comes under 51% attack?

If the 2x fork really does have the support of >66% of miners (which
remains to be seen), it seems like they'd have spare capacity to perform
such an attack. In which case, a rushed hard fork might be the only option
to guarantee the survival of the chain, would it not?

I'm aware of Luke's work on BitcoinHardfork
<https://github.com/BitcoinHardfork>, but not aware of whether this has
actually been tested in the field by anyone - ie whether anyone actually
has even run the code much / created a testnet. What are the options for an
emergency hard fork, and how much testing has each seen?

*Ben Kloester*

On 10 October 2017 at 13:19, Mark Friedenbach via bitcoin-dev <
bitcoin-***@lists.linuxfoundation.org> wrote:

> The problem of fast acting but non vulnerable difficulty adjustment
> algorithms is interesting. I would certainly like to see this space further
> explored, and even have some ideas myself.
>
> However without commenting on the technical merits of this specific
> proposal, I think it must be said upfront that the stated goal is not good.
> The largest technical concern (ignoring governance) over B2X is that it is
> a rushed, poorly reviewed hard fork. Hard forks should not be rushed, and
> they should receive more than the usual level of expert and community
> review.
>
> I’m that light, doing an even more rushed hard fork on an even newer idea
> with even less review would be hypocritical at best. I would suggest
> reframing as a hardfork wishlist research problem for the next properly
> planned hard fork, if one occurs. You might also find the hardfork research
> group a more accommodating venue for this discussion:
>
> https://bitcoinhardforkresearch.github.io/
>
> On Oct 9, 2017, at 3:57 PM, Scott Roberts via bitcoin-dev <
> bitcoin-***@lists.linuxfoundation.org> wrote:
>
> Sorry, my previous email did not have the plain text I intended.
>
> Background:
>
> The bitcoin difficulty algorithm does not seem to be a good one. If there
> is a fork due to miners seeking maximum profit without due regard to
> security, users, and nodes, the "better" coin could end up being the
> minority chain. If 90% of hashrate is really going to at least initially
> go
> towards using SegWit2x, BTC would face 10x delays in confirmations
> until the next difficulty adjustment, negatively affecting its price
> relative
> to BTC1, causing further delays from even more miner abandonment
> (until the next adjustment). The 10% miners remaining on BTC do not
> inevitably lose by staying to endure 10x delays because they have 10x
> less competition, and the same situation applies to BTC1 miners. If the
> prices are the same and stable, all seems well for everyone, other things
> aside. But if the BTC price does not fall to reflect the decreased
> hashrate,
> he situation seems to be a big problem for both coins: BTC1 miners will
> jump back to BTC when the difficulty adjustment occurs, initiating a
> potentially never-ending oscillation between the two coins, potentially
> worse than what BCH is experiencing. They will not issue coins too fast
> like BCH because that is a side effect of the asymmetry in BCH's rise and
> fall algorithm.
>
> Solution:
>
> Hard fork to implement a new difficulty algorithm that uses a simple
> rolling
> average with a much smaller window. Many small coins have done this as
> a way to stop big miners from coming on and then suddenly leaving, leaving
> constant miners stuck with a high difficulty for the rest of a (long)
> averaging
> window. Even better, adjust the reward based on recent solvetimes to
> motivate more mining (or less) if the solvetimes are too slow (or too
> fast).
> This will keep keep coin issuance rate perfectly on schedule with real
> time.
>
> I recommend the following for Bitcoin, as fast, simple, and better than
> any
> other difficulty algorithm I'm aware of. This is the result of a lot of
> work the
> past year.
>
> === Begin difficulty algorithm ===
> # Zawy v6 difficulty algorithm (modified for bitcoin)
> # Unmodified Zawy v6 for alt coins:
> # http://zawy1.blogspot.com/2017/07/best-difficulty-
> algorithm-zawy-v1b.html
> # All my failed attempts at something better:
> # https://github.com/seredat/karbowanec/commit/
> 231db5270acb2e673a641a1800be910ce345668a
> #
> # Keep negative solvetimes to correct bad timestamps.
> # Do not be tempted to use:
> # next_D = sum(last N Ds) * T / [max(last N TSs) - min(last N TSs];
> # ST= Solvetime, TS = timestamp
>
> # set constants until next hard fork:
>
> T=600; # coin's TargetSolvetime
> N=30; # Averaging window. Smoother than N=15, faster response than N=60.
> X=5;
> limit = X^(2/N); # limit rise and fall in case of timestamp manipulation
> adjust = 1/(1+0.67/N); # keeps avg solvetime on track
>
> # begin difficulty algorithm
>
> avg_ST=0; avg_D=0;
> for ( i=height; i > height-N; i--) { # go through N most recent blocks
> avg_ST += (TS[i] - TS[i-1]) / N;
> avg_D += D[i]/N;
> }
> avg_ST = T*limit if avg_ST > T*limit;
> avg_ST = T/limit if avg_ST < T/limit;
>
> next_D = avg_D * T / avg_ST * adjust;
>
> # Tim Olsen suggested changing reward to protect against hash attacks.
> # Karbowanek coin suggested something similar.
> # I could not find anything better than the simplest idea below.
> # It was a great surprise that coin issuance rate came out perfect.
> # BaseReward = coins per block
>
> next_reward = BaseReward * avg_ST / T;
>
> ======= end algo ====
>
> Due to the limit and keeping negative solvetimes in a true average,
> timestamp errors resulting in negative solvetimes are corrected in the
> next
> block. Otherwise, one would need to do like Zcash and cause a 5-block
> delay in the response by resorting to the median of past 11 blocks (MPT)
> as the most recent timestamp, offsetting the timestamps from their
> corresponding difficulties by 5 blocks. (it does not cause an averaging
> problem, but it does cause a 5-block delay in the response.)
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-***@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-***@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
>
greg misiorek via bitcoin-dev
2017-10-10 10:34:35 UTC
Reply
Permalink
Raw Message
Yes, I agree. Hard forks should be as much scrutinized by fellow bitcoiners, i.e. developers and holders and not only rushed by miners or some other investment gurus, whose incentives are not entirely clear, to remain as decentralized as economically possible.

________________________________
From: bitcoin-dev-***@lists.linuxfoundation.org <bitcoin-dev-***@lists.linuxfoundation.org> on behalf of Mark Friedenbach via bitcoin-dev <bitcoin-***@lists.linuxfoundation.org>
Sent: Monday, October 9, 2017 10:19 PM
To: Scott Roberts; Bitcoin Protocol Discussion
Subject: Re: [bitcoin-dev] New difficulty algorithm needed for SegWit2x fork? (reformatted text)

The problem of fast acting but non vulnerable difficulty adjustment algorithms is interesting. I would certainly like to see this space further explored, and even have some ideas myself.

However without commenting on the technical merits of this specific proposal, I think it must be said upfront that the stated goal is not good. The largest technical concern (ignoring governance) over B2X is that it is a rushed, poorly reviewed hard fork. Hard forks should not be rushed, and they should receive more than the usual level of expert and community review.

I’m that light, doing an even more rushed hard fork on an even newer idea with even less review would be hypocritical at best. I would suggest reframing as a hardfork wishlist research problem for the next properly planned hard fork, if one occurs. You might also find the hardfork research group a more accommodating venue for this discussion:

https://bitcoinhardforkresearch.github.io/
Welcome - Bitcoin Hard Fork Research<https://bitcoinhardforkresearch.github.io/>
bitcoinhardforkresearch.github.io
Bitcoin Hard Fork Research This website will be updated with relevant ongoing information about Bitcoin hard fork research. Resources: BIP-MMHF, draft patch last ...



On Oct 9, 2017, at 3:57 PM, Scott Roberts via bitcoin-dev <bitcoin-***@lists.linuxfoundation.org<mailto:bitcoin-***@lists.linuxfoundation.org>> wrote:

Sorry, my previous email did not have the plain text I intended.

Background:

The bitcoin difficulty algorithm does not seem to be a good one. If there
is a fork due to miners seeking maximum profit without due regard to
security, users, and nodes, the "better" coin could end up being the
minority chain. If 90% of hashrate is really going to at least initially go
towards using SegWit2x, BTC would face 10x delays in confirmations
until the next difficulty adjustment, negatively affecting its price relative
to BTC1, causing further delays from even more miner abandonment
(until the next adjustment). The 10% miners remaining on BTC do not
inevitably lose by staying to endure 10x delays because they have 10x
less competition, and the same situation applies to BTC1 miners. If the
prices are the same and stable, all seems well for everyone, other things
aside. But if the BTC price does not fall to reflect the decreased hashrate,
he situation seems to be a big problem for both coins: BTC1 miners will
jump back to BTC when the difficulty adjustment occurs, initiating a
potentially never-ending oscillation between the two coins, potentially
worse than what BCH is experiencing. They will not issue coins too fast
like BCH because that is a side effect of the asymmetry in BCH's rise and
fall algorithm.

Solution:

Hard fork to implement a new difficulty algorithm that uses a simple rolling
average with a much smaller window. Many small coins have done this as
a way to stop big miners from coming on and then suddenly leaving, leaving
constant miners stuck with a high difficulty for the rest of a (long) averaging
window. Even better, adjust the reward based on recent solvetimes to
motivate more mining (or less) if the solvetimes are too slow (or too fast).
This will keep keep coin issuance rate perfectly on schedule with real time.

I recommend the following for Bitcoin, as fast, simple, and better than any
other difficulty algorithm I'm aware of. This is the result of a lot of work the
past year.

=== Begin difficulty algorithm ===
# Zawy v6 difficulty algorithm (modified for bitcoin)
# Unmodified Zawy v6 for alt coins:
# http://zawy1.blogspot.com/2017/07/best-difficulty-algorithm-zawy-v1b.html
# All my failed attempts at something better:
# https://github.com/seredat/karbowanec/commit/231db5270acb2e673a641a1800be910ce345668a
#
# Keep negative solvetimes to correct bad timestamps.
# Do not be tempted to use:
# next_D = sum(last N Ds) * T / [max(last N TSs) - min(last N TSs];
# ST= Solvetime, TS = timestamp

# set constants until next hard fork:

T=600; # coin's TargetSolvetime
N=30; # Averaging window. Smoother than N=15, faster response than N=60.
X=5;
limit = X^(2/N); # limit rise and fall in case of timestamp manipulation
adjust = 1/(1+0.67/N); # keeps avg solvetime on track

# begin difficulty algorithm

avg_ST=0; avg_D=0;
for ( i=height; i > height-N; i--) { # go through N most recent blocks
avg_ST += (TS[i] - TS[i-1]) / N;
avg_D += D[i]/N;
}
avg_ST = T*limit if avg_ST > T*limit;
avg_ST = T/limit if avg_ST < T/limit;

next_D = avg_D * T / avg_ST * adjust;

# Tim Olsen suggested changing reward to protect against hash attacks.
# Karbowanek coin suggested something similar.
# I could not find anything better than the simplest idea below.
# It was a great surprise that coin issuance rate came out perfect.
# BaseReward = coins per block

next_reward = BaseReward * avg_ST / T;

======= end algo ====

Due to the limit and keeping negative solvetimes in a true average,
timestamp errors resulting in negative solvetimes are corrected in the next
block. Otherwise, one would need to do like Zcash and cause a 5-block
delay in the response by resorting to the median of past 11 blocks (MPT)
as the most recent timestamp, offsetting the timestamps from their
corresponding difficulties by 5 blocks. (it does not cause an averaging
problem, but it does cause a 5-block delay in the response.)
Ben Kloester via bitcoin-dev
2017-10-11 01:44:52 UTC
Reply
Permalink
Raw Message
Mark, this seems an awful lot like an answer of "no", to my question "Is
there a contingency plan in the case that the incumbent chain following the
Bitcoin Core consensus rules comes under 51% attack?" - is this a correct
interpretation?

In fact, beyond a no, it seems like a "no, and I disagree with the idea of
creating one".

So if Bitcoin comes under successful 51%, the project, in your vision, has
simply failed?

*Ben Kloester*

On 10 October 2017 at 13:19, Mark Friedenbach via bitcoin-dev <
bitcoin-***@lists.linuxfoundation.org> wrote:

> The problem of fast acting but non vulnerable difficulty adjustment
> algorithms is interesting. I would certainly like to see this space further
> explored, and even have some ideas myself.
>
> However without commenting on the technical merits of this specific
> proposal, I think it must be said upfront that the stated goal is not good.
> The largest technical concern (ignoring governance) over B2X is that it is
> a rushed, poorly reviewed hard fork. Hard forks should not be rushed, and
> they should receive more than the usual level of expert and community
> review.
>
> I’m that light, doing an even more rushed hard fork on an even newer idea
> with even less review would be hypocritical at best. I would suggest
> reframing as a hardfork wishlist research problem for the next properly
> planned hard fork, if one occurs. You might also find the hardfork research
> group a more accommodating venue for this discussion:
>
> https://bitcoinhardforkresearch.github.io/
>
> On Oct 9, 2017, at 3:57 PM, Scott Roberts via bitcoin-dev <
> bitcoin-***@lists.linuxfoundation.org> wrote:
>
> Sorry, my previous email did not have the plain text I intended.
>
> Background:
>
> The bitcoin difficulty algorithm does not seem to be a good one. If there
> is a fork due to miners seeking maximum profit without due regard to
> security, users, and nodes, the "better" coin could end up being the
> minority chain. If 90% of hashrate is really going to at least initially
> go
> towards using SegWit2x, BTC would face 10x delays in confirmations
> until the next difficulty adjustment, negatively affecting its price
> relative
> to BTC1, causing further delays from even more miner abandonment
> (until the next adjustment). The 10% miners remaining on BTC do not
> inevitably lose by staying to endure 10x delays because they have 10x
> less competition, and the same situation applies to BTC1 miners. If the
> prices are the same and stable, all seems well for everyone, other things
> aside. But if the BTC price does not fall to reflect the decreased
> hashrate,
> he situation seems to be a big problem for both coins: BTC1 miners will
> jump back to BTC when the difficulty adjustment occurs, initiating a
> potentially never-ending oscillation between the two coins, potentially
> worse than what BCH is experiencing. They will not issue coins too fast
> like BCH because that is a side effect of the asymmetry in BCH's rise and
> fall algorithm.
>
> Solution:
>
> Hard fork to implement a new difficulty algorithm that uses a simple
> rolling
> average with a much smaller window. Many small coins have done this as
> a way to stop big miners from coming on and then suddenly leaving, leaving
> constant miners stuck with a high difficulty for the rest of a (long)
> averaging
> window. Even better, adjust the reward based on recent solvetimes to
> motivate more mining (or less) if the solvetimes are too slow (or too
> fast).
> This will keep keep coin issuance rate perfectly on schedule with real
> time.
>
> I recommend the following for Bitcoin, as fast, simple, and better than
> any
> other difficulty algorithm I'm aware of. This is the result of a lot of
> work the
> past year.
>
> === Begin difficulty algorithm ===
> # Zawy v6 difficulty algorithm (modified for bitcoin)
> # Unmodified Zawy v6 for alt coins:
> # http://zawy1.blogspot.com/2017/07/best-difficulty-
> algorithm-zawy-v1b.html
> # All my failed attempts at something better:
> # https://github.com/seredat/karbowanec/commit/
> 231db5270acb2e673a641a1800be910ce345668a
> #
> # Keep negative solvetimes to correct bad timestamps.
> # Do not be tempted to use:
> # next_D = sum(last N Ds) * T / [max(last N TSs) - min(last N TSs];
> # ST= Solvetime, TS = timestamp
>
> # set constants until next hard fork:
>
> T=600; # coin's TargetSolvetime
> N=30; # Averaging window. Smoother than N=15, faster response than N=60.
> X=5;
> limit = X^(2/N); # limit rise and fall in case of timestamp manipulation
> adjust = 1/(1+0.67/N); # keeps avg solvetime on track
>
> # begin difficulty algorithm
>
> avg_ST=0; avg_D=0;
> for ( i=height; i > height-N; i--) { # go through N most recent blocks
> avg_ST += (TS[i] - TS[i-1]) / N;
> avg_D += D[i]/N;
> }
> avg_ST = T*limit if avg_ST > T*limit;
> avg_ST = T/limit if avg_ST < T/limit;
>
> next_D = avg_D * T / avg_ST * adjust;
>
> # Tim Olsen suggested changing reward to protect against hash attacks.
> # Karbowanek coin suggested something similar.
> # I could not find anything better than the simplest idea below.
> # It was a great surprise that coin issuance rate came out perfect.
> # BaseReward = coins per block
>
> next_reward = BaseReward * avg_ST / T;
>
> ======= end algo ====
>
> Due to the limit and keeping negative solvetimes in a true average,
> timestamp errors resulting in negative solvetimes are corrected in the
> next
> block. Otherwise, one would need to do like Zcash and cause a 5-block
> delay in the response by resorting to the median of past 11 blocks (MPT)
> as the most recent timestamp, offsetting the timestamps from their
> corresponding difficulties by 5 blocks. (it does not cause an averaging
> problem, but it does cause a 5-block delay in the response.)
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-***@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-***@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
>
ZmnSCPxj via bitcoin-dev
2017-10-11 02:48:13 UTC
Reply
Permalink
Raw Message
Good morning Ben,

I am not Mark, and I am nowhere near being a true Core developer yet, but I would like to point out that even under a 51% attack, there is a practical limit to the number of blocks that can be orphaned. It would still take years to rewrite history from the Genesis block, for instance.

What little data we have (BT1 / BT2 price ratio on BitFinex) suggests that tokens solely on the 2X chain will not be valued as highly as tokens solely on the Core chain. As miners generate tokens that are only for a specific chain, they will have higher incentive to gain tokens on the Core chain rather than the 2X chain.

As is commonly said, hodling is free, whereas mining is not. Hodlers have much greater power in hardfork situations than miners have: simply by selling their tokens on the 2X chain and not in the Core chain, hodlers can impose economic disincentives for mining of the 2X chain.

Miners can switch to BCH, but that is valued even less than BT2 tokens are, and thus even less attractive to mine on.

We should also pay attention, that BCH changed its difficulty algorithm, and it is often considered to be to its detriment due to sudden hashpower oscillations on that chain. We should be wary of difficulty algorithm changes, as it is the difficulty which determines the security of the chain.

--

If we attempt to deploy a difficulty change, that is a hardfork, and hodlers will be divided on this situation. Some will sell the tokens on the difficulty-change hardfork, some will sell the tokens on the non-difficulty-change hardfork. Thus the economic punishment for mining the 2X chain will be diluted due to the introduction of the difficulty-change hardfork, due to splitting of the hodler base that passes judgment over development.

Thus, strategy-wise, it is better to not hardfork (whether difficulty adjustment, PoW change, or so on) in response to a contentious hardfork, as hodlers can remain united against or for the contentious hardfork. Instead, it is better to let the market decide, which automatically imposes economic sanctions on miners who choose against the market's decision. Thus, it is better to simply let 2X die under the hands of our benevolent hodlers.

Later, when it is obvious which fate is sealed, we can reconsider such changes (difficulty adjustment, PoW change, block size) when things are calmer. However, such changes cannot be safely done in response to a contentious hardfork.

--

If indeed the Core chain is eradicated, then Bitcoin indeed has failed and I would very much rather sell my hodlings and find some other means to amuse myself.

Regards,
ZmnSCPxj
Mark Friedenbach via bitcoin-dev
2017-10-11 04:08:52 UTC
Reply
Permalink
Raw Message
You phrase the question as if gdeploying a hard fork to bitcoinh would protect the bitcoin chain from the attack. But thatfs not what happens. If you are hard forking from the perspective of deployed nodes, you are an different ledger, regardless of circumstance or who did it. Instead of there being one altcoin fighting to take hashpower from bitcoin, therefd now be 2. It is not at all obvious to me that this would be a better outcome.

If that isnft reason enough, changing the difficulty adjustment algorithm doesnft solve the underlying issue\hashpower not being aligned with usersf (or even its ownersf) interests. Propose a fix to the underlying cause and that might be worth considering, if it passes peer review. But without that youfd just be making the state of affairs arguably worse.

And so yes, *if* this incentive problem canft be solved, and the unaltered bitcoin chain dies from disuse after suffering a hashpower attack, especially a centrally and/or purposefully instigated one, then bitcoin would be failed a failed project.

The thesis (and value proposition) of bitcoin is that a particular category of economic incentives can be used to solve the problem of creating a secure trustess ledger. If those incentives failed, then he thesis of bitcoin would have been experimentally falsified, yes. Maybe the incentives can be made better to save the project, but wefd have to fix the source of the problem not the symptoms.

> On Oct 10, 2017, at 6:44 PM, Ben Kloester <***@gmail.com> wrote:
>
> Mark, this seems an awful lot like an answer of "no", to my question "Is there a contingency plan in the case that the incumbent chain following the Bitcoin Core consensus rules comes under 51% attack?" - is this a correct interpretation?
>
> In fact, beyond a no, it seems like a "no, and I disagree with the idea of creating one".
>
> So if Bitcoin comes under successful 51%, the project, in your vision, has simply failed?
>
> Ben Kloester
>
>
>> On 10 October 2017 at 13:19, Mark Friedenbach via bitcoin-dev <bitcoin-***@lists.linuxfoundation.org> wrote:
>> The problem of fast acting but non vulnerable difficulty adjustment algorithms is interesting. I would certainly like to see this space further explored, and even have some ideas myself.
>>
>> However without commenting on the technical merits of this specific proposal, I think it must be said upfront that the stated goal is not good. The largest technical concern (ignoring governance) over B2X is that it is a rushed, poorly reviewed hard fork. Hard forks should not be rushed, and they should receive more than the usual level of expert and community review.
>>
>> Ifm that light, doing an even more rushed hard fork on an even newer idea with even less review would be hypocritical at best. I would suggest reframing as a hardfork wishlist research problem for the next properly planned hard fork, if one occurs. You might also find the hardfork research group a more accommodating venue for this discussion:
>>
>> https://bitcoinhardforkresearch.github.io/
>>
>>> On Oct 9, 2017, at 3:57 PM, Scott Roberts via bitcoin-dev <bitcoin-***@lists.linuxfoundation.org> wrote:
>>>
>>> Sorry, my previous email did not have the plain text I intended.
>>>
>>> Background:
>>>
>>> The bitcoin difficulty algorithm does not seem to be a good one. If there
>>> is a fork due to miners seeking maximum profit without due regard to
>>> security, users, and nodes, the "better" coin could end up being the
>>> minority chain. If 90% of hashrate is really going to at least initially go
>>> towards using SegWit2x, BTC would face 10x delays in confirmations
>>> until the next difficulty adjustment, negatively affecting its price relative
>>> to BTC1, causing further delays from even more miner abandonment
>>> (until the next adjustment). The 10% miners remaining on BTC do not
>>> inevitably lose by staying to endure 10x delays because they have 10x
>>> less competition, and the same situation applies to BTC1 miners. If the
>>> prices are the same and stable, all seems well for everyone, other things
>>> aside. But if the BTC price does not fall to reflect the decreased hashrate,
>>> he situation seems to be a big problem for both coins: BTC1 miners will
>>> jump back to BTC when the difficulty adjustment occurs, initiating a
>>> potentially never-ending oscillation between the two coins, potentially
>>> worse than what BCH is experiencing. They will not issue coins too fast
>>> like BCH because that is a side effect of the asymmetry in BCH's rise and
>>> fall algorithm.
>>>
>>> Solution:
>>>
>>> Hard fork to implement a new difficulty algorithm that uses a simple rolling
>>> average with a much smaller window. Many small coins have done this as
>>> a way to stop big miners from coming on and then suddenly leaving, leaving
>>> constant miners stuck with a high difficulty for the rest of a (long) averaging
>>> window. Even better, adjust the reward based on recent solvetimes to
>>> motivate more mining (or less) if the solvetimes are too slow (or too fast).
>>> This will keep keep coin issuance rate perfectly on schedule with real time.
>>>
>>> I recommend the following for Bitcoin, as fast, simple, and better than any
>>> other difficulty algorithm I'm aware of. This is the result of a lot of work the
>>> past year.
>>>
>>> === Begin difficulty algorithm ===
>>> # Zawy v6 difficulty algorithm (modified for bitcoin)
>>> # Unmodified Zawy v6 for alt coins:
>>> # http://zawy1.blogspot.com/2017/07/best-difficulty-algorithm-zawy-v1b.html
>>> # All my failed attempts at something better:
>>> # https://github.com/seredat/karbowanec/commit/231db5270acb2e673a641a1800be910ce345668a
>>> #
>>> # Keep negative solvetimes to correct bad timestamps.
>>> # Do not be tempted to use:
>>> # next_D = sum(last N Ds) * T / [max(last N TSs) - min(last N TSs];
>>> # ST= Solvetime, TS = timestamp
>>>
>>> # set constants until next hard fork:
>>>
>>> T=600; # coin's TargetSolvetime
>>> N=30; # Averaging window. Smoother than N=15, faster response than N=60.
>>> X=5;
>>> limit = X^(2/N); # limit rise and fall in case of timestamp manipulation
>>> adjust = 1/(1+0.67/N); # keeps avg solvetime on track
>>>
>>> # begin difficulty algorithm
>>>
>>> avg_ST=0; avg_D=0;
>>> for ( i=height; i > height-N; i--) { # go through N most recent blocks
>>> avg_ST += (TS[i] - TS[i-1]) / N;
>>> avg_D += D[i]/N;
>>> }
>>> avg_ST = T*limit if avg_ST > T*limit;
>>> avg_ST = T/limit if avg_ST < T/limit;
>>>
>>> next_D = avg_D * T / avg_ST * adjust;
>>>
>>> # Tim Olsen suggested changing reward to protect against hash attacks.
>>> # Karbowanek coin suggested something similar.
>>> # I could not find anything better than the simplest idea below.
>>> # It was a great surprise that coin issuance rate came out perfect.
>>> # BaseReward = coins per block
>>>
>>> next_reward = BaseReward * avg_ST / T;
>>>
>>> ======= end algo ====
>>>
>>> Due to the limit and keeping negative solvetimes in a true average,
>>> timestamp errors resulting in negative solvetimes are corrected in the next
>>> block. Otherwise, one would need to do like Zcash and cause a 5-block
>>> delay in the response by resorting to the median of past 11 blocks (MPT)
>>> as the most recent timestamp, offsetting the timestamps from their
>>> corresponding difficulties by 5 blocks. (it does not cause an averaging
>>> problem, but it does cause a 5-block delay in the response.)
>>> _______________________________________________
>>> bitcoin-dev mailing list
>>> bitcoin-***@lists.linuxfoundation.org
>>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>>
>> _______________________________________________
>> bitcoin-dev mailing list
>> bitcoin-***@lists.linuxfoundation.org
>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>>
>
Moral Agent via bitcoin-dev
2017-10-11 15:28:08 UTC
Reply
Permalink
Raw Message
>Instead of there being one altcoin fighting to take hashpower from
bitcoin, there’d now be 2

Yes, there would be 2. One of which would (in the scenario we are
discussing) be producing blocks excruciatingly slowly but be the same in
all other aspects.

>changing the difficulty adjustment algorithm doesn’t solve the underlying
issue—hashpower not being aligned with users’ (or even its owners’)
interests

I disagree. Changing the difficulty adjustment algorithm could improve the
functionality of a chain, which could be an important prerequisite to using
and trading the tokens on the chain. This property could help keep the
price of the token high, which is what pressures hashpower to align with
user interests.

>And so yes, *if* this incentive problem can’t be solved, and the unaltered
bitcoin chain dies from disuse after suffering a hashpower attack,
especially a centrally and/or purposefully instigated one, then bitcoin
would be failed a failed project.

IF the incentive problem could not be resolved then Bitcoin would be a
failed project.

But here is a bit of good news.

Bitcoin has developers!

And those developers can publish a contingency plan!

And that contingency plan can be an emergency hard fork to a different
retarget algorithm.

And that emergency hard fork can gain consensus if it is broadly preferred
over the status quo.

If 90% of the hash power follows NYA, blocks are going to take 100 minutes
until difficulty adjusts after 4.5 months.

That is quite a handicap, even for a honey badger. Emergency hard fork
carries a risk, but depending on the scenario in November, it could be a
risk worth taking.

One more thing. If miners think they are going to succeed in starving the
legacy chain to death, they might be more likely to try. If they get a
credible signal that the legacy chain will react by changing the retarget
function and thereby be more likely to survive, they might feel less
committed to a strategy of starving the legacy chain. This could be
especially true if they are giving up profit for what they fervently hope
will be a short period of time.

On Wed, Oct 11, 2017 at 12:08 AM, Mark Friedenbach via bitcoin-dev <
bitcoin-***@lists.linuxfoundation.org> wrote:

> You phrase the question as if “deploying a hard fork to bitcoin” would
> protect the bitcoin chain from the attack. But that’s not what happens. If
> you are hard forking from the perspective of deployed nodes, you are an
> different ledger, regardless of circumstance or who did it. Instead of
> there being one altcoin fighting to take hashpower from bitcoin, there’d
> now be 2. It is not at all obvious to me that this would be a better
> outcome.
>
> If that isn’t reason enough, changing the difficulty adjustment algorithm
> doesn’t solve the underlying issue—hashpower not being aligned with users’
> (or even its owners’) interests. Propose a fix to the underlying cause and
> that might be worth considering, if it passes peer review. But without that
> you’d just be making the state of affairs arguably worse.
>
> And so yes, *if* this incentive problem can’t be solved, and the unaltered
> bitcoin chain dies from disuse after suffering a hashpower attack,
> especially a centrally and/or purposefully instigated one, then bitcoin
> would be failed a failed project.
>
> The thesis (and value proposition) of bitcoin is that a particular
> category of economic incentives can be used to solve the problem of
> creating a secure trustess ledger. If those incentives failed, then he
> thesis of bitcoin would have been experimentally falsified, yes. Maybe the
> incentives can be made better to save the project, but we’d have to fix the
> source of the problem not the symptoms.
>
> On Oct 10, 2017, at 6:44 PM, Ben Kloester <***@gmail.com> wrote:
>
> Mark, this seems an awful lot like an answer of "no", to my question "Is
> there a contingency plan in the case that the incumbent chain following the
> Bitcoin Core consensus rules comes under 51% attack?" - is this a correct
> interpretation?
>
> In fact, beyond a no, it seems like a "no, and I disagree with the idea of
> creating one".
>
> So if Bitcoin comes under successful 51%, the project, in your vision, has
> simply failed?
>
> *Ben Kloester*
>
> On 10 October 2017 at 13:19, Mark Friedenbach via bitcoin-dev <
> bitcoin-***@lists.linuxfoundation.org> wrote:
>
>> The problem of fast acting but non vulnerable difficulty adjustment
>> algorithms is interesting. I would certainly like to see this space further
>> explored, and even have some ideas myself.
>>
>> However without commenting on the technical merits of this specific
>> proposal, I think it must be said upfront that the stated goal is not good.
>> The largest technical concern (ignoring governance) over B2X is that it is
>> a rushed, poorly reviewed hard fork. Hard forks should not be rushed, and
>> they should receive more than the usual level of expert and community
>> review.
>>
>> I’m that light, doing an even more rushed hard fork on an even newer idea
>> with even less review would be hypocritical at best. I would suggest
>> reframing as a hardfork wishlist research problem for the next properly
>> planned hard fork, if one occurs. You might also find the hardfork research
>> group a more accommodating venue for this discussion:
>>
>> https://bitcoinhardforkresearch.github.io/
>>
>> On Oct 9, 2017, at 3:57 PM, Scott Roberts via bitcoin-dev <
>> bitcoin-***@lists.linuxfoundation.org> wrote:
>>
>> Sorry, my previous email did not have the plain text I intended.
>>
>> Background:
>>
>> The bitcoin difficulty algorithm does not seem to be a good one. If there
>> is a fork due to miners seeking maximum profit without due regard to
>> security, users, and nodes, the "better" coin could end up being the
>> minority chain. If 90% of hashrate is really going to at least initially
>> go
>> towards using SegWit2x, BTC would face 10x delays in confirmations
>> until the next difficulty adjustment, negatively affecting its price
>> relative
>> to BTC1, causing further delays from even more miner abandonment
>> (until the next adjustment). The 10% miners remaining on BTC do not
>> inevitably lose by staying to endure 10x delays because they have 10x
>> less competition, and the same situation applies to BTC1 miners. If the
>> prices are the same and stable, all seems well for everyone, other things
>> aside. But if the BTC price does not fall to reflect the decreased
>> hashrate,
>> he situation seems to be a big problem for both coins: BTC1 miners will
>> jump back to BTC when the difficulty adjustment occurs, initiating a
>> potentially never-ending oscillation between the two coins, potentially
>> worse than what BCH is experiencing. They will not issue coins too fast
>> like BCH because that is a side effect of the asymmetry in BCH's rise and
>> fall algorithm.
>>
>> Solution:
>>
>> Hard fork to implement a new difficulty algorithm that uses a simple
>> rolling
>> average with a much smaller window. Many small coins have done this as
>> a way to stop big miners from coming on and then suddenly leaving,
>> leaving
>> constant miners stuck with a high difficulty for the rest of a (long)
>> averaging
>> window. Even better, adjust the reward based on recent solvetimes to
>> motivate more mining (or less) if the solvetimes are too slow (or too
>> fast).
>> This will keep keep coin issuance rate perfectly on schedule with real
>> time.
>>
>> I recommend the following for Bitcoin, as fast, simple, and better than
>> any
>> other difficulty algorithm I'm aware of. This is the result of a lot of
>> work the
>> past year.
>>
>> === Begin difficulty algorithm ===
>> # Zawy v6 difficulty algorithm (modified for bitcoin)
>> # Unmodified Zawy v6 for alt coins:
>> # http://zawy1.blogspot.com/2017/07/best-difficulty-algorithm-
>> zawy-v1b.html
>> # All my failed attempts at something better:
>> # https://github.com/seredat/karbowanec/commit/231db5270acb2e6
>> 73a641a1800be910ce345668a
>> #
>> # Keep negative solvetimes to correct bad timestamps.
>> # Do not be tempted to use:
>> # next_D = sum(last N Ds) * T / [max(last N TSs) - min(last N TSs];
>> # ST= Solvetime, TS = timestamp
>>
>> # set constants until next hard fork:
>>
>> T=600; # coin's TargetSolvetime
>> N=30; # Averaging window. Smoother than N=15, faster response than N=60.
>> X=5;
>> limit = X^(2/N); # limit rise and fall in case of timestamp manipulation
>> adjust = 1/(1+0.67/N); # keeps avg solvetime on track
>>
>> # begin difficulty algorithm
>>
>> avg_ST=0; avg_D=0;
>> for ( i=height; i > height-N; i--) { # go through N most recent blocks
>> avg_ST += (TS[i] - TS[i-1]) / N;
>> avg_D += D[i]/N;
>> }
>> avg_ST = T*limit if avg_ST > T*limit;
>> avg_ST = T/limit if avg_ST < T/limit;
>>
>> next_D = avg_D * T / avg_ST * adjust;
>>
>> # Tim Olsen suggested changing reward to protect against hash attacks.
>> # Karbowanek coin suggested something similar.
>> # I could not find anything better than the simplest idea below.
>> # It was a great surprise that coin issuance rate came out perfect.
>> # BaseReward = coins per block
>>
>> next_reward = BaseReward * avg_ST / T;
>>
>> ======= end algo ====
>>
>> Due to the limit and keeping negative solvetimes in a true average,
>> timestamp errors resulting in negative solvetimes are corrected in the
>> next
>> block. Otherwise, one would need to do like Zcash and cause a 5-block
>> delay in the response by resorting to the median of past 11 blocks (MPT)
>> as the most recent timestamp, offsetting the timestamps from their
>> corresponding difficulties by 5 blocks. (it does not cause an averaging
>> problem, but it does cause a 5-block delay in the response.)
>> _______________________________________________
>> bitcoin-dev mailing list
>> bitcoin-***@lists.linuxfoundation.org
>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>>
>>
>> _______________________________________________
>> bitcoin-dev mailing list
>> bitcoin-***@lists.linuxfoundation.org
>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>>
>>
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-***@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
>
Scott Roberts via bitcoin-dev
2017-10-11 01:29:54 UTC
Reply
Permalink
Raw Message
I agree: a new difficulty algorithm starting from zero is inconceivably
rushed. But it's also inconceivable to not have one ready in two months
if my understanding of our current situation is correct. Is there any
complaint or suggestion about this algorithm or the appropriate goals of
an ideal difficulty algorithm? I feel like there is a discussion that needs
to be hashed out before a draft BIP at the HF page, but I do not know
where is best or who would be interested. If the community shows it is
receptive and supportive I think I could get Karbowanek coin to put it
into live action and solicit hash attacks. They are currently using a
simpler N=17 like this since last November. They have tested all my
attempted improvements the past few months, so they are familiar with all
the in and outs.

This particular coin split is looking different. Assuming users currently
prefer SW, it still seems like miner support is going to convince enough
users that SegWit2x is a worthy if not superior alternative. The result
could be both coins oscillating with long delays, as long as the price is
similar. As soon as it is not similar, maybe the loser will be in a death
spiral, pushed to the margin like previous coins. This might be a bitcoin
feature. But the 2016 window seems like it is too brutal. It seems like it
will result in an accidental winner before the better coin can be
determined by more rational means.
Scott Roberts via bitcoin-dev
2017-10-12 08:51:33 UTC
Reply
Permalink
Raw Message
Since there is no surviving argument in this thread contrary to my original
post, I'll begin work on a BIP.
Loading...