[Bitcoin-development] Proposed alternatives to the 20MB step function

[Matt Whitlock]

Between all the flames on this list, several ideas were raised that did not get much attention. I hereby resubmit these ideas for consideration and discussion.

- Perhaps the hard block size limit should be a function of the actual block sizes over some trailing sampling period. For example, take the median block size among the most recent 2016 blocks and multiply it by 1.5. This allows Bitcoin to scale up gradually and organically, rather than having human beings guessing at what is an appropriate limit.

- Perhaps the hard block size limit should be determined by a vote of the miners. Each miner could embed a desired block size limit in the coinbase transactions of the blocks it publishes. The effective hard block size limit would be that size having the greatest number of votes within a sliding window of most recent blocks.

- Perhaps the hard block size limit should be a function of block-chain length, so that it can scale up smoothly rather than jumping immediately to 20 MB. This function could be linear (anticipating a breakdown of Moore's Law) or quadratic.

I would be in support of any of the above, but I do not support Mike Hearn's proposed jump to 20 MB. Hearn's proposal kicks the can down the road without actually solving the problem, and it does so in a controversial (step function) way.

[Mike Hearn]

There are certainly arguments to be made for and against all of these
proposals.

The fixed 20mb cap isn't actually my proposal at all, it is from Gavin. I
am supporting it because anything is better than nothing. Gavin originally
proposed the block size be a function of time. That got dropped, I suppose
to make the process of getting consensus easier. It is "the simplest thing
that can possibly work".

I would like to see the process of chain forking becoming less traumatic. I
remember Gavin, Jeff and I once considered (on stage at a conference??)
that maybe there should be a scheduled fork every year, so people know when
to expect them.

If everything goes well, I see no reason why 20mb would be the limit
forever.

[Clément Elbaz]

Matt : I think proposal #1 and #3 are a lot better than #2, and #1 is my
favorite.

I see two problems with proposal #2.
The first problem with proposal #2 is that, as we see in democracies,
there is often a mismatch between the people conscious vote and these same
people behavior.

Relying on an intentional vote made consciously by miners by choosing a
configuration value can lead to twisted results if their actual behavior
doesn't correlate with their vote (eg, they all vote for a small block size
because it is the default configuration of their software, and then they
fill it completely all the time and everything crashes).

The second problem with proposal #2 is that if Gavin and Mike are right,
there is simply no time to gather a meaningful amount of votes over the
coinbases, after the fork but before the Bitcoin scalability crash.

I like proposal #1 because the "vote" is made using already available data.
Also there is no possible mismatch between behavior and vote. As a miner
you vote by choosing to create a big (or small) block, and your actions
reflect your vote. It is simple and straightforward.

My feelings on proposal #3 is it is a little bit mixing apples and oranges,
but I may not seeing all the implications.

Post by Matt Whitlock
Between all the flames on this list, several ideas were raised that did
not get much attention. I hereby resubmit these ideas for consideration and
discussion.
- Perhaps the hard block size limit should be a function of the actual
block sizes over some trailing sampling period. For example, take the
median block size among the most recent 2016 blocks and multiply it by 1.5.
This allows Bitcoin to scale up gradually and organically, rather than
having human beings guessing at what is an appropriate limit.
- Perhaps the hard block size limit should be determined by a vote of the
miners. Each miner could embed a desired block size limit in the coinbase
transactions of the blocks it publishes. The effective hard block size
limit would be that size having the greatest number of votes within a
sliding window of most recent blocks.
- Perhaps the hard block size limit should be a function of block-chain
length, so that it can scale up smoothly rather than jumping immediately to
20 MB. This function could be linear (anticipating a breakdown of Moore's
Law) or quadratic.
I would be in support of any of the above, but I do not support Mike
Hearn's proposed jump to 20 MB. Hearn's proposal kicks the can down the
road without actually solving the problem, and it does so in a
controversial (step function) way.
------------------------------------------------------------------------------
One dashboard for servers and applications across Physical-Virtual-Cloud
Widest out-of-the-box monitoring support with 50+ applications
Performance metrics, stats and reports that give you Actionable Insights
Deep dive visibility with transaction tracing using APM Insight.
http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
_______________________________________________
Bitcoin-development mailing list
https://lists.sourceforge.net/lists/listinfo/bitcoin-development

[Joel Joonatan Kaartinen]

Matt,

It seems you missed my suggestion about basing the maximum block size on
the bitcoin days destroyed in transactions that are included in the block.
I think it has potential for both scaling as well as keeping up a constant
fee pressure. If tuned properly, it should both stop spamming and increase
block size maximum when there are a lot of real transactions waiting for
inclusion.

- Joel

Post by Clément Elbaz
Matt : I think proposal #1 and #3 are a lot better than #2, and #1 is my
favorite.
I see two problems with proposal #2.
The first problem with proposal #2 is that, as we see in democracies,
there is often a mismatch between the people conscious vote and these same
people behavior.
Relying on an intentional vote made consciously by miners by choosing a
configuration value can lead to twisted results if their actual behavior
doesn't correlate with their vote (eg, they all vote for a small block size
because it is the default configuration of their software, and then they
fill it completely all the time and everything crashes).
The second problem with proposal #2 is that if Gavin and Mike are right,
there is simply no time to gather a meaningful amount of votes over the
coinbases, after the fork but before the Bitcoin scalability crash.
I like proposal #1 because the "vote" is made using already available
data. Also there is no possible mismatch between behavior and vote. As a
miner you vote by choosing to create a big (or small) block, and your
actions reflect your vote. It is simple and straightforward.
My feelings on proposal #3 is it is a little bit mixing apples and
oranges, but I may not seeing all the implications.

Post by Matt Whitlock
Between all the flames on this list, several ideas were raised that did
not get much attention. I hereby resubmit these ideas for consideration and
discussion.
- Perhaps the hard block size limit should be a function of the actual
block sizes over some trailing sampling period. For example, take the
median block size among the most recent 2016 blocks and multiply it by 1.5.
This allows Bitcoin to scale up gradually and organically, rather than
having human beings guessing at what is an appropriate limit.
- Perhaps the hard block size limit should be determined by a vote of the
miners. Each miner could embed a desired block size limit in the coinbase
transactions of the blocks it publishes. The effective hard block size
limit would be that size having the greatest number of votes within a
sliding window of most recent blocks.
- Perhaps the hard block size limit should be a function of block-chain
length, so that it can scale up smoothly rather than jumping immediately to
20 MB. This function could be linear (anticipating a breakdown of Moore's
Law) or quadratic.
I would be in support of any of the above, but I do not support Mike
Hearn's proposed jump to 20 MB. Hearn's proposal kicks the can down the
road without actually solving the problem, and it does so in a
controversial (step function) way.
------------------------------------------------------------------------------
One dashboard for servers and applications across Physical-Virtual-Cloud
Widest out-of-the-box monitoring support with 50+ applications
Performance metrics, stats and reports that give you Actionable Insights
Deep dive visibility with transaction tracing using APM Insight.
http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
_______________________________________________
Bitcoin-development mailing list
https://lists.sourceforge.net/lists/listinfo/bitcoin-development

------------------------------------------------------------------------------
One dashboard for servers and applications across Physical-Virtual-Cloud
Widest out-of-the-box monitoring support with 50+ applications
Performance metrics, stats and reports that give you Actionable Insights
Deep dive visibility with transaction tracing using APM Insight.
http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
_______________________________________________
Bitcoin-development mailing list
https://lists.sourceforge.net/lists/listinfo/bitcoin-development

[Gavin Andresen]

I like the bitcoin days destroyed idea.

I like lots of the ideas that have been presented here, on the bitcointalk
forums, etc etc etc.

It is easy to make a proposal, it is hard to wade through all of the
proposals. I'm going to balance that equation by completely ignoring any
proposal that isn't accompanied by code that implements the proposal (with
appropriate tests).

However, I'm not the bottleneck-- you need to get the attention of the
other committers and convince THEM:

a) something should be done "now-ish"
b) your idea is good

We are stuck on (a) right now, I think.


On Fri, May 8, 2015 at 8:32 AM, Joel Joonatan Kaartinen <

Post by Joel Joonatan Kaartinen
Matt,
It seems you missed my suggestion about basing the maximum block size on
the bitcoin days destroyed in transactions that are included in the block.
I think it has potential for both scaling as well as keeping up a constant
fee pressure. If tuned properly, it should both stop spamming and increase
block size maximum when there are a lot of real transactions waiting for
inclusion.
- Joel

[Matt Whitlock]

Post by Joel Joonatan Kaartinen
It seems you missed my suggestion about basing the maximum block size on
the bitcoin days destroyed in transactions that are included in the block.
I think it has potential for both scaling as well as keeping up a constant
fee pressure. If tuned properly, it should both stop spamming and increase
block size maximum when there are a lot of real transactions waiting for
inclusion.

I saw it. I apologize for not including it in my list. I should have, for sake of discussion, even though I have a problem with it.

My problem with it is that "bitcoin days destroyed" is not a measure of demand for space in the block chain. In the distant future, when Bitcoin is the predominant global currency, bitcoins will have such high velocity that the number of bitcoin days destroyed in each block will be much lower than at present. Does this mean that the block size limit should be lower in the future than it is now? Clearly this would be incorrect.

Perhaps I am misunderstanding your proposal. Could you describe it more explicitly?

[Matt Whitlock]

Post by Matt Whitlock

Post by Joel Joonatan Kaartinen
It seems you missed my suggestion about basing the maximum block size on
the bitcoin days destroyed in transactions that are included in the block.
I think it has potential for both scaling as well as keeping up a constant
fee pressure. If tuned properly, it should both stop spamming and increase
block size maximum when there are a lot of real transactions waiting for
inclusion.

My problem with it is that "bitcoin days destroyed" is not a measure of demand for space in the block chain. In the distant future, when Bitcoin is the predominant global currency, bitcoins will have such high velocity that the number of bitcoin days destroyed in each block will be much lower than at present. Does this mean that the block size limit should be lower in the future than it is now? Clearly this would be incorrect.

I feel a need to point out something that may be obvious to some but not to others: the cumulative total number of "bitcoin days destroyed" since the genesis block is bounded by the cumulative total number of "bitcoin days created" since the genesis block. (You can't destroy something that hasn't yet been created.) After all coins have been mined, bitcoin days will be created at a rate of 21M bitcoin days per day. In the long run, bitcoin days will be destroyed at a rate not exceeding 21M bitcoin days per day. This is so because bitcoin days cannot be destroyed at a rate faster than they are created for an indefinitely long time. This upper limit on the rate of bitcoin days destruction is irrespective of bitcoin adoption and the growth in demand for space in the block chain.

Even ignoring the fact that "bitcoin days destroyed" is bounded whereas demand for block-chain space is not, we'd still have to answer the question of whether the rate of bitcoin days destroyed is a good estimator of demand for block-chain space. Why would it be? Suppose some day Satoshi moves his 1M coins to a new address. Would this huge destruction of bitcoin days imply anything about future demand for space in the block chain? No.

[Peter Todd]

Post by Joel Joonatan Kaartinen
Matt,
It seems you missed my suggestion about basing the maximum block size on
the bitcoin days destroyed in transactions that are included in the block.
I think it has potential for both scaling as well as keeping up a constant
fee pressure. If tuned properly, it should both stop spamming and increase
block size maximum when there are a lot of real transactions waiting for
inclusion.

The problem with gating block creation on Bitcoin days destroyed is
there's a strong potential of giving big mining pools an huge advantage,
because they can contract with large Bitcoin owners and buy dummy
transactions with large numbers of Bitcoin days destroyed on demand
whenever they need more days-destroyed to create larger blocks.
Similarly, with appropriate SIGHASH flags such contracting can be done
by modifying *existing* transactions on demand.

Ultimately bitcoin days destroyed just becomes a very complex version of
transaction fees, and it's already well known that gating blocksize on
total transaction fees doesn't work.

[Joel Joonatan Kaartinen]

such a contract is a possibility, but why would big owners give an
exclusive right to such pools? It seems to me it'd make sense to offer
those for any miner as long as the get paid a little for it. Especially
when it's as simple as offering an incomplete transaction with the
appropriate SIGHASH flags.

a part of the reason I like this idea is because it will allow stakeholders
a degree of influence on how large the fees are. At least from the surface,
it looks like incentives are pretty well matched. They have an incentive to
not let the fees drop too low so the network continues to be usable and
they also have an incentive to not raise them too high because it'll push
users into using other systems. Also, there'll be competition between
stakeholders, which should keep the fees reasonable.

I think this would at least be preferable to the "let the miner decide"
model.

- Joel

Post by Joel Joonatan Kaartinen

Post by Joel Joonatan Kaartinen
Matt,
It seems you missed my suggestion about basing the maximum block size on
the bitcoin days destroyed in transactions that are included in the

block.

Post by Joel Joonatan Kaartinen
I think it has potential for both scaling as well as keeping up a

constant

Post by Joel Joonatan Kaartinen
fee pressure. If tuned properly, it should both stop spamming and

increase

Post by Joel Joonatan Kaartinen
block size maximum when there are a lot of real transactions waiting for
inclusion.

The problem with gating block creation on Bitcoin days destroyed is
there's a strong potential of giving big mining pools an huge advantage,
because they can contract with large Bitcoin owners and buy dummy
transactions with large numbers of Bitcoin days destroyed on demand
whenever they need more days-destroyed to create larger blocks.
Similarly, with appropriate SIGHASH flags such contracting can be done
by modifying *existing* transactions on demand.
Ultimately bitcoin days destroyed just becomes a very complex version of
transaction fees, and it's already well known that gating blocksize on
total transaction fees doesn't work.
--
00000000000000000f53e2d214685abf15b6d62d32453a03b0d472e374e10e94

[Peter Todd]

This post might be inappropriate. Click to display it.

[Steven Pine]

Block size scaling should be as transparent and simple as possible, like
pegging it to total transactions per difficulty change.

[Alex Mizrahi]

Adaptive schedules, i.e. those where block size limit depends not only on
block height, but on other parameters as well, are surely attractive in the
sense that the system can adapt to the actual use, but they also open a
possibility of a manipulation.

E.g. one of mining companies might try to bankrupt other companies by
making mining non-profitable. To do that they will accept transactions with
ridiculously low fees (e.g. 1 satoshi per transaction). Of course, they
will suffer losees themselves, but the they might be able to survive that
if they have access to financial resources. (E.g. companies backed by banks
and such will have an advantage).
Once competitors close down their mining operations, they can drive fees
upwards.

So if you don't want to open room for manipulation (which is very hard to
analyze), it is better to have a block size hard limit which depends only
on block height.
On top of that there might be a soft limit which is enforced by the
majority of miners.

[Bryan Bishop]

Post by Matt Whitlock
- Perhaps the hard block size limit should be a function of the actual block sizes over some
trailing sampling period. For example, take the median block size among the most recent
2016 blocks and multiply it by 1.5. This allows Bitcoin to scale up gradually and organically,
rather than having human beings guessing at what is an appropriate limit.

Block contents can be grinded much faster than hashgrinding and
mining. There is a significant run-away effect there, and it also
works in the gradual sense as a miner probabilistically mines large
blocks that get averaged into that 2016 median block size computation.
At least this proposal would be a slower way of pushing out miners and
network participants that can't handle 100 GB blocks immediately.. As
the size of the blocks are increased, low-end hardware participants
have to fall off the network because they no longer meet the minimum
performance requirements. Adjustment might become severely mismatched
with general economic trends in data storage device development or
availability or even current-market-saturation of said storage
devices. With the assistance of transaction stuffing or grinding, that
2016 block median metric can be gamed to increase faster than other
participants can keep up with or, perhaps worse, in a way that was
unintended by developers yet known to be a failure mode. These are
just some issues to keep and mind and consider.

- Bryan
http://heybryan.org/
1 512 203 0507

[Mark Friedenbach]

It is my professional opinion that raising the block size by merely
adjusting a constant without any sort of feedback mechanism would be a
dangerous and foolhardy thing to do. We are custodians of a multi-billion
dollar asset, and it falls upon us to weigh the consequences of our own
actions against the combined value of the entire bitcoin ecosystem. Ideally
we would take no action for which we are not absolutely certain of the
ramifications, with the information that can be made available to us. But
of course that is not always possible: there are unknown-unknowns, time
pressures, and known-unknowns where information has too high a marginal
cost. So where certainty is unobtainable, we must instead hedge against
unwanted outcomes.

The proposal to raise the block size now by redefining a constant carries
with it risk associated with infrastructure scaling, centralization
pressures, and delaying the necessary development of a constraint-based fee
economy. It also simply kicks the can down the road in settling these
issues because a larger but realistic hard limit must still exist, meaning
a future hard fork may still be required.

But whatever new hard limit is chosen, there is also a real possibility
that it may be too high. The standard response is that it is a soft-fork
change to impose a lower block size limit, which miners could do with a
minimal amount of coordination. This is however undermined by the
unfortunate reality that so many mining operations are absentee-run
businesses, or run by individuals without a strong background in bitcoin
protocol policy, or with interests which are not well aligned with other
users or holders of bitcoin. We cannot rely on miners being vigilant about
issues that develop, as they develop, or able to respond in the appropriate
fashion that someone with full domain knowledge and an objective
perspective would.

The alternative then is to have some sort of dynamic block size limit
controller, and ideally one which applies a cost to raising the block size
in some way the preserves the decentralization and/or long-term stability
features that we care about. I will now describe one such proposal:

* For each block, the miner is allowed to select a different difficulty
(nBits) within a certain range, e.g. +/- 25% of the expected difficulty,
and this miner-selected difficulty is used for the proof of work check. In
addition to adjusting the hashcash target, selecting a different difficulty
also raises or lowers the maximum block size for that block by a function
of the difference in difficulty. So increasing the difficulty of the block
by an additional 25% raises the block limit for that block from 100% of the
current limit to 125%, and lowering the difficulty by 10% would also lower
the maximum block size for that block from 100% to 90% of the current
limit. For simplicity I will assume a linear identity transform as the
function, but a quadratic or other function with compounding marginal cost
may be preferred.

* The default maximum block size limit is then adjusted at regular
intervals. For simplicity I will assume an adjustment at the end of each
2016 block interval, at the same time that difficulty is adjusted, but
there is no reason these have to be aligned. The adjustment algorithm
itself is either the selection of the median, or perhaps some sort of
weighted average that respects the "middle majority." There would of course
be limits on how quickly the block size limit can adjusted in any one
period, just as there are min/max limits on the difficulty adjustment.

* To prevent perverse mining incentives, the original difficulty without
adjustment is used in the aggregate work calculations for selecting the
most-work chain, and the allowable miner-selected adjustment to difficulty
would have to be tightly constrained.

These rules create an incentive environment where raising the block size
has a real cost associated with it: a more difficult hashcash target for
the same subsidy reward. For rational miners that cost must be
counter-balanced by additional fees provided in the larger block. This
allows block size to increase, but only within the confines of a
self-supporting fee economy.

When the subsidy goes away or is reduced to an insignificant fraction of
the block reward, this incentive structure goes away. Hopefully at that
time we would have sufficient information to soft-fork set a hard block
size maximum. But in the mean time, the block size limit controller
constrains the maximum allowed block size to be within a range supported by
fees on the network, providing an emergency relief valve that we can be
assured will only be used at significant cost.

Mark Friedenbach

* There has over time been various discussions on the bitcointalk forums
about dynamically adjusting block size limits. The true origin of the idea
is unclear at this time (citations would be appreciated!) but a form of it
was implemented in Bytecoin / Monero using subsidy burning to increase the
block size. That approach has various limitations. These were corrected in
Greg Maxwell's suggestion to adjust the difficulty/nBits field directly,
which also has the added benefit of providing incentive for bidirectional
movement during the subsidy period. The description in this email and any
errors are my own.

Post by Matt Whitlock
Between all the flames on this list, several ideas were raised that did
not get much attention. I hereby resubmit these ideas for consideration and
discussion.
- Perhaps the hard block size limit should be a function of the actual
block sizes over some trailing sampling period. For example, take the
median block size among the most recent 2016 blocks and multiply it by 1.5.
This allows Bitcoin to scale up gradually and organically, rather than
having human beings guessing at what is an appropriate limit.
- Perhaps the hard block size limit should be determined by a vote of the
miners. Each miner could embed a desired block size limit in the coinbase
transactions of the blocks it publishes. The effective hard block size
limit would be that size having the greatest number of votes within a
sliding window of most recent blocks.
- Perhaps the hard block size limit should be a function of block-chain
length, so that it can scale up smoothly rather than jumping immediately to
20 MB. This function could be linear (anticipating a breakdown of Moore's
Law) or quadratic.
I would be in support of any of the above, but I do not support Mike
Hearn's proposed jump to 20 MB. Hearn's proposal kicks the can down the
road without actually solving the problem, and it does so in a
controversial (step function) way.
------------------------------------------------------------------------------
One dashboard for servers and applications across Physical-Virtual-Cloud
Widest out-of-the-box monitoring support with 50+ applications
Performance metrics, stats and reports that give you Actionable Insights
Deep dive visibility with transaction tracing using APM Insight.
http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
_______________________________________________
Bitcoin-development mailing list
https://lists.sourceforge.net/lists/listinfo/bitcoin-development

[Aaron Voisine]

This is a clever way to tie block size to fees.

I would just like to point out though that it still fundamentally is using
hard block size limits to enforce scarcity. Transactions with below market
fees will hang in limbo for days and fail, instead of failing immediately
by not propagating, or seeing degraded, long confirmation times followed by
eventual success.


Aaron Voisine
co-founder and CEO
breadwallet.com

Post by Mark Friedenbach
It is my professional opinion that raising the block size by merely
adjusting a constant without any sort of feedback mechanism would be a
dangerous and foolhardy thing to do. We are custodians of a multi-billion
dollar asset, and it falls upon us to weigh the consequences of our own
actions against the combined value of the entire bitcoin ecosystem. Ideally
we would take no action for which we are not absolutely certain of the
ramifications, with the information that can be made available to us. But
of course that is not always possible: there are unknown-unknowns, time
pressures, and known-unknowns where information has too high a marginal
cost. So where certainty is unobtainable, we must instead hedge against
unwanted outcomes.
The proposal to raise the block size now by redefining a constant carries
with it risk associated with infrastructure scaling, centralization
pressures, and delaying the necessary development of a constraint-based fee
economy. It also simply kicks the can down the road in settling these
issues because a larger but realistic hard limit must still exist, meaning
a future hard fork may still be required.
But whatever new hard limit is chosen, there is also a real possibility
that it may be too high. The standard response is that it is a soft-fork
change to impose a lower block size limit, which miners could do with a
minimal amount of coordination. This is however undermined by the
unfortunate reality that so many mining operations are absentee-run
businesses, or run by individuals without a strong background in bitcoin
protocol policy, or with interests which are not well aligned with other
users or holders of bitcoin. We cannot rely on miners being vigilant about
issues that develop, as they develop, or able to respond in the appropriate
fashion that someone with full domain knowledge and an objective
perspective would.
The alternative then is to have some sort of dynamic block size limit
controller, and ideally one which applies a cost to raising the block size
in some way the preserves the decentralization and/or long-term stability
* For each block, the miner is allowed to select a different difficulty
(nBits) within a certain range, e.g. +/- 25% of the expected difficulty,
and this miner-selected difficulty is used for the proof of work check. In
addition to adjusting the hashcash target, selecting a different difficulty
also raises or lowers the maximum block size for that block by a function
of the difference in difficulty. So increasing the difficulty of the block
by an additional 25% raises the block limit for that block from 100% of the
current limit to 125%, and lowering the difficulty by 10% would also lower
the maximum block size for that block from 100% to 90% of the current
limit. For simplicity I will assume a linear identity transform as the
function, but a quadratic or other function with compounding marginal cost
may be preferred.
* The default maximum block size limit is then adjusted at regular
intervals. For simplicity I will assume an adjustment at the end of each
2016 block interval, at the same time that difficulty is adjusted, but
there is no reason these have to be aligned. The adjustment algorithm
itself is either the selection of the median, or perhaps some sort of
weighted average that respects the "middle majority." There would of course
be limits on how quickly the block size limit can adjusted in any one
period, just as there are min/max limits on the difficulty adjustment.
* To prevent perverse mining incentives, the original difficulty without
adjustment is used in the aggregate work calculations for selecting the
most-work chain, and the allowable miner-selected adjustment to difficulty
would have to be tightly constrained.
These rules create an incentive environment where raising the block size
has a real cost associated with it: a more difficult hashcash target for
the same subsidy reward. For rational miners that cost must be
counter-balanced by additional fees provided in the larger block. This
allows block size to increase, but only within the confines of a
self-supporting fee economy.
When the subsidy goes away or is reduced to an insignificant fraction of
the block reward, this incentive structure goes away. Hopefully at that
time we would have sufficient information to soft-fork set a hard block
size maximum. But in the mean time, the block size limit controller
constrains the maximum allowed block size to be within a range supported by
fees on the network, providing an emergency relief valve that we can be
assured will only be used at significant cost.
Mark Friedenbach
* There has over time been various discussions on the bitcointalk forums
about dynamically adjusting block size limits. The true origin of the idea
is unclear at this time (citations would be appreciated!) but a form of it
was implemented in Bytecoin / Monero using subsidy burning to increase the
block size. That approach has various limitations. These were corrected in
Greg Maxwell's suggestion to adjust the difficulty/nBits field directly,
which also has the added benefit of providing incentive for bidirectional
movement during the subsidy period. The description in this email and any
errors are my own.

Post by Matt Whitlock
Between all the flames on this list, several ideas were raised that did
not get much attention. I hereby resubmit these ideas for consideration and
discussion.
- Perhaps the hard block size limit should be a function of the actual
block sizes over some trailing sampling period. For example, take the
median block size among the most recent 2016 blocks and multiply it by 1.5.
This allows Bitcoin to scale up gradually and organically, rather than
having human beings guessing at what is an appropriate limit.
- Perhaps the hard block size limit should be determined by a vote of the
miners. Each miner could embed a desired block size limit in the coinbase
transactions of the blocks it publishes. The effective hard block size
limit would be that size having the greatest number of votes within a
sliding window of most recent blocks.
- Perhaps the hard block size limit should be a function of block-chain
length, so that it can scale up smoothly rather than jumping immediately to
20 MB. This function could be linear (anticipating a breakdown of Moore's
Law) or quadratic.
I would be in support of any of the above, but I do not support Mike
Hearn's proposed jump to 20 MB. Hearn's proposal kicks the can down the
road without actually solving the problem, and it does so in a
controversial (step function) way.
------------------------------------------------------------------------------
One dashboard for servers and applications across Physical-Virtual-Cloud
Widest out-of-the-box monitoring support with 50+ applications
Performance metrics, stats and reports that give you Actionable Insights
Deep dive visibility with transaction tracing using APM Insight.
http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
_______________________________________________
Bitcoin-development mailing list
https://lists.sourceforge.net/lists/listinfo/bitcoin-development

------------------------------------------------------------------------------
One dashboard for servers and applications across Physical-Virtual-Cloud
Widest out-of-the-box monitoring support with 50+ applications
Performance metrics, stats and reports that give you Actionable Insights
Deep dive visibility with transaction tracing using APM Insight.
http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
_______________________________________________
Bitcoin-development mailing list
https://lists.sourceforge.net/lists/listinfo/bitcoin-development

[Mark Friedenbach]

Post by Aaron Voisine
This is a clever way to tie block size to fees.
I would just like to point out though that it still fundamentally is using
hard block size limits to enforce scarcity. Transactions with below market
fees will hang in limbo for days and fail, instead of failing immediately
by not propagating, or seeing degraded, long confirmation times followed by
eventual success.

There are already solutions to this which are waiting to be deployed as
default policy to bitcoind, and need to be implemented in other clients:
replace-by-fee and child-pays-for-parent.

[Aaron Voisine]

That's fair, and we've implemented child-pays-for-parent for spending
unconfirmed inputs in breadwallet. But what should the behavior be when
those options aren't understood/implemented/used?

My argument is that the less risky, more conservative default fallback
behavior should be either non-propagation or delayed confirmation, which is
generally what we have now, until we hit the block size limit. We still
have lots of safe, non-controversial, easy to experiment with options to
add fee pressure, causing users to economize on block space without
resorting to dropping transactions after a prolonged delay.

Aaron Voisine
co-founder and CEO
breadwallet.com

Post by Mark Friedenbach

Post by Aaron Voisine
This is a clever way to tie block size to fees.
I would just like to point out though that it still fundamentally is
using hard block size limits to enforce scarcity. Transactions with below
market fees will hang in limbo for days and fail, instead of failing
immediately by not propagating, or seeing degraded, long confirmation times
followed by eventual success.

There are already solutions to this which are waiting to be deployed as
replace-by-fee and child-pays-for-parent.

[Mark Friedenbach]

In a fee-dominated future, replace-by-fee is not an opt-in feature. When
you create a transaction, the wallet presents a range of fees that it
expects you might pay. It then signs copies of the transaction with spaced
fees from this interval and broadcasts the lowest fee first. In the user
interface, the transaction is shown with its transacted amount and the
approved fee range. All of the inputs used are placed on hold until the
transaction gets a confirmation. As time goes by and it looks like the
transaction is not getting accepted, successively higher fee versions are
released. You can opt-out and send a no-fee or base-fee-only transaction,
but that should not be the default.

On the receiving end, local policy controls how much fee should be spent
trying to obtain confirmations before alerting the user, if there are fees
available in the hot wallet to do this. The receiving wallet then adds its
own fees via a spend if it thinks insufficient fees were provided to get a
confirmation. Again, this should all be automated so long as there is a hot
wallet on the receiving end.

Is this more complicated than now, where blocks are not full and clients
generally don't have to worry about their transactions eventually
confirming? Yes, it is significantly more complicated. But such
complication is unavoidable. It is a simple fact that the block size cannot
increase so much as to cover every single use by every single person in the
world, so there is no getting around the reality that we will have to
transition into an economy where at least one side has to pay up for a
transaction to get confirmation, at all. We are going to have to deal with
this issue whether it is now at 1MB or later at 20MB. And frankly, it'll be
much easier to do now.

Post by Aaron Voisine
That's fair, and we've implemented child-pays-for-parent for spending
unconfirmed inputs in breadwallet. But what should the behavior be when
those options aren't understood/implemented/used?
My argument is that the less risky, more conservative default fallback
behavior should be either non-propagation or delayed confirmation, which is
generally what we have now, until we hit the block size limit. We still
have lots of safe, non-controversial, easy to experiment with options to
add fee pressure, causing users to economize on block space without
resorting to dropping transactions after a prolonged delay.
Aaron Voisine
co-founder and CEO
breadwallet.com

Post by Mark Friedenbach

Post by Aaron Voisine
This is a clever way to tie block size to fees.
I would just like to point out though that it still fundamentally is
using hard block size limits to enforce scarcity. Transactions with below
market fees will hang in limbo for days and fail, instead of failing
immediately by not propagating, or seeing degraded, long confirmation times
followed by eventual success.

There are already solutions to this which are waiting to be deployed as
replace-by-fee and child-pays-for-parent.

[Gregory Maxwell]

These rules create an incentive environment where raising the block size has
a real cost associated with it: a more difficult hashcash target for the
same subsidy reward. For rational miners that cost must be counter-balanced
by additional fees provided in the larger block. This allows block size to
increase, but only within the confines of a self-supporting fee economy.
When the subsidy goes away or is reduced to an insignificant fraction of the
block reward, this incentive structure goes away. Hopefully at that time we
would have sufficient information to soft-fork set a hard block size
maximum. But in the mean time, the block size limit controller constrains
the maximum allowed block size to be within a range supported by fees on the
network, providing an emergency relief valve that we can be assured will
only be used at significant cost.

Though I'm a fan of this class of techniques(*) and think using something
in this space is strictly superior to not, and I think it makes larger
sizes safer long term; I do not think it adequately obviates the need
for a hard upper limit for two reasons:

(1) for software engineering and operational reasons it is very
difficult to develop, test for, or provision for something without
knowing limits. There would in fact be hard limits on real deployments
but they'd be opaque to their operators and you could easily imagine
the network forking by surprise as hosts crossed those limits.

(2) At best this approach mitigates the collective action problem between
miners around fees; it does not correct the incentive alignment between
miners and everyone else (miners can afford huge node costs because they
have income; but the full-node-using-users that need to exist in plenty
to keep miners honest do not), or the centralization pressures (N miners
can reduce their storage/bandwidth/cpu costs N fold by centralizing).

A dynamic limit can be combined with a hard upper to at least be no
worse than a hard upper with respect to those two points.


Another related point which has been tendered before but seems to have
been ignored is that changing how the size limit is computed can help
better align incentives and thus reduce risk. E.g. a major cost to the
network is the UTXO impact of transactions, but since the limit is blind
to UTXO impact a miner would gain less income if substantially factoring
UTXO impact into its fee calculations; and without fee impact users have
little reason to optimize their UTXO behavior. This can be corrected
by augmenting the "size" used for limit calculations. An example would
be tx_size = MAX( real_size >> 1, real_size + 4*utxo_created_size -
3*utxo_consumed_size). The reason for the MAX is so that a block
which cleaned a bunch of big UTXO could not break software by being
super large, the utxo_consumed basically lets you credit your fees by
cleaning the utxo set; but since you get less credit than you cost the
pressure should be downward but not hugely so. The 1/2, 4, 3 I regard
as parameters which I don't have very strong opinions on which could be
set based on observations in the network today (e.g. adjusted so that a
normal cleaning transaction can hit the minimum size). One way to think
about this is that it makes it so that every output you create "prepays"
the transaction fees needed to spend it by shifting "space" from the
current block to a future block. The fact that the prepayment is not
perfectly efficient reduces the incentive for miners to create lots of
extra outputs when they have room left in their block in order to store
space to use later [an issue that is potentially less of a concern with a
dynamic size limit]. With the right parameters there would never be such
at thing as a dust output (one which costs more to spend than its worth).

(likewise the sigops limit should be counted correctly and turned into
size augmentation (ones that get run by the txn); which would greatly
simplify selection rules: maximize income within a single scalar limit)

(*) I believe my currently favored formulation of general dynamic control
idea is that each miner expresses in their coinbase a preferred size
between some minimum (e.g. 500k) and the miner's effective-maximum;
the actual block size can be up to the effective maximum even if the
preference is lower (you're not forced to make a lower block because you
stated you wished the limit were lower). There is a computed maximum
which is the 33-rd percentile of the last 2016 coinbase preferences
minus computed_max/52 (rounding up to 1) bytes-- or 500k if thats
larger. The effective maximum is X bytes more, where X on the range
[0, computed_maximum] e.g. the miner can double the size of their
block at most. If X > 0, then the miners must also reach a target
F(x/computed_maximum) times the bits-difficulty; with F(x) = x^2+1 ---
so the maximum penalty is 2, with a quadratic shape; for a given mempool
there will be some value that maximizes expected income. (obviously all
implemented with precise fixed point arithmetic). The percentile is
intended to give the preferences of the 33% least preferring miners a
veto on increases (unless a majority chooses to soft-fork them out). The
minus-comp_max/52 provides an incentive to slowly shrink the maximum
if its too large-- x/52 would halve the size in one year if miners
were doing the lowest difficulty mining. The parameters 500k/33rd,
-computed_max/52 bytes, and f(x) I have less strong opinions about;
and would love to hear reasoned arguments for particular parameters.

[Gavin Andresen]

RE: fixing sigop counting, and building in UTXO cost: great idea! One of
the problems with this debate is it is easy for great ideas get lost in all
the noise.

RE: a hard upper limit, with a dynamic limit under it:

I like that idea. Can we drill down on the hard upper limit?

There are lots of people who want a very high upper limit, right now (all
the big Bitcoin companies, and anybody who thinks as-rapid-as-possible
growth now is the best path to long-term success). This is the "it is OK if
you have to run full nodes in a data center" camp.

There are also lots of people who want an upper limit low enough that they
can continue to run Bitcoin on the hardware and Internet connection that
they have (or are concerned about centralization, so want to make sure
OTHER people can continue to run....).

Is there an upper limit "we" can choose to make both sets of people mostly
happy? I've proposed "must be inexpensive enough that a 'hobbyist' can
afford to run a full node" ...

Is the limit chosen once, now, via hard-fork, or should we expect multiple
hard-forks to change it "when necessary" ?

The economics change every time the block reward halves, which make me
think that might be a good time to adjust the hard upper limit. If we have
a hard upper limit and a lower dynamic limit, perhaps adjusting the hard
upper limit (up or down) to account for the block reward halving, based on
the dynamic limit....



RE: the lower dynamic limit algorithm: I REALLY like that idea.

[Tier Nolan]

Post by Gavin Andresen
RE: fixing sigop counting, and building in UTXO cost: great idea! One of
the problems with this debate is it is easy for great ideas get lost in all
the noise.

If the UTXO set cost is built in, UTXO database entries suddenly are worth
something, in addition to the bitcoin held in that entry.

A user's client might display how many they own. When sending money to a
merchant, the user might demand the merchant indicate a slot to pay to.

The user could send an ANYONE_CAN_PAY partial transaction. The transaction
would guarantee that the user has at least as many UTXOs as before.

Discussing the possibility of doing this creates an incentive to bloat the
UTXO set right now, since UTXOs would be valuable in the future.

The objective would be to make them valuable enough to encourage
conservation, but not so valuable that the UTXO contains more value than
the bitcoins in the output.

Gmaxwell's suggested "tx_size = MAX( real_size >> 1, real_size +
4*utxo_created_size - 3*utxo_consumed_size)" for a 250 byte transaction
with 1 input and 2 outputs has very little effect.

real_size + 4 * (2) - 3 * 1 = 255

That gives a 2% size penalty for adding an extra UTXO. I doubt that is
enough to change behavior.

The UTXO set growth could be limited directly. A block would be invalid if
it increases the number of UTXO entries above the charted path.

RE: a hard upper limit, with a dynamic limit under it:
If the block is greater than 32MB, then it means an update to how blocks
are broadcast, so that could be a reasonable hard upper limit (or maybe
31MB, or just the 20MB already suggested).

[Owen Gunden]

Post by Gregory Maxwell
Another related point which has been tendered before but seems to have
been ignored is that changing how the size limit is computed can help
better align incentives and thus reduce risk. E.g. a major cost to the
network is the UTXO impact of transactions, but since the limit is blind
to UTXO impact a miner would gain less income if substantially factoring
UTXO impact into its fee calculations; and without fee impact users have
little reason to optimize their UTXO behavior.

Along the lines of aligning incentives with a diversity of costs to a
variety of network participants, I am curious about reactions to Justus'
general approach:

http://bitcoinism.liberty.me/2015/02/09/economic-fallacies-and-the-block-size-limit-part-2-price-discovery/

I realize it relies on pie-in-the-sky ideas like micropayment channels,
but I wonder if it's a worthy long-term ideal direction for this stuff.

[Mark Friedenbach]

Micropayment channels are not pie in the sky proposals. They work today on
Bitcoin as it is deployed without any changes. People just need to start
using them.

Post by Owen Gunden

Post by Gregory Maxwell
Another related point which has been tendered before but seems to have
been ignored is that changing how the size limit is computed can help
better align incentives and thus reduce risk. E.g. a major cost to the
network is the UTXO impact of transactions, but since the limit is blind
to UTXO impact a miner would gain less income if substantially factoring
UTXO impact into its fee calculations; and without fee impact users have
little reason to optimize their UTXO behavior.

Along the lines of aligning incentives with a diversity of costs to a
variety of network participants, I am curious about reactions to Justus'
http://bitcoinism.liberty.me/2015/02/09/economic-fallacies-and-the-block-size-limit-part-2-price-discovery/
I realize it relies on pie-in-the-sky ideas like micropayment channels,
but I wonder if it's a worthy long-term ideal direction for this stuff.
------------------------------------------------------------------------------
One dashboard for servers and applications across Physical-Virtual-Cloud
Widest out-of-the-box monitoring support with 50+ applications
Performance metrics, stats and reports that give you Actionable Insights
Deep dive visibility with transaction tracing using APM Insight.
http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
_______________________________________________
Bitcoin-development mailing list
https://lists.sourceforge.net/lists/listinfo/bitcoin-development

[Gavin Andresen]

Post by Gregory Maxwell
(*) I believe my currently favored formulation of general dynamic control
idea is that each miner expresses in their coinbase a preferred size
between some minimum (e.g. 500k) and the miner's effective-maximum;
the actual block size can be up to the effective maximum even if the
preference is lower (you're not forced to make a lower block because you
stated you wished the limit were lower). There is a computed maximum
which is the 33-rd percentile of the last 2016 coinbase preferences
minus computed_max/52 (rounding up to 1) bytes-- or 500k if thats
larger. The effective maximum is X bytes more, where X on the range
[0, computed_maximum] e.g. the miner can double the size of their
block at most. If X > 0, then the miners must also reach a target
F(x/computed_maximum) times the bits-difficulty; with F(x) = x^2+1 ---
so the maximum penalty is 2, with a quadratic shape; for a given mempool
there will be some value that maximizes expected income. (obviously all
implemented with precise fixed point arithmetic). The percentile is
intended to give the preferences of the 33% least preferring miners a
veto on increases (unless a majority chooses to soft-fork them out). The
minus-comp_max/52 provides an incentive to slowly shrink the maximum
if its too large-- x/52 would halve the size in one year if miners
were doing the lowest difficulty mining. The parameters 500k/33rd,
-computed_max/52 bytes, and f(x) I have less strong opinions about;
and would love to hear reasoned arguments for particular parameters.

I'm going to try to figure out how much transaction fee a transaction would
have to pay to bribe a miner to include it. Greg, please let me know if
I've misinterpreted the proposed algorithm. And everybody, please let me
know if I'm making a bone-headed mistake in how I'm computing anything:

Lets say miners are expressing a desire for 600,000 byte blocks in their
coinbases.

computed_max = 600,000 - 600,000/52 = 588,462 bytes.
--> this is about 23 average-size (500-byte) transactions less than
600,000.
effective_max = 1,176,923

Lets say I want to maintain status quo at 600,000 bytes; how much penalty
do I have?
((600,000-588,462)/588,462)^2 + 1 = 1.00038

How much will that cost me?
The network is hashing at 310PetaHash/sec right now.
Takes 600 seconds to find a block, so 186,000PH per block
186,000 * 0.00038 = 70 extra PH

If it takes 186,000 PH to find a block, and a block is worth 25.13 BTC
(reward plus fees), that 70 PH costs:
(25.13 BTC/block / 186,000 PH/block) * 70 PH = 0.00945 BTC
or at $240 / BTC: $2.27

... so average transaction fee will have to be about ten cents ($2.27
spread across 23 average-sized transactions) for miners to decide to stay
at 600K blocks. If they fill up 588,462 bytes and don't have some
ten-cent-fee transactions left, they should express a desire to create a
588,462-byte-block and mine with no penalty.

Is that too much? Not enough? Average transaction fees today are about 3
cents per transaction.
I created a spreadsheet playing with the parameters:

https://docs.google.com/spreadsheets/d/1zYZfb44Uns8ai0KnoQ-LixDwdhqO5iTI3ZRcihQXlgk/edit?usp=sharing

"We" could tweak the constants or function to get a transaction fee we
think is reasonable... but we really shouldn't be deciding whether
transaction fees are too high, too low, or just right, and after thinking
about this for a while I think any algorithm that ties difficulty to block
size is just a complicated way of dictating minimum fees.

As for some other dynamic algorithm: OK with me. How do we get consensus on
what the best algorithm is? I'm ok with any "don't grow too quickly, give
some reasonable-percentage-minority of miners the ability to block further
increases."

Also relevant here:
"The curious task of economics is to demonstrate to men how little they
really know about what they imagine they can design." - Friedrich August
von Hayek

[Gregory Maxwell]

a while I think any algorithm that ties difficulty to block size is just a
complicated way of dictating minimum fees.

Thats not the long term effect or the motivation-- what you're seeing
is that the subsidy gets in the way here. Consider how the procedure
behaves with subsidy being negligible compared to fees. What it
accomplishes in that case is that it incentivizes increasing the size
until the marginal "value" to miners of the transaction-data being
left out is not enormously smaller than the "value" of the data in the
block on average. Value in quotes because it's blind to the "fees"
the transaction claims.

With a large subsidy, the marginal value of the first byte in the
block is HUGE; and so that pushes up the average-- and creates the
"base fee effect" that you're looking at. It's not that anyone is
picking a fee there, it's that someone picked the subsidy there. :)
As the subsidy goes down the only thing fees are relative to is fees.

An earlier version of the proposal took subsidy out of the picture
completely by increasing it linearly with the increased difficulty;
but that creates additional complexity both to implement and to
explain to people (e.g. that the setup doesn't change the supply of
coins); ... I suppose without it that starting disadvantage parameter
(the offset that reduces the size if you're indifferent) needs to be
much smaller, unfortunately.

[Rob Golding]

Post by Gavin Andresen
How much will that cost me?
The network is hashing at 310PetaHash/sec right now.
Takes 600 seconds to find a block, so 186,000PH per block
186,000 * 0.00038 = 70 extra PH
If it takes 186,000 PH to find a block, and a block is worth 25.13 BTC
(25.13 BTC/block / 186,000 PH/block) * 70 PH = 0.00945 BTC
or at $240 / BTC: $2.27
... so average transaction fee will have to be about ten cents ($2.27
spread across 23 average-sized transactions) for miners to decide to
stay at 600K blocks

Surely that's an *extra* $2.27 as you've already included .13BTC
($31.20) in fees in the calculation ?

Rob

[Tier Nolan]

Post by Gregory Maxwell
An example would
be tx_size = MAX( real_size >> 1, real_size + 4*utxo_created_size -
3*utxo_consumed_size).

This could be implemented as a soft fork too.

* 1MB hard size limit
* 900kB soft limit

S = block size
U = UTXO_adjusted_size = S + 4 * outputs - 3 * inputs

A block is valid if S < 1MB and U < 1MB

A 250 byte transaction with 2 inputs and 2 outputs would have an adjusted
size of 252 bytes.

The memory pool could be sorted by fee per adjusted_size.

Coin selection could be adjusted so it tries to have at least 2 inputs
when creating transactions, unless the input is worth more than a threshold
(say 0.001 BTC).

This is a pretty weak incentive, especially if the block size is
increased. Maybe it will cause a "nudge"

[Rusty Russell]

Post by Tier Nolan

Post by Gregory Maxwell
An example would
be tx_size = MAX( real_size >> 1, real_size + 4*utxo_created_size -
3*utxo_consumed_size).

This could be implemented as a soft fork too.
* 1MB hard size limit
* 900kB soft limit

I like this too.

Some tweaks:

1) Nomenclature: call tx_size "tx_cost" and real_size "tx_bytes"?

2) If we have a reasonable hard *byte* limit, I don't think that we need
the MAX(). In fact, it's probably OK to go negative.

3) ... or maybe not, if any consumed UTXO was generated before the soft
fork (reducing Tier's perverse incentive).

4) How do we measure UTXO size? There are some constant-ish things in
there (eg. txid as key, height, outnum, amount). Maybe just add 32
to scriptlen?

5) Add a CHECKSIG cost. Naively, since we allow 20,000 CHECKSIGs and
1MB blocks, that implies a cost of 50 bytes per CHECKSIG (but counted
correctly, unlike now).

This last one implies that the initial cost limit would be 2M, but in
practice probably somewhere in the middle.

tx_cost = 50*num-CHECKSIG
+ tx_bytes
+ 4*utxo_created_size
- 3*utxo_consumed_size

Post by Tier Nolan
A 250 byte transaction with 2 inputs and 2 outputs would have an adjusted
size of 252 bytes.

Now cost == 352.

Cheers,
Rusty.

[Tier Nolan]

Post by Rusty Russell
1) Nomenclature: call tx_size "tx_cost" and real_size "tx_bytes"?

Fair enough.

Post by Rusty Russell
2) If we have a reasonable hard *byte* limit, I don't think that we need
the MAX(). In fact, it's probably OK to go negative.

I agree, we want people to compress the UTXO space and a transaction with
100 inputs and one output is great.

It may have privacy problem though.

Post by Rusty Russell
3) ... or maybe not, if any consumed UTXO was generated before the soft
fork (reducing Tier's perverse incentive).

The incentive problem can be fixed by excluding UTXOs from blocks before a
certain count.

UTXOs in blocks before 375000 don't count.

Post by Rusty Russell
4) How do we measure UTXO size? There are some constant-ish things in
there (eg. txid as key, height, outnum, amount). Maybe just add 32
to scriptlen?

They can be stored as a fixed digest. That can be any size, depending on
security requirements.

Gmaxwell's cost proposal is 3-4 bytes per UTXO change. It isn't
4*UXTO.size - 3*UTXO.size

It is only a small nudge. With only 10% of the block space to play with it
can't be massive.

This requires that transactions include scriptPubKey information when
broadcasting them.

Post by Rusty Russell
5) Add a CHECKSIG cost. Naively, since we allow 20,000 CHECKSIGs and
1MB blocks, that implies a cost of 50 bytes per CHECKSIG (but counted
correctly, unlike now).
This last one implies that the initial cost limit would be 2M, but in
practice probably somewhere in the middle.
tx_cost = 50*num-CHECKSIG
+ tx_bytes
+ 4*utxo_created_size
- 3*utxo_consumed_size

Post by Tier Nolan
A 250 byte transaction with 2 inputs and 2 outputs would have an adjusted
size of 252 bytes.

Now cost == 352.

That is to large a cost for a 10% block change. It could be included in
the block size hard fork though. I think have one combined "cost" for
transactions is good. It means much fewer spread out transaction checks.
The code for the cost formula would be in one place.

[Rusty Russell]

Post by Tier Nolan

Post by Rusty Russell
3) ... or maybe not, if any consumed UTXO was generated before the soft
fork (reducing Tier's perverse incentive).

The incentive problem can be fixed by excluding UTXOs from blocks before a
certain count.
UTXOs in blocks before 375000 don't count.

OK. Be nice if these were cleaned up, but I guess it's a sunk cost.

Post by Tier Nolan

Post by Rusty Russell
4) How do we measure UTXO size? There are some constant-ish things in
there (eg. txid as key, height, outnum, amount). Maybe just add 32
to scriptlen?

They can be stored as a fixed digest. That can be any size, depending on
security requirements.
Gmaxwell's cost proposal is 3-4 bytes per UTXO change. It isn't
4*UXTO.size - 3*UTXO.size

He said "utxo_created_size" not "utxo_created" so I assumed scriptlen?

Post by Tier Nolan
It is only a small nudge. With only 10% of the block space to play with it
can't be massive.

But you made that number up? The soft cap and hard byte limit are
different beasts, so there's no need for soft cost cap < hard byte
limit.

Post by Tier Nolan
This requires that transactions include scriptPubKey information when
broadcasting them.

Brilliant! I completely missed that possibility...

Post by Tier Nolan

Post by Rusty Russell
5) Add a CHECKSIG cost. Naively, since we allow 20,000 CHECKSIGs and
1MB blocks, that implies a cost of 50 bytes per CHECKSIG (but counted
correctly, unlike now).
This last one implies that the initial cost limit would be 2M, but in
practice probably somewhere in the middle.
tx_cost = 50*num-CHECKSIG
+ tx_bytes
+ 4*utxo_created_size
- 3*utxo_consumed_size

Post by Tier Nolan
A 250 byte transaction with 2 inputs and 2 outputs would have an adjusted
size of 252 bytes.

Now cost == 352.

That is to large a cost for a 10% block change. It could be included in
the block size hard fork though.

I don't think so. Again, you're mixing units.

Post by Tier Nolan
I think have one combined "cost" for
transactions is good. It means much fewer spread out transaction checks.
The code for the cost formula would be in one place.

Agreed! Unfortunately there'll always be 2, because we really do want a
hard byte limit: it's total tx bytes which brings most concerns about
centralization. But ideally it'll be so rarely hit that it can be ~
ignored (and certainly not optimized for).

Cheers,
Rusty.

[Tier Nolan]

Post by Rusty Russell
OK. Be nice if these were cleaned up, but I guess it's a sunk cost.

Yeah.

On the plus side, as people spend their money, old UTXOs would be used up
and then they would be included in the cost function. It is only people
who are storing their money long term that wouldn't.

They are unlikely to have consumed their UTXOs anyway, unless miners
started paying for UTXOs.

We could make it a range.

UTXOs from below 355,000 and above 375,000 are included. That can create
incentive problems for the next similar change, I think a future threshold
is better.

Post by Rusty Russell
He said "utxo_created_size" not "utxo_created" so I assumed scriptlen?

Maybe I mis-read.

Post by Rusty Russell
But you made that number up? The soft cap and hard byte limit are
different beasts, so there's no need for soft cost cap < hard byte
limit.

I was thinking about it being a soft-fork.

If it was combined with the 20MB limit change, then it can be anything.

I made a suggestion somewhere (her or forums not sure), that transactions
should be allowed to store bytes.

For example, a new opcode could be added, <byte_count> OP_LOCK_BYTES.

This makes the transaction seem <byte_count> larger. However, when
spending the UTXO, that transaction counts as <byte_count> smaller, even
against the hard-cap.

This would be useful for channels. If channels were 100-1000X the
blockchain volume and someone caused lots of channels to close, there
mightn't be enough space for all the close channel transactions. Some
people might be able to get their refund transactions included in the
blockchain because the timeout expires.

If transactions could store enough space to be spent, then a mass channel
close would cause some very large blocks, but then they would have to be
followed by lots of tiny blocks.

The block limit would be an average not fixed per block. There would be 3
limits

Absolute hard limit (max bytes no matter what): 100MB
Hard limit (max bytes after stored bytes offset): 30MB
Soft limit (max bytes equivalents): 10MB

Blocks lager than ~32MB require a new network protocol, which makes the
hard fork even "harder". The protocol change could be "messages can now be
150MB max" though, so maybe not so complex.

Post by Rusty Russell

Post by Tier Nolan
This requires that transactions include scriptPubKey information when
broadcasting them.

Brilliant! I completely missed that possibility...

I have written a BIP about it. It is still in the draft stage. I had a
look into writing up the code for the protocol change.

https://github.com/TierNolan/bips/blob/extended_transactions/bip-etx.mediawiki
https://github.com/TierNolan/bips/blob/extended_transactions/bip-etx-fork.mediawiki

[Thomas Voegtlin]

Post by Mark Friedenbach
* For each block, the miner is allowed to select a different difficulty
(nBits) within a certain range, e.g. +/- 25% of the expected difficulty,
and this miner-selected difficulty is used for the proof of work check. In
addition to adjusting the hashcash target, selecting a different difficulty
also raises or lowers the maximum block size for that block by a function
of the difference in difficulty. So increasing the difficulty of the block
by an additional 25% raises the block limit for that block from 100% of the
current limit to 125%, and lowering the difficulty by 10% would also lower
the maximum block size for that block from 100% to 90% of the current
limit. For simplicity I will assume a linear identity transform as the
function, but a quadratic or other function with compounding marginal cost
may be preferred.

Sorry but I fail to see how a linear identity transform between block
size and difficulty would work.

The miner's reward for finding a block is the sum of subsidy and fees:

R = S + F

The probability that the miner will find a block over a time interval is
inversely proportional to the difficulty D:

P = K / D

where K is a constant that depends on the miner's hashrate. The expected
reward of the miner is:

E = P * R

Consider that the miner chooses a new difficulty:

D' = D(1 + x).

With a linear identity transform between block size and difficulty, the
miner will be allowed to collect fees from a block of size: S'=S(1+x)

In the best case, collected will be proportional to block size:

F' = F(1+x)

Thus we get:

E' = P' * R' = K/(D(1+x)) * (S + F(1+x))

E' = E - x/(1+x) * S * K / D

So with this linear identity transform, increasing block size never
increases the miners gain. As long as the subsidy exists, the best
strategy for miners is to reduce block size (i.e. to choose x<0).

[Mark Friedenbach]

I'm on my phone today so I'm somewhat constrained in my reply, but the key
takeaway is that the proposal is a mechanism for miners to trade subsidy
for the increased fees of a larger block. Necessarily it only makes sense
to do so when the marginal fee per KB exceeds the subsidy fee per KB. It
correspondingly makes sense to use a smaller block size if fees are less
than subsidy, but note that fees are not uniform and as the block shrinks
the marginal fee rate goes up..

Limits on both the relative and absolute amount a miner can trade subsidy
for block size prevent incentive edge cases as well as prevent a sharp
shock to the current fee-poor economy (by disallowing adjustment below 1MB).

Also the identity transform was used only for didactic purposes. I fully
expect there to be other, more interesting functions to use.

Post by Steven Pine

Post by Mark Friedenbach
* For each block, the miner is allowed to select a different difficulty
(nBits) within a certain range, e.g. +/- 25% of the expected difficulty,
and this miner-selected difficulty is used for the proof of work check.

In

Post by Mark Friedenbach
addition to adjusting the hashcash target, selecting a different

difficulty

Post by Mark Friedenbach
also raises or lowers the maximum block size for that block by a function
of the difference in difficulty. So increasing the difficulty of the

block

Post by Mark Friedenbach
by an additional 25% raises the block limit for that block from 100% of

the

Post by Mark Friedenbach
current limit to 125%, and lowering the difficulty by 10% would also

lower

Post by Mark Friedenbach
the maximum block size for that block from 100% to 90% of the current
limit. For simplicity I will assume a linear identity transform as the
function, but a quadratic or other function with compounding marginal

cost

Post by Mark Friedenbach
may be preferred.

Sorry but I fail to see how a linear identity transform between block
size and difficulty would work.
R = S + F
The probability that the miner will find a block over a time interval is
P = K / D
where K is a constant that depends on the miner's hashrate. The expected
E = P * R
D' = D(1 + x).
With a linear identity transform between block size and difficulty, the
miner will be allowed to collect fees from a block of size: S'=S(1+x)
F' = F(1+x)
E' = P' * R' = K/(D(1+x)) * (S + F(1+x))
E' = E - x/(1+x) * S * K / D
So with this linear identity transform, increasing block size never
increases the miners gain. As long as the subsidy exists, the best
strategy for miners is to reduce block size (i.e. to choose x<0).
------------------------------------------------------------------------------
One dashboard for servers and applications across Physical-Virtual-Cloud
Widest out-of-the-box monitoring support with 50+ applications
Performance metrics, stats and reports that give you Actionable Insights
Deep dive visibility with transaction tracing using APM Insight.
http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
_______________________________________________
Bitcoin-development mailing list
https://lists.sourceforge.net/lists/listinfo/bitcoin-development

[Thomas Voegtlin]

Post by Mark Friedenbach
I'm on my phone today so I'm somewhat constrained in my reply, but the key
takeaway is that the proposal is a mechanism for miners to trade subsidy
for the increased fees of a larger block. Necessarily it only makes sense
to do so when the marginal fee per KB exceeds the subsidy fee per KB. It
correspondingly makes sense to use a smaller block size if fees are less
than subsidy, but note that fees are not uniform and as the block shrinks
the marginal fee rate goes up..

Oh I see, you expect the sign of the dE/dx to change depending on
whether fees exceed the subsidy. This is possible, but instead of the
linear identity, you have to increase the block size twice as fast as
the difficulty. In that case we would get (using the notations of my
previous email):

D' = D(1+x)
F' = F(1+2x)

and thus:

E' - E = x/(1+x)P(F-S)

The presence of the (F-S) factor means that the sign reversal occurs
when fees exceed subsidy.

Post by Mark Friedenbach
Limits on both the relative and absolute amount a miner can trade subsidy
for block size prevent incentive edge cases as well as prevent a sharp
shock to the current fee-poor economy (by disallowing adjustment below 1MB).
Also the identity transform was used only for didactic purposes. I fully
expect there to be other, more interesting functions to use.

[Gavin Andresen]

Post by Matt Whitlock
Between all the flames on this list, several ideas were raised that did
not get much attention. I hereby resubmit these ideas for consideration and
discussion.
- Perhaps the hard block size limit should be a function of the actual
block sizes over some trailing sampling period. For example, take the
median block size among the most recent 2016 blocks and multiply it by 1.5.
This allows Bitcoin to scale up gradually and organically, rather than
having human beings guessing at what is an appropriate limit.

A lot of people like this idea, or something like it. It is nice and
simple, which is really important for consensus-critical code.

With this rule in place, I believe there would be more "fee pressure"
(miners would be creating smaller blocks) today. I created a couple of
histograms of block sizes to infer what policy miners are ACTUALLY
following today with respect to block size:

Last 1,000 blocks:
http://bitcoincore.org/~gavin/sizes_last1000.html

Notice a big spike at 750K -- the default size for Bitcoin Core.
This graph might be misleading, because transaction volume or fees might
not be high enough over the last few days to fill blocks to whatever limit
miners are willing to mine.

So I graphed a time when (according to statoshi.info) there WERE a lot of
transactions waiting to be confirmed:
http://bitcoincore.org/~gavin/sizes_357511.html

That might also be misleading, because it is possible there were a lot of
transactions waiting to be confirmed because miners who choose to create
small blocks got lucky and found more blocks than normal. In fact, it
looks like that is what happened: more smaller-than-normal blocks were
found, and the memory pool backed up.

So: what if we had a dynamic maximum size limit based on recent history?

The average block size is about 400K, so a 1.5x rule would make the max
block size 600K; miners would definitely be squeezing out transactions /
putting pressure to increase transaction fees. Even a 2x rule (implying
800K max blocks) would, today, be squeezing out transactions / putting
pressure to increase fees.

Using a median size instead of an average means the size can increase or
decrease more quickly. For example, imagine the rule is "median of last
2016 blocks" and 49% of miners are producing 0-size blocks and 51% are
producing max-size blocks. The median is max-size, so the 51% have total
control over making blocks bigger. Swap the roles, and the median is
min-size.

Because of that, I think using an average is better-- it means the max size
will change (up or down) more slowly.

I also think 2016 blocks is too long, because transaction volumes change
quicker than that. An average over 144 blocks (last 24 hours) would be
better able to handle increased transaction volume around major holidays,
and would also be able to react more quickly if an economically irrational
attacker attempted to flood the network with fee-paying transactions.

So my straw-man proposal would be: max size 2x average size over last 144
blocks, calculated at every block.

There are a couple of other changes I'd pair with that consensus change:

+ Make the default mining policy for Bitcoin Core neutral-- have its target
block size be the average size, so miners that don't care will "go along
with the people who do care."

+ Use something like Greg's formula for size instead of bytes-on-the-wire,
to discourage bloating the UTXO set.


---------

When I've proposed (privately, to the other core committers) some dynamic
algorithm the objection has been "but that gives miners complete control
over the max block size."

I think that worry is unjustified right now-- certainly, until we have
size-independent new block propagation there is an incentive for miners to
keep their blocks small, and we see miners creating small blocks even when
there are fee-paying transactions waiting to be confirmed.

I don't even think it will be a problem if/when we do have size-independent
new block propagation, because I think the combination of the random timing
of block-finding plus a dynamic limit as described above will create a
healthy system.

If I'm wrong, then it seems to me the miners will have a very strong
incentive to, collectively, impose whatever rules are necessary (maybe a
soft-fork to put a hard cap on block size) to make the system healthy again.

[Mike Hearn]

Even a 2x rule (implying 800K max blocks) would, today, be squeezing out
transactions / putting pressure to increase fees .....
So my straw-man proposal would be: max size 2x average size over last 144
blocks, calculated at every block.

Isn't that a step backwards, then? I see no reason for fee pressure to
exist at the moment. All it's doing is turning away users for no purpose:
mining isn't supported by fees, and the tiny fees we use right now seem to
be good enough to stop penny flooding.

Why not set the max size to be 20x the average size? Why 2x, given you just
pointed out that'd result in blocks shrinking rather than growing.

[Gavin Andresen]

Post by Mike Hearn
Isn't that a step backwards, then? I see no reason for fee pressure to

Post by Mike Hearn
mining isn't supported by fees, and the tiny fees we use right now seem to
be good enough to stop penny flooding.

Why not set the max size to be 20x the average size? Why 2x, given you
just pointed out that'd result in blocks shrinking rather than growing.

Twenty is scary.

And two is a very neutral number: if 50% of hashpower want the max size to
grow as fast as possible and 50% are dead-set opposed to any increase in
max size, then half produce blocks 2 times as big, half produce empty
blocks, and the max size doesn't change. If it was 20, then a small
minority of miners could force a max size increase. (if it is less than 2,
then a minority of minors can force the block size down)


As for whether there "should" be fee pressure now or not: I have no
opinion, besides "we should make block propagation faster so there is no
technical reason for miners to produce tiny blocks." I don't think us
developers should be deciding things like whether or not fees are too high,
too low, .....

[Mike Hearn]

Post by Gavin Andresen
Twenty is scary.

To whom? The only justification for the max size is DoS attacks, right?
Back when Bitcoin had an average block size of 10kb, the max block size was
100x the average. Things worked fine, nobody was scared.

The max block size is really a limit set by hardware capability, which is
something that's difficult to measure in software. I think I preferred your
original formula that guesstimated based on previous trends to one that
just tries to follow some average.

As noted, many miners just accept the defaults. With your proposed change
their target would effectively *drop* from 1mb to 800kb today, which seems
crazy. That's the exact opposite of what is needed right now.

I am very skeptical about this idea.

Post by Gavin Andresen
I don't think us developers should be deciding things like whether or not
fees are too high, too low,

Miners can already attempt to apply fee pressure by just not mining
transactions that they feel don't pay enough. Some sort of auto-cartel that
attempts to restrict supply based on everyone looking at everyone else
feels overly complex and prone to strange situations: it looks a lot like
some kind of Mexican standoff to me.

Additionally, the justification for the block size limit was DoS by someone
mining "troll blocks". It was never meant to be about fee pressure.
Resource management inside Bitcoin Core is certainly something to be
handled by developers.

[Gavin Andresen]

Post by Mike Hearn
As noted, many miners just accept the defaults. With your proposed change

Post by Mike Hearn
their target would effectively *drop* from 1mb to 800kb today, which
seems crazy. That's the exact opposite of what is needed right now.

I am very skeptical about this idea.

By the time a hard fork can happen, I expect average block size will be
above 500K.

Would you support a rule that was "larger of 1MB or 2x average size" ? That
is strictly better than the situation we're in today.

[Mike Hearn]

Post by Gavin Andresen
By the time a hard fork can happen, I expect average block size will be
above 500K.

Yes, possibly.

Post by Gavin Andresen
Would you support a rule that was "larger of 1MB or 2x average size" ?
That is strictly better than the situation we're in today.

It is, but only by a trivial amount - hitting the limit is still very
likely. I don't want to see this issue come up over and over again. Ideally
never. We shouldn't be artificially throttling organic growth of the
network, especially not by accident.

IMO it's not even clear there needs to be a size limit at all. Currently
the 32mb message cap imposes one anyway, but if miners can always just
discourage blocks over some particular size if they want to.

But I can get behind a 20mb limit (or 20mb+N) as it represents a reasonable
compromise: the limit still exists, it's far below VISA capacity etc, but
it should also free up enough space that everyone can get back to what we
*should* be focusing on, which is user growth!

[Tier Nolan]

Post by Mike Hearn
IMO it's not even clear there needs to be a size limit at all. Currently
the 32mb message cap imposes one anyway

If the plan is a fix once and for all, then that should be changed too. It
could be set so that it is at least some multiple of the max block size
allowed.

Alternatively, the merkle block message already incorporates the required
functionality.

Send
- headers message (with 1 header)
- merkleblock messages (max 1MB per message)

The transactions for each merkleblock could be sent directly before each
merkleblock, as is currently the case.

That system can send a block of any size. It would require a change to the
processing of any merkleblocks received.

[Mike Hearn]

Post by Tier Nolan
If the plan is a fix once and for all, then that should be changed too.
It could be set so that it is at least some multiple of the max block size
allowed.

Well, but RAM is not infinite :-) Effectively what these caps are doing is
setting the minimum hardware requirements for running a Bitcoin node.

That's OK by me - I don't think we are actually going to exhaust the
hardware abilities of any reasonable computer any time soon, but still,
having the software recognise the finite nature of a computing machine
doesn't seem unwise.

Post by Tier Nolan
That system can send a block of any size. It would require a change to
the processing of any merkleblocks received.

Not "any" size because, again, the remote node must buffer things up and
have the transaction data actually in memory in order to digest it. But a
much larger size, yes.

However, that's a bigger change.

[Gavin Andresen]

What do other people think?


If we can't come to an agreement soon, then I'll ask for help
reviewing/submitting patches to Mike's Bitcoin-Xt project that implement a
big increase now that grows over time so we may never have to go through
all this rancor and debate again.

I'll then ask for help lobbying the merchant services and exchanges and
hosted wallet companies and other bitcoind-using-infrastructure companies
(and anybody who agrees with me that we need bigger blocks sooner rather
than later) to run Bitcoin-Xt instead of Bitcoin Core, and state that they
are running it. We'll be able to see uptake on the network by monitoring
client versions.

Perhaps by the time that happens there will be consensus bigger blocks are
needed sooner rather than later; if so, great! The early deployment will
just serve as early testing, and all of the software already deployed will
ready for bigger blocks.

But if there is still no consensus among developers but the "bigger blocks
now" movement is successful, I'll ask for help getting big miners to do the
same, and use the soft-fork block version voting mechanism to (hopefully)
get a majority and then a super-majority willing to produce bigger blocks.
The purpose of that process is to prove to any doubters that they'd better
start supporting bigger blocks or they'll be left behind, and to give them
a chance to upgrade before that happens.


Because if we can't come to consensus here, the ultimate authority for
determining consensus is what code the majority of merchants and exchanges
and miners are running.

[i***@national.shitposting.agency]

Are you really that pig headed that you are going to try and blow up the
entire system just to get your way? A bunch of ignorant redditors do not
make consensus, mercifully.

Post by Gavin Andresen
What do other people think?
If we can't come to an agreement soon, then I'll ask for help
reviewing/submitting patches to Mike's Bitcoin-Xt project that
implement a big increase now that grows over time so we may never have
to go through all this rancor and debate again.
I'll then ask for help lobbying the merchant services and exchanges
and hosted wallet companies and other bitcoind-using-infrastructure
companies (and anybody who agrees with me that we need bigger blocks
sooner rather than later) to run Bitcoin-Xt instead of Bitcoin Core,
and state that they are running it. We'll be able to see uptake on the
network by monitoring client versions.
Perhaps by the time that happens there will be consensus bigger blocks
are needed sooner rather than later; if so, great! The early
deployment will just serve as early testing, and all of the software
already deployed will ready for bigger blocks.
But if there is still no consensus among developers but the "bigger
blocks now" movement is successful, I'll ask for help getting big
miners to do the same, and use the soft-fork block version voting
mechanism to (hopefully) get a majority and then a super-majority
willing to produce bigger blocks. The purpose of that process is to
prove to any doubters that they'd better start supporting bigger
blocks or they'll be left behind, and to give them a chance to upgrade
before that happens.
Because if we can't come to consensus here, the ultimate authority for
determining consensus is what code the majority of merchants and
exchanges and miners are running.
--
--
Gavin Andresen
------------------------------------------------------------------------------
_______________________________________________
Bitcoin-development mailing list
https://lists.sourceforge.net/lists/listinfo/bitcoin-development

------------------------------------------------------------------------------

[Braun Brelin]

How is this being pigheaded? In my opinion, this is leadership. If
*something* isn't implemented soon, the network is going to have some real
problems, right at the
time when adoption is starting to accelerate. I've been seeing nothing but
navel-gazing and circlejerks on this issue for weeks now. Gavin or Mike or
someone at some
point needs to step up and say "follow me".

Braun Brelin

Post by i***@national.shitposting.agency
Are you really that pig headed that you are going to try and blow up the
entire system just to get your way? A bunch of ignorant redditors do not
make consensus, mercifully.

Post by Gavin Andresen
What do other people think?
If we can't come to an agreement soon, then I'll ask for help
reviewing/submitting patches to Mike's Bitcoin-Xt project that
implement a big increase now that grows over time so we may never have
to go through all this rancor and debate again.
I'll then ask for help lobbying the merchant services and exchanges
and hosted wallet companies and other bitcoind-using-infrastructure
companies (and anybody who agrees with me that we need bigger blocks
sooner rather than later) to run Bitcoin-Xt instead of Bitcoin Core,
and state that they are running it. We'll be able to see uptake on the
network by monitoring client versions.
Perhaps by the time that happens there will be consensus bigger blocks
are needed sooner rather than later; if so, great! The early
deployment will just serve as early testing, and all of the software
already deployed will ready for bigger blocks.
But if there is still no consensus among developers but the "bigger
blocks now" movement is successful, I'll ask for help getting big
miners to do the same, and use the soft-fork block version voting
mechanism to (hopefully) get a majority and then a super-majority
willing to produce bigger blocks. The purpose of that process is to
prove to any doubters that they'd better start supporting bigger
blocks or they'll be left behind, and to give them a chance to upgrade
before that happens.
Because if we can't come to consensus here, the ultimate authority for
determining consensus is what code the majority of merchants and
exchanges and miners are running.
--
--
Gavin Andresen

------------------------------------------------------------------------------

Post by Gavin Andresen
_______________________________________________
Bitcoin-development mailing list
https://lists.sourceforge.net/lists/listinfo/bitcoin-development

------------------------------------------------------------------------------
_______________________________________________
Bitcoin-development mailing list
https://lists.sourceforge.net/lists/listinfo/bitcoin-development

[Tier Nolan]

Post by Gavin Andresen
But if there is still no consensus among developers but the "bigger blocks
now" movement is successful, I'll ask for help getting big miners to do the
same, and use the soft-fork block version voting mechanism to (hopefully)
get a majority and then a super-majority willing to produce bigger blocks.
The purpose of that process is to prove to any doubters that they'd better
start supporting bigger blocks or they'll be left behind, and to give them
a chance to upgrade before that happens.

How do you define that the movement is successful?

For

Post by Gavin Andresen
Because if we can't come to consensus here, the ultimate authority for
determining consensus is what code the majority of merchants and exchanges
and miners are running.

The measure is miner consensus. How do you intend to measure
exchange/merchant acceptance?

[Gavin Andresen]

How do you intend to measure exchange/merchant acceptance?

Public statements saying "we're running software that is ready for bigger
blocks."

And looking at the version (aka user-agent) strings of publicly reachable
nodes on the network.
(e.g. see the count at https://getaddr.bitnodes.io/nodes/ )

[Mike Hearn]

Post by Gavin Andresen
And looking at the version (aka user-agent) strings of publicly reachable
nodes on the network.
(e.g. see the count at https://getaddr.bitnodes.io/nodes/ )

Yeah, though FYI Luke informed me last week that I somehow managed to take
out the change to the user-agent string in Bitcoin XT, presumably I made a
mistake during a rebase of the rebranding change. So the actual number of
XT nodes is a bit higher than counting user-agent strings would suggest.

I sort of neglected XT lately. If we go ahead with this then I'll fix
things like this.

[Mike Hearn]

Post by Tier Nolan
The measure is miner consensus. How do you intend to measure
exchange/merchant acceptance?

Asking them.

In fact, we already have. I have been talking to well known people and CEOs
in the Bitcoin community for some time now. *All* of them support bigger
blocks, this includes:

- Every wallet developer I have asked (other than Bitcoin Core)
- So far, every payment processor and every exchange company

I know Gavin has also been talking to people about this.

There's a feeling on this list that there's no consensus, or that Gavin and
myself are on the wrong side of it. I'd put it differently - there's very
strong consensus out in the wider community and this list is something of
an aberration.

[Tier Nolan]

Post by Tier Nolan

Post by Gavin Andresen
But if there is still no consensus among developers but the "bigger
blocks now" movement is successful, I'll ask for help getting big miners to
do the same, and use the soft-fork block version voting mechanism to
(hopefully) get a majority and then a super-majority willing to produce
bigger blocks. The purpose of that process is to prove to any doubters that
they'd better start supporting bigger blocks or they'll be left behind, and
to give them a chance to upgrade before that happens.

How do you define that the movement is successful?

Sorry again, I keep auto-sending from gmail when trying to delete.

In theory, using the "nuclear option", the block size can be increased via
soft fork.

Version 4 blocks would contain the hash of the a valid extended block in
the coinbase.

<block height> <32 byte extended hash>

To send coins to the auxiliary block, you send them to some template.

OP_P2SH_EXTENDED <scriptPubKey hash> OP_TRUE

This transaction can be spent by anyone (under the current rules). The
soft fork would lock the transaction output unless it transferred money
from the extended block.

To unlock the transaction output, you need to include the txid of
transaction(s) in the extended block and signature(s) in the scriptSig.

The transaction output can be spent in the extended block using P2SH
against the scriptPubKey hash.

This means that people can choose to move their money to the extended
block. It might have lower security than leaving it in the root chain.

The extended chain could use the updated script language too.

This is obviously more complex than just increasing the size though, but it
could be a fallback option if no consensus is reached. It has the
advantage of giving people a choice. They can move their money to the
extended chain or not, as they wish.

[Raystonn .]

Regarding Tier’s proposal: The lower security you mention for extended blocks would delay, possibly forever, the larger blocks maximum block size that we want for the entire network. That doesn’t sound like an optimal solution.

Regarding consensus for larger maximum block size, what we are seeing on this list is typical of what we see in the U.S. Congress. Support for changes by the stakeholders (support for bills by the citizens as a whole) has become irrelevant to the probability of these changes being adopted. Lobbyists have all the sway in getting their policies enacted. In our case, I would bet on some lobbying of core developers by wealthy miners.

Someone recently proposed that secret ballots could help eliminate the power of lobbyists in Congress. Nobody invests in that which cannot be confirmed. Secret ballots mean the vote you are buying cannot be confirmed. Perhaps this will work for Bitcoin Core as well.


From: Tier Nolan
Sent: Friday, May 29, 2015 7:22 AM
Cc: Bitcoin Dev
Subject: Re: [Bitcoin-development] Proposed alternatives to the 20MB stepfunction

On Fri, May 29, 2015 at 3:09 PM, Tier Nolan <***@gmail.com> wrote:



On Fri, May 29, 2015 at 1:39 PM, Gavin Andresen <***@gmail.com> wrote:

But if there is still no consensus among developers but the "bigger blocks now" movement is successful, I'll ask for help getting big miners to do the same, and use the soft-fork block version voting mechanism to (hopefully) get a majority and then a super-majority willing to produce bigger blocks. The purpose of that process is to prove to any doubters that they'd better start supporting bigger blocks or they'll be left behind, and to give them a chance to upgrade before that happens.

How do you define that the movement is successful?


Sorry again, I keep auto-sending from gmail when trying to delete.


In theory, using the "nuclear option", the block size can be increased via soft fork.


Version 4 blocks would contain the hash of the a valid extended block in the coinbase.


<block height> <32 byte extended hash>


To send coins to the auxiliary block, you send them to some template.


OP_P2SH_EXTENDED <scriptPubKey hash> OP_TRUE


This transaction can be spent by anyone (under the current rules). The soft fork would lock the transaction output unless it transferred money from the extended block.


To unlock the transaction output, you need to include the txid of transaction(s) in the extended block and signature(s) in the scriptSig.


The transaction output can be spent in the extended block using P2SH against the scriptPubKey hash.


This means that people can choose to move their money to the extended block. It might have lower security than leaving it in the root chain.


The extended chain could use the updated script language too.


This is obviously more complex than just increasing the size though, but it could be a fallback option if no consensus is reached. It has the advantage of giving people a choice. They can move their money to the extended chain or not, as they wish.



--------------------------------------------------------------------------------

[Tier Nolan]

Post by Raystonn .
Regarding Tier’s proposal: The lower security you mention for extended
blocks would delay, possibly forever, the larger blocks maximum block size
that we want for the entire network. That doesn’t sound like an optimal
solution.

I don't think so. The lower security is the potential centralisation
risk. If you have your money in the "root" chain, then you can watch it.
You can probably also watch it in a 20MB chain.

Full nodes would still verify the entire block (root + extended). It is a
"nuclear option", since you can make any changes you want to the rules for
the extended chain. The only safe guard is that people have to voluntarly
transfer coins to the extended block.

The extended block might have 10-15% of the total bitcoins, but still be
useful, since they would be the ones that move the most. If you want to
store your coins long term, you move them back to the root block where you
can watch them more closely.

It does make things more complex though. Wallets would have to list 2
balances.

[Admin Istrator]

What about trying the dynamic scaling method within the 20MB range + 1 year
with a 40% increase of that cap? Until a way to dynamically scale is
found, the cap will only continue to be an issue. With 20 MB + 40% yoy,
we're either imposing an arbitrary cap later, or achieving less than great
DOS protection always. Why not set that policy as a maximum for 2 years as
a protection against the possibility of dynamic scaling abuse, and see what
happens with a dynamic method in the mean time. The policy of Max(1MB,
(average size over previous 144 blocks) * 2) calculated at each block seems
pretty reasonable.

As an outsider, the real 'median' here seems to be 'keeping the cap as
small as possible while allowing for larger blocks still'. We know
miners will want to keep space in their blocks relatively scarce, but we
also know that doesn't exclude the more powerful miners from
including superfluous transactions to increase their effective share of the
network. I have the luck of not being drained by this topic over the past
three years, so it looks to me as if its two poles of 'block size must
increase' and 'block size must not increase' are forcing what is the clear
route to establishing the 'right' block size off the table.

--Andrew Len
(sorry if anybody received this twice, sent as the wrong email the first
time around).

Post by Gavin Andresen
What do other people think?
If we can't come to an agreement soon, then I'll ask for help
reviewing/submitting patches to Mike's Bitcoin-Xt project that implement a
big increase now that grows over time so we may never have to go through
all this rancor and debate again.
I'll then ask for help lobbying the merchant services and exchanges and
hosted wallet companies and other bitcoind-using-infrastructure companies
(and anybody who agrees with me that we need bigger blocks sooner rather
than later) to run Bitcoin-Xt instead of Bitcoin Core, and state that they
are running it. We'll be able to see uptake on the network by monitoring
client versions.
Perhaps by the time that happens there will be consensus bigger blocks are
needed sooner rather than later; if so, great! The early deployment will
just serve as early testing, and all of the software already deployed will
ready for bigger blocks.
But if there is still no consensus among developers but the "bigger blocks
now" movement is successful, I'll ask for help getting big miners to do the
same, and use the soft-fork block version voting mechanism to (hopefully)
get a majority and then a super-majority willing to produce bigger blocks.
The purpose of that process is to prove to any doubters that they'd better
start supporting bigger blocks or they'll be left behind, and to give them
a chance to upgrade before that happens.
Because if we can't come to consensus here, the ultimate authority for
determining consensus is what code the majority of merchants and exchanges
and miners are running.
--
--
Gavin Andresen
------------------------------------------------------------------------------
_______________________________________________
Bitcoin-development mailing list
https://lists.sourceforge.net/lists/listinfo/bitcoin-development

[Aaron Voisine]

or achieving less than great DOS protection

Right now a bunch of redditors can DOS the network at the cost of a few
thousand dollars per day, shared between them. Since the cost of validating
transactions is far lower than current minimum relay fees, then increasing
the block size increases the cost of DOSing the network.


Aaron Voisine
co-founder and CEO
breadwallet.com

What about trying the dynamic scaling method within the 20MB range + 1
year with a 40% increase of that cap? Until a way to dynamically scale is
found, the cap will only continue to be an issue. With 20 MB + 40% yoy,
we're either imposing an arbitrary cap later, or achieving less than great
DOS protection always. Why not set that policy as a maximum for 2 years as
a protection against the possibility of dynamic scaling abuse, and see what
happens with a dynamic method in the mean time. The policy of Max(1MB,
(average size over previous 144 blocks) * 2) calculated at each block seems
pretty reasonable.
As an outsider, the real 'median' here seems to be 'keeping the cap as
small as possible while allowing for larger blocks still'. We know
miners will want to keep space in their blocks relatively scarce, but we
also know that doesn't exclude the more powerful miners from
including superfluous transactions to increase their effective share of the
network. I have the luck of not being drained by this topic over the past
three years, so it looks to me as if its two poles of 'block size must
increase' and 'block size must not increase' are forcing what is the clear
route to establishing the 'right' block size off the table.
--Andrew Len
(sorry if anybody received this twice, sent as the wrong email the first
time around).

Post by Gavin Andresen
What do other people think?
If we can't come to an agreement soon, then I'll ask for help
reviewing/submitting patches to Mike's Bitcoin-Xt project that implement a
big increase now that grows over time so we may never have to go through
all this rancor and debate again.
I'll then ask for help lobbying the merchant services and exchanges and
hosted wallet companies and other bitcoind-using-infrastructure companies
(and anybody who agrees with me that we need bigger blocks sooner rather
than later) to run Bitcoin-Xt instead of Bitcoin Core, and state that they
are running it. We'll be able to see uptake on the network by monitoring
client versions.
Perhaps by the time that happens there will be consensus bigger blocks
are needed sooner rather than later; if so, great! The early deployment
will just serve as early testing, and all of the software already deployed
will ready for bigger blocks.
But if there is still no consensus among developers but the "bigger
blocks now" movement is successful, I'll ask for help getting big miners to
do the same, and use the soft-fork block version voting mechanism to
(hopefully) get a majority and then a super-majority willing to produce
bigger blocks. The purpose of that process is to prove to any doubters that
they'd better start supporting bigger blocks or they'll be left behind, and
to give them a chance to upgrade before that happens.
Because if we can't come to consensus here, the ultimate authority for
determining consensus is what code the majority of merchants and exchanges
and miners are running.
--
--
Gavin Andresen
------------------------------------------------------------------------------
_______________________________________________
Bitcoin-development mailing list
https://lists.sourceforge.net/lists/listinfo/bitcoin-development

------------------------------------------------------------------------------
_______________________________________________
Bitcoin-development mailing list
https://lists.sourceforge.net/lists/listinfo/bitcoin-development

[Ricardo Filipe]

I've been following the discussion of the block size limit and IMO it
is clear that any constant block size limit is, as many have said
before, just kicking the can down the road.
My problem with the dynamic lower limit solution based on past blocks
is that it doesn't account for usage spikes. I would like to propose
another dynamic lower limit scheme:
Let the block size limit be a function of the number of current
transactions in the mempool. This way, bitcoin usage regulates the
block size limit.

I'm sorry i don't have the knowledge of the code base or time to make
simulations on this kind of approach, but nevertheless I would like to
leave it here for discussion or foster other ideas.

cheers

------------------------------------------------------------------------------

[Marcel Jamin]

Let the block size limit be a function of the number of current transactions

in the mempool.

There is no single mempool which transactions could be counted and there is
no consensus about the average number of unconfirmed transactions.

I've been following the discussion of the block size limit and IMO it
is clear that any constant block size limit is, as many have said
before, just kicking the can down the road.
My problem with the dynamic lower limit solution based on past blocks
is that it doesn't account for usage spikes. I would like to propose
Let the block size limit be a function of the number of current
transactions in the mempool. This way, bitcoin usage regulates the
block size limit.
I'm sorry i don't have the knowledge of the code base or time to make
simulations on this kind of approach, but nevertheless I would like to
leave it here for discussion or foster other ideas.
cheers
------------------------------------------------------------------------------
_______________________________________________
Bitcoin-development mailing list
https://lists.sourceforge.net/lists/listinfo/bitcoin-development

[Bryan Cheng]

Post by Gavin Andresen
What do other people think?
If we can't come to an agreement soon, then I'll ask for help
reviewing/submitting patches to Mike's Bitcoin-Xt project that implement a
big increase now that grows over time so we may never have to go through
all this rancor and debate again.
I'll then ask for help lobbying the merchant services and exchanges and
hosted wallet companies and other bitcoind-using-infrastructure companies
(and anybody who agrees with me that we need bigger blocks sooner rather
than later) to run Bitcoin-Xt instead of Bitcoin Core, and state that they
are running it. We'll be able to see uptake on the network by monitoring
client versions.

While I think we'd all prefer Core to make changes like this, the current
environment may make that impossible. If this change happens in XT, we will
support the necessary changes in our own implementation. The block size
limit is a problem _today_, and I'd rather we solve today's problems with
today's understanding rather than let speculation about future unknowns
stop our ability to respond to known issues.

[Cameron Garnham]

First off, I am glad that the idea of dynamic block size adjustment is
gaining some attention, in particular the model that I proposed.

I wanted to take some time and explain some of the philosophy of how,
and why, I proposed this this particular model.

When Bitcoin was first made, there was a 32MB block size limit; this
was quickly found to be open to spam (and potentially DOS, as the code
was not-at-all optimized to support large blocks), and was reduced to
1MB, this was a quick fix that was never intended to last; at some
point the network should come to an understanding, a consensus if you
will, of what (and how much) belongs in a block.
The core point of this is that miners have always, and will always;
hold the power, to decide what goes into blocks; this implicitly,
obviously, includes how large blocks are. Miners are able to come any
sort of agreement they wish, providing the bitcoin clients accept
their blocks as valid.

Say if Satoshi never decided to place the 1MB block limit: It would be
up to the miners to decide what they consider a ‘reasonable’ block is.
However, they would need to find some way to communicate this and
reach an agreement; some protocol. They, say, could have done this
informally on what is now the bitcointalk forum, or used Twitter.
However, what they really need is indeed a "consensus protocol". Some
simple terms to define what is acceptable and what is not.

Hence, the proposal introducing a consensus protocol for block sizes;
instead of just having a hard limit (enforced by everyone), instead,
we have a constant factor above the average block size over a fixed
intervals that is soft-forked by only the miners. (The next simplest
mathematical construct).
This proposal is entirely a soft-fork and may be implemented without
changing any client code what so ever. In-fact, it could be
implemented by only a simple 51% majority of miners, with-or-without
gaining the wider community consensus. (Assuming that the 1MB block
size rule still applies).
The nice thing about this is that it really is impossible to stop,
for-example, if pre-relaying of block headers is implemented; the
miners could always soft-fork to include the block-size in the
coinbase. The only reason that the miners have not done this yet, is
that there has not yet been a strong will to increase transaction fees.

If we assume the miners will operate in a way to collectively maximize
profit; then we can assume they will not try to maximize utility of
the network (having as many transactions as possible), rather have as
few transactions as the total economy can support the cost. Meaning
that limiting to much smaller blocks will probably be much more
profitable than having large blocks.

Since there is no requirement for the clients to know about the block
size consensus protocol, this truly can be a
‘bi-directional-soft-fork’, in that the miners can choose to change
the rules at any time, with only a simple 51% majority. Therefore, any
parameters that we pick are always up for debate.

Why the 1.5x over 2016 blocks? - Using some game theory, and
deduction: I wished to pick the type of agreement that would be
natural for the miners to come to (selfishly).

First, Why 1.5x, this means that only a super-majority of miners can
easily increase the block size. – There is no natural incentive for
miners to produce large blocks that have very few fees.

Second, Why 2016 blocks for adjusting the average: Miners HATE
unpredictability, for shorter time periods the miner will need to have
infrastructure ready to support potentially much larger block almost
immediately. 2016 blocks is a period that the miners are already well
used to, meaning that it will take slightly less than a month for
blocks of double size to be permitted.

This entire infrastructure can be implemented without needing to
update any clients; once implemented, tested, solid, and well accepted
by the (mining) community then we can revisit increasing the 1M hard
limit. (If we still have demand for it, maybe the average block size
will reduce to say, 100KB).


Cam.

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256
While being in the Bitcoin community for a long time, I haven't
been so directly involved in the development. However I wish to
Set a 'block size cap' in the similar same way as we set
difficulty.
Every 2016 blocks take the average size of the blocks and multiply
the size by 1.5x, rejecting blocks that are larger than this size,
for the next 2016 period.
I would of-course suggest that we keep the limits at min 100kb and
max (initially) 990kb (not 1mb on purpose, as this should become
the new limit), rounding up to the nearest 10kb.
A: we don't have pressure at the 1mb limit, (we reduce the limit in
a flexible manner to 990kb).
B: we can upgrade the network to XYZ hard-limit, then slowly raze
the soft-limit after being sure the network, as-a-whole is ready.
If we on-day remove the block-size limit, this rule will stop a
rouge miner from making 10mb, or 100mb blocks, or 1gb blocks.
This could be implemented by the miners without breaking any of
the clients, and would tend to produce a better dynamic fee
pressure.
This will give the mechanics to the miners to create consensus to
agree what block-sizes they believe are best for the network, and
allows the block-sizes to dynamically grow in response to larger
demand.

I would not modify my node if the change introduced a
perpetual 100 BTC subsidy per block, even if 99% of miners
went along with it.

Surely, in that scenario Bitcoin is dead. If the fork you
prefer has only 1% of the hash power it is trivially vulnerably
not just to a 51% attack but to a 501% attack, not to mention
the fact that you'd only be getting one block every 16 hours.

Yes, indeed, Bitcoin would be dead if this actually happens. But
that is still where the power lies: before anyone (miners or
others) would think about trying such a change, they would need
to convince people and be sure they will effectively modify
their code.
----------------------------------------------------------------------

- --------

One dashboard for servers and applications across
Physical-Virtual-Cloud

Widest out-of-the-box monitoring support with 50+ applications
Performance metrics, stats and reports that give you Actionable
Insights Deep dive visibility with transaction tracing using APM
Insight. http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
_______________________________________________
Bitcoin-development mailing list
https://lists.sourceforge.net/lists/listinfo/bitcoin-development

-----BEGIN PGP SIGNATURE----- Version: GnuPG v2
iF4EAREIAAYFAlVMKZYACgkQBJ8cMDO159aTiQEApTITEBrhE1DRbj/w+GncNeqB
0hGvmIBa1z0hGww0kaMBAOhxjn/K5leRJgdt1fKhNEDKKHdeCOIX3QRgry90D3NO
=p0+H -----END PGP SIGNATURE-----

[Raystonn .]

I agree that developers should avoid imposing economic policy. It is dangerous for Bitcoin and the core developers themselves to become such a central point of attack for those wishing to disrupt Bitcoin. My opinion is these things are better left to a decentralized free market anyhow.


From: Gavin Andresen
Sent: Thursday, May 28, 2015 10:19 AM
To: Mike Hearn
Cc: Bitcoin Dev
Subject: Re: [Bitcoin-development] Proposed alternatives to the 20MB stepfunction

On Thu, May 28, 2015 at 1:05 PM, Mike Hearn <***@plan99.net> wrote:

Isn't that a step backwards, then? I see no reason for fee pressure to exist at the moment. All it's doing is turning away users for no purpose: mining isn't supported by fees, and the tiny fees we use right now seem to be good enough to stop penny flooding.


Why not set the max size to be 20x the average size? Why 2x, given you just pointed out that'd result in blocks shrinking rather than growing.

Twenty is scary.

And two is a very neutral number: if 50% of hashpower want the max size to grow as fast as possible and 50% are dead-set opposed to any increase in max size, then half produce blocks 2 times as big, half produce empty blocks, and the max size doesn't change. If it was 20, then a small minority of miners could force a max size increase. (if it is less than 2, then a minority of minors can force the block size down)


As for whether there "should" be fee pressure now or not: I have no opinion, besides "we should make block propagation faster so there is no technical reason for miners to produce tiny blocks." I don't think us developers should be deciding things like whether or not fees are too high, too low, .....

[Pieter Wuille]

Post by Raystonn .
I agree that developers should avoid imposing economic policy. It is

dangerous for Bitcoin and the core developers themselves to become such a
central point of attack for those wishing to disrupt Bitcoin.

I could not agree more that developers should not be in charge of the
network rules.

Which is why - in my opinion - hard forks cannot be controversial things. A
controversial change to the software, forced to be adopted by the public
because the only alternative is a permanent chain fork, is a use of power
that developers (or anyone) should not have, and an incredibly dangerous
precedent for other changes that only a subset of participants would want.

The block size is also not just an economic policy. It is the compromise
the _network_ chooses to make between utility and various forms of
centralization pressure, and we should treat it as a compromise, and not as
some limit that is inferior to scaling demands.

I personally think the block size should increase, by the way, but only if
we can do it under a policy of doing it after technological growth has been
shown to be sufficient to support it without increased risk.

[Gavin Andresen]

Post by Pieter Wuille
I personally think the block size should increase, by the way, but only if
we can do it under a policy of doing it after technological growth has been
shown to be sufficient to support it without increased risk.
Can you be more specific about this? What risks are you worried about?

I've tried to cover all that I've heard about in my blog posts about why I
think the risks of 20MB blocks are outweighed by the benefits, am I missing
something?
(blog posts are linked from
http://gavinandresen.ninja/time-to-roll-out-bigger-blocks )

There is the "a sudden jump to a 20MB max might have unforseen
consequences" risk that I don't address, but a dynamic increase would fix
that.

[Peter Todd]

Post by Gavin Andresen
As for whether there "should" be fee pressure now or not: I have no
opinion, besides "we should make block propagation faster so there is no
technical reason for miners to produce tiny blocks." I don't think us
developers should be deciding things like whether or not fees are too high,
too low, .....

Note that the majority of hashing power is using Matt Corallo's block
relay network, something I confirmed the other day through my mining
contacts. Interestingly, the miners that aren't using it include some of
the largest pools; I haven't yet gotten an answer as to what their
rational for not using it was exactly.

Importantly, this does mean that block propagation is probably fairly
close to optimal already, modulo major changes to the consensus
protocol; IBLT won't improve the situation much, if any.

It's also notable that we're already having issues with miners turning
validation off as a way to lower their latency; I've been asked myself
about the possibility of creating an "SPV miner" that skips validation
while new blocks are propagating to shave off time and builds directly
off of block headers corresponding to blocks with unknown contents.

[Thomas Voegtlin]

Post by Gavin Andresen
So my straw-man proposal would be: max size 2x average size over last 144
blocks, calculated at every block.

I like that idea.

Average is a better choice than median. The median is not well defined
on discrete sets, as shown in your example, and there is no need to be
robust to outliers, thanks to the max size.


------------------------------------------------------------------------------

[Pieter Wuille]

until we have size-independent new block propagation

I don't really believe that is possible. I'll argue why below. To be clear,
this is not an argument against increasing the block size, only against
using the assumption of size-independent propagation.

There are several significant improvements likely possible to various
aspects of block propagation, but I don't believe you can make any part
completely size-independent. Perhaps the remaining aspects result in terms
in the total time that vanish compared to the link latencies for 1 MB
blocks, but there will be some block sizes for which this is no longer the
case, and we need to know where that is the case.

* You can't assume that every transaction is pre-relayed and pre-validated.
This can happen due to non-uniform relay policies (different codebases, and
future things like size-limited mempools), double spend attempts, and
transactions generated before a block had time to propagate. You've
previously argued for a policy of not including too recent transactions,
but that requires a bound on network diameter, and if these late
transactions are profitable, it has exactly the same problem as making
larger blocks non-proportionally more economic for larger pools groups if
propagation time is size dependent).
* This results in extra bandwidth usage for efficient relay protocols,
and if discrepancy estimation mispredicts the size of IBLT or error
correction data needed, extra roundtrips.
* Signature validation for unrelayed transactions will be needed at block
relay time.
* Database lookups for the inputs of unrelayed transactions cannot be
cached in advance.

* Block validation with 100% known and pre-validated transactions is not
constant time, due to updates that need to be made to the UTXO set (and
future ideas like UTXO commitments would make this effect an order of
magnitude worse).

* More efficient relay protocols also have higher CPU cost for
encoding/decoding.

Again, none of this is a reason why the block size can't increase. If
availability of hardware with higher bandwidth, faster disk/ram access
times, and faster CPUs increases, we should be able to have larger blocks
with the same propagation profile as smaller blocks with earlier technology.

But we should know how technology scales with larger blocks, and I don't
believe we do, apart from microbenchmarks in laboratory conditions.
--
Pieter

Between all the flames on this list, several ideas were raised that did
not get much attention. I hereby resubmit these ideas for consideration and
discussion.
- Perhaps the hard block size limit should be a function of the actual
block sizes over some trailing sampling period. For example, take the
median block size among the most recent 2016 blocks and multiply it by 1.5.
This allows Bitcoin to scale up gradually and organically, rather than
having human beings guessing at what is an appropriate limit.

A lot of people like this idea, or something like it. It is nice and
simple, which is really important for consensus-critical code.

With this rule in place, I believe there would be more "fee pressure"
(miners would be creating smaller blocks) today. I created a couple of
histograms of block sizes to infer what policy miners are ACTUALLY
following today with respect to block size:

Last 1,000 blocks:
http://bitcoincore.org/~gavin/sizes_last1000.html

Notice a big spike at 750K -- the default size for Bitcoin Core.
This graph might be misleading, because transaction volume or fees might
not be high enough over the last few days to fill blocks to whatever limit
miners are willing to mine.

So I graphed a time when (according to statoshi.info) there WERE a lot of
transactions waiting to be confirmed:
http://bitcoincore.org/~gavin/sizes_357511.html

That might also be misleading, because it is possible there were a lot of
transactions waiting to be confirmed because miners who choose to create
small blocks got lucky and found more blocks than normal. In fact, it
looks like that is what happened: more smaller-than-normal blocks were
found, and the memory pool backed up.

So: what if we had a dynamic maximum size limit based on recent history?

The average block size is about 400K, so a 1.5x rule would make the max
block size 600K; miners would definitely be squeezing out transactions /
putting pressure to increase transaction fees. Even a 2x rule (implying
800K max blocks) would, today, be squeezing out transactions / putting
pressure to increase fees.

Using a median size instead of an average means the size can increase or
decrease more quickly. For example, imagine the rule is "median of last
2016 blocks" and 49% of miners are producing 0-size blocks and 51% are
producing max-size blocks. The median is max-size, so the 51% have total
control over making blocks bigger. Swap the roles, and the median is
min-size.

Because of that, I think using an average is better-- it means the max size
will change (up or down) more slowly.

I also think 2016 blocks is too long, because transaction volumes change
quicker than that. An average over 144 blocks (last 24 hours) would be
better able to handle increased transaction volume around major holidays,
and would also be able to react more quickly if an economically irrational
attacker attempted to flood the network with fee-paying transactions.

So my straw-man proposal would be: max size 2x average size over last 144
blocks, calculated at every block.

There are a couple of other changes I'd pair with that consensus change:

+ Make the default mining policy for Bitcoin Core neutral-- have its target
block size be the average size, so miners that don't care will "go along
with the people who do care."

+ Use something like Greg's formula for size instead of bytes-on-the-wire,
to discourage bloating the UTXO set.


---------

When I've proposed (privately, to the other core committers) some dynamic
algorithm the objection has been "but that gives miners complete control
over the max block size."

I think that worry is unjustified right now-- certainly, until we have
size-independent new block propagation there is an incentive for miners to
keep their blocks small, and we see miners creating small blocks even when
there are fee-paying transactions waiting to be confirmed.

I don't even think it will be a problem if/when we do have size-independent
new block propagation, because I think the combination of the random timing
of block-finding plus a dynamic limit as described above will create a
healthy system.

If I'm wrong, then it seems to me the miners will have a very strong
incentive to, collectively, impose whatever rules are necessary (maybe a
soft-fork to put a hard cap on block size) to make the system healthy again.

[Aaron Voisine]

miners would definitely be squeezing out transactions / putting pressure

to increase transaction fees

I'd just like to re-iterate that transactions getting "squeezed out"
(failure after a lengthy period of uncertainty) is a radical change from
the current behavior of the network. There are plenty of avenues to create
fee pressure without resorting to such a drastic change in how the network
works today.


Aaron Voisine
co-founder and CEO
breadwallet.com

Post by Matt Whitlock
Between all the flames on this list, several ideas were raised that did
not get much attention. I hereby resubmit these ideas for consideration and
discussion.
- Perhaps the hard block size limit should be a function of the actual
block sizes over some trailing sampling period. For example, take the
median block size among the most recent 2016 blocks and multiply it by 1.5.
This allows Bitcoin to scale up gradually and organically, rather than
having human beings guessing at what is an appropriate limit.

A lot of people like this idea, or something like it. It is nice and
simple, which is really important for consensus-critical code.
With this rule in place, I believe there would be more "fee pressure"
(miners would be creating smaller blocks) today. I created a couple of
histograms of block sizes to infer what policy miners are ACTUALLY
http://bitcoincore.org/~gavin/sizes_last1000.html
Notice a big spike at 750K -- the default size for Bitcoin Core.
This graph might be misleading, because transaction volume or fees might
not be high enough over the last few days to fill blocks to whatever limit
miners are willing to mine.
So I graphed a time when (according to statoshi.info) there WERE a lot of
http://bitcoincore.org/~gavin/sizes_357511.html
That might also be misleading, because it is possible there were a lot of
transactions waiting to be confirmed because miners who choose to create
small blocks got lucky and found more blocks than normal. In fact, it
looks like that is what happened: more smaller-than-normal blocks were
found, and the memory pool backed up.
So: what if we had a dynamic maximum size limit based on recent history?
The average block size is about 400K, so a 1.5x rule would make the max
block size 600K; miners would definitely be squeezing out transactions /
putting pressure to increase transaction fees. Even a 2x rule (implying
800K max blocks) would, today, be squeezing out transactions / putting
pressure to increase fees.
Using a median size instead of an average means the size can increase or
decrease more quickly. For example, imagine the rule is "median of last
2016 blocks" and 49% of miners are producing 0-size blocks and 51% are
producing max-size blocks. The median is max-size, so the 51% have total
control over making blocks bigger. Swap the roles, and the median is
min-size.
Because of that, I think using an average is better-- it means the max
size will change (up or down) more slowly.
I also think 2016 blocks is too long, because transaction volumes change
quicker than that. An average over 144 blocks (last 24 hours) would be
better able to handle increased transaction volume around major holidays,
and would also be able to react more quickly if an economically irrational
attacker attempted to flood the network with fee-paying transactions.
So my straw-man proposal would be: max size 2x average size over last 144
blocks, calculated at every block.
+ Make the default mining policy for Bitcoin Core neutral-- have its
target block size be the average size, so miners that don't care will "go
along with the people who do care."
+ Use something like Greg's formula for size instead of bytes-on-the-wire,
to discourage bloating the UTXO set.
---------
When I've proposed (privately, to the other core committers) some dynamic
algorithm the objection has been "but that gives miners complete control
over the max block size."
I think that worry is unjustified right now-- certainly, until we have
size-independent new block propagation there is an incentive for miners to
keep their blocks small, and we see miners creating small blocks even when
there are fee-paying transactions waiting to be confirmed.
I don't even think it will be a problem if/when we do have
size-independent new block propagation, because I think the combination of
the random timing of block-finding plus a dynamic limit as described above
will create a healthy system.
If I'm wrong, then it seems to me the miners will have a very strong
incentive to, collectively, impose whatever rules are necessary (maybe a
soft-fork to put a hard cap on block size) to make the system healthy again.
--
--
Gavin Andresen
------------------------------------------------------------------------------
_______________________________________________
Bitcoin-development mailing list
https://lists.sourceforge.net/lists/listinfo/bitcoin-development

[Steven Pine]

I would support a dynamic block size increase as outlined. I have a few
questions though.

Is scaling by average block size the best and easiest method, why not scale
by transactions confirmed instead? Anyone can write and relay a
transaction, and those are what we want to scale for, why not measure it
directly?

I would prefer changes every 2016 blocks, it is a well known change and a
reasonable time period for planning on changes. Two weeks is plenty fast,
especially at a 50% rate increase, in a few months the block size could be
dramatically larger.

Daily change to size seems confusing especially considering that max block
size will be dipping up and down. Also if something breaks trying to fix it
in a day seems problematic. The hard fork database size difference error
comes to mind. Finally daily 50% increases could quickly crowd out smaller
nodes if changes happen too quickly to adapt for.

Date: Thu, 28 May 2015 11:53:41 -0400
<
Content-Type: text/plain; charset="utf-8"

Post by Matt Whitlock
Between all the flames on this list, several ideas were raised that did
not get much attention. I hereby resubmit these ideas for consideration

and

Post by Matt Whitlock
discussion.
- Perhaps the hard block size limit should be a function of the actual
block sizes over some trailing sampling period. For example, take the
median block size among the most recent 2016 blocks and multiply it by

1.5.

Post by Matt Whitlock
This allows Bitcoin to scale up gradually and organically, rather than
having human beings guessing at what is an appropriate limit.

A lot of people like this idea, or something like it. It is nice and
simple, which is really important for consensus-critical code.
With this rule in place, I believe there would be more "fee pressure"
(miners would be creating smaller blocks) today. I created a couple of
histograms of block sizes to infer what policy miners are ACTUALLY
http://bitcoincore.org/~gavin/sizes_last1000.html
Notice a big spike at 750K -- the default size for Bitcoin Core.
This graph might be misleading, because transaction volume or fees might
not be high enough over the last few days to fill blocks to whatever limit
miners are willing to mine.
So I graphed a time when (according to statoshi.info) there WERE a lot of
http://bitcoincore.org/~gavin/sizes_357511.html
That might also be misleading, because it is possible there were a lot of
transactions waiting to be confirmed because miners who choose to create
small blocks got lucky and found more blocks than normal. In fact, it
looks like that is what happened: more smaller-than-normal blocks were
found, and the memory pool backed up.
So: what if we had a dynamic maximum size limit based on recent history?
The average block size is about 400K, so a 1.5x rule would make the max
block size 600K; miners would definitely be squeezing out transactions /
putting pressure to increase transaction fees. Even a 2x rule (implying
800K max blocks) would, today, be squeezing out transactions / putting
pressure to increase fees.
Using a median size instead of an average means the size can increase or
decrease more quickly. For example, imagine the rule is "median of last
2016 blocks" and 49% of miners are producing 0-size blocks and 51% are
producing max-size blocks. The median is max-size, so the 51% have total
control over making blocks bigger. Swap the roles, and the median is
min-size.
Because of that, I think using an average is better-- it means the max

size

will change (up or down) more slowly.
I also think 2016 blocks is too long, because transaction volumes change
quicker than that. An average over 144 blocks (last 24 hours) would be
better able to handle increased transaction volume around major holidays,
and would also be able to react more quickly if an economically irrational
attacker attempted to flood the network with fee-paying transactions.
So my straw-man proposal would be: max size 2x average size over last 144
blocks, calculated at every block.
+ Make the default mining policy for Bitcoin Core neutral-- have its

target

block size be the average size, so miners that don't care will "go along
with the people who do care."
+ Use something like Greg's formula for size instead of bytes-on-the-wire,
to discourage bloating the UTXO set.
---------
When I've proposed (privately, to the other core committers) some dynamic
algorithm the objection has been "but that gives miners complete control
over the max block size."
I think that worry is unjustified right now-- certainly, until we have
size-independent new block propagation there is an incentive for miners to
keep their blocks small, and we see miners creating small blocks even when
there are fee-paying transactions waiting to be confirmed.
I don't even think it will be a problem if/when we do have

size-independent

new block propagation, because I think the combination of the random

timing

of block-finding plus a dynamic limit as described above will create a
healthy system.
If I'm wrong, then it seems to me the miners will have a very strong
incentive to, collectively, impose whatever rules are necessary (maybe a
soft-fork to put a hard cap on block size) to make the system healthy

again.

--
--
Gavin Andresen
-------------- next part --------------
An HTML attachment was scrubbed...
------------------------------

------------------------------------------------------------------------------

------------------------------
_______________________________________________
Bitcoin-development mailing list
https://lists.sourceforge.net/lists/listinfo/bitcoin-development
End of Bitcoin-development Digest, Vol 48, Issue 122
****************************************************

[Steven Pine]

My understanding, which is very likely wrong in one way or another, is
transaction size and block size are two slightly different things but
perhaps it's so negligible that block size is a fine stand-in for total
transaction throughput.

Potentially Doubling the block size everyday is frankly imprudent. The
logarithmic increases in difficulty, which were often closer to 10% or 20%
every 2016 blocks was and is plenty fast, potentially changing blocksize by
twice daily is the mentality I would expect from a startup with the move
fast break things motto.

Infrastructure takes time, not everyone wants to run a node on a virtual
amazon instance, provisioning additional hard drive and bandwidth can't
happen overnight and trying to plan when block size from one week to the
next is a total mystery would be extremely difficult.

Anyone who has spent time examining the mining difficulty increases and
trajectory knows future planning is very very hard, allowing block size to
double daily would make it impossible.

Perhaps a middle way would be 300% increase every 2016 blocks, that will
scale to 20mbs within a month or two

The problem is logarithmic increases seem slow until they seem fast. If the
network begins to grow and block size hits 20, then the next day 40, 80...
Small nodes could get swamped within a week or less.

As for your point about Christmas, Bitcoin is a global network, Christmas,
while widely celebrated, isn't the only holiday, and planning around
American buying habits seems short sighted and no different from developers
trying to choose what the right fee pressure is.

Post by Steven Pine
I would support a dynamic block size increase as outlined. I have a few

questions though.

Post by Steven Pine
Is scaling by average block size the best and easiest method, why not

scale by transactions confirmed instead? Anyone can write and relay a
transaction, and those are what we want to scale for, why not measure it
directly?

What do you mean? Transactions aren't confirmed until they're in a

block...

Post by Steven Pine
I would prefer changes every 2016 blocks, it is a well known change and

a reasonable time period for planning on changes. Two weeks is plenty fast,
especially at a 50% rate increase, in a few months the block size could be
dramatically larger.

What type of planning do you imagine is necessary?
And have you looked at transaction volumes for credit-card payment

networks around Christmas?

Post by Steven Pine
Daily change to size seems confusing especially considering that max

block size will be dipping up and down. Also if something breaks trying to
fix it in a day seems problematic. The hard fork database size difference
error comes to mind. Finally daily 50% increases could quickly crowd out
smaller nodes if changes happen too quickly to adapt for.

The bottleneck is transaction volume; blocks won't get bigger unless

there are fee-paying transactions around to pay them. What scenario are you
imagining where transaction volume increases by 50% a day for a sustained
period of time?

--
--
Gavin Andresen

[Gavin Andresen]

Can we hold off on bike-shedding the particular choice of parameters until
people have a chance to weigh in on whether or not there is SOME set of
dynamic parameters they would support right now?