Discussion:
[bitcoin-dev] Bitcoin Core 0.12.0 release candidate 1 available
Wladimir J. van der Laan via bitcoin-dev
2016-01-17 10:08:08 UTC
Permalink
Binaries for bitcoin Core version 0.12.0rc1 are available from:

https://bitcoin.org/bin/bitcoin-core-0.12.0/test/

Source code can be found on github under the signed tag

https://github.com/bitcoin/bitcoin/tree/v0.12.0rc1

This is a release candidate for a new major version release, bringing new
features, bug fixes, as well as other improvements.

Preliminary release notes for the release can be found here:

https://github.com/bitcoin/bitcoin/blob/0.12/doc/release-notes.md

Release candidates are test versions for releases. When no critical problems
are found, this release candidate will be tagged as 0.12.0.

Please report bugs using the issue tracker at github:

https://github.com/bitcoin/bitcoin/issues
xor--- via bitcoin-dev
2016-01-17 22:57:28 UTC
Permalink
On Sunday, January 17, 2016 11:08:08 AM Wladimir J. van der Laan via bitcoin-
Post by Wladimir J. van der Laan via bitcoin-dev
https://github.com/bitcoin/bitcoin/blob/0.12/doc/release-notes.md
#6057 ac5476e re-enable wallet in autoprune
But the main, handwritten part does not mention this.
Is pruning really finished, i.e. could I safely use it as a wallet "end-user"?

IMHO it would be one of the most interesting feature for users, as it could
fix the issue of taking >60 GB of disk space.

So if it is finished, please mention that
- it's finished
- how to enable it.


Thanks for your hard work! :)
Wladimir J. van der Laan via bitcoin-dev
2016-01-18 11:14:16 UTC
Permalink
Post by xor--- via bitcoin-dev
On Sunday, January 17, 2016 11:08:08 AM Wladimir J. van der Laan via bitcoin-
Post by Wladimir J. van der Laan via bitcoin-dev
https://github.com/bitcoin/bitcoin/blob/0.12/doc/release-notes.md
#6057 ac5476e re-enable wallet in autoprune
But the main, handwritten part does not mention this.
Is pruning really finished, i.e. could I safely use it as a wallet "end-user"?
It has been tested in git for almost half a year. This RC is the first binary
release that contains the functionality.

It is extremely unlikely that the wallet will eat your coins (always backup
nevertheless), but I can't guarantee there won't be some issue where the wallet
and chain get out of sync and you're forced to redownload the blockchain.
Post by xor--- via bitcoin-dev
IMHO it would be one of the most interesting feature for users, as it could
fix the issue of taking >60 GB of disk space.
So if it is finished, please mention that
- it's finished
- how to enable it.
How to enable it is still the same as mentioned in the 0.11 release notes:
https://github.com/bitcoin/bitcoin/blob/v0.11.0/doc/release-notes.md#block-file-pruning

Wladimir
xor--- via bitcoin-dev
2016-01-19 06:06:50 UTC
Permalink
Post by Wladimir J. van der Laan via bitcoin-dev
It has been tested in git for almost half a year. This RC is the first
binary release that contains the functionality.
It is extremely unlikely that the wallet will eat your coins (always backup
nevertheless), but I can't guarantee there won't be some issue where the
wallet and chain get out of sync and you're forced to redownload the
blockchain.
I think I asked the wrong way, sorry: My question was not really meant at
whether it is bug-free (testing that is the purpose of a release candidate, so
we of course don't know yet), but rather whether it is at least feature
complete now.
Post by Wladimir J. van der Laan via bitcoin-dev
Block pruning is currently incompatible with running a wallet due to the
fact that block data is used for rescanning the wallet and importing keys
or addresses (which require a rescan.) However, running the wallet with
block pruning will be supported in the near future, subject to those
limitations.
So I'm interested whether this limitation has been lifted, and the whole
feature is considered as finished.

If yes, I would highly recommend advertising it in the new release notes - as
said, the disk space reduction is a big deal.


Thank you!
Wladimir J. van der Laan via bitcoin-dev
2016-01-25 12:03:17 UTC
Permalink
Post by xor--- via bitcoin-dev
So I'm interested whether this limitation has been lifted, and the whole
feature is considered as finished.
Yes, it's exactly that limitation that has been lifted!
Post by xor--- via bitcoin-dev
If yes, I would highly recommend advertising it in the new release notes - as
said, the disk space reduction is a big deal.
Good idea, has been added by Marco Falke in commit fa31133,

Wladimir
xor--- via bitcoin-dev
2016-01-25 12:27:18 UTC
Permalink
Post by Wladimir J. van der Laan via bitcoin-dev
Post by xor--- via bitcoin-dev
If yes, I would highly recommend advertising it in the new release notes -
as said, the disk space reduction is a big deal.
Good idea, has been added by Marco Falke in commit fa31133,
To enable block pruning set prune=<N> on the command line or in
bitcoin.conf, where N is the number of MiB to allot for raw block & undo
data.
From having read the Bitcoin whitepaper quite a few months ago ago, I have the
very very basic understanding that pruning is meant to:
- delete old transaction data which merely "moves coins around"
- instead only store the "origin" (= block where coins were mined) and
"current location" of the coins, i.e. the unspent transactions. Notably, I
understood it as "this is as secure as storing everything, since we know where
the coins were created, and where they are".

So from that point of view, I would assume that there is a "natural" amount of
megabytes which a fully pruned blockchain consists of: It would be defined by
the final amount of unspent coins.
I thereby am confused why it is possible to configure a number of megabytes
"to allot for raw block & undo data". I would rather expect pruning just to be
a boolean on/off flag, and the number of megabytes to be an automatically
computed result from the natural size of the dataset.
And especially, I fear that I could set N too low, and as a result, it would
delete "too much". I mean could this result in even security relevant
transaction data being deleted?

Thus, it would be nice if you could yet once more edit the release notes to:
- explain why a N must be given
- what a "safe" value of N is. I.e. how large must N be at least to not delete
security-relevant stuff?
- maybe mention if there is a "auto" setting for N to ensure that it choses a
safe value on its own?

Sorry if my descriptions are from a layman's point of view. I intentionally
did *not* re-read the Bitcoin whitepaper to have a better understanding:
I think having a layman's understanding is a good usability test for such
stuff.
Wladimir J. van der Laan via bitcoin-dev
2016-01-25 15:05:59 UTC
Permalink
Post by xor--- via bitcoin-dev
Post by Wladimir J. van der Laan via bitcoin-dev
To enable block pruning set prune=<N> on the command line or in
bitcoin.conf, where N is the number of MiB to allot for raw block & undo
data.
From having read the Bitcoin whitepaper quite a few months ago ago, I have the
- delete old transaction data which merely "moves coins around"
- instead only store the "origin" (= block where coins were mined) and
"current location" of the coins, i.e. the unspent transactions. Notably, I
understood it as "this is as secure as storing everything, since we know where
the coins were created, and where they are".
So from that point of view, I would assume that there is a "natural" amount of
megabytes which a fully pruned blockchain consists of: It would be defined by
the final amount of unspent coins.
I thereby am confused why it is possible to configure a number of megabytes
"to allot for raw block & undo data". I would rather expect pruning just to be
a boolean on/off flag, and the number of megabytes to be an automatically
computed result from the natural size of the dataset.
And especially, I fear that I could set N too low, and as a result, it would
delete "too much". I mean could this result in even security relevant
transaction data being deleted?
The term 'pruning', unfortunately is very much overused and overloaded in the
bitcoin ecosystem. Satoshi's paper refers to UTXO pruning. This is Pieter Wuille's "ultraprune",
which has been part of Bitcoin Core for more than three years.

Block pruning is not mentioned in that paper because it is just administrative,
the only reason that nodes store historical blocks at all is to serve it to other nodes,
as well as to catch up the wallet status and for -reindexes.
I don't have time to work on the release notes right now, but if someone else
wants to contribute that'd be awesome.
Post by xor--- via bitcoin-dev
- explain why a N must be given
To give a quotum. The point is that the user can choose how much harddisk space they want to
dedicate to block storage.

Block data that is stored can be used by other software, or potentially be
served to other nodes. The latter is not implemented at the moment - it would require
a change to the P2P protocol, thus right now pruning nodes don't serve block
data at all.
Post by xor--- via bitcoin-dev
- what a "safe" value of N is. I.e. how large must N be at least to not delete
security-relevant stuff?
There is no security compromise with pruning. Any value of N is intended to be safe.

Very low values would delete undo data that may be necessary in a reorganization,
but this is prohibited by argument checks.

Release notes are not meant as a replacement or supplement for documentation.
The documentation for -prune is this:

-prune=<n>
Reduce storage requirements by pruning (deleting) old blocks. This mode
is incompatible with -txindex and -rescan. Warning: Reverting this
setting requires re-downloading the entire blockchain. (default: 0 =
disable pruning blocks, >550 = target size in MiB to use for block
files)
Post by xor--- via bitcoin-dev
- maybe mention if there is a "auto" setting for N to ensure that it choses a
safe value on its own?
As said, there is no safe or unsafe value. The lowest acceptable value is just as safe
as storing everything.

Wladimir
Simon Selitsky via bitcoin-dev
2016-01-25 15:57:53 UTC
Permalink
Post by Wladimir J. van der Laan via bitcoin-dev
Post by Wladimir J. van der Laan via bitcoin-dev
Block data that is stored can be used by other software, or potentially be
served to other nodes. The latter is not implemented at the moment - it would require a change to the P2P protocol, thus right now pruning nodes don't serve block data at all.
Why is the minimum storage quota of 550 MiB necessary for pruning nodes
if the block data is not served to other nodes ? Could the client just do transaction verification and transaction relaying and only keep the block(s)
being verified on disk ?
Post by Wladimir J. van der Laan via bitcoin-dev
Post by Wladimir J. van der Laan via bitcoin-dev
Post by Wladimir J. van der Laan via bitcoin-dev
To enable block pruning set prune=<N> on the command line or in
bitcoin.conf, where N is the number of MiB to allot for raw block & undo
data.
From having read the Bitcoin whitepaper quite a few months ago ago, I have the
- delete old transaction data which merely "moves coins around"
- instead only store the "origin" (= block where coins were mined) and
"current location" of the coins, i.e. the unspent transactions. Notably, I
understood it as "this is as secure as storing everything, since we know where
the coins were created, and where they are".
So from that point of view, I would assume that there is a "natural" amount of
megabytes which a fully pruned blockchain consists of: It would be defined by
the final amount of unspent coins.
I thereby am confused why it is possible to configure a number of megabytes
"to allot for raw block & undo data". I would rather expect pruning just to be
a boolean on/off flag, and the number of megabytes to be an automatically
computed result from the natural size of the dataset.
And especially, I fear that I could set N too low, and as a result, it would
delete "too much". I mean could this result in even security relevant
transaction data being deleted?
The term 'pruning', unfortunately is very much overused and overloaded in the
bitcoin ecosystem. Satoshi's paper refers to UTXO pruning. This is Pieter Wuille's "ultraprune",
which has been part of Bitcoin Core for more than three years.
Block pruning is not mentioned in that paper because it is just administrative,
the only reason that nodes store historical blocks at all is to serve it to other nodes,
as well as to catch up the wallet status and for -reindexes.
I don't have time to work on the release notes right now, but if someone else
wants to contribute that'd be awesome.
Post by Wladimir J. van der Laan via bitcoin-dev
- explain why a N must be given
To give a quotum. The point is that the user can choose how much harddisk space they want to
dedicate to block storage.
Block data that is stored can be used by other software, or potentially be
served to other nodes. The latter is not implemented at the moment - it would require
a change to the P2P protocol, thus right now pruning nodes don't serve block
data at all.
Post by Wladimir J. van der Laan via bitcoin-dev
- what a "safe" value of N is. I.e. how large must N be at least to not delete
security-relevant stuff?
There is no security compromise with pruning. Any value of N is intended to be safe.
Very low values would delete undo data that may be necessary in a reorganization,
but this is prohibited by argument checks.
Release notes are not meant as a replacement or supplement for documentation.
-prune=<n>
Reduce storage requirements by pruning (deleting) old blocks. This mode
is incompatible with -txindex and -rescan. Warning: Reverting this
setting requires re-downloading the entire blockchain. (default: 0 =
disable pruning blocks, >550 = target size in MiB to use for block
files)
Post by Wladimir J. van der Laan via bitcoin-dev
- maybe mention if there is a "auto" setting for N to ensure that it choses a
safe value on its own?
As said, there is no safe or unsafe value. The lowest acceptable value is just as safe
as storing everything.
Wladimir
_______________________________________________
bitcoin-dev mailing list
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
Jonas Schnelli via bitcoin-dev
2016-01-25 16:12:01 UTC
Permalink
Post by Simon Selitsky via bitcoin-dev
Why is the minimum storage quota of 550 MiB necessary for pruning
nodes if the block data is not served to other nodes ? Could the
client just do transaction verification and transaction relaying
and only keep the block(s) being verified on disk ?
We try to allow reorgs ~288 blocks deep also in pruning:

- From code comments:
Require that user allocate at least 550MB for block & undo files
(blk???.dat and rev???.dat)
At 1MB per block, 288 blocks = 288MB.
Add 15% for Undo data = 331MB
Add 20% for Orphan block rate = 397MB
We want the low water mark after pruning to be at least 397 MB and
since we prune in
full block file chunks, we need the high water mark which triggers the
prune to be
one 128MB block file + added 15% undo data = 147MB greater for a total
of 545MB
Setting the target to > than 550MB will make it likely we can respect
the target.


</jonas>
xor--- via bitcoin-dev
2016-01-25 17:33:09 UTC
Permalink
Post by Wladimir J. van der Laan via bitcoin-dev
I don't have time to work on the release notes right now, but if someone
else wants to contribute that'd be awesome.
I cooked my first pull request to resolve this:
https://github.com/bitcoin/bitcoin/pull/7416

Thanks for all the explanations you folks provided! :)

Marco Falke via bitcoin-dev
2016-01-25 14:44:08 UTC
Permalink
All of this is already implemented in the bitcoind and bitcoin gui.

The theoretic minimum for the prune target would be 0 (just the header
of the current best block) as Bitcoin Core already stores the
chainstate (about 2 GiB) regardless of what you set for -prune=<N>.

In practice, the minimum is 510, so reorgs and small rescans (may not
be implemented in 0.12) are still possible.

The clients won't let you set it below that target:
"Prune configured below the minimum of 550 MiB. Please use a higher number."

Also, keep in mind Bitcoin Core comes with a help message explaining
-prune and other command line options

--Marco

2016-01-25 13:27 GMT+01:00 xor--- via bitcoin-dev
Post by xor--- via bitcoin-dev
Post by Wladimir J. van der Laan via bitcoin-dev
Post by xor--- via bitcoin-dev
If yes, I would highly recommend advertising it in the new release notes -
as said, the disk space reduction is a big deal.
Good idea, has been added by Marco Falke in commit fa31133,
To enable block pruning set prune=<N> on the command line or in
bitcoin.conf, where N is the number of MiB to allot for raw block & undo
data.
From having read the Bitcoin whitepaper quite a few months ago ago, I have the
- delete old transaction data which merely "moves coins around"
- instead only store the "origin" (= block where coins were mined) and
"current location" of the coins, i.e. the unspent transactions. Notably, I
understood it as "this is as secure as storing everything, since we know where
the coins were created, and where they are".
So from that point of view, I would assume that there is a "natural" amount of
megabytes which a fully pruned blockchain consists of: It would be defined by
the final amount of unspent coins.
I thereby am confused why it is possible to configure a number of megabytes
"to allot for raw block & undo data". I would rather expect pruning just to be
a boolean on/off flag, and the number of megabytes to be an automatically
computed result from the natural size of the dataset.
And especially, I fear that I could set N too low, and as a result, it would
delete "too much". I mean could this result in even security relevant
transaction data being deleted?
- explain why a N must be given
- what a "safe" value of N is. I.e. how large must N be at least to not delete
security-relevant stuff?
- maybe mention if there is a "auto" setting for N to ensure that it choses a
safe value on its own?
Sorry if my descriptions are from a layman's point of view. I intentionally
I think having a layman's understanding is a good usability test for such
stuff.
_______________________________________________
bitcoin-dev mailing list
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
Loading...