Discussion:
BIP Status updates (including to Active/Final Status) - BIP 39, BIP 43, BIP 44, BIP 67, BIP 111, BIP 125, BIP 130
(too old to reply)
Luke Dashjr via bitcoin-dev
2016-08-23 20:12:11 UTC
Permalink
A number of BIPs seem ready for updating to Final Status. If there are no
objections, I will update these in 2 weeks:

BIP 39: Mnemonic code for generating deterministic keys
- Used by many wallets and hundreds of thousands of users.

BIP 44: Multi-Account Hierarchy for Deterministic Wallets
- Appears to be implemented by multiple wallets.

BIP 67: Deterministic Pay-to-script-hash multi-signature addresses through
public key sorting
- Implementations in multiple wallet software exist.

BIP 125: Opt-in Full Replace-by-Fee Signaling
- Implemented in Bitcoin Core and derivatives; appears to be in regular use on
the network.

BIP 130: sendheaders message
- Implemented in Bitcoin Core and derivatives.

Also, BIP 43 (Purpose Field for Deterministic Wallets) is an informational BIP
which appears to be guiding to some extent the creation of new BIPs; therefore
I propose its Status be upgraded to Active. I will make this update in 2 weeks
also, if no objections.

Additionally, BIP 111 (NODE_BLOOM service bit) has been implemented in Bitcoin
Core and derivatives; it is unclear if used by clients yet. Can developers of
such clients please comment and let me know: 1) if their software supports
this BIP already; 2) if not, do they intend to support it in the future?
If and only if there are any clients using this service bit already, I will
update BIP 111 to Final Status in 2 weeks also.

Thanks,

Luke
Kenneth Heutmaker via bitcoin-dev
2016-08-23 20:54:50 UTC
Permalink
Post by Luke Dashjr via bitcoin-dev
Additionally, BIP 111 (NODE_BLOOM service bit) has been implemented in Bitcoin
Core and derivatives; it is unclear if used by clients yet. Can developers of
such clients please comment and let me know: 1) if their software supports
this BIP already; 2) if not, do they intend to support it in the future?
If and only if there are any clients using this service bit already, I will
update BIP 111 to Final Status in 2 weeks also.
Multibit is adding detection of the NODE_BLOOM bit in the next 2-3 weeks.

SPV is kinda broken if the wallet doesn’t do this detection. If your wallet connects only to nodes that don’t support bloom filtering, the wallet never gets updates. We have had a spike in users reporting that their wallet isn't getting updated. To compound the problem, they rescan the blockchain and lose all of their transaction history. It has caused much panic among less technical users.

We believe that failing to detect the NODE_BLOOM bit is the culprit, although it is non-deterministic, so we aren't certain.

I imagine that other SPV wallets are having similar issues. BIP 111 really isn’t optional at this point, so it should be marked final.
Jonas Schnelli via bitcoin-dev
2016-08-24 06:59:56 UTC
Permalink
Post by Kenneth Heutmaker via bitcoin-dev
Post by Luke Dashjr via bitcoin-dev
Additionally, BIP 111 (NODE_BLOOM service bit) has been implemented in Bitcoin
Core and derivatives; it is unclear if used by clients yet. Can developers of
such clients please comment and let me know: 1) if their software supports
this BIP already; 2) if not, do they intend to support it in the future?
If and only if there are any clients using this service bit already, I will
update BIP 111 to Final Status in 2 weeks also.
Multibit is adding detection of the NODE_BLOOM bit in the next 2-3 weeks.
SPV is kinda broken if the wallet doesn’t do this detection. If your wallet connects only to nodes that don’t support bloom filtering, the wallet never gets updates. We have had a spike in users reporting that their wallet isn't getting updated. To compound the problem, they rescan the blockchain and lose all of their transaction history. It has caused much panic among less technical users.
We believe that failing to detect the NODE_BLOOM bit is the culprit, although it is non-deterministic, so we aren't certain.
I imagine that other SPV wallets are having similar issues. BIP 111 really isn’t optional at this point, so it should be marked final.
SPV Wallets should definitively update to respect NODE_BLOOM. Bloom
filtering is CPU and disk intense and some node operators have disabled
it (or will disabled it) because there is no direct p2p network-health
benefit.

SPV wallets should probably also make use of the new DNS seeder filter
option.
It is running at least on seed.bitcoin.sipa.be and
seed.bitcoin.jonasschnelli.ch.

The filter option allows SPV Wallets to only get nodes that signal
support for NODE_BLOOM.

The syntax is

x<networkservice_flags_filter_as_int>.seed.bitcoin....

Example for NODE_NETWORK together with NODE_BLOOM
dig x5.seed.bitcoin.jonasschnelli.ch
(NETWORK = (1 << 0), NODE_BLOOM = (1 << 2)) = (bin00000101 = (int)5)


</jonas>
Gregory Maxwell via bitcoin-dev
2016-08-24 08:34:20 UTC
Permalink
On Tue, Aug 23, 2016 at 8:54 PM, Kenneth Heutmaker via bitcoin-dev
SPV is kinda broken if the wallet doesn’t do this detection. If your wallet connects only to nodes that don’t support bloom filtering, the wallet never gets updates. We have had a spike in users reporting that their wallet isn't getting updated. To compound the problem, they rescan the blockchain and lose all of their transaction history. It has caused much panic among less technical users.
We believe that failing to detect the NODE_BLOOM bit is the culprit, although it is non-deterministic, so we aren't certain.
There are almost no NODE_BLOOM supporting bloom-off nodes on the
network currently. So, while supporting this is important, I am
doubtful that its the current problem you've suffered.

There are a great many fake nodes which appear to exist purely to
monitor transactions. Many do not implement enough of the protocol to
support scanning or transaction relay. (and, in fact, relaying
transactions would make monitoring less effective).

You can't count on peers on a peer to peer network to be honest and
cooperative. Implementations need to work hard to be robust to abusive
peers. Unfortunately, the design of the bloom filtering is such that
it isn't always easy (or even possible) to be robust.
Thomas Voegtlin via bitcoin-dev
2016-08-24 12:51:32 UTC
Permalink
Post by Luke Dashjr via bitcoin-dev
BIP 39: Mnemonic code for generating deterministic keys
- Used by many wallets and hundreds of thousands of users.
BIP 44: Multi-Account Hierarchy for Deterministic Wallets
- Appears to be implemented by multiple wallets.
I personally believe that BIP39/BIP44 is a bad design. There is limited
support for these BIPs in Electrum, in order to provide compatibility
with hardware wallets. However, I do not plan to use BIP39/BIP44 for
default Electrum wallets, for the following reasons.

(Note that it does not make sense to consider BIP39 and BIP44
independently. Any wallet that decides to implement one without the
other would be considered as broken.)

Here is why I rejected this design:

1 - BIP44 uses multiple accounts. This means that in order to be
compatible with the standard, a wallet *must* implement multiple
accounts. A wallet that decides to keep things simple and use only one
account, will not allow users to recover all their funds when they
restore from a BIP39 seed, and will be considered as broken.

2 - An appealing feature of deterministic wallets is that you can use
the same instance of your wallet on different devices. Two instances of
your wallet can automatically synchronize their Bitcoin addresses, and
display the same balance. The problem is that hardened derivations break
this property. Indeed, with hardened derivations, software wallets need
to ask the user's password in order to derive new accounts. Therefore,
in order to implement automated detection of newly created accounts, a
BIP44-compatible software wallets would need to ask the user's password
whenever a new account is detected. This means that the wallet would ask
the password without the user initiating any action. This seems to be an
avenue for malware.

Of course, hardware wallets do not have that issue, because they can
derive new accounts without requesting a password from the user. BIP44
is a standard that has been designed for hardware wallets, but that
makes things really difficult for software wallets.

3 - Unneeded complexity. From an end user perspective, the multiple
accounts in BIP44 achieve the same result as using different derivation
passphrases with the same BIP39 seed phrase. The only real difference is
that BIP44 accounts can be enumerated deterministically, while
passphrases in general cannot. However, this property is of limited
interest, because automatic synchronization of multiple accounts cannot
be guaranteed for bip44 software wallets, as explained in 2.

4 - BIP39 is inconsistent. It uses a hash of the utf8 encoded 'seed
phrase' in order to derive the BIP32 seed. This hash-based derivation
was added on my suggestion, in order to make the BIP independent from
the particular wordlist used to generate the seed phrases. However,
BIP39 also requires the implementation of a checksum, in order to verify
that a seed phrase is valid. Suprisingly, the specification of the
checksum involves wordlist indices. This means the checksum (and thus
the BIP) requires a fixed wordlist. This defeats the purpose of using a
hash for the derivation of the seed.

The authors of the BIP should either have used hash functions for both
the seed AND the checksum (that is what Electrum does), or for none of
them (in that case case, you can have a bidirectional function between
seed phrases and entropy, which is nice if you want to perform Shamir
secret sharing of seed phrases, at the expenses of a fixed wordlist). In
its current state, BIP39 takes the worst of both worlds.

5 - The fact that the wordlist must be part of BIP39, and cannot be
changed in the future, seems a terrible idea to me. I believe that a
specification should always try to be minimal. In that case, the
specification includes a 2000+ words dictionary, when it could have
avoided that.

Even if you decide that BIP39 is final, there will always be users
requiring the addition of wordlists for new languages. So, in practice,
this BIP will never be final.

6 - Finally, and most importantly, BIP39 seed phrases do not have a
version number. Without a version number, how are you going to derive
addresses from a BIP39 seed phrase, when wallets start to use to new
derivation methods (such as SegWit, or Schnorr signatures)? Does it mean
that a BIP39 compatible wallet will have to check addresses from all the
derivation methods that ever existed in the past, in order to ensure
that all coins are correctly retrieved? Or will there be users that
cannot access their coins because their BIP39 seed phrase is too old for
newer software?
Andreas Schildbach via bitcoin-dev
2016-08-24 13:47:08 UTC
Permalink
FWIW, BIP44 also doesn't encode a seed birthday. This needed so that SPV
wallets do not need to scan from the beginning of the blockchain.

That doesn't mean BIP44 could not be final. There are some wallets that
interoperate on that standard and that's fine. The whole reason I
insisted on separating BIP43 from BIP44 is that someone else can come up
with a better "BIP44+" standard and not get into the way of existing
standards. I think BIP43 should be made final as well, if it isn't already.
Post by Thomas Voegtlin via bitcoin-dev
Post by Luke Dashjr via bitcoin-dev
BIP 39: Mnemonic code for generating deterministic keys
- Used by many wallets and hundreds of thousands of users.
BIP 44: Multi-Account Hierarchy for Deterministic Wallets
- Appears to be implemented by multiple wallets.
I personally believe that BIP39/BIP44 is a bad design. There is limited
support for these BIPs in Electrum, in order to provide compatibility
with hardware wallets. However, I do not plan to use BIP39/BIP44 for
default Electrum wallets, for the following reasons.
(Note that it does not make sense to consider BIP39 and BIP44
independently. Any wallet that decides to implement one without the
other would be considered as broken.)
1 - BIP44 uses multiple accounts. This means that in order to be
compatible with the standard, a wallet *must* implement multiple
accounts. A wallet that decides to keep things simple and use only one
account, will not allow users to recover all their funds when they
restore from a BIP39 seed, and will be considered as broken.
2 - An appealing feature of deterministic wallets is that you can use
the same instance of your wallet on different devices. Two instances of
your wallet can automatically synchronize their Bitcoin addresses, and
display the same balance. The problem is that hardened derivations break
this property. Indeed, with hardened derivations, software wallets need
to ask the user's password in order to derive new accounts. Therefore,
in order to implement automated detection of newly created accounts, a
BIP44-compatible software wallets would need to ask the user's password
whenever a new account is detected. This means that the wallet would ask
the password without the user initiating any action. This seems to be an
avenue for malware.
Of course, hardware wallets do not have that issue, because they can
derive new accounts without requesting a password from the user. BIP44
is a standard that has been designed for hardware wallets, but that
makes things really difficult for software wallets.
3 - Unneeded complexity. From an end user perspective, the multiple
accounts in BIP44 achieve the same result as using different derivation
passphrases with the same BIP39 seed phrase. The only real difference is
that BIP44 accounts can be enumerated deterministically, while
passphrases in general cannot. However, this property is of limited
interest, because automatic synchronization of multiple accounts cannot
be guaranteed for bip44 software wallets, as explained in 2.
4 - BIP39 is inconsistent. It uses a hash of the utf8 encoded 'seed
phrase' in order to derive the BIP32 seed. This hash-based derivation
was added on my suggestion, in order to make the BIP independent from
the particular wordlist used to generate the seed phrases. However,
BIP39 also requires the implementation of a checksum, in order to verify
that a seed phrase is valid. Suprisingly, the specification of the
checksum involves wordlist indices. This means the checksum (and thus
the BIP) requires a fixed wordlist. This defeats the purpose of using a
hash for the derivation of the seed.
The authors of the BIP should either have used hash functions for both
the seed AND the checksum (that is what Electrum does), or for none of
them (in that case case, you can have a bidirectional function between
seed phrases and entropy, which is nice if you want to perform Shamir
secret sharing of seed phrases, at the expenses of a fixed wordlist). In
its current state, BIP39 takes the worst of both worlds.
5 - The fact that the wordlist must be part of BIP39, and cannot be
changed in the future, seems a terrible idea to me. I believe that a
specification should always try to be minimal. In that case, the
specification includes a 2000+ words dictionary, when it could have
avoided that.
Even if you decide that BIP39 is final, there will always be users
requiring the addition of wordlists for new languages. So, in practice,
this BIP will never be final.
6 - Finally, and most importantly, BIP39 seed phrases do not have a
version number. Without a version number, how are you going to derive
addresses from a BIP39 seed phrase, when wallets start to use to new
derivation methods (such as SegWit, or Schnorr signatures)? Does it mean
that a BIP39 compatible wallet will have to check addresses from all the
derivation methods that ever existed in the past, in order to ensure
that all coins are correctly retrieved? Or will there be users that
cannot access their coins because their BIP39 seed phrase is too old for
newer software?
_______________________________________________
bitcoin-dev mailing list
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
Luke Dashjr via bitcoin-dev
2016-08-24 18:22:39 UTC
Permalink
On Wednesday, August 24, 2016 1:47:08 PM Andreas Schildbach via bitcoin-dev
Post by Andreas Schildbach via bitcoin-dev
FWIW, BIP44 also doesn't encode a seed birthday. This needed so that SPV
wallets do not need to scan from the beginning of the blockchain.
That doesn't mean BIP44 could not be final. There are some wallets that
interoperate on that standard and that's fine.
Right. The Status doesn't depend on whether it is a good idea or not, only
whether or not people are de facto using it.

BIP 2's BIP Comments would have provided a place for Thomas and yourself to
criticise the BIP, but unfortunately this was too controversial.
Post by Andreas Schildbach via bitcoin-dev
I think BIP43 should be made final as well, if it isn't already.
BIP 43 merely advises other BIPs how they might do things, so it goes into the
Draft->Active Status flow rather than Draft->Accepted->Final.

Luke
Jonas Schnelli via bitcoin-dev
2016-08-24 14:18:26 UTC
Permalink
Hi
Post by Thomas Voegtlin via bitcoin-dev
6 - Finally, and most importantly, BIP39 seed phrases do not have a
version number. Without a version number, how are you going to derive
addresses from a BIP39 seed phrase, when wallets start to use to new
derivation methods (such as SegWit, or Schnorr signatures)? Does it mean
that a BIP39 compatible wallet will have to check addresses from all the
derivation methods that ever existed in the past, in order to ensure
that all coins are correctly retrieved? Or will there be users that
cannot access their coins because their BIP39 seed phrase is too old for
newer software?
I totally agree with Thomas.

Another thing that I think could be a BIP misdesign:

BIP44 Gap Limits
From the BIP:

----------
"Address gap limit is currently set to 20. If the software hits 20
unused addresses in a row, it expects there are no used addresses beyond
this point and stops searching the address chain."
----------

* Does that mean, we do a transaction rescan back to the genesis block
for every 20 addresses?
* Or is it a prerequirement to do a wallet recovery after BIP44's to
have access to a full address-indexed blockchain?

Or maybe I'm missing something.

</jonas>
Jochen Hoenicke via bitcoin-dev
2016-08-24 14:42:25 UTC
Permalink
Post by Jonas Schnelli via bitcoin-dev
BIP44 Gap Limits
----------
"Address gap limit is currently set to 20. If the software hits 20
unused addresses in a row, it expects there are no used addresses beyond
this point and stops searching the address chain."
----------
* Does that mean, we do a transaction rescan back to the genesis block
for every 20 addresses?
As I understand it, you can scan sequentially starting with the genesis
block (or with a block at around the time when BIP44 was written). Then
if you find a new transaction, which requires to generate new addresses,
you generate them and scan further from that point on. This way you can
scan in a single pass if the scanning process calls you back when it
finds a transaction and allows you to change the set of addresses on the
fly.

Jochen
Jonas Schnelli via bitcoin-dev
2016-08-25 07:39:28 UTC
Permalink
Sorry for hijacking the thread again
Post by Jochen Hoenicke via bitcoin-dev
As I understand it, you can scan sequentially starting with the genesis
block (or with a block at around the time when BIP44 was written). Then
if you find a new transaction, which requires to generate new addresses,
you generate them and scan further from that point on. This way you can
scan in a single pass if the scanning process calls you back when it
finds a transaction and allows you to change the set of addresses on the
fly.
(I think this case if not completely unrealistic):

What would happen, if a user gave out 21 addresses, then address0 had
receive funds in +180 days after generation where address21 had receive
funds immediately (all other addresses never received a tx).

In a scan, address0 would be detected at <address-birthday>+180 days
which would trigger the resize+20 of the address-lookup-window, but, we
would require to go back 180day in order to detect received transaction
of address21 (new lookup-window) in that case.

Or do I misunderstand something?


</jonas>
Thomas Voegtlin via bitcoin-dev
2016-08-25 07:56:17 UTC
Permalink
Post by Jonas Schnelli via bitcoin-dev
What would happen, if a user gave out 21 addresses, then address0 had
receive funds in +180 days after generation where address21 had receive
funds immediately (all other addresses never received a tx).
In a scan, address0 would be detected at <address-birthday>+180 days
which would trigger the resize+20 of the address-lookup-window, but, we
would require to go back 180day in order to detect received transaction
of address21 (new lookup-window) in that case.
Or do I misunderstand something?
That case is not unrealistic; a merchant might generate addresses that
are beyond their gap limit, and orders get filled at a later date.

In that case you will not get the same result when restoring your wallet
in a block-scanning wallet and in Electrum.

The lack of consideration for these cases is another issue with BIP44.
Jonas Schnelli via bitcoin-dev
2016-08-25 08:08:21 UTC
Permalink
Post by Thomas Voegtlin via bitcoin-dev
Post by Jonas Schnelli via bitcoin-dev
What would happen, if a user gave out 21 addresses, then address0 had
receive funds in +180 days after generation where address21 had receive
funds immediately (all other addresses never received a tx).
In a scan, address0 would be detected at <address-birthday>+180 days
which would trigger the resize+20 of the address-lookup-window, but, we
would require to go back 180day in order to detect received transaction
of address21 (new lookup-window) in that case.
Or do I misunderstand something?
That case is not unrealistic; a merchant might generate addresses that
are beyond their gap limit, and orders get filled at a later date.
In that case you will not get the same result when restoring your wallet
in a block-scanning wallet and in Electrum.
The lack of consideration for these cases is another issue with BIP44.
The development paradigm of "maybe detect funds" is not something we
should *not* encourage for Bitcoin IMO.

Users might sweep their existing bip32/bip44 seed (which could miss
funds according to the problem above) to a new wallet and discard the
previous seed.

But I agree with Luke-Jr.
This Thread is not about specification, it's about what's used and what
should be marked as standard.

New BIPs could cover "overhauled" proposals for BIP39 and BIP44.
Otherwise – very likely – nothing will happen.

</jonas>
Jonas Schnelli via bitcoin-dev
2016-08-25 08:12:22 UTC
Permalink
Post by Jonas Schnelli via bitcoin-dev
The development paradigm of "maybe detect funds" is not something we
should *not* encourage for Bitcoin IMO.
Sorry. That was one "not" to many.

</jonas>
Marek Palatinus via bitcoin-dev
2016-08-25 08:50:43 UTC
Permalink
As Luke pointed, BIP44 is already used by many wallets and to my knowledge
people don't have any real world issues with that, including loading funds
in another BIP44 wallet. I'm not saying that BIP44 is perfect from all
points of view, but IMO it just works for most use cases. Let's set it as
final, and propose competing standards which cover all your concerns.

slush

On Thu, Aug 25, 2016 at 10:12 AM, Jonas Schnelli via bitcoin-dev <
Post by Jonas Schnelli via bitcoin-dev
Post by Jonas Schnelli via bitcoin-dev
The development paradigm of "maybe detect funds" is not something we
should *not* encourage for Bitcoin IMO.
Sorry. That was one "not" to many.
</jonas>
_______________________________________________
bitcoin-dev mailing list
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
Pieter Wuille via bitcoin-dev
2016-08-25 09:02:27 UTC
Permalink
This is not the place to discuss the merits and/or issues of these BIPs,
only whether they should be treated as final.

On Aug 25, 2016 10:51, "Marek Palatinus via bitcoin-dev" <
Post by Marek Palatinus via bitcoin-dev
As Luke pointed, BIP44 is already used by many wallets and to my knowledge
people don't have any real world issues with that, including loading funds
in another BIP44 wallet. I'm not saying that BIP44 is perfect from all
points of view, but IMO it just works for most use cases. Let's set it as
final, and propose competing standards which cover all your concerns.
slush
On Thu, Aug 25, 2016 at 10:12 AM, Jonas Schnelli via bitcoin-dev <
Post by Jonas Schnelli via bitcoin-dev
Post by Jonas Schnelli via bitcoin-dev
The development paradigm of "maybe detect funds" is not something we
should *not* encourage for Bitcoin IMO.
Sorry. That was one "not" to many.
</jonas>
_______________________________________________
bitcoin-dev mailing list
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
_______________________________________________
bitcoin-dev mailing list
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
Continue reading on narkive:
Loading...