Discussion:
[bitcoin-dev] BloomFilter issue with segwit addresses
Andreas Schildbach via bitcoin-dev
2018-04-13 15:32:15 UTC
Permalink
Anton, a developer on the bitcoinj maiing list, recently made me aware
[1] of a compatibility issue between segwit and BIP37 (Bloom Filtering).

The issue affects only P2WPKH and the special case of transactions
without change outputs (such as when emptying a wallet). In this case,
neither inputs not outputs contain any data elements that would cause a
match for the filter. The public key, which would match, goes to the
witness but not to the input.

My suggestion was to include an OP_RETURN output with a matching public
key in such transactions. Anton confirmed that this workaround is indeed
working. But of course it nullifies some of the segwit's size improvements.

I wonder if Bitcoin Core would be willing to extend the BIP37 matching
rules such that data elements in the witness are also matched against?


[1] https://groups.google.com/d/msg/bitcoinj/SJpLgjowc1I/V7u2BavvAwAJ
Jonas Schnelli via bitcoin-dev
2018-04-13 19:12:47 UTC
Permalink
Hi Andreas

Thanks for bringing this up and this seems indeed to be suboptimal.
Post by Andreas Schildbach via bitcoin-dev
I wonder if Bitcoin Core would be willing to extend the BIP37 matching
rules such that data elements in the witness are also matched against?
Bitcoin Core is not an identity that can be „willing to extend“ (or reject) a feature.
Someone needs to come up with a proposal (pull request).

Maybe an extension for BIP37 would make sense (*meh*).
Just inserting the witness data into the bloom filter seems to be an easy solution (CBloomFilter::IsRelevantAndUpdate())

/jonas
Jim Posen via bitcoin-dev
2018-04-13 22:15:50 UTC
Permalink
Why not add the outpoints owned by the wallet to the filter and watch for
those instead of elements in the input script or witness data?

On Fri, Apr 13, 2018 at 12:12 PM, Jonas Schnelli via bitcoin-dev <
Post by Jonas Schnelli via bitcoin-dev
Hi Andreas
Thanks for bringing this up and this seems indeed to be suboptimal.
Post by Andreas Schildbach via bitcoin-dev
I wonder if Bitcoin Core would be willing to extend the BIP37 matching
rules such that data elements in the witness are also matched against?
Bitcoin Core is not an identity that can be „willing to extend“ (or
reject) a feature.
Someone needs to come up with a proposal (pull request).
Maybe an extension for BIP37 would make sense (*meh*).
Just inserting the witness data into the bloom filter seems to be an easy
solution (CBloomFilter::IsRelevantAndUpdate())
/jonas
_______________________________________________
bitcoin-dev mailing list
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
Christian Decker via bitcoin-dev
2018-04-14 16:14:04 UTC
Permalink
Note that this would compound the privacy leak that Jonas Nick used to
identify address clusters via the bloom filters in one of his publications.
By reducing the false positives when matching you can get very detailed
clusters. Then again we know that bloom filters aren't good for privacy
anyway, so this might be a non-issue.

On Sat, Apr 14, 2018, 00:17 Jim Posen via bitcoin-dev <
Post by Jim Posen via bitcoin-dev
Why not add the outpoints owned by the wallet to the filter and watch for
those instead of elements in the input script or witness data?
On Fri, Apr 13, 2018 at 12:12 PM, Jonas Schnelli via bitcoin-dev <
Post by Jonas Schnelli via bitcoin-dev
Hi Andreas
Thanks for bringing this up and this seems indeed to be suboptimal.
Post by Andreas Schildbach via bitcoin-dev
I wonder if Bitcoin Core would be willing to extend the BIP37 matching
rules such that data elements in the witness are also matched against?
Bitcoin Core is not an identity that can be „willing to extend“ (or
reject) a feature.
Someone needs to come up with a proposal (pull request).
Maybe an extension for BIP37 would make sense (*meh*).
Just inserting the witness data into the bloom filter seems to be an easy
solution (CBloomFilter::IsRelevantAndUpdate())
/jonas
_______________________________________________
bitcoin-dev mailing list
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
_______________________________________________
bitcoin-dev mailing list
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
Jim Posen via bitcoin-dev
2018-04-14 19:46:01 UTC
Permalink
To Christian's point about privacy, I'll take this opportunity to
shamelessly review beg on https://github.com/bitcoin/bitcoin/pull/12254,
the PR for BIP 158 implementation (but not 157).

On Sat, Apr 14, 2018 at 9:14 AM, Christian Decker <
Post by Christian Decker via bitcoin-dev
Note that this would compound the privacy leak that Jonas Nick used to
identify address clusters via the bloom filters in one of his publications.
By reducing the false positives when matching you can get very detailed
clusters. Then again we know that bloom filters aren't good for privacy
anyway, so this might be a non-issue.
Post by Jim Posen via bitcoin-dev
Why not add the outpoints owned by the wallet to the filter and watch for
those instead of elements in the input script or witness data?
On Fri, Apr 13, 2018 at 12:12 PM, Jonas Schnelli via bitcoin-dev <
Post by Jonas Schnelli via bitcoin-dev
Hi Andreas
Thanks for bringing this up and this seems indeed to be suboptimal.
Post by Andreas Schildbach via bitcoin-dev
I wonder if Bitcoin Core would be willing to extend the BIP37 matching
rules such that data elements in the witness are also matched against?
Bitcoin Core is not an identity that can be „willing to extend“ (or
reject) a feature.
Someone needs to come up with a proposal (pull request).
Maybe an extension for BIP37 would make sense (*meh*).
Just inserting the witness data into the bloom filter seems to be an
easy solution (CBloomFilter::IsRelevantAndUpdate())
/jonas
_______________________________________________
bitcoin-dev mailing list
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
_______________________________________________
bitcoin-dev mailing list
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
Andreas Schildbach via bitcoin-dev
2018-04-15 18:37:45 UTC
Permalink
Yes, I guess the quicker filter exhaustion must be the reason why
bitcoinj doesn't make use of outpoints in filters for standard
transactions. I'll look into if I can change that.
Post by Christian Decker via bitcoin-dev
Note that this would compound the privacy leak that Jonas Nick used to
identify address clusters via the bloom filters in one of his
publications. By reducing the false positives when matching you can get
very detailed clusters. Then again we know that bloom filters aren't
good for privacy anyway, so this might be a non-issue.
On Sat, Apr 14, 2018, 00:17 Jim Posen via bitcoin-dev
Why not add the outpoints owned by the wallet to the filter and
watch for those instead of elements in the input script or witness data?
On Fri, Apr 13, 2018 at 12:12 PM, Jonas Schnelli via bitcoin-dev
Hi Andreas
Thanks for bringing this up and this seems indeed to be suboptimal.
Post by Andreas Schildbach via bitcoin-dev
I wonder if Bitcoin Core would be willing to extend the BIP37 matching
rules such that data elements in the witness are also matched against?
Bitcoin Core is not an identity that can be „willing to extend“
(or reject) a feature.
Someone needs to come up with a proposal (pull request).
Maybe an extension for BIP37 would make sense (*meh*).
Just inserting the witness data into the bloom filter seems to
be an easy solution (CBloomFilter::IsRelevantAndUpdate())
/jonas
_______________________________________________
bitcoin-dev mailing list
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
_______________________________________________
bitcoin-dev mailing list
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
_______________________________________________
bitcoin-dev mailing list
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
Luke Dashjr via bitcoin-dev
2018-04-13 22:52:23 UTC
Permalink
As I understand it, the plan is to deprecated and remove BIP37 entirely once
BIP158 is implemented and deployed.

In the meantime, Bitcoin Knots supports the MSG_FILTERED_WITNESS_BLOCK
extension to download witness data. (Note that light clients currently have no
way to verify the witness data is correct.)

As far as matching goes, why not look for the specific COutPoints? That should
work already with standard BIP37.

Luke
Post by Andreas Schildbach via bitcoin-dev
Anton, a developer on the bitcoinj maiing list, recently made me aware
[1] of a compatibility issue between segwit and BIP37 (Bloom Filtering).
The issue affects only P2WPKH and the special case of transactions
without change outputs (such as when emptying a wallet). In this case,
neither inputs not outputs contain any data elements that would cause a
match for the filter. The public key, which would match, goes to the
witness but not to the input.
My suggestion was to include an OP_RETURN output with a matching public
key in such transactions. Anton confirmed that this workaround is indeed
working. But of course it nullifies some of the segwit's size improvements.
I wonder if Bitcoin Core would be willing to extend the BIP37 matching
rules such that data elements in the witness are also matched against?
[1] https://groups.google.com/d/msg/bitcoinj/SJpLgjowc1I/V7u2BavvAwAJ
_______________________________________________
bitcoin-dev mailing list
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
Loading...