Discussion:
BIP proposal: NODE_NETWORK_LIMITED service bits
Add Reply
Jonas Schnelli via bitcoin-dev
2017-05-11 15:13:12 UTC
Reply
Permalink
Raw Message
Hi

Currently, pruned peers have no way how to signal their (valuable) service.
A BIP proposal to improve this (draft):
https://github.com/jonasschnelli/bips/wiki/NODE_NETWORK_LIMITED-BIP-DRAFT <https://github.com/jonasschnelli/bips/wiki/NODE_NETWORK_LIMITED-BIP-DRAFT>

Feedback is highly welcome.

</jonas>
Gregory Maxwell via bitcoin-dev
2017-05-11 18:17:19 UTC
Reply
Permalink
Raw Message
It probably should be stated in terms of what you're promising to do--
288 and 1152 blocks, not what we hope it will accomplish. Then advise
clients to use peers with headroom because their estimates could be
wrong and reorgs.

Reorgs aren't the only concerns that drive larger numbers: The peak
at syncing is at ~24 hours, but sometimes there are quite a few more
than 144 blocks in 24 hours. Also, new blocks show up in the chain:
you think you're 144 behind but by the time you connect you find
you're 146 behind from that peer's perspective.

I think it's a bit ambiguous what it's saying about the headers,
especially because it goes into detail about address relay. I believe
nodes with any of these settings should be willing to serve headers
for their entire best chain. Perhaps you could say that this is
equivalent to NODE_NETWORK except that they aren't necessarily willing
to server historical blocks.

I'm unsure about the third depth level. Perhaps that should be left
undefined for sending right now and treated as least 1152 blocks by
receivers-- I don't have any reason to think 7056 is a particularly
useful choice, and we'll need another (longer) level for UTXO based
sync. You could probably go further and say that nodes shouldn't
send it now, but if sent it means they intend to keep 2016*2 blocks.
(Not sending because the requirement for sending it may be that the
node is able to send you a UTXO data feed.)
consider to switch a low percentage
That isn't grammatical, you want "switching". But I think it would be
better to say that when a node believe it is in sync enough to use
NODE_NETWORK_LIMITED_X it should just treat them identically to
NODE_NETWORK in peer selection. We don't really need any more
topology distortion than that. In particular, I don't want to be in
a case where NODE_NETWORK peers suddenly find themselves far less well
connected.

In terms of making room, a node network peer could choose to
disconnect the least useful peers that aren't syncing from them to
make more room for ones that are. This lets them decide what
connections they want, based on how full they are and what is useful
to them, rather than finding themselves all lonely because nodes
decided to avoid them to be "helpful", and we already use
disconnections to manage fullness.

On Thu, May 11, 2017 at 3:13 PM, Jonas Schnelli via bitcoin-dev
Hi
Currently, pruned peers have no way how to signal their (valuable) service.
https://github.com/jonasschnelli/bips/wiki/NODE_NETWORK_LIMITED-BIP-DRAFT
Feedback is highly welcome.
</jonas>
_______________________________________________
bitcoin-dev mailing list
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
Luke Dashjr via bitcoin-dev
2017-05-11 19:24:21 UTC
Reply
Permalink
Raw Message
A peer signaling NODE_NETWORK_LIMITED_LOW & NODE_NETWORK_LIMITED_HIGH MUST
be capable of serving at least the last 7'056 blocks (~49 days)
(NODE_NETWORK_LIMITED_HIGH's value ^2).
Is 49 days particularly useful? Would it be a problem to instead leave both-
bits undefined? I'm thinking this might be better as a way to indicate "7
days, plus a deterministically chosen set of historical blocks"...
Current Bitcoin-Core pruned full nodes guarantees a minimum of 288 blocks,
thus allowing to signal NODE_NETWORK_LIMITED_LOW with the current minimum
configuration.
This is technically true right now, but as soon as segwit activates, it will
no longer be... Therefore, I suggest striking it from the BIP, expounding on
it in greater detail, or making it true for the longer term.
Peers following this BIP SHOULD connect a limited amount of their available
outbound connections to peers signaling one or both of the
NODE_NETWORK_LIMITED_* service bits if they expect to request less blocks
than the signaled number.
This isn't entirely clear whether it refers to peers downloading blocks, or
peers serving them. (I assume the former, but it should be clarified.)
Light clients (and such) who are not checking the nServiceFlags (service
bits) from a relayed addr-message may unwillingly connect to a pruned peer
and ask for (filtered) blocks at a depth below their pruned depth.
Wouldn't this already be a problem, without the BIP?

Luke
Jonas Schnelli via bitcoin-dev
2017-05-11 20:10:08 UTC
Reply
Permalink
Raw Message
Post by Luke Dashjr via bitcoin-dev
Is 49 days particularly useful? Would it be a problem to instead leave both-
bits undefined? I'm thinking this might be better as a way to indicate "7
days, plus a deterministically chosen set of historical blocks"

I though two service bits allow three states and we should define all three combinations.
But I guess an adequate „definition“ would be to reserve it for future „definitions“.
Or use Gregory's proposal of min 2016*2 blocks & keep it „undefined“ for now.

49 days was chosen to allow SPV peers to be „offline“ for a month and still be capable to catch-up with a peer pruned to a datadir of ~10GB.
Post by Luke Dashjr via bitcoin-dev
This is technically true right now, but as soon as segwit activates, it will
no longer be... Therefore, I suggest striking it from the BIP, expounding on
it in greater detail, or making it true for the longer term.
AFAIK Core does also guaranteed the 288 blocks post segwit activation:
https://github.com/bitcoin/bitcoin/blob/08a7316c144f9f2516db8fa62400893f4358c5ae/src/validation.h#L204
But maybe I’m confused.
Post by Luke Dashjr via bitcoin-dev
Peers following this BIP SHOULD connect a limited amount of their available
outbound connections to peers signaling one or both of the
NODE_NETWORK_LIMITED_* service bits if they expect to request less blocks
than the signaled number.
This isn't entirely clear whether it refers to peers downloading blocks, or
peers serving them. (I assume the former, but it should be clarified.)
Indeed. I’ll try to make that more clear.
Post by Luke Dashjr via bitcoin-dev
Light clients (and such) who are not checking the nServiceFlags (service
bits) from a relayed addr-message may unwillingly connect to a pruned peer
and ask for (filtered) blocks at a depth below their pruned depth.
Wouldn't this already be a problem, without the BIP?
AFAIK, Core does currently only relay NODE_NETWORK addresses.
But yes, It may be a problem already.

</jonas>
Aymeric Vitte via bitcoin-dev
2017-05-11 20:36:33 UTC
Reply
Permalink
Raw Message
Sorry again, is this discussion really serious?

In this thread, previous ones or others, the level of approximation is
obvious, often shadowed by useless maths/papers and strange calculations
that are not helping, at the end nobody knows what would happen "if",
how far the chain can roll back, etc

Then don't make pruning the default if it's the current concern, pruning
is of no use

Again, the priority should be to scale the full nodes
Post by Jonas Schnelli via bitcoin-dev
Post by Luke Dashjr via bitcoin-dev
Is 49 days particularly useful? Would it be a problem to instead leave both-
bits undefined? I'm thinking this might be better as a way to indicate "7
days, plus a deterministically chosen set of historical blocks"…
I though two service bits allow three states and we should define all three combinations.
But I guess an adequate „definition“ would be to reserve it for future „definitions“.
Or use Gregory's proposal of min 2016*2 blocks & keep it „undefined“ for now.
49 days was chosen to allow SPV peers to be „offline“ for a month and still be capable to catch-up with a peer pruned to a datadir of ~10GB.
Post by Luke Dashjr via bitcoin-dev
This is technically true right now, but as soon as segwit activates, it will
no longer be... Therefore, I suggest striking it from the BIP, expounding on
it in greater detail, or making it true for the longer term.
https://github.com/bitcoin/bitcoin/blob/08a7316c144f9f2516db8fa62400893f4358c5ae/src/validation.h#L204
But maybe I’m confused.
Post by Luke Dashjr via bitcoin-dev
Peers following this BIP SHOULD connect a limited amount of their available
outbound connections to peers signaling one or both of the
NODE_NETWORK_LIMITED_* service bits if they expect to request less blocks
than the signaled number.
This isn't entirely clear whether it refers to peers downloading blocks, or
peers serving them. (I assume the former, but it should be clarified.)
Indeed. I’ll try to make that more clear.
Post by Luke Dashjr via bitcoin-dev
Light clients (and such) who are not checking the nServiceFlags (service
bits) from a relayed addr-message may unwillingly connect to a pruned peer
and ask for (filtered) blocks at a depth below their pruned depth.
Wouldn't this already be a problem, without the BIP?
AFAIK, Core does currently only relay NODE_NETWORK addresses.
But yes, It may be a problem already.
</jonas>
_______________________________________________
bitcoin-dev mailing list
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
--
Zcash wallets made simple: https://github.com/Ayms/zcash-wallets
Bitcoin wallets made simple: https://github.com/Ayms/bitcoin-wallets
Get the torrent dynamic blocklist: http://peersm.com/getblocklist
Check the 10 M passwords list: http://peersm.com/findmyass
Anti-spies and private torrents, dynamic blocklist: http://torrent-live.org
Peersm : http://www.peersm.com
torrent-live: https://github.com/Ayms/torrent-live
node-Tor : https://www.github.com/Ayms/node-Tor
GitHub : https://www.github.com/Ayms
Eric Voskuil via bitcoin-dev
2017-05-11 21:05:09 UTC
Reply
Permalink
Raw Message
Post by Aymeric Vitte via bitcoin-dev
Sorry again, is this discussion really serious?
In this thread, previous ones or others, the level of approximation
is obvious, often shadowed by useless maths/papers and strange
calculations that are not helping, at the end nobody knows what
would happen "if", how far the chain can roll back, etc
Then don't make pruning the default if it's the current concern,
pruning is of no use
Again, the priority should be to scale the full nodes
I agree. Every full node operator should (and likely would at some
point) simply never connect to, or relay the address of, any "peer"
advertising itself as diminished. Why on earth would a full node
operator want to encourage shrinking support for bootstrapping and
deep reorgs, resulting in greater load for himself. That's a race to
the bottom.

We are literally talking about $7.50 for the *entire chain* with
breathing room. If someone wants to save a few dollars they are better
off attempting to hide their pruning.

e
Post by Aymeric Vitte via bitcoin-dev
Post by Jonas Schnelli via bitcoin-dev
Post by Luke Dashjr via bitcoin-dev
Is 49 days particularly useful? Would it be a problem to
instead leave both- bits undefined? I'm thinking this might be
better as a way to indicate "7 days, plus a deterministically
chosen set of historical blocks"…
I though two service bits allow three states and we should define
all three combinations. But I guess an adequate „definition“
would be to reserve it for future „definitions“. Or use Gregory's
proposal of min 2016*2 blocks & keep it „undefined“ for now.
49 days was chosen to allow SPV peers to be „offline“ for a month
and still be capable to catch-up with a peer pruned to a datadir
of ~10GB.
Post by Luke Dashjr via bitcoin-dev
This is technically true right now, but as soon as segwit
activates, it will no longer be... Therefore, I suggest
striking it from the BIP, expounding on it in greater detail,
or making it true for the longer term.
AFAIK Core does also guaranteed the 288 blocks post segwit
https://github.com/bitcoin/bitcoin/blob/08a7316c144f9f2516db8fa624008
93f4358c5ae/src/validation.h#L204
But maybe I’m confused.
Post by Aymeric Vitte via bitcoin-dev
Post by Jonas Schnelli via bitcoin-dev
Post by Luke Dashjr via bitcoin-dev
Peers following this BIP SHOULD connect a limited amount of
their available outbound connections to peers signaling one
or both of the NODE_NETWORK_LIMITED_* service bits if they
expect to request less blocks than the signaled number.
This isn't entirely clear whether it refers to peers
downloading blocks, or peers serving them. (I assume the
former, but it should be clarified.)
Indeed. I’ll try to make that more clear.
Post by Luke Dashjr via bitcoin-dev
Light clients (and such) who are not checking the
nServiceFlags (service bits) from a relayed addr-message may
unwillingly connect to a pruned peer and ask for (filtered)
blocks at a depth below their pruned depth.
Wouldn't this already be a problem, without the BIP?
AFAIK, Core does currently only relay NODE_NETWORK addresses. But
yes, It may be a problem already.
</jonas>
_______________________________________________ bitcoin-dev
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
https://github.com/Ayms/bitcoin-wallets Get the torrent dynamic
blocklist: http://peersm.com/getblocklist Check the 10 M passwords
list: http://peersm.com/findmyass Anti-spies and private torrents,
https://www.github.com/Ayms
_______________________________________________ bitcoin-dev mailing
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
Gregory Maxwell via bitcoin-dev
2017-05-12 02:22:15 UTC
Reply
Permalink
Raw Message
On Thu, May 11, 2017 at 3:13 PM, Jonas Schnelli via bitcoin-dev
Post by Jonas Schnelli via bitcoin-dev
Hi
Currently, pruned peers have no way how to signal their (valuable) service.
https://github.com/jonasschnelli/bips/wiki/NODE_NETWORK_LIMITED-BIP-DRAFT
The instructions for relay addresses should not instruct you to relay
these addresses but rather that you should relay addresses you would
connect to, under the generalized assumption that if it is useful to
you it will be useful to others.

This avoids instructing someone who might not consider
non-node-network peers useful from being directed by the BIP to relay
things that they don't find useful. (In particular, it means that the
obvious implementation of just throwing out the 'useless' information
works fine.) I think would better reflect what people are likely to
actually do.

Loading...