Discussion:
Segregated witness p2p layer compatibility
(too old to reply)
Suhas Daftuar via bitcoin-dev
2017-03-27 19:22:43 UTC
Permalink
Raw Message
Hi,

There have been two threads recently that have made references to
peer-to-peer implementation details in Bitcoin Core's Segregated Witness
code that I would like to clarify.

In the thread "Issolated Bitcoin Nodes" (
https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-March/013765.html),
there was some discussion about how Bitcoin Core's block download logic
behaves after segwit activation. After segwit activation, Bitcoin Core
nodes will not (currently) attempt to download any blocks from non-segwit
peers (nodes that do not set the NODE WITNESS service bit). This is a
bandwidth optimization to prevent a node from downloading a block that may
be invalid only because the sender omitted the witness, requiring
re-download until the block is received with the required witness data.

But to be clear, non-segwit blocks -- that is, blocks without a witness
commitment in the coinbase, and whose transactions are serialized without
witnesses, and whose transactions are not spending segwit outputs which
require a witness -- are evaluated under the same rules as prior,
pre-segwit versions of the software. So such non-segwit blocks that are
valid to older, pre-segwit nodes are also valid to segwit-nodes.

In
https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-March/013796.html,
Eric Voskuil wrote:

Given the protocol requirements of the segwit proposal this is not the
case. A miner running pre-segwit code will produce blocks that no
segwit node will ever receive.
The phrase "protocol requirements of segwit" is confusing here, because
there are two different layers that need consideration: the consensus
protocol layer and the peer-to-peer protocol layer. But in neither layer
is the behavior of not downloading blocks from non-NODE WITNESS peers a
"requirement". This is an implementation detail in the Bitcoin Core code
that alternate implementations compliant with BIP 144 could implement
differently.

At the consensus layer, non-segwit blocks (described above) that are valid
to older nodes are also valid to segwit nodes. That means that if a miner
was using an older, pre-segwit version of Bitcoin Core to produce blocks
after segwit activates, that blocks they find will be valid to all nodes.

At the p2p layer, though, segwit-enabled Bitcoin Core nodes will only try
to download those blocks if announced by a segwit-enabled peer. But this
is not a protocol requirement; other implementations can remain compatible
even they take different approaches here. (As an example, I could imagine
an implementation that downloaded a new block from any peer, but if the
block has a witness commitment in the coinbase and was received from a
non-segwit peer, then the node would attempt re-download from a segwit
peer. I'm sure many other reasonable block download strategies could be
devised.)

Still, if a miner wants to continue mining post-segwit activation, but
using pre-segwit software, they would need a way to relay their blocks to a
segwit-enabled peer.

There are a few ways to do this that I can think of:

- Use the RPC call "submitblock" on a segwit-enabled node. Calling
"submitblock" on a Bitcoin Core 0.13.1 (0.13.0 in the case of testnet) or
later node works fine as long as the block is valid (whether or not it has
a witness commitment or witness transactions), and once a segwit-enabled
peer has the block it will relay to other segwit peers.

- Explicitly deliver the block to a segwit node over the p2p network, even
if unrequested. Currently Bitcoin Core at least will process unrequested
blocks, and valid blocks that update the tip will then be relayed to other
peers.

- Run a bridge node, which advertises NODE_WITNESS and can serialize blocks
with witness data, which downloads blocks even from non-NODE WITNESS
peers. Anyone can do this to bridge the networks for the benefit of the
whole network (I have personally been running a few nodes that do this, for
several months now), but miners concerned about this issue for their own
blocks could explicitly do this themselves to ensure that their own blocks
propagate to the segwit-enabled network.

- Peer directly with other miners, bypassing the p2p network. Many miners
do this already, using protocols which may already serve to bridge the
network.

So saying that "A miner running pre-segwit code will produce blocks that no
segwit node will ever receive" is not really correct, in my view. If the
whole network were just running Bitcoin Core software releases, and the
miner was not able/willing to deliver their block to a segwit-enabled node
(eg by using the RPC call "submitblock", or one of the other suggestions I
had above), then I would agree with the statement. But given that there
are bridge nodes on the network, and that miners have other options to
relay their block, I think this is not an accurate portrayal of what would
actually happen on the network -- I would expect that non-segwit miners'
blocks would still get relayed post-segwit activation, even if only by the
handful of bridge nodes that I expect are currently running.

All that said, I do think this is an important detail to highlight and that
this behavior should be better documented (I believe it deserves specific
mention in a BIP), as this is an important issue for miners to be aware of.
Matt Corallo via bitcoin-dev
2017-03-27 19:32:30 UTC
Permalink
Raw Message
Just to expand a tiny bit here, while the testnet setup of only a few nodes acting as "bridges", mainnet already has many systems which act as effective bridges today - there are several relay networks in use which effectively bypass the P2P network, including my legacy relay network (which many miners historically have used, and I'd expect those who aren't paying attention and don't upgrade will not turn off, fixing the issue for them), ViaBTC's super aggressive bandwidth-wasting block announcement network which pushes blocks from several pools to many nodes globally, and Bitcoin.com's private relay network. (Of course many other miners and pools have private relay networks, but the several other such networks I'm aware of are already segwit-compatible, even for pools not signaling segwit).

Matt
Post by Suhas Daftuar via bitcoin-dev
Hi,
There have been two threads recently that have made references to
peer-to-peer implementation details in Bitcoin Core's Segregated Witness
code that I would like to clarify.
In the thread "Issolated Bitcoin Nodes" (
https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-March/013765.html),
there was some discussion about how Bitcoin Core's block download logic
behaves after segwit activation. After segwit activation, Bitcoin Core
nodes will not (currently) attempt to download any blocks from
non-segwit
peers (nodes that do not set the NODE WITNESS service bit). This is a
bandwidth optimization to prevent a node from downloading a block that may
be invalid only because the sender omitted the witness, requiring
re-download until the block is received with the required witness data.
But to be clear, non-segwit blocks -- that is, blocks without a witness
commitment in the coinbase, and whose transactions are serialized without
witnesses, and whose transactions are not spending segwit outputs which
require a witness -- are evaluated under the same rules as prior,
pre-segwit versions of the software. So such non-segwit blocks that are
valid to older, pre-segwit nodes are also valid to segwit-nodes.
In
https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-March/013796.html,
Given the protocol requirements of the segwit proposal this is not the
case. A miner running pre-segwit code will produce blocks that no
segwit node will ever receive.
The phrase "protocol requirements of segwit" is confusing here, because
there are two different layers that need consideration: the consensus
protocol layer and the peer-to-peer protocol layer. But in neither layer
is the behavior of not downloading blocks from non-NODE WITNESS peers a
"requirement". This is an implementation detail in the Bitcoin Core code
that alternate implementations compliant with BIP 144 could implement
differently.
At the consensus layer, non-segwit blocks (described above) that are valid
to older nodes are also valid to segwit nodes. That means that if a miner
was using an older, pre-segwit version of Bitcoin Core to produce blocks
after segwit activates, that blocks they find will be valid to all nodes.
At the p2p layer, though, segwit-enabled Bitcoin Core nodes will only try
to download those blocks if announced by a segwit-enabled peer. But this
is not a protocol requirement; other implementations can remain
compatible
even they take different approaches here. (As an example, I could imagine
an implementation that downloaded a new block from any peer, but if the
block has a witness commitment in the coinbase and was received from a
non-segwit peer, then the node would attempt re-download from a segwit
peer. I'm sure many other reasonable block download strategies could be
devised.)
Still, if a miner wants to continue mining post-segwit activation, but
using pre-segwit software, they would need a way to relay their blocks to a
segwit-enabled peer.
- Use the RPC call "submitblock" on a segwit-enabled node. Calling
"submitblock" on a Bitcoin Core 0.13.1 (0.13.0 in the case of testnet) or
later node works fine as long as the block is valid (whether or not it has
a witness commitment or witness transactions), and once a
segwit-enabled
peer has the block it will relay to other segwit peers.
- Explicitly deliver the block to a segwit node over the p2p network, even
if unrequested. Currently Bitcoin Core at least will process
unrequested
blocks, and valid blocks that update the tip will then be relayed to other
peers.
- Run a bridge node, which advertises NODE_WITNESS and can serialize blocks
with witness data, which downloads blocks even from non-NODE WITNESS
peers. Anyone can do this to bridge the networks for the benefit of the
whole network (I have personally been running a few nodes that do this, for
several months now), but miners concerned about this issue for their own
blocks could explicitly do this themselves to ensure that their own blocks
propagate to the segwit-enabled network.
- Peer directly with other miners, bypassing the p2p network. Many miners
do this already, using protocols which may already serve to bridge the
network.
So saying that "A miner running pre-segwit code will produce blocks that no
segwit node will ever receive" is not really correct, in my view. If the
whole network were just running Bitcoin Core software releases, and the
miner was not able/willing to deliver their block to a segwit-enabled node
(eg by using the RPC call "submitblock", or one of the other
suggestions I
had above), then I would agree with the statement. But given that there
are bridge nodes on the network, and that miners have other options to
relay their block, I think this is not an accurate portrayal of what would
actually happen on the network -- I would expect that non-segwit miners'
blocks would still get relayed post-segwit activation, even if only by the
handful of bridge nodes that I expect are currently running.
All that said, I do think this is an important detail to highlight and that
this behavior should be better documented (I believe it deserves specific
mention in a BIP), as this is an important issue for miners to be aware of.
Eric Voskuil via bitcoin-dev
2017-03-27 21:03:08 UTC
Permalink
Raw Message
Post by Suhas Daftuar via bitcoin-dev
Post by Suhas Daftuar via bitcoin-dev
Given the protocol requirements of the segwit proposal this is
not the case. A miner running pre-segwit code will produce
blocks that no segwit node will ever receive.
The phrase "protocol requirements of segwit" is confusing here,
the consensus protocol layer and the peer-to-peer protocol layer.
But in neither layer is the behavior of not downloading blocks
from non-NODE WITNESS peers a "requirement". This is an
implementation detail in the Bitcoin Core code that alternate
implementations compliant with BIP 144 could implement
differently.
I agree, and thanks for the detailed clarification. Clearly it is
possible for segwit blocks to be relayed. It is the implementation of
Bitcoin Core (at least), in the absence of sufficient relay, that
produces this outcome.
Post by Suhas Daftuar via bitcoin-dev
This is an implementation detail in the Bitcoin Core code that
alternate implementations compliant with BIP 144 could implement
differently.
At the consensus layer, non-segwit blocks (described above) that
are valid to older nodes are also valid to segwit nodes. That
means that if a miner was using an older, pre-segwit version of
Bitcoin Core to produce blocks after segwit activates, that blocks
they find will be valid to all nodes.
IOW Bitcoin Core has been implemented so that it will not see valid
blocks announced by certain of its peers. Forcing it to see such
blocks requires the p2p network work around its implementation. I
agree that this is not inherent in the specifications for segwit, but
it reads more like a bug than an "implementation detail" to me.

e

Loading...