Discussion:
[bitcoin-dev] Bitcoin Node Speed Test
slurms--- via bitcoin-dev
2015-07-23 14:19:59 UTC
Permalink
On this day, the Bitcoin network was crawled and reachable nodes surveyed to find their maximum throughput in order to determine if it can safely support a faster block rate. Specifically this is an attempt to prove or disprove the common statement that 1MB blocks were only suitable slower internet connections in 2009 when Bitcoin launched, and that connection speeds have improved to the point of obviously supporting larger blocks.


The testing methodology is as follows:

 * Nodes were randomly selected from a peers.dat, 5% of the reachable nodes in the network were contacted.

 * A random selection of blocks was downloaded from each peer.

 * There is some bias towards higher connection speeds, very slow connections (<30KB/s) timed out in order to run the test at a reasonable rate.

 * The connecting node was in Amsterdam with a 1GB NIC.

 
Results:

* 37% of connected nodes failed to upload blocks faster than 1MB/s.

* 16% of connected nodes uploaded blocks faster than 10MB/s.

* Raw data, one line per connected node, kilobytes per second http://pastebin.com/raw.php?i=6b4NuiVQ


This does not support the theory that the network has the available bandwidth for increased block sizes, as in its current state 37% of nodes would fail to upload a 20MB block to a single peer in under 20 seconds (referencing a number quoted by Gavin). If the bar for suitability is placed at taking only 1% of the block time (6 seconds) to upload one block to one peer, then 69% of the network fails for 20MB blocks. For comparison, only 10% fail this metric for 1MB blocks.
Gavin Andresen via bitcoin-dev
2015-07-23 15:04:39 UTC
Permalink
Ahh, data... a breath of fresh air...

Can you re-analyze for 8MB blocks? There is no current proposal for 20MB
blocks.

Also, most hashing power is now using Matt Corallo's fast block propagation
network; slow 'block' propagation to merchants/end-users doesn't really
matter (as long as it doesn't get anywhere near the 10-minute block time).

On Thu, Jul 23, 2015 at 10:19 AM, slurms--- via bitcoin-dev <
bitcoin-***@lists.linuxfoundation.org> wrote:

> On this day, the Bitcoin network was crawled and reachable nodes surveyed
> to find their maximum throughput in order to determine if it can safely
> support a faster block rate. Specifically this is an attempt to prove or
> disprove the common statement that 1MB blocks were only suitable slower
> internet connections in 2009 when Bitcoin launched, and that connection
> speeds have improved to the point of obviously supporting larger blocks.
>
>
> The testing methodology is as follows:
>
> * Nodes were randomly selected from a peers.dat, 5% of the reachable
> nodes in the network were contacted.
>
> * A random selection of blocks was downloaded from each peer.
>
> * There is some bias towards higher connection speeds, very slow
> connections (<30KB/s) timed out in order to run the test at a reasonable
> rate.
>
> * The connecting node was in Amsterdam with a 1GB NIC.
>
>
> Results:
>
> * 37% of connected nodes failed to upload blocks faster than 1MB/s.
>
> * 16% of connected nodes uploaded blocks faster than 10MB/s.
>
> * Raw data, one line per connected node, kilobytes per second
> http://pastebin.com/raw.php?i=6b4NuiVQ
>
>
> This does not support the theory that the network has the available
> bandwidth for increased block sizes, as in its current state 37% of nodes
> would fail to upload a 20MB block to a single peer in under 20 seconds
> (referencing a number quoted by Gavin). If the bar for suitability is
> placed at taking only 1% of the block time (6 seconds) to upload one block
> to one peer, then 69% of the network fails for 20MB blocks. For comparison,
> only 10% fail this metric for 1MB blocks.
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-***@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>



--
--
Gavin Andresen
Jameson Lopp via bitcoin-dev
2015-07-23 15:55:14 UTC
Permalink
Are you willing to share the code that you used to run the test?

- Jameson

On Thu, Jul 23, 2015 at 10:19 AM, slurms--- via bitcoin-dev <
bitcoin-***@lists.linuxfoundation.org> wrote:

> On this day, the Bitcoin network was crawled and reachable nodes surveyed
> to find their maximum throughput in order to determine if it can safely
> support a faster block rate. Specifically this is an attempt to prove or
> disprove the common statement that 1MB blocks were only suitable slower
> internet connections in 2009 when Bitcoin launched, and that connection
> speeds have improved to the point of obviously supporting larger blocks.
>
>
> The testing methodology is as follows:
>
> * Nodes were randomly selected from a peers.dat, 5% of the reachable
> nodes in the network were contacted.
>
> * A random selection of blocks was downloaded from each peer.
>
> * There is some bias towards higher connection speeds, very slow
> connections (<30KB/s) timed out in order to run the test at a reasonable
> rate.
>
> * The connecting node was in Amsterdam with a 1GB NIC.
>
>
> Results:
>
> * 37% of connected nodes failed to upload blocks faster than 1MB/s.
>
> * 16% of connected nodes uploaded blocks faster than 10MB/s.
>
> * Raw data, one line per connected node, kilobytes per second
> http://pastebin.com/raw.php?i=6b4NuiVQ
>
>
> This does not support the theory that the network has the available
> bandwidth for increased block sizes, as in its current state 37% of nodes
> would fail to upload a 20MB block to a single peer in under 20 seconds
> (referencing a number quoted by Gavin). If the bar for suitability is
> placed at taking only 1% of the block time (6 seconds) to upload one block
> to one peer, then 69% of the network fails for 20MB blocks. For comparison,
> only 10% fail this metric for 1MB blocks.
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-***@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
Slurms MacKenzie via bitcoin-dev
2015-07-23 17:14:31 UTC
Permalink
_______________________________________________
bitcoin-dev mailing list
bitcoin-***@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
Pindar Wong via bitcoin-dev
2015-07-23 17:41:33 UTC
Permalink
This looks like the beginnings of some great analysis.

Per Peter's remarks, I think it would be productive to run the test(s) on a
simulated network with worst case network failure(s) so that we can
determine the safety margin needed.

I have potential access to h/w resources that would be available for
running such tests at the necessary scales.

Cheers,

p.


On Fri, Jul 24, 2015 at 1:14 AM, Slurms MacKenzie via bitcoin-dev <
bitcoin-***@lists.linuxfoundation.org> wrote:

> The library used isn't open source, so unfortunately not. It shouldn't be
> too hard to replicate in python-bitcoinlib or bitcoinj though.
>
> *Sent:* Thursday, July 23, 2015 at 6:55 PM
> *From:* "Jameson Lopp" <***@gmail.com>
> *To:* ***@gmx.us
> *Cc:* bitcoin-***@lists.linuxfoundation.org
> *Subject:* Re: [bitcoin-dev] Bitcoin Node Speed Test
> Are you willing to share the code that you used to run the test?
>
> - Jameson
>
> On Thu, Jul 23, 2015 at 10:19 AM, slurms--- via bitcoin-dev <
> bitcoin-***@lists.linuxfoundation.org> wrote:
>>
>> On this day, the Bitcoin network was crawled and reachable nodes surveyed
>> to find their maximum throughput in order to determine if it can safely
>> support a faster block rate. Specifically this is an attempt to prove or
>> disprove the common statement that 1MB blocks were only suitable slower
>> internet connections in 2009 when Bitcoin launched, and that connection
>> speeds have improved to the point of obviously supporting larger blocks.
>>
>>
>> The testing methodology is as follows:
>>
>> * Nodes were randomly selected from a peers.dat, 5% of the reachable
>> nodes in the network were contacted.
>>
>> * A random selection of blocks was downloaded from each peer.
>>
>> * There is some bias towards higher connection speeds, very slow
>> connections (<30KB/s) timed out in order to run the test at a reasonable
>> rate.
>>
>> * The connecting node was in Amsterdam with a 1GB NIC.
>>
>>
>> Results:
>>
>> * 37% of connected nodes failed to upload blocks faster than 1MB/s.
>>
>> * 16% of connected nodes uploaded blocks faster than 10MB/s.
>>
>> * Raw data, one line per connected node, kilobytes per second
>> http://pastebin.com/raw.php?i=6b4NuiVQ
>>
>>
>> This does not support the theory that the network has the available
>> bandwidth for increased block sizes, as in its current state 37% of nodes
>> would fail to upload a 20MB block to a single peer in under 20 seconds
>> (referencing a number quoted by Gavin). If the bar for suitability is
>> placed at taking only 1% of the block time (6 seconds) to upload one block
>> to one peer, then 69% of the network fails for 20MB blocks. For comparison,
>> only 10% fail this metric for 1MB blocks.
>> _______________________________________________
>> bitcoin-dev mailing list
>> bitcoin-***@lists.linuxfoundation.org
>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-***@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
>
Peter Todd via bitcoin-dev
2015-07-23 16:05:11 UTC
Permalink
On 23 July 2015 10:19:59 GMT-04:00, slurms--- via bitcoin-dev <bitcoin-***@lists.linuxfoundation.org> wrote:
>This does not support the theory that the network has the available
>bandwidth for increased block sizes, as in its current state 37% of
>nodes would fail to upload a 20MB block to a single peer in under 20
>seconds (referencing a number quoted by Gavin). If the bar for
>suitability is placed at taking only 1% of the block time (6 seconds)
>to upload one block to one peer, then 69% of the network fails for 20MB
>blocks. For comparison, only 10% fail this metric for 1MB blocks.

Note how due to bandwidth being generally asymetric your findings are probably optimistic - you've measured download capacity. On top of that upload is further reduced by the fact that multiple peers at once need to be sent blocks for reliability.

Secondly you're measuring a network that isn't under attack - we need significant additional margin to resist attack as performance is consensus-critical.
Marcel Jamin via bitcoin-dev
2015-07-23 19:56:36 UTC
Permalink
He measured the upload capacity of the peers by downloading from them, or
am I being dumb? :)


2015-07-23 18:05 GMT+02:00 Peter Todd via bitcoin-dev <
bitcoin-***@lists.linuxfoundation.org>:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
>
>
>
> On 23 July 2015 10:19:59 GMT-04:00, slurms--- via bitcoin-dev <
> bitcoin-***@lists.linuxfoundation.org> wrote:
> >This does not support the theory that the network has the available
> >bandwidth for increased block sizes, as in its current state 37% of
> >nodes would fail to upload a 20MB block to a single peer in under 20
> >seconds (referencing a number quoted by Gavin). If the bar for
> >suitability is placed at taking only 1% of the block time (6 seconds)
> >to upload one block to one peer, then 69% of the network fails for 20MB
> >blocks. For comparison, only 10% fail this metric for 1MB blocks.
>
> Note how due to bandwidth being generally asymetric your findings are
> probably optimistic - you've measured download capacity. On top of that
> upload is further reduced by the fact that multiple peers at once need to
> be sent blocks for reliability.
>
> Secondly you're measuring a network that isn't under attack - we need
> significant additional margin to resist attack as performance is
> consensus-critical.
>
> -----BEGIN PGP SIGNATURE-----
>
> iQE9BAEBCAAnIBxQZXRlciBUb2RkIDxwZXRlQHBldGVydG9kZC5vcmc+BQJVsRCj
> AAoJEMCF8hzn9Lnc47AIAIQbznavjd2Rbqxeq5a3GLqeYoI4BZIQYqfWky+6OQtq
> yGRKaqPtGuES5y9L0k7efivT385mOl87PWnWMy61xxZ9FJgoS+YHkEx8K4tfgfA2
> yLOKzeFSar2ROCcjHYyPWa2XXjRbNmiLzfNuQyIBArg/Ch9//iXUUM+GG0mChF5k
> nUxLstXgXDNh5H8xkHeLi4lEbt9HFiwcZnT1Tzeo2dvVTujrtyNb/zEhNZScMXDc
> UOlT8rBLxzHlytKdXt1GNKIq0feTRJNbreBh7/EB4nYTT54CItaaVXul0LdHd5/2
> kgKtdbUdeyaRUKrKcvxiuIwclyoOuRQp0DZThsB262o=
> =tBUM
> -----END PGP SIGNATURE-----
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-***@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
Joseph Gleason ⑈ via bitcoin-dev
2015-07-23 19:57:38 UTC
Permalink
That is how I read it as well.


On Thu, Jul 23, 2015 at 12:56 PM Marcel Jamin via bitcoin-dev <
bitcoin-***@lists.linuxfoundation.org> wrote:

> He measured the upload capacity of the peers by downloading from them, or
> am I being dumb? :)
>
>
> 2015-07-23 18:05 GMT+02:00 Peter Todd via bitcoin-dev <
> bitcoin-***@lists.linuxfoundation.org>:
>
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA256
>>
>>
>>
>> On 23 July 2015 10:19:59 GMT-04:00, slurms--- via bitcoin-dev <
>> bitcoin-***@lists.linuxfoundation.org> wrote:
>> >This does not support the theory that the network has the available
>> >bandwidth for increased block sizes, as in its current state 37% of
>> >nodes would fail to upload a 20MB block to a single peer in under 20
>> >seconds (referencing a number quoted by Gavin). If the bar for
>> >suitability is placed at taking only 1% of the block time (6 seconds)
>> >to upload one block to one peer, then 69% of the network fails for 20MB
>> >blocks. For comparison, only 10% fail this metric for 1MB blocks.
>>
>> Note how due to bandwidth being generally asymetric your findings are
>> probably optimistic - you've measured download capacity. On top of that
>> upload is further reduced by the fact that multiple peers at once need to
>> be sent blocks for reliability.
>>
>> Secondly you're measuring a network that isn't under attack - we need
>> significant additional margin to resist attack as performance is
>> consensus-critical.
>>
>> -----BEGIN PGP SIGNATURE-----
>>
>> iQE9BAEBCAAnIBxQZXRlciBUb2RkIDxwZXRlQHBldGVydG9kZC5vcmc+BQJVsRCj
>> AAoJEMCF8hzn9Lnc47AIAIQbznavjd2Rbqxeq5a3GLqeYoI4BZIQYqfWky+6OQtq
>> yGRKaqPtGuES5y9L0k7efivT385mOl87PWnWMy61xxZ9FJgoS+YHkEx8K4tfgfA2
>> yLOKzeFSar2ROCcjHYyPWa2XXjRbNmiLzfNuQyIBArg/Ch9//iXUUM+GG0mChF5k
>> nUxLstXgXDNh5H8xkHeLi4lEbt9HFiwcZnT1Tzeo2dvVTujrtyNb/zEhNZScMXDc
>> UOlT8rBLxzHlytKdXt1GNKIq0feTRJNbreBh7/EB4nYTT54CItaaVXul0LdHd5/2
>> kgKtdbUdeyaRUKrKcvxiuIwclyoOuRQp0DZThsB262o=
>> =tBUM
>> -----END PGP SIGNATURE-----
>>
>> _______________________________________________
>> bitcoin-dev mailing list
>> bitcoin-***@lists.linuxfoundation.org
>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>>
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-***@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
Peter Todd via bitcoin-dev
2015-07-24 02:55:14 UTC
Permalink
On Thu, Jul 23, 2015 at 09:56:36PM +0200, Marcel Jamin wrote:
> He measured the upload capacity of the peers by downloading from them, or
> am I being dumb? :)

Lol, no, I'm being dumb. :)

--
'peter'[:-1]@petertodd.org
00000000000000000402fe6fb9ad613c93e12bddfc6ec02a2bd92f002050594d
Leo Wandersleb via bitcoin-dev
2015-07-23 16:36:53 UTC
Permalink
Thank you a lot for doing this test!

Two questions:

1) A node is typically connected to many nodes that would all in parallel
download said block. In your test you measured how fast new blocks that
presumably are being uploaded in parallel to all those other nodes are being
uploaded? Or did you download blocks while those nodes were basically idle?

2) What is your percentage of the very slow connections?

On 07/23/2015 11:19 AM, slurms--- via bitcoin-dev wrote:
> On this day, the Bitcoin network was crawled and reachable nodes surveyed to find their maximum throughput in order to determine if it can safely support a faster block rate. Specifically this is an attempt to prove or disprove the common statement that 1MB blocks were only suitable slower internet connections in 2009 when Bitcoin launched, and that connection speeds have improved to the point of obviously supporting larger blocks.
>
>
> The testing methodology is as follows:
>
> * Nodes were randomly selected from a peers.dat, 5% of the reachable nodes in the network were contacted.
>
> * A random selection of blocks was downloaded from each peer.
>
> * There is some bias towards higher connection speeds, very slow connections (<30KB/s) timed out in order to run the test at a reasonable rate.
>
> * The connecting node was in Amsterdam with a 1GB NIC.
>
>
> Results:
>
> * 37% of connected nodes failed to upload blocks faster than 1MB/s.
>
> * 16% of connected nodes uploaded blocks faster than 10MB/s.
>
> * Raw data, one line per connected node, kilobytes per second http://pastebin.com/raw.php?i=6b4NuiVQ
>
>
> This does not support the theory that the network has the available bandwidth for increased block sizes, as in its current state 37% of nodes would fail to upload a 20MB block to a single peer in under 20 seconds (referencing a number quoted by Gavin). If the bar for suitability is placed at taking only 1% of the block time (6 seconds) to upload one block to one peer, then 69% of the network fails for 20MB blocks. For comparison, only 10% fail this metric for 1MB blocks.
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-***@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
Slurms MacKenzie via bitcoin-dev
2015-07-23 17:12:04 UTC
Permalink
_______________________________________________
bitcoin-dev mailing list
bitcoin-***@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
Matt Corallo via bitcoin-dev
2015-07-24 02:48:03 UTC
Permalink
You may see much better throughput if you run a few servers around the
globe and test based on closest-by-geoip. TCP throughput is rather
significantly effected by latency, though I'm not really sure what you
should be testing here, ideally.

On 07/23/15 14:19, slurms--- via bitcoin-dev wrote:
> On this day, the Bitcoin network was crawled and reachable nodes surveyed to find their maximum throughput in order to determine if it can safely support a faster block rate. Specifically this is an attempt to prove or disprove the common statement that 1MB blocks were only suitable slower internet connections in 2009 when Bitcoin launched, and that connection speeds have improved to the point of obviously supporting larger blocks.
>
>
> The testing methodology is as follows:
>
> * Nodes were randomly selected from a peers.dat, 5% of the reachable nodes in the network were contacted.
>
> * A random selection of blocks was downloaded from each peer.
>
> * There is some bias towards higher connection speeds, very slow connections (<30KB/s) timed out in order to run the test at a reasonable rate.
>
> * The connecting node was in Amsterdam with a 1GB NIC.
>
>
> Results:
>
> * 37% of connected nodes failed to upload blocks faster than 1MB/s.
>
> * 16% of connected nodes uploaded blocks faster than 10MB/s.
>
> * Raw data, one line per connected node, kilobytes per second http://pastebin.com/raw.php?i=6b4NuiVQ
>
>
> This does not support the theory that the network has the available bandwidth for increased block sizes, as in its current state 37% of nodes would fail to upload a 20MB block to a single peer in under 20 seconds (referencing a number quoted by Gavin). If the bar for suitability is placed at taking only 1% of the block time (6 seconds) to upload one block to one peer, then 69% of the network fails for 20MB blocks. For comparison, only 10% fail this metric for 1MB blocks.
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-***@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
Slurms MacKenzie via bitcoin-dev
2015-07-24 03:19:35 UTC
Permalink
Yes that is completely doable for the next crawl, however I am not sure how much that reflects the behavior bitcoind would see when making connections. Nodes do not make any attempt to sync with close peers, which is an undesirable property if you are attempting to be sybil resistant. With some quick playing around it seems that you do get the expected speedup with close proximity, but it's not a particularly huge difference at present. I'll keep working on it and see where I get.


> Sent: Friday, July 24, 2015 at 4:48 AM
> From: "Matt Corallo via bitcoin-dev" <bitcoin-***@lists.linuxfoundation.org>
> To: bitcoin-***@lists.linuxfoundation.org
> Subject: Re: [bitcoin-dev] Bitcoin Node Speed Test
>
> You may see much better throughput if you run a few servers around the
> globe and test based on closest-by-geoip. TCP throughput is rather
> significantly effected by latency, though I'm not really sure what you
> should be testing here, ideally.
grarpamp via bitcoin-dev
2015-07-24 03:54:31 UTC
Permalink
On Thu, Jul 23, 2015 at 10:19 AM, slurms--- via bitcoin-dev
<bitcoin-***@lists.linuxfoundation.org> wrote:
> * A random selection of blocks was downloaded from each peer.

If the selection is different for each peer the results will be broken.
Loading...