Discussion:
[bitcoin-dev] Do we really need a mempool? (for relay nodes)
Peter Todd via bitcoin-dev
2015-07-18 18:52:59 UTC
Permalink
As in, do relay nodes need to keep a record of the transactions they've
relayed? Strictly speaking, the answer is no: one a tx is relayed modulo
DoS concerns the entire thing can be discarded by the node. (unconfirmed
txs spending other unconfirmed txs can be handled by creating packages
of transactions, evaluated as a whole)

To mitigate DoS concerns, we of course have to have some per-UTXO limit
on bandwidth relayed, but that task can be accomplished by simply
maintaining some kind of per-UTXO record of bandwidth used. For instance
if the weighted fee and fee/KB were recorded, and forced to - say -
double for each additional tx relayed that spent a given UTXO you would
have a clear and simple upper limit of lifetime bandwidth. Equally it's
easy to limit bandwidth moment to moment by asking peers for highest
fee/KB transactions they advertise first, stopping when our bandwidth
limit is reached.

You probably could even remove IsStandard() pretty much entirely with
the right increasingly expensive "replacement" policy, relying on it
alone to provide anti-DoS. Obviously this would simplify some of the
debates around mining policy! This could even be re-used for scalable a
general-purpose messaging network paid by coin ownership if the UTXO set
is split up, and some kind of expiration over time policy is
implemented.

Miners of course would still want to have a mempool, but that codebase
may prove simpler if it doesn't have to work double-duty for relaying as
well.
--
'peter'[:-1]@petertodd.org
00000000000000000b675c4d825a10c278b8d63ee4df90a19393f3b6498fd073
Patrick Strateman via bitcoin-dev
2015-07-18 19:46:01 UTC
Permalink
Relay nodes do not need a mempool, but do need some mechanism to avoid
DoS issues.

Wallet nodes can use the mempool for fee estimation (in addition to
looking at past blocks).
Post by Peter Todd via bitcoin-dev
As in, do relay nodes need to keep a record of the transactions they've
relayed? Strictly speaking, the answer is no: one a tx is relayed modulo
DoS concerns the entire thing can be discarded by the node. (unconfirmed
txs spending other unconfirmed txs can be handled by creating packages
of transactions, evaluated as a whole)
To mitigate DoS concerns, we of course have to have some per-UTXO limit
on bandwidth relayed, but that task can be accomplished by simply
maintaining some kind of per-UTXO record of bandwidth used. For instance
if the weighted fee and fee/KB were recorded, and forced to - say -
double for each additional tx relayed that spent a given UTXO you would
have a clear and simple upper limit of lifetime bandwidth. Equally it's
easy to limit bandwidth moment to moment by asking peers for highest
fee/KB transactions they advertise first, stopping when our bandwidth
limit is reached.
You probably could even remove IsStandard() pretty much entirely with
the right increasingly expensive "replacement" policy, relying on it
alone to provide anti-DoS. Obviously this would simplify some of the
debates around mining policy! This could even be re-used for scalable a
general-purpose messaging network paid by coin ownership if the UTXO set
is split up, and some kind of expiration over time policy is
implemented.
Miners of course would still want to have a mempool, but that codebase
may prove simpler if it doesn't have to work double-duty for relaying as
well.
_______________________________________________
bitcoin-dev mailing list
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
odinn via bitcoin-dev
2015-07-19 08:59:49 UTC
Permalink
Some brief thoughts, adding here a suggestion for a dynamic approach
to the issue. (e.g. each additional tx relayed has some thing
associated with it, that is, a "doubling" for each additional tx
relayed that spends a given UTXO, doesn't sound like it would be the
most dynamic approach to the issue; considering that full nodes use
the (UTXOs) to establish if transactions are valid – all inputs to a
transaction must be in the UTXO database for it to be valid, but
rather, would end up ratcheting upward the fee/kB for each additional
tx relayed, as proposed).

A more dynamic fee approach would be a better one, imho, but how is
this to occur?

Quoting from Gavin Andresen's http://gavinandresen.ninja/utxo-uhoh,
"The entire UTXO set doesn’t have to be in RAM; it can be stored on an
SSD or spinning hard disk. The access pattern to the UTXO is not
random; outputs that were spent recently are more likely to be
re-spent than outputs that have not been spent in a long time. Bitcoin
Core already has a multi-level UTXO cache, thanks to the hard work of
Pieter Wuille."

The relay nodes would, in this scenario that is proposed here in this
message, be confirming and discarding; the wallet nodes, if I
understand properly, in this scenario, as proposed, should be
retaining (keeping a record of the transactions they've relayed and
using a mempool).

There are some questions here:

- - How is the mempool to be limited?
- - What is the mechanism by which the UTXO set is stored (or proposed
to be stored)?
- - How would dynamic fee determinations be calculated?
- - Please describe more the general purpose messaging network?

Thank you
Post by Patrick Strateman via bitcoin-dev
Relay nodes do not need a mempool, but do need some mechanism to
avoid DoS issues.
Wallet nodes can use the mempool for fee estimation (in addition
to looking at past blocks).
Post by Peter Todd via bitcoin-dev
As in, do relay nodes need to keep a record of the transactions
they've relayed? Strictly speaking, the answer is no: one a tx is
relayed modulo DoS concerns the entire thing can be discarded by
the node. (unconfirmed txs spending other unconfirmed txs can be
handled by creating packages of transactions, evaluated as a
whole)
To mitigate DoS concerns, we of course have to have some per-UTXO
limit on bandwidth relayed, but that task can be accomplished by
simply maintaining some kind of per-UTXO record of bandwidth
used. For instance if the weighted fee and fee/KB were recorded,
and forced to - say - double for each additional tx relayed that
spent a given UTXO you would have a clear and simple upper limit
of lifetime bandwidth. Equally it's easy to limit bandwidth
moment to moment by asking peers for highest fee/KB transactions
they advertise first, stopping when our bandwidth limit is
reached.
You probably could even remove IsStandard() pretty much entirely
with the right increasingly expensive "replacement" policy,
relying on it alone to provide anti-DoS. Obviously this would
simplify some of the debates around mining policy! This could
even be re-used for scalable a general-purpose messaging network
paid by coin ownership if the UTXO set is split up, and some kind
of expiration over time policy is implemented.
Miners of course would still want to have a mempool, but that
codebase may prove simpler if it doesn't have to work double-duty
for relaying as well.
_______________________________________________ bitcoin-dev
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
_______________________________________________ bitcoin-dev mailing
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
- --
http://abis.io ~
"a protocol concept to enable decentralization
and expansion of a giving economy, and a new social good"
https://keybase.io/odinn
Jorge Timón via bitcoin-dev
2015-07-20 22:14:41 UTC
Permalink
On Sat, Jul 18, 2015 at 9:46 PM, Patrick Strateman via bitcoin-dev
Relay nodes do not need a mempool, but do need some mechanism to avoid DoS
issues.
Wallet nodes can use the mempool for fee estimation (in addition to looking
at past blocks).
Exactly, so an anti-DoS mechanism that would be sufficient for a
non-mempool node would be also useful for small values in -maxmempool.
I think a simple cache for transaction validations should be enough.
Please, review a draft for that in the newest #6448.

https://github.com/bitcoin/bitcoin/pull/6448

I would be happy to rebase it back to 0.11 and even 0.10.

Loading...