[bitcoin-dev] Revisiting BIP 125 RBF policy.

[Russell O'Connor via bitcoin-dev]

I think it is worth revisiting BIP 125's replace-by-fee policy for when to
replace transactions.

The current policy can be problematic. As noted earlier by Rhavar,
sometimes one's transaction becomes pinned making it infeasible to fee bump
with RBF. This happens when one makes a normal payment to a large
commercial service, and, while the transaction remains unconfirmed, the
commercial service creates a low-fee-rate sweep of one's payment, among a
collection of others. If one wants to RBF this original payment, for
example to get confirmation of the change output for use in further
transactions, the current BIP 125 rules require that you make a fee bump
that exceeds the combined total fees of the original transaction and the
low-fee-rate sweep of the commercial service.

The problem is that, while the fee rate of the sweep is low, the absolute
size of the fee can still be large, making it infeasible to RBF the
original transaction. BIP 125 was defined back in 2015, when perhaps
rational miners did care about absolute fee amounts. However, today we are
in an era where rational miners care about fee-rates more than absolute
fees. The fee-rate of the large sweep transaction is low enough that we do
not expect that miners will be mining it in the same block as the original
transaction. Today, a rational miner will prefer a fee-bumped version of
original transaction without consideration of the low-fee sweep transaction
(or at least discounting the low-fee sweep in proportion to the miner's
hash-rate fraction).

Let me quote the five rules that define the current BIP 125 policy:

One or more transactions currently in the mempool (original transactions)

will be replaced by a new transaction (replacement transaction) that spends
one or more of the same inputs if,
1. The original transactions signal replaceability explicitly or
through inheritance as described in the above Summary section.
2. The replacement transaction does not contain any new unconfirmed
inputs that did not previously appear in the mempool. (Unconfirmed inputs
are inputs spending outputs from currently unconfirmed transactions.)
3. The replacement transaction pays an absolute fee of at least the
sum paid by the original transactions.
4. The replacement transaction must also pay for its own bandwidth at
or above the rate set by the node's minimum relay fee setting. For example,
if the minimum relay fee is 1 satoshi/byte and the replacement transaction
is 500 bytes total, then the replacement must pay a fee at least 500
satoshis higher than the sum of the originals.
5. The number of original transactions to be replaced and their
descendant transactions which will be evicted from the mempool must not
exceed a total of 100 transactions.
To address the new reality of rational miners' consideration, I propose

changing rules 3 and 4 to something like the following.

3'. The replacement transaction pays a fee rate of at least the effective
fee rate of any chain of transactions from the set of original transactions
that begins with the root of the original transaction set.

4'. The replacement transaction must also pay for replacing the original
transactions at or above the rate set by the node's minimum relay fee
setting. For example, if the minimum relay fee is 1 satoshi/byte and the
replacement transaction and the original transactions are 1000 bytes total,
then the replacement must pay a fee at least 1000 satoshis higher than the
fee of the root transaction of the original transactions.

Rule 3' is a fancy way of saying that the replacement transaction must have
a fee rate that is larger than the package fee rate of the root of the set
of transactions it replaces, where the package fee rate is the fee rate
implied by considering CPFP.

Rule 4' is an amended anti-spam rule that is intended to avoid DOS attacks
from churning the mempool. I don't know if it is really necessary to pay
for the size of the original transactions being evicted, but some people I
chatted with thought it could be important.

Other people on the mailing list have been thinking about RBF policy for
far longer than I have, so I wouldn't be surprised if my proposal above is
naive. However, I think it can start a conversation about addressing the
problems with the current RBF policy.

[Ryan Havar via bitcoin-dev]

Thank you very much for writing this up.  It's worth noting that there can be multiple roots for the transactions that are getting replaced.

So for rule 3, you probably want a feeRate >= the max "package fee rate" of all replaced roots.


I am very happy with this proposal in general, as it's clearly a step in the right direction for making transaction replacement practically usable for todays services.


However, I think your new rule 4 is a bit weak. The logical extension of your proposal would be to allow a transaction (say B) be able to replace transactions (say A) by purely paying a higher fee rate, /even if it's less absolute fee/. In this simple example of B replacing A -- B should pay at least: (a.FeeRate * b.size) + relayFeeRate*(a.size + b.size)



​-Ryan

​

-------- Original Message --------

I think it is worth revisiting BIP 125's replace-by-fee policy for when to replace transactions.
The current policy can be problematic. As noted earlier by Rhavar, sometimes one's transaction becomes pinned making it infeasible to fee bump with RBF.  This happens when one makes a normal payment to a large commercial service, and, while the transaction remains unconfirmed, the commercial service creates a low-fee-rate sweep of one's payment, among a collection of others.  If one wants to RBF this original payment, for example to get confirmation of the change output for use in further transactions, the current BIP 125 rules require that you make a fee bump that exceeds the combined total fees of the original transaction and the low-fee-rate sweep of the commercial service.
The problem is that, while the fee rate of the sweep is low, the absolute size of the fee can still be large, making it infeasible to RBF the original transaction.  BIP 125 was defined back in 2015, when perhaps rational miners did care about absolute fee amounts. However, today we are in an era where rational miners care about fee-rates more than absolute fees.  The fee-rate of the large sweep transaction is low enough that we do not expect that miners will be mining it in the same block as the original transaction.  Today, a rational miner will prefer a fee-bumped version of original transaction without consideration of the low-fee sweep transaction (or at least discounting the low-fee sweep in proportion to the miner's hash-rate fraction).

Post by Russell O'Connor via bitcoin-dev
One or more transactions currently in the mempool (original
transactions) will be replaced by a new transaction (replacement
transaction) that spends one or more of the same inputs if,
1. The original transactions signal replaceability explicitly or through inheritance as described in the above Summary section.
2. The
replacement transaction does not contain any new unconfirmed inputs
that did not previously appear in the mempool. (Unconfirmed inputs are
inputs spending outputs from currently unconfirmed transactions.)
3. The replacement transaction pays an absolute fee of at least the sum paid by the original transactions.
4. The
replacement transaction must also pay for its own bandwidth at or above
the rate set by the node's minimum relay fee setting. For example, if
the minimum relay fee is 1 satoshi/byte and the replacement transaction
is 500 bytes total, then the replacement must pay a fee at least 500
satoshis higher than the sum of the originals.
5. The number of
original transactions to be replaced and their descendant transactions
which will be evicted from the mempool must not exceed a total of 100
transactions.
To address the new reality of rational miners' consideration, I propose changing rules 3 and 4 to something like the following.

3'. The replacement transaction pays a fee rate of at least the effective fee rate of any chain of transactions from the set of original transactions that begins with the root of the original transaction set.
4'. The
replacement transaction must also pay for replacing the original transactions at or above
the rate set by the node's minimum relay fee setting. For example, if
the minimum relay fee is 1 satoshi/byte and the replacement transaction and the original transactions are 1000 bytes total, then the replacement must pay a fee at least 1000
satoshis higher than the fee of the root transaction of the original transactions.
Rule 3' is a fancy way of saying that the replacement transaction must have a fee rate that is larger than the package fee rate of the root of the set of transactions it replaces, where the package fee rate is the fee rate implied by considering CPFP.
Rule 4' is an amended anti-spam rule that is intended to avoid DOS attacks from churning the mempool. I don't know if it is really necessary to pay for the size of the original transactions being evicted, but some people I chatted with thought it could be important.
Other people on the mailing list have been thinking about RBF policy for far longer than I have, so I wouldn't be surprised if my proposal above is naive.  However, I think it can start a conversation about addressing the problems with the current RBF policy.

[Peter Todd via bitcoin-dev]

Post by Russell O'Connor via bitcoin-dev
I think it is worth revisiting BIP 125's replace-by-fee policy for when to
replace transactions.
The current policy can be problematic. As noted earlier by Rhavar,
sometimes one's transaction becomes pinned making it infeasible to fee bump
with RBF. This happens when one makes a normal payment to a large
commercial service, and, while the transaction remains unconfirmed, the
commercial service creates a low-fee-rate sweep of one's payment, among a
collection of others. If one wants to RBF this original payment, for
example to get confirmation of the change output for use in further
transactions, the current BIP 125 rules require that you make a fee bump
that exceeds the combined total fees of the original transaction and the
low-fee-rate sweep of the commercial service.
The problem is that, while the fee rate of the sweep is low, the absolute
size of the fee can still be large, making it infeasible to RBF the
original transaction. BIP 125 was defined back in 2015, when perhaps
rational miners did care about absolute fee amounts. However, today we are
in an era where rational miners care about fee-rates more than absolute
fees. The fee-rate of the large sweep transaction is low enough that we do
not expect that miners will be mining it in the same block as the original
transaction. Today, a rational miner will prefer a fee-bumped version of
original transaction without consideration of the low-fee sweep transaction
(or at least discounting the low-fee sweep in proportion to the miner's
hash-rate fraction).

I don't actually see where the problem is here. First of all, suppose we have a
transaction T_a that already pays Alice with a feerate sufficiently high that
we expect it to get mined in the near future. If we want to pay Bob, we can do
that by simply creating a double-spend of T_a that pays both Bob and Alice,
T_{ab}. BIP125 only requires that double-spend to have an absolute fee higher
than the minimum relay feerate * size of the transaction.

I just checked one of my nodes, and the absolute minimum relay fee is about
1/5th that of what estimatefee returns for the longest possible estimate, 48
blocks. Depends on the exact circumstances, but it'll likely be worth it to pay
Bob with a replacement of T_a rather than create a second transaction due to
that difference.

Secondly, if for some reason you need to broadcast a separate transaction
paying Bob before you do the replacement, again I don't see an issue: just make
a minimum fee T_b that pays Bob, and replace both with T_{ab}. Again, the big
difference between minimum fee and what you might actually pay in fees means
that you'll still save money in most cases, so long as your wallet is
intelligent enough to pick a low feerate for T_b.

Post by Russell O'Connor via bitcoin-dev
One or more transactions currently in the mempool (original transactions)

will be replaced by a new transaction (replacement transaction) that spends
one or more of the same inputs if,
1. The original transactions signal replaceability explicitly or
through inheritance as described in the above Summary section.
2. The replacement transaction does not contain any new unconfirmed
inputs that did not previously appear in the mempool. (Unconfirmed inputs
are inputs spending outputs from currently unconfirmed transactions.)
3. The replacement transaction pays an absolute fee of at least the
sum paid by the original transactions.
4. The replacement transaction must also pay for its own bandwidth at
or above the rate set by the node's minimum relay fee setting. For example,
if the minimum relay fee is 1 satoshi/byte and the replacement transaction
is 500 bytes total, then the replacement must pay a fee at least 500
satoshis higher than the sum of the originals.
5. The number of original transactions to be replaced and their
descendant transactions which will be evicted from the mempool must not
exceed a total of 100 transactions.
To address the new reality of rational miners' consideration, I propose

changing rules 3 and 4 to something like the following.
3'. The replacement transaction pays a fee rate of at least the effective
fee rate of any chain of transactions from the set of original transactions
that begins with the root of the original transaction set.

I think what you mean here should be the effective fee rate of the maximum
feerate package that can be built from the set of transactions that begins with
the candidate replacement. But actually calculating this is I believe
non-trivial, which is why I didn't implement it this way when RBF was first
implemented.

Post by Russell O'Connor via bitcoin-dev
4'. The replacement transaction must also pay for replacing the original
transactions at or above the rate set by the node's minimum relay fee
setting. For example, if the minimum relay fee is 1 satoshi/byte and the
replacement transaction and the original transactions are 1000 bytes total,
then the replacement must pay a fee at least 1000 satoshis higher than the
fee of the root transaction of the original transactions.

So the previous version of condition #4 does this implicitly because the
absolute fee isn't allowed to go down; you're effectively re-adding this
condition. But as I've shown above, you can get the same *behavior* by simply
ensuring that the transactions you broadcast that you'll want to double-spend
have a minimum feerate in the first place.

Post by Russell O'Connor via bitcoin-dev
Rule 3' is a fancy way of saying that the replacement transaction must have
a fee rate that is larger than the package fee rate of the root of the set
of transactions it replaces, where the package fee rate is the fee rate
implied by considering CPFP.
Rule 4' is an amended anti-spam rule that is intended to avoid DOS attacks
from churning the mempool. I don't know if it is really necessary to pay
for the size of the original transactions being evicted, but some people I
chatted with thought it could be important.

I think this is very important. For example, without this condition I could do
a DoS attack by repeatedly broadcasting a transaction, then spending the
outputs of that transaction with a very large number of child transactions, all
of minimum fee. With up to 100 transactions allowed for consideration, and a
100KB max transaction size, that could be up to ~10MB of transactions.

Next I double spend the root, increasing it's feerate but *not* paying for the
child transactions. Those ~10MB are now evicted from the mempool, and I can
repeat the cycle again. The cost is whatever the root tx replacement cost,
which will be much less than the cost of broadcasting 10MB should have been.

Post by Russell O'Connor via bitcoin-dev
Other people on the mailing list have been thinking about RBF policy for
far longer than I have, so I wouldn't be surprised if my proposal above is
naive. However, I think it can start a conversation about addressing the
problems with the current RBF policy.

A better way to solve this class of problems may be diffed tx replacement
propagation: basically broadcast a diff between the original tx and the
proposed replacement, allowing you to do the minimum bandwidth accounting based
on the size of the diff instead.

[Russell O'Connor via bitcoin-dev]

This post might be inappropriate. Click to display it.

[Peter Todd via bitcoin-dev]

Post by Russell O'Connor via bitcoin-dev

Post by Peter Todd via bitcoin-dev
I don't actually see where the problem is here. First of all, suppose we have a
transaction T_a that already pays Alice with a feerate sufficiently high that
we expect it to get mined in the near future. If we want to pay Bob, we can do
that by simply creating a double-spend of T_a that pays both Bob and Alice,
T_{ab}. BIP125 only requires that double-spend to have an absolute fee higher
than the minimum relay feerate * size of the transaction.

The problem is that rule 3 of BIP 125 requires you pay a fee that is higher
than the the fee of T_a *plus* the fee of the sweep-transaction that the
Alice has added as a unconfirmed child transaction to T_a because
double-spending to pay Alice and Bob invalidates Alice's
sweep-transaction. Alice's sweep-transaction is very large, and hence pays
a large absolute fee even though her fee-rate is very low. We do not have
any control over its value, hence Alice has "pinned" our RBF transaction.

Ah ok, I misunderstood and didn't realise you were talking about the case where
Alice re-spends her unconfirmed payment. Unfortunately I don't think that case
is possible to solve without putting some kind of restriction on spending
unconfirmed outputs; with a restriction it's fairly simple to solve.

Post by Russell O'Connor via bitcoin-dev

Post by Peter Todd via bitcoin-dev
I think what you mean here should be the effective fee rate of the maximum
feerate package that can be built from the set of transactions that begins with
the candidate replacement. But actually calculating this is I believe
non-trivial, which is why I didn't implement it this way when RBF was first
implemented.

Yes, that is what I mean. My proposal was off-the-mark.
Surely CPFP is already computing the package-fee rates of mempool
transactions. That is the value we need to compute.

True, maybe we can just reuse the CPFP calculation now. That said, AFAIK that's
only done in the miner code, not the mempool, so that may not be trivial to
actually do.

[Russell O'Connor via bitcoin-dev]

Post by Peter Todd via bitcoin-dev

Post by Russell O'Connor via bitcoin-dev

Post by Peter Todd via bitcoin-dev
I don't actually see where the problem is here. First of all, suppose

we

Post by Russell O'Connor via bitcoin-dev

Post by Peter Todd via bitcoin-dev
have a
transaction T_a that already pays Alice with a feerate sufficiently

high

Post by Russell O'Connor via bitcoin-dev

Post by Peter Todd via bitcoin-dev
that
we expect it to get mined in the near future. If we want to pay Bob, we can do
that by simply creating a double-spend of T_a that pays both Bob and

Alice,

Post by Russell O'Connor via bitcoin-dev

Post by Peter Todd via bitcoin-dev
T_{ab}. BIP125 only requires that double-spend to have an absolute fee higher
than the minimum relay feerate * size of the transaction.

The problem is that rule 3 of BIP 125 requires you pay a fee that is

higher

Post by Russell O'Connor via bitcoin-dev
than the the fee of T_a *plus* the fee of the sweep-transaction that the
Alice has added as a unconfirmed child transaction to T_a because
double-spending to pay Alice and Bob invalidates Alice's
sweep-transaction. Alice's sweep-transaction is very large, and hence

pays

Post by Russell O'Connor via bitcoin-dev
a large absolute fee even though her fee-rate is very low. We do not

have

Post by Russell O'Connor via bitcoin-dev
any control over its value, hence Alice has "pinned" our RBF transaction.

Ah ok, I misunderstood and didn't realise you were talking about the case where
Alice re-spends her unconfirmed payment. Unfortunately I don't think that case
is possible to solve without putting some kind of restriction on spending
unconfirmed outputs; with a restriction it's fairly simple to solve.

Adding such a restriction was Rhavar's original suggestion in
https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-July/014688.html,
but it seems the proposal wasn't well received because it kinda destroys
CPFP.

[Russell O'Connor via bitcoin-dev]

Post by Peter Todd via bitcoin-dev

Post by Russell O'Connor via bitcoin-dev
Surely CPFP is already computing the package-fee rates of mempool
transactions. That is the value we need to compute.

True, maybe we can just reuse the CPFP calculation now. That said, AFAIK that's
only done in the miner code, not the mempool, so that may not be trivial to
actually do.

Do you (or anyone else) know if the package fee rate is considered when
ejecting transactions from the bottom of the mempool when the mempool gets
too large?

[Greg Sanders via bitcoin-dev]

Yes.

On Wed, Feb 14, 2018 at 9:08 AM, Russell O'Connor via bitcoin-dev <

Post by Russell O'Connor via bitcoin-dev

Post by Peter Todd via bitcoin-dev

Post by Russell O'Connor via bitcoin-dev
Surely CPFP is already computing the package-fee rates of mempool
transactions. That is the value we need to compute.

True, maybe we can just reuse the CPFP calculation now. That said, AFAIK that's
only done in the miner code, not the mempool, so that may not be trivial to
actually do.

Do you (or anyone else) know if the package fee rate is considered when
ejecting transactions from the bottom of the mempool when the mempool gets
too large?
_______________________________________________
bitcoin-dev mailing list
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

[Russell O'Connor via bitcoin-dev]

Post by Peter Todd via bitcoin-dev
Ah ok, I misunderstood and didn't realise you were talking about the case where
Alice re-spends her unconfirmed payment. Unfortunately I don't think that case
is possible to solve without putting some kind of restriction on spending
unconfirmed outputs; with a restriction it's fairly simple to solve.

When you say that you don't think it is possible to solve, do you mean that
there is a specific problem with this proposal of replacing transactions
when offered a new transaction whose fee rate exceeds the package fee rate
of the original transaction (and ensuring that the fee increase covers the
size of the transactions being ejected)? Is your concern only about the
ability to computing and track the package fee rate for transactions within
the mempool or is there some other issue you foresee?

[Peter Todd via bitcoin-dev]

Post by Russell O'Connor via bitcoin-dev

Post by Peter Todd via bitcoin-dev
Ah ok, I misunderstood and didn't realise you were talking about the case where
Alice re-spends her unconfirmed payment. Unfortunately I don't think that case
is possible to solve without putting some kind of restriction on spending
unconfirmed outputs; with a restriction it's fairly simple to solve.

When you say that you don't think it is possible to solve, do you mean that
there is a specific problem with this proposal of replacing transactions
when offered a new transaction whose fee rate exceeds the package fee rate
of the original transaction (and ensuring that the fee increase covers the
size of the transactions being ejected)? Is your concern only about the
ability to computing and track the package fee rate for transactions within
the mempool or is there some other issue you foresee?

I mean, I think in general solving this problem is probably not possible.
Basically, the fundamental problem is someone else has consumed network
bandwidth that should be paid for with fees. What you're trying to do is
replace a transaction without paying those fees, which is identical to what an
attacker is trying to do, and thus any such scheme will be as vulnerable to
attack as not having that protection in the first place.

...which does give you an out: maybe the attack isn't important enough to
matter. :)

[Russell O'Connor via bitcoin-dev]

Post by Russell O'Connor via bitcoin-dev

Post by Russell O'Connor via bitcoin-dev
When you say that you don't think it is possible to solve, do you mean

that

Post by Russell O'Connor via bitcoin-dev
there is a specific problem with this proposal of replacing transactions
when offered a new transaction whose fee rate exceeds the package fee

rate

Post by Russell O'Connor via bitcoin-dev
of the original transaction (and ensuring that the fee increase covers

the

Post by Russell O'Connor via bitcoin-dev
size of the transactions being ejected)? Is your concern only about the
ability to computing and track the package fee rate for transactions

within

Post by Russell O'Connor via bitcoin-dev
the mempool or is there some other issue you foresee?

I mean, I think in general solving this problem is probably not possible.
Basically, the fundamental problem is someone else has consumed network
bandwidth that should be paid for with fees. What you're trying to do is
replace a transaction without paying those fees, which is identical to what an
attacker is trying to do, and thus any such scheme will be as vulnerable to
attack as not having that protection in the first place.
...which does give you an out: maybe the attack isn't important enough to
matter. :)

Thanks, that makes sense.

I still think it is worthwhile pursuing this proposed change in RBF policy
as it would seem that the current policy is problematic in practice today
where participants are just performing normal transactions and are not
trying to attack each other.

[Peter Todd via bitcoin-dev]

Post by Russell O'Connor via bitcoin-dev

Post by Peter Todd via bitcoin-dev
I mean, I think in general solving this problem is probably not possible.
Basically, the fundamental problem is someone else has consumed network
bandwidth that should be paid for with fees. What you're trying to do is
replace a transaction without paying those fees, which is identical to what an
attacker is trying to do, and thus any such scheme will be as vulnerable to
attack as not having that protection in the first place.
...which does give you an out: maybe the attack isn't important enough to
matter. :)

Thanks, that makes sense.
I still think it is worthwhile pursuing this proposed change in RBF policy
as it would seem that the current policy is problematic in practice today
where participants are just performing normal transactions and are not
trying to attack each other.

But that's not a good argument: whether or not normal users are trying to
attack each other has nothing to do with whether or not you're opening up an
attack by relaxing anti-DoS protections.

Equally, how often are normal users who aren't attacking each other creating
issues anyway? You can always have your wallet code just skip use of RBF
replacements in the event that someone does spend an unconfirmed output that
you sent them; how often does this actually happen in practice? Not many
wallets let you spend unconfirmed outputs that you didn't create.

[Russell O'Connor via bitcoin-dev]

Post by Peter Todd via bitcoin-dev

Post by Russell O'Connor via bitcoin-dev

Post by Peter Todd via bitcoin-dev
I mean, I think in general solving this problem is probably not

possible.

Post by Russell O'Connor via bitcoin-dev

Post by Peter Todd via bitcoin-dev
Basically, the fundamental problem is someone else has consumed network
bandwidth that should be paid for with fees. What you're trying to do

is

Post by Russell O'Connor via bitcoin-dev

Post by Peter Todd via bitcoin-dev
replace a transaction without paying those fees, which is identical to what an
attacker is trying to do, and thus any such scheme will be as

vulnerable to

Post by Russell O'Connor via bitcoin-dev

Post by Peter Todd via bitcoin-dev
attack as not having that protection in the first place.
...which does give you an out: maybe the attack isn't important enough

to

Post by Russell O'Connor via bitcoin-dev

Post by Peter Todd via bitcoin-dev
matter. :)

Thanks, that makes sense.
I still think it is worthwhile pursuing this proposed change in RBF

policy

Post by Russell O'Connor via bitcoin-dev
as it would seem that the current policy is problematic in practice today
where participants are just performing normal transactions and are not
trying to attack each other.

But that's not a good argument: whether or not normal users are trying to
attack each other has nothing to do with whether or not you're opening up an
attack by relaxing anti-DoS protections.

I'm not suggesting removing the anti-DoS protections. I'm suggesting that
replaced transaction require a fee increase of at least the min-fee-rate
times the size of all the transactions being ejected (in addition to the
other proposed requirements).

Post by Peter Todd via bitcoin-dev
Equally, how often are normal users who aren't attacking each other creating
issues anyway? You can always have your wallet code just skip use of RBF

replacements in the event that someone does spend an unconfirmed output that

Post by Peter Todd via bitcoin-dev
you sent them; how often does this actually happen in practice?

Just ask rhavar. It happens regularly.

Not many wallets let you spend unconfirmed outputs that you didn't create.
The problem is with institutional wallets sweeping incoming payments. It
seems that in practice they are happy to sweep unconfirmed outputs.

Setting all of the above aside for a moment. We need to understand that
rational miners are going to prefer to transactions with higher package fee
rates regardless of whatever your personal preferred RBF policy is. If we
do not bring the RBF policy to alignment with what is economically
rational, then miners are going to change their own policies anyways,
probably all in slightly different ways. It behooves everyone to develop a
reasonable standard RBF policy, that is still robust against possible DoS
vectors, and aligns with miner incentives, so that all participants know
what behaviour they can reasonably expect. It is simply a bonus that this
change in RBF policy also partially mitigates the problem of pinned
transactions.

[Peter Todd via bitcoin-dev]

Post by Russell O'Connor via bitcoin-dev

Post by Peter Todd via bitcoin-dev
But that's not a good argument: whether or not normal users are trying to
attack each other has nothing to do with whether or not you're opening up an
attack by relaxing anti-DoS protections.

I'm not suggesting removing the anti-DoS protections. I'm suggesting that
replaced transaction require a fee increase of at least the min-fee-rate
times the size of all the transactions being ejected (in addition to the
other proposed requirements).

Fair: you're not removing them entirely, but you are weakening them compared to
the status quo.

Post by Russell O'Connor via bitcoin-dev

Post by Peter Todd via bitcoin-dev
Equally, how often are normal users who aren't attacking each other creating
issues anyway? You can always have your wallet code just skip use of RBF

replacements in the event that someone does spend an unconfirmed output that

Post by Peter Todd via bitcoin-dev
you sent them; how often does this actually happen in practice?

Just ask rhavar. It happens regularly.
Not many wallets let you spend unconfirmed outputs that you didn't create.
The problem is with institutional wallets sweeping incoming payments. It
seems that in practice they are happy to sweep unconfirmed outputs.

Pity, that does sound like a problem. :(

Post by Russell O'Connor via bitcoin-dev
Setting all of the above aside for a moment. We need to understand that
rational miners are going to prefer to transactions with higher package fee
rates regardless of whatever your personal preferred RBF policy is. If we
do not bring the RBF policy to alignment with what is economically
rational, then miners are going to change their own policies anyways,
probably all in slightly different ways. It behooves everyone to develop a
reasonable standard RBF policy, that is still robust against possible DoS
vectors, and aligns with miner incentives, so that all participants know
what behaviour they can reasonably expect. It is simply a bonus that this
change in RBF policy also partially mitigates the problem of pinned
transactions.

Miners and full nodes have slightly different priorities here; it's not clear
to me why it matters that they implement slightly different policies.


Still, re-reading your initital post, I'm convinced that the weakening of the
DoS protections is probably not a huge problem, so maybe lets try this in a
release and see what happens.

Notably, if people actually use this new replacement behavior, the institutions
doing these sweeps of unconfirmed outputs might stop doing that! That's
probably a good thing, as respends of potentially conflicted unconfirmed
outputs can be dangerous in reorgs; we're better off if outputs are buried
deeply before being spent again.

[Ryan Havar via bitcoin-dev]

Post by Peter Todd via bitcoin-dev
Still, re-reading your initital post, I'm convinced that the weakening of the
DoS protections is probably not a huge problem, so maybe lets try this in a
release and see what happens.

Awesome! I very much agree. The relaxation of some of these DoS prevention rules I think will really open up a lot of use cases and adoption

Post by Peter Todd via bitcoin-dev
Notably, if people actually use this new replacement behavior, the institutions
doing these sweeps of unconfirmed outputs might stop doing that!

Agree, I'm pretty sure it's unintentional. I know a lot of services struggle with coin selection, so what they do is conceptually have a receive wallet from which they can sweep to their hot wallet (or cold storage) to keep their utxo manageable.

Currently some of them are sweeping unconfirmed inputs with it, but I don't think it's a conscious design choice, just something that happens to be working well now.

(FWIW I observed this behavior like 6+ months ago, I haven't kept track of if it's still happening or how often. But at the time I had to write off the idea of low-fee rbf batch transactions as it was happening too often to be feasible)


​-Ryan​

‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐

Post by Peter Todd via bitcoin-dev
​​

Post by Peter Todd via bitcoin-dev
But that's not a good argument: whether or not normal users are trying to
attack each other has nothing to do with whether or not you're opening up
an
attack by relaxing anti-DoS protections.

I'm not suggesting removing the anti-DoS protections. I'm suggesting that
replaced transaction require a fee increase of at least the min-fee-rate
times the size of all the transactions being ejected (in addition to the
other proposed requirements).

Fair: you're not removing them entirely, but you are weakening them compared to
the status quo.

Post by Peter Todd via bitcoin-dev
Equally, how often are normal users who aren't attacking each other
creating
issues anyway? You can always have your wallet code just skip use of RBF

replacements in the event that someone does spend an unconfirmed output that

Post by Peter Todd via bitcoin-dev
you sent them; how often does this actually happen in practice?

Just ask rhavar. It happens regularly.
Not many wallets let you spend unconfirmed outputs that you didn't create.
The problem is with institutional wallets sweeping incoming payments. It
seems that in practice they are happy to sweep unconfirmed outputs.

Pity, that does sound like a problem. :(

Setting all of the above aside for a moment. We need to understand that
rational miners are going to prefer to transactions with higher package fee
rates regardless of whatever your personal preferred RBF policy is. If we
do not bring the RBF policy to alignment with what is economically
rational, then miners are going to change their own policies anyways,
probably all in slightly different ways. It behooves everyone to develop a
reasonable standard RBF policy, that is still robust against possible DoS
vectors, and aligns with miner incentives, so that all participants know
what behaviour they can reasonably expect. It is simply a bonus that this
change in RBF policy also partially mitigates the problem of pinned
transactions.

Miners and full nodes have slightly different priorities here; it's not clear
to me why it matters that they implement slightly different policies.
Still, re-reading your initital post, I'm convinced that the weakening of the
DoS protections is probably not a huge problem, so maybe lets try this in a
release and see what happens.
Notably, if people actually use this new replacement behavior, the institutions
doing these sweeps of unconfirmed outputs might stop doing that! That's
probably a good thing, as respends of potentially conflicted unconfirmed
outputs can be dangerous in reorgs; we're better off if outputs are buried
deeply before being spent again.
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
bitcoin-dev mailing list
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

[Ryan Havar via bitcoin-dev]

Post by Peter Todd via bitcoin-dev
I don't actually see where the problem is here. First of all, suppose we have a
transaction T_a that already pays Alice with a feerate sufficiently high that
we expect it to get mined in the near future. If we want to pay Bob, we can do
that by simply creating a double-spend of T_a that pays both Bob and Alice,
T_{ab}. BIP125 only requires that double-spend to have an absolute fee higher
than the minimum relay feerate * size of the transaction.

It's a bit of a made up term, but when Russel said "pinned" it refers to a child transaction that was made (i.e. in this case by Alice) and for us to replace our transaction we need to "unpin" it.

However to unpin it, the current bip125 rules require you pay not only the relay of the transactions you're throwing away, but the absolute fee. This is general is cost prohibitive, as even a normalish transaction can be spending $20 in fees.

Also FWIW this whole idea of T_a and T_ab is good, but it's also pretty impractical at the moment due to the sheer amount complexity it introduces (i.e. monitoring, seeing which confirms, trying to rebroadcast the missing one in a way that is safe against reorgs, blah blah).

Post by Peter Todd via bitcoin-dev
I just checked one of my nodes, and the absolute minimum relay fee is about
1/5th that of what estimatefee returns for the longest possible estimate, 48
blocks.

If you use estimatesmartfee you should be able to get estimates all the way to 1008 or something btw

Post by Peter Todd via bitcoin-dev
[...]
I think this is very important. For example, without this condition I could do
a DoS attack by repeatedly broadcasting a transaction, then spending the
outputs of that transaction with a very large number of child transactions, all
of minimum fee.

I agree.

Post by Peter Todd via bitcoin-dev
A better way to solve this class of problems may be diffed tx replacement
propagation: basically broadcast a diff between the original tx and the
proposed replacement, allowing you to do the minimum bandwidth accounting based
on the size of the diff instead.

This would definitely work for some specific use-case. For instance currently if you do n replacements of a transaction, each time adding an additional output .. you need to pay something like O(n^2) relay fee. If you used a diff instead, you could probably get it to O(n)ish.

But relay fee (and n) at the moment, mean it's not a big deal at all. The big flaw (imo) in bip125 is that you need to pay the absolute fee from the transactions you are evicting. And that can be from transactions you didn't even generate yourself. We can already compactly represent the diff (the new transaction invalidates it) the debate is more "Should you have to pay the absolute fee or just relay fee for the stuff you invalidate"

[Peter Todd via bitcoin-dev]

Post by Ryan Havar via bitcoin-dev

Post by Peter Todd via bitcoin-dev
I don't actually see where the problem is here. First of all, suppose we have a
transaction T_a that already pays Alice with a feerate sufficiently high that
we expect it to get mined in the near future. If we want to pay Bob, we can do
that by simply creating a double-spend of T_a that pays both Bob and Alice,
T_{ab}. BIP125 only requires that double-spend to have an absolute fee higher
than the minimum relay feerate * size of the transaction.

It's a bit of a made up term, but when Russel said "pinned" it refers to a child transaction that was made (i.e. in this case by Alice) and for us to replace our transaction we need to "unpin" it.

Yeah, sorry, I just misread what scenario you guys were talking about. IIRC the
term "pinned" may have even been invented by myself, as IIRC I noticed the
issue when the RBF patch was being developed years ago. I don't think I had a
solution at the time so I just punted on it.

Post by Ryan Havar via bitcoin-dev
However to unpin it, the current bip125 rules require you pay not only the relay of the transactions you're throwing away, but the absolute fee. This is general is cost prohibitive, as even a normalish transaction can be spending $20 in fees.
Also FWIW this whole idea of T_a and T_ab is good, but it's also pretty impractical at the moment due to the sheer amount complexity it introduces (i.e. monitoring, seeing which confirms, trying to rebroadcast the missing one in a way that is safe against reorgs, blah blah).

I'm not sure that's actually true, as you're only creating transactions sets
that are reorg safe. Though I don't have a detailed design in mind so I may be
missing something.

Post by Ryan Havar via bitcoin-dev

Post by Peter Todd via bitcoin-dev
A better way to solve this class of problems may be diffed tx replacement
propagation: basically broadcast a diff between the original tx and the
proposed replacement, allowing you to do the minimum bandwidth accounting based
on the size of the diff instead.

This would definitely work for some specific use-case. For instance currently if you do n replacements of a transaction, each time adding an additional output .. you need to pay something like O(n^2) relay fee. If you used a diff instead, you could probably get it to O(n)ish.
But relay fee (and n) at the moment, mean it's not a big deal at all. The big flaw (imo) in bip125 is that you need to pay the absolute fee from the transactions you are evicting. And that can be from transactions you didn't even generate yourself. We can already compactly represent the diff (the new transaction invalidates it) the debate is more "Should you have to pay the absolute fee or just relay fee for the stuff you invalidate"

Yes, the diff approach doesn't help for the pinned case.

Unfortunately the only solution I have is basically the same as what you
proposed(1) months ago: limit spends of unconfirmed outputs in some way.

So here's a question: how many wallets have actually implemented CPFP fee bumps
for incoming transactions?

1) https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-July/014688.html

[Ryan Havar via bitcoin-dev]

Post by Peter Todd via bitcoin-dev
Yeah, sorry, I just misread what scenario you guys were talking about. IIRC the
term "pinned" may have even been invented by myself, as IIRC I noticed the
issue when the RBF patch was being developed years ago. I don't think I had a
solution at the time so I just punted on it.

Yeah. I posted that before it was clarified, it's just my message got held up in the moderation queue so it came out of order at an inconvenient time ><

Post by Peter Todd via bitcoin-dev
I'm not sure that's actually true, as you're only creating transactions sets
that are reorg safe. Though I don't have a detailed design in mind so I may be
missing something.

It is. T_a and T_ab are "reorg" safe, but if T_a confirms you will still need to pay Bob in way. But you need to pay him such that in a reorg occurs and suddenly T_ab is mined, you haven't doubled paid him.

I've been working on it's implementation, but it's honestly really complex and hard to test. I outlined the procedure here: https://gist.github.com/RHavar/cff76a026ece8446c898470db4f35682 which I call "Super Withdrawals".


My point though isn't that it's impossible, it's that it's sufficiently complex that it's unreasonable to expect anyone to be doing it any time soon. By relaxing any unnecessary restrictions on bip125, just makes it _drastically_ easier to do certain things.

Post by Peter Todd via bitcoin-dev
So here's a question: how many wallets have actually implemented CPFP fee bumps
for incoming transactions?

Never tried it, but I recall seeing it in the electrum gui. I originally tried supporting this myself, but it's kind of annoying. It's generally a bit cost-prohibitive to create a transaction specifically for the purpose of a CPFP fee bump, but since I made transactions pretty frequently (averaged say every 8 minutes) it doesn't add an additional input for the purpose of bumping selected incoming transactions.

The work flow is reasonably smooth: Alice has sent me 1 BTC with low fees, I owe Bob some money. I source Alice's output in the payment to Bob, giving her transaction a fee bump. Both transactions confirm, everyone is happy.

However during the whole time I need to watch Alice's transaction because if it ever is replaced/conflicted, I need to immediately pay Bob (in a reorg safe way, so I don't double-pay). It's not terribly hard to do, by making sure when I pay Bob I use an additional input that I also use for any "repayment" but it's enough complexity and hard enough to test that I gave up.

The really nice thing of most current send systems (and now especially so with segwit) is everything is pretty much fire and forget. (although I just schedule in 0.5, 1, 2, 4, .... 32 hours fee bump attempts. But that's just background that can fail/succeed blindly)

Post by Peter Todd via bitcoin-dev
1. https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-July/014688.html
--