Discussion:
How accurate are the Bitcoin timestamps?
Add Reply
Neiman via bitcoin-dev
2018-01-29 13:34:20 UTC
Reply
Permalink
Raw Message
First time posting here, please be gentle.

I'm doing a research project about blockchain timestamping. There are many
such projects, including the fantastic OpenTimestamps.

All of the projects essentially save some data in a block, and rely on the
block timestamp as a proof that this data existed at some specific time.

But how accurate are Bitcoins timestamps?

I didn't find any discussion or research regarding Bitcoin timestamp
accuracy (also not in the history of this mailing list). I share here a
simple analysis of timestamp accuracy, and a suggestion how to improve it.

Basic observations and questions:
-------------------------------------------
*1.* It seems to me that the timestamp is not the time that the block was
created. Ideally, it's the time that the miner started to try to mine the
block. However, as timestamps may also be used as a source of variety for
hashes, the exact meaning of its value is unclear.

If this is true, then there's a strange phenomena to observe in
blockchain.info and blockexplorer.com: the timestamps of blocks equals the
receiving times.

Am I wrong in my understanding, or is there a mistake in those websites?

*2.* Timestamps are not necessary to avoid double-spending. A simple
ordering of blocks is sufficient, so exchanging timestamps with enumeration
would work double-spending wise. Permissioned consensus protocols, such as
hyperledger, indeed have no timestamps (in version 1.0).

As far as I could tell, timestamps are included in Bitcoin's protocol
*only* to adjust the difficulty of PoW.

Direct control of timestamp accuracy:
-----------------------------------------------
The only element in the protocol that I found to control timestamp accuracy
is based on the network time concept.

The Bitcoin protocol defines “network time” for each node. The network time
is the median time of the other clients, but only if
1. there are at least 5 connected, and
2. the difference between the median time and the nodes own system time
is less than 70 minutes.

Then new blocks are accepted by the peers if their timestamps is
1. less than the network time plus 2 hours, and
2. greater than the median timestamp of previous 11 blocks.

The first rule supplies a 2 hour upper bound for timestamp accuracy.

However, the second rule doesn't give a tight lower bound. Actually, no
lower bound is given at all if no assumption is made about the median. If
we assume the median to be accurate enough at some timepoint, then we're
only assured that any future timestamp is no bigger than this specific
median, which is not much information.

Further analysis can be made under different assumptions. For example,
what's the accuracy if holders of 51% of the computational power create
honest timestamps? But unfortunately, I don't see any good reason to work
under such an assumptions.

The second rule cannot be strengthened to be similar to the first one
(i.e., nodes don't accept blocks less than network time minus 2 hours). The
reason is that nodes cannot differentiate if it's a new block with
dishonest timestamp, an old block with an old timestamps (with many other
blocks coming) or simply a new block that took a long time to mine.

Indirect control of timestamps accuracy:
--------------------------------------------------
If we assume that miners have no motive to increase difficulty
artificially, then the PoW adjusting algorithm yields a second mechanism of
accuracy control.

The adjustment rules are given in pow.cpp (bitcoin-core source, version
0.15.1), in the function 'CalculateNextWorkRequired', by the formula (with
some additional adjustments which I omit):

(old_target* (time_of_last_block_in_2016_blocks_interval -
time_of_first_block_in_2016_blocks_interval) )/time_of_two_weeks

It uses a simple average of block time in the last 2016 blocks. But such
averages ignore any values besides the first and last one in the interval.
Hence, if the difficulty is constant, the following sequence is valid from
both the protocol and the miners incentives point of views:

1, 2, 3,
., 2015, 1209600 (time of two weeks), 2017, 2018, 2019,
.,
4031, 1209600*2, 4033, 4044, 


If we want to be pedantic, the best lower bound for a block timestamp is
the timestamp of the block that closes the adjustment interval in which it
resides.

Possible improvement:
-----------------------------
We may consider exchanging average with standard deviation in the
difficulty adjustment formula. It both better mirrors changes in the hash
power along the interval, and disables the option to manipulate timestamps
without affecting the difficulty.

I'm aware that this change requires a hardfork, and won't happen any time
soon. But does it make sense to add it to a potential future hard fork?
Tier Nolan via bitcoin-dev
2018-01-29 21:40:44 UTC
Reply
Permalink
Raw Message
On Mon, Jan 29, 2018 at 1:34 PM, Neiman via bitcoin-dev <
Post by Neiman via bitcoin-dev
*2.* Timestamps are not necessary to avoid double-spending. A simple
ordering of blocks is sufficient, so exchanging timestamps with enumeration
would work double-spending wise. Permissioned consensus protocols, such as
hyperledger, indeed have no timestamps (in version 1.0).
The timestamps simply needs to be reasonably accurate. Their main purpose
is to allow difficulty updates.

They can also be used to check that the node has caught up.
Post by Neiman via bitcoin-dev
It uses a simple average of block time in the last 2016 blocks. But such
averages ignore any values besides the first and last one in the interval.
Hence, if the difficulty is constant, the following sequence is valid from
1, 2, 3,
., 2015, 1209600 (time of two weeks), 2017, 2018, 2019,
.,
4031, 1209600*2, 4033, 4044, 

Much of Bitcoin operates on the assumption that a majority of miners are
honest. If 50%+ of miners set their timestamp reasonably accurately (say
within 10 mins), then the actual timestamp will move forward at the same
rate as real time.

Dishonest miners could set their timestamp as low as possible, but the
median would move foward if more than half of the timestamps move forward.
Post by Neiman via bitcoin-dev
If we want to be pedantic, the best lower bound for a block timestamp is
the timestamp of the block that closes the adjustment interval in which it
resides.
If you are assuming that the miners are majority dishonest, then they can
set the limit to anything as long as they don't move it more than 2 hours
into the future.

The miners could set their timestamps so that they increase 1 week fake
time every 2 weeks real time and reject any blocks more than 2 hours ahead
of their fake time. The difficulty would settle so that one block occurs
every 20 mins.
Post by Neiman via bitcoin-dev
-----------------------------
We may consider exchanging average with standard deviation in the
difficulty adjustment formula. It both better mirrors changes in the hash
power along the interval, and disables the option to manipulate timestamps
without affecting the difficulty.
I'm aware that this change requires a hardfork, and won't happen any time
soon. But does it make sense to add it to a potential future hard fork?
For check locktime, the median of the last 11 blocks is used as an improved
indicator of what the actual real time is. Again, it assumes that a
majority of the miners are honest.
Post by Neiman via bitcoin-dev
_______________________________________________
bitcoin-dev mailing list
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
Gregory Maxwell via bitcoin-dev
2018-01-29 21:54:23 UTC
Reply
Permalink
Raw Message
On Mon, Jan 29, 2018 at 9:40 PM, Tier Nolan via bitcoin-dev
Post by Tier Nolan via bitcoin-dev
For check locktime, the median of the last 11 blocks is used as an improved
indicator of what the actual real time is. Again, it assumes that a
majority of the miners are honest.
It would be more accurate to say that the median is not used for
improved accuracy but to mitigate a consensus incompatibility:

If the block's own timestamp were used for nlocktime and time based
nlocks were common on the network each miner would maximize their fee
income by setting the value as high as they could get away with. What
concerned us wasn't so much that this would make the times less
accurate (though it would) but rather that it would create an
incentive for a runaway situation that could harm network stability
(e.g. with all miners cranking times against the 2hr window, then
creating pressure for miners to accept further and further in the
future; each responding to his own local incentives).

This incentive incompatibility could have been addressed e.g. by using
the prior block's time, but since the protocol doesn't require times
to be monotone (and for good reason!) the simple implementation of
that wouldn't have been a soft-fork. The 11 block MTP worked out
nicely because the protocol already required new times to be larger
than that.

The timestamps in Bitcoin aren't intended to be particularly accurate.
They're used only for controlling the difficulty, and the adjustment
window is large enough that there isn't much distortion that can be
accomplished there. It's not clear to me that much better can really
be done... if there were tighter time requirements in the protocol
miners would address them by running NTP which as an _astounding_ lack
of security in terms of how it is commonly deployed. As far as I
know, I'm the only person whos ever mined blocks with their own
stratum 1 time source.

If times need to be accurate Bitcoin would need to use a rather
different design (e.g. each block would commit to the observation time
of the prior N blocks, and an iterative algorithm would solve for each
blocks time and each miners local offset).

IIRC open-timestamp calendar servers provide more precise
time-stamping under the assumption that the calendar server is
behaving correctly.
Peter Todd via bitcoin-dev
2018-01-30 07:27:16 UTC
Reply
Permalink
Raw Message
Post by Gregory Maxwell via bitcoin-dev
If times need to be accurate Bitcoin would need to use a rather
different design (e.g. each block would commit to the observation time
of the prior N blocks, and an iterative algorithm would solve for each
blocks time and each miners local offset).
IIRC open-timestamp calendar servers provide more precise
time-stamping under the assumption that the calendar server is
behaving correctly.
That is incorrect. The OpenTimestamps servers are specifically designed not to
be trusted, and thus do not make any cryptographically verifiable attestations
as to when timestamps were created.

In the future I expect to add a trusted timestamping scheme via disposable keys
to the OpenTimestamps protocol, but that work isn't yet complete:

https://lists.opentimestamps.org/pipermail/ots-dev/2017-May/000001.html
--
https://petertodd.org 'peter'[:-1]@petertodd.org
Neiman via bitcoin-dev
2018-01-30 10:53:41 UTC
Reply
Permalink
Raw Message
On Mon, Jan 29, 2018 at 10:54 PM, Gregory Maxwell via bitcoin-dev <
Post by Gregory Maxwell via bitcoin-dev
On Mon, Jan 29, 2018 at 9:40 PM, Tier Nolan via bitcoin-dev
if there were tighter time requirements in the protocol
miners would address them by running NTP which as an _astounding_ lack
of security in terms of how it is commonly deployed.
Could you say a few more words about this lack of security? Or share a link
if you have one. I know very little about NTPs.
Post by Gregory Maxwell via bitcoin-dev
_______________________________________________
bitcoin-dev mailing list
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
Neiman via bitcoin-dev
2018-01-30 10:52:21 UTC
Reply
Permalink
Raw Message
On Mon, Jan 29, 2018 at 10:40 PM, Tier Nolan via bitcoin-dev <
Post by Tier Nolan via bitcoin-dev
Much of Bitcoin operates on the assumption that a majority of miners are
honest. If 50%+ of miners set their timestamp reasonably accurately (say
within 10 mins), then the actual timestamp will move forward at the same
rate as real time.
Thank you for replying. I agree that under the 50%+ assumption, timestamps
are reasonably accurately, but I fail to see a reason to make this
assumption.

I'm comfortable with the 50%+ assumption regarding ledger manipulation
(double-spending, deletion of transactions etc.). I'm much less comfortable
with it regarding timestamps manipulation.

Consider the following situation:
(1) miners are selfish,
(2) miners have a financial incentive to be dishonest.

(1) is a common state on how miners function nowadays. (2) is the case that
interests us when coming to do this analysis.

In the case of ledger manipulation, the 50%+ assumption is not because we
assume that miners are good-hearted (this violates (1)). It is there due to
an assumption that the financial damage to a miner would be bigger than the
gain in (2). This happens since a ledge manipulation may cause miners to
lose block rewards, and certainly will devaluate Bitcoin, an asset which
they possess.

In the case of timestamps manipulation, I don't see any financial damage
caused to miners. Timestamps manipulation (besides the 2016*n blocks) won't
harm the function of Bitcoin, and may even go undetected (it seems to me
that the main blockchain explorers don't track it). I don't see a
justification for the 50%+ assumption here.
Post by Tier Nolan via bitcoin-dev
Dishonest miners could set their timestamp as low as possible, but the
median would move foward if more than half of the timestamps move forward.
Post by Neiman via bitcoin-dev
If we want to be pedantic, the best lower bound for a block timestamp is
the timestamp of the block that closes the adjustment interval in which it
resides.
If you are assuming that the miners are majority dishonest, then they can
set the limit to anything as long as they don't move it more than 2 hours
into the future.
The miners could set their timestamps so that they increase 1 week fake
time every 2 weeks real time and reject any blocks more than 2 hours ahead
of their fake time. The difficulty would settle so that one block occurs
every 20 mins.
Post by Neiman via bitcoin-dev
-----------------------------
We may consider exchanging average with standard deviation in the
difficulty adjustment formula. It both better mirrors changes in the hash
power along the interval, and disables the option to manipulate timestamps
without affecting the difficulty.
I'm aware that this change requires a hardfork, and won't happen any time
soon. But does it make sense to add it to a potential future hard fork?
For check locktime, the median of the last 11 blocks is used as an
improved indicator of what the actual real time is. Again, it assumes that
a majority of the miners are honest.
Post by Neiman via bitcoin-dev
_______________________________________________
bitcoin-dev mailing list
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
_______________________________________________
bitcoin-dev mailing list
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
Bryan Bishop via bitcoin-dev
2018-01-29 22:23:46 UTC
Reply
Permalink
Raw Message
On Mon, Jan 29, 2018 at 7:34 AM, Neiman via bitcoin-dev <
Post by Neiman via bitcoin-dev
But how accurate are Bitcoins timestamps?
A perspective on block timestamp and opentimestamps can be found here:
https://lists.w3.org/Archives/Public/public-blockchain/2016Sep/0076.html

- Bryan
http://heybryan.org/
1 512 203 0507
George Balch via bitcoin-dev
2018-01-29 21:53:06 UTC
Reply
Permalink
Raw Message
The terms "simple ordering of blocks" and timestamp are essentially the
same thing.

On Jan 29, 2018 1:16 PM, "Neiman via bitcoin-dev" <
Post by Neiman via bitcoin-dev
First time posting here, please be gentle.
I'm doing a research project about blockchain timestamping. There are many
such projects, including the fantastic OpenTimestamps.
All of the projects essentially save some data in a block, and rely on the
block timestamp as a proof that this data existed at some specific time.
But how accurate are Bitcoins timestamps?
I didn't find any discussion or research regarding Bitcoin timestamp
accuracy (also not in the history of this mailing list). I share here a
simple analysis of timestamp accuracy, and a suggestion how to improve it.
-------------------------------------------
*1.* It seems to me that the timestamp is not the time that the block was
created. Ideally, it's the time that the miner started to try to mine the
block. However, as timestamps may also be used as a source of variety for
hashes, the exact meaning of its value is unclear.
If this is true, then there's a strange phenomena to observe in
blockchain.info and blockexplorer.com: the timestamps of blocks equals
the receiving times.
Am I wrong in my understanding, or is there a mistake in those websites?
*2.* Timestamps are not necessary to avoid double-spending. A simple
ordering of blocks is sufficient, so exchanging timestamps with enumeration
would work double-spending wise. Permissioned consensus protocols, such as
hyperledger, indeed have no timestamps (in version 1.0).
As far as I could tell, timestamps are included in Bitcoin's protocol
*only* to adjust the difficulty of PoW.
-----------------------------------------------
The only element in the protocol that I found to control timestamp
accuracy is based on the network time concept.
The Bitcoin protocol defines “network time” for each node. The network
time is the median time of the other clients, but only if
1. there are at least 5 connected, and
2. the difference between the median time and the nodes own system
time is less than 70 minutes.
Then new blocks are accepted by the peers if their timestamps is
1. less than the network time plus 2 hours, and
2. greater than the median timestamp of previous 11 blocks.
The first rule supplies a 2 hour upper bound for timestamp accuracy.
However, the second rule doesn't give a tight lower bound. Actually, no
lower bound is given at all if no assumption is made about the median. If
we assume the median to be accurate enough at some timepoint, then we're
only assured that any future timestamp is no bigger than this specific
median, which is not much information.
Further analysis can be made under different assumptions. For example,
what's the accuracy if holders of 51% of the computational power create
honest timestamps? But unfortunately, I don't see any good reason to work
under such an assumptions.
The second rule cannot be strengthened to be similar to the first one
(i.e., nodes don't accept blocks less than network time minus 2 hours). The
reason is that nodes cannot differentiate if it's a new block with
dishonest timestamp, an old block with an old timestamps (with many other
blocks coming) or simply a new block that took a long time to mine.
--------------------------------------------------
If we assume that miners have no motive to increase difficulty
artificially, then the PoW adjusting algorithm yields a second mechanism of
accuracy control.
The adjustment rules are given in pow.cpp (bitcoin-core source, version
0.15.1), in the function 'CalculateNextWorkRequired', by the formula (with
(old_target* (time_of_last_block_in_2016_blocks_interval -
time_of_first_block_in_2016_blocks_interval) )/time_of_two_weeks
It uses a simple average of block time in the last 2016 blocks. But such
averages ignore any values besides the first and last one in the interval.
Hence, if the difficulty is constant, the following sequence is valid from
1, 2, 3,
., 2015, 1209600 (time of two weeks), 2017, 2018, 2019,
.,
4031, 1209600*2, 4033, 4044, 

If we want to be pedantic, the best lower bound for a block timestamp is
the timestamp of the block that closes the adjustment interval in which it
resides.
-----------------------------
We may consider exchanging average with standard deviation in the
difficulty adjustment formula. It both better mirrors changes in the hash
power along the interval, and disables the option to manipulate timestamps
without affecting the difficulty.
I'm aware that this change requires a hardfork, and won't happen any time
soon. But does it make sense to add it to a potential future hard fork?
_______________________________________________
bitcoin-dev mailing list
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
Loading...