Discussion:
Would anyone object to adding a dlopen message hook system?
(too old to reply)
Erik Aronesty via bitcoin-dev
2017-08-13 18:46:37 UTC
Permalink
Raw Message
I was thinking about something like this that could add the ability for
module extensions in the core client.

When messages are received, modules hooks are called with the message data.


They can then handle, mark the peer invalid, push a message to the peer or
pass through an alternate command. Also, modules could have their own
private commands prefixed by "x:" or something like that.

The idea is that the base P2P layer is left undisturbed, but there is now a
way to create "enhanced features" that some peers support.

My end goal is to support using lightning network micropayments to allow
people to pay for better node access - creating a market for node services.


But I don't think this should be "baked in" to core. Nor do I think it
should be a "patch". It should be a linked-in module, optionally compiled
and added to bitcoin conf, then loaded via dlopen(). Modules should be
slightly robust to Bitcoin versions changing out from under them, but not
if the network layer is changed. This can be ensured by a) keeping a
module version number, and b) treating module responses as if they were
just received from the network. Any module incompatibility should throw
an exception...ensuring broken peers don't stay online.

In general I think the core reference would benefit from the ability to
create subnetworks within the Bitcoin ecosystem. Right now, we have two
choices... full node and get slammed with traffic, or listen-only node, and
do nothing.

Adding a module/hook system would allow a complex ecosystem of
participation - and it would seem to be far more robust in the long term.

Something like this???

class MessageHookIn {
public:
int hookversion;
int64_t nodeid;
int nVersion;
int64_t serviceflags;
const char *strCommand;
const char *nodeaddr;
const char *vRecv;
int vRecvLen;
int64_t nTimeReceived;
};

class MessageHookOut {
public:
int hookversion;
int misbehaving;
const char *logMsg;
const char *pushCommand;
const unsigned char *pushData;
int pushDataLen;
const char *passCommand;
CDataStream passStream;
};

class MessageHook {
public:
int hookversion;
std::string name;
typedef bool (*HandlerType)(const MessageHookIn *in, MessageHookOut
*out);
HandlerType handle;
};
Jonas Schnelli via bitcoin-dev
2017-08-13 20:00:41 UTC
Permalink
Raw Message
Hi Erik

Thanks for your proposal.
In general, modularisation is a good thing, though proposing core to add modules wie dlopen() seems the wrong direction.
Core already has the problem of running to many things in the same process. The consensus logic, p2p system as well as the wallet AND the GUI do all share the same process (!).

A module approach like you describe would be a security nightmare (and Core is currently in the process of separating out the wallet and the GUI into its own process).

What does speak against using the existing IPC interfaces like RPC/ZMQ?
RPC can be bidirectional using long poll.

/jonas
I was thinking about something like this that could add the ability for module extensions in the core client.
When messages are received, modules hooks are called with the message data.
They can then handle, mark the peer invalid, push a message to the peer or pass through an alternate command. Also, modules could have their own private commands prefixed by "x:" or something like that.
The idea is that the base P2P layer is left undisturbed, but there is now a way to create "enhanced features" that some peers support.
My end goal is to support using lightning network micropayments to allow people to pay for better node access - creating a market for node services.
But I don't think this should be "baked in" to core. Nor do I think it should be a "patch". It should be a linked-in module, optionally compiled and added to bitcoin conf, then loaded via dlopen(). Modules should be slightly robust to Bitcoin versions changing out from under them, but not if the network layer is changed. This can be ensured by a) keeping a module version number, and b) treating module responses as if they were just received from the network. Any module incompatibility should throw an exception...ensuring broken peers don't stay online.
Mark Friedenbach via bitcoin-dev
2017-08-13 20:56:39 UTC
Permalink
Raw Message
Jonas, I think his proposal is to enable extending the P2P layer, e.g.
adding new message types. Are you suggesting having externalized
message processing? That could be done via RPC/ZMQ while opening up a
much more narrow attack surface than dlopen, although I imagine such
an interface would require a very complex API specification.

On Sun, Aug 13, 2017 at 1:00 PM, Jonas Schnelli via bitcoin-dev
Post by Jonas Schnelli via bitcoin-dev
Hi Erik
Thanks for your proposal.
In general, modularisation is a good thing, though proposing core to add modules wie dlopen() seems the wrong direction.
Core already has the problem of running to many things in the same process. The consensus logic, p2p system as well as the wallet AND the GUI do all share the same process (!).
A module approach like you describe would be a security nightmare (and Core is currently in the process of separating out the wallet and the GUI into its own process).
What does speak against using the existing IPC interfaces like RPC/ZMQ?
RPC can be bidirectional using long poll.
/jonas
I was thinking about something like this that could add the ability for module extensions in the core client.
When messages are received, modules hooks are called with the message data.
They can then handle, mark the peer invalid, push a message to the peer or pass through an alternate command. Also, modules could have their own private commands prefixed by "x:" or something like that.
The idea is that the base P2P layer is left undisturbed, but there is now a way to create "enhanced features" that some peers support.
My end goal is to support using lightning network micropayments to allow people to pay for better node access - creating a market for node services.
But I don't think this should be "baked in" to core. Nor do I think it should be a "patch". It should be a linked-in module, optionally compiled and added to bitcoin conf, then loaded via dlopen(). Modules should be slightly robust to Bitcoin versions changing out from under them, but not if the network layer is changed. This can be ensured by a) keeping a module version number, and b) treating module responses as if they were just received from the network. Any module incompatibility should throw an exception...ensuring broken peers don't stay online.
_______________________________________________
bitcoin-dev mailing list
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
Erik Aronesty via bitcoin-dev
2017-08-15 01:33:34 UTC
Permalink
Raw Message
Actually the more I think about it, the more I realize that all I need is
to listen on a new port, and use the RPC api to affect Bitcoin:

- ban a peer (# of hours)
- unban a peer (# of hours)

As long as I have those two functions, I can do everything I need.
Post by Mark Friedenbach via bitcoin-dev
Jonas, I think his proposal is to enable extending the P2P layer, e.g.
adding new message types. Are you suggesting having externalized
message processing? That could be done via RPC/ZMQ while opening up a
much more narrow attack surface than dlopen, although I imagine such
an interface would require a very complex API specification.
On Sun, Aug 13, 2017 at 1:00 PM, Jonas Schnelli via bitcoin-dev
Post by Jonas Schnelli via bitcoin-dev
Hi Erik
Thanks for your proposal.
In general, modularisation is a good thing, though proposing core to add
modules wie dlopen() seems the wrong direction.
Post by Jonas Schnelli via bitcoin-dev
Core already has the problem of running to many things in the same
process. The consensus logic, p2p system as well as the wallet AND the GUI
do all share the same process (!).
Post by Jonas Schnelli via bitcoin-dev
A module approach like you describe would be a security nightmare (and
Core is currently in the process of separating out the wallet and the GUI
into its own process).
Post by Jonas Schnelli via bitcoin-dev
What does speak against using the existing IPC interfaces like RPC/ZMQ?
RPC can be bidirectional using long poll.
/jonas
Post by Erik Aronesty via bitcoin-dev
I was thinking about something like this that could add the ability for
module extensions in the core client.
Post by Jonas Schnelli via bitcoin-dev
Post by Erik Aronesty via bitcoin-dev
When messages are received, modules hooks are called with the message
data.
Post by Jonas Schnelli via bitcoin-dev
Post by Erik Aronesty via bitcoin-dev
They can then handle, mark the peer invalid, push a message to the peer
or pass through an alternate command. Also, modules could have their own
private commands prefixed by "x:" or something like that.
Post by Jonas Schnelli via bitcoin-dev
Post by Erik Aronesty via bitcoin-dev
The idea is that the base P2P layer is left undisturbed, but there is
now a way to create "enhanced features" that some peers support.
Post by Jonas Schnelli via bitcoin-dev
Post by Erik Aronesty via bitcoin-dev
My end goal is to support using lightning network micropayments to
allow people to pay for better node access - creating a market for node
services.
Post by Jonas Schnelli via bitcoin-dev
Post by Erik Aronesty via bitcoin-dev
But I don't think this should be "baked in" to core. Nor do I think
it should be a "patch". It should be a linked-in module, optionally
compiled and added to bitcoin conf, then loaded via dlopen(). Modules
should be slightly robust to Bitcoin versions changing out from under them,
but not if the network layer is changed. This can be ensured by a)
keeping a module version number, and b) treating module responses as if
they were just received from the network. Any module incompatibility
should throw an exception...ensuring broken peers don't stay online.
Post by Jonas Schnelli via bitcoin-dev
_______________________________________________
bitcoin-dev mailing list
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
Erik Aronesty via bitcoin-dev
2017-08-15 04:44:52 UTC
Permalink
Raw Message
The idea is that some peers, when you connect to them will work fine for
some time, but you need to find out the rate for services and send a
micropayment to maintain the connection. This creates an optional pay
layer for high quality services, and also creates DDOS resistance in this
fallback layer.

Loading...