Collecting Gossip at Scale
Draft — placeholder content to verify rendering
To observe gossip propagation, we need a passive collector node that connects to as many Lightning peers as possible and records exactly when each peer delivers each gossip message.
The entire collection infrastructure described in this chapter was designed and built by Jonathan Harvey-Buschel (@jharveyb) as part of the gossip_observer project. The LN Gossip Visualizer builds its visualization layer on top of this foundation.
The Observer Node
Our collector is built on LDK (Lightning Dev Kit) and operates as a silent listener — it never opens channels or routes payments. Its only job is to connect to peers and log incoming gossip.
What Gets Recorded
For every gossip message received, we store:
| Field | Description |
|---|---|
msg_hash | SHA-256 hash identifying the unique message |
peer | Public key of the peer that delivered it |
net_timestamp | Nanosecond-precision arrival time |
collector | Which collector instance received it |
The same message arrives from many peers at different times — that’s the core data that powers our visualizations.