Bläddra i källkod

add tvu port binding info to docs (#4948)

* add tvu port binding info
* add notes about TPU and quic
Co-authored-by: Greg Cusack <greg.cusack@anza.xyz>
Alex Pyattaev 8 månader sedan
förälder
incheckning
14432a4a08
2 ändrade filer med 70 tillägg och 15 borttagningar
  1. 13 13
      docs/art/tvu.bob
  2. 57 2
      docs/src/validator/tvu.md

+ 13 - 13
docs/art/tvu.bob

@@ -1,28 +1,28 @@
                                                                          +------------+
                                                                          |   Gossip   |
                                                                          |   Service  |
-                                 +-------------+                         |            |
-                                 |Child        |    +------------+       +-+-----+----+
-                                 |Validators   |    |Neighborhood|         |     ^
-                                 |             |    |Validators  |         |     |
-                                 +-------+-----+    |            |         |     |
-                                         ^          +---------+--+     Peer|     |Votes
-                                         |                    ^        List|     |
-                                         +-----------------+  |            |     |
-                                       shreds(forward=true)|  |            |     |
+                          +-------------+                                |            |
+                          |Child        |           +------------+       +-+-----+----+
+                          |Validators   |           |Neighborhood|         |     ^
+                          |             |           |Validators  |         |     |
+                          +-------+-----+           |            |         |     |
+                                  ^                 +---------+--+     Peer|     |Votes
+                                  |                           ^        List|     |
+                                  +------------------------+  |            |     |
+                                   shreds(forward = true)  |  |            |     |
                          +---------------------------------------------------------------------------------------------+
                          |                                 |  |            |     |                                     |
                          |  TVU                            |  |   +--------+     |                                     |
                          |                                 |  |   v              |                                     |
 +------------+           |  +-------+  +------------+    +-+--+---+---+      +---+--------------+      +-------------+ |
-|            |    Repair |  |       |  |            |    | Retransmit |      | Replay           |      | Transaction | |
+|            |   Repair  |  |       |  |            |    | Retransmit |      | Replay           |      | Transaction | |
 | Upstream   +------------->+       +->+ Shred      +--->+ Stage      +----->+ Stage            +----->+ Status      | |
-| Validators |     TVU   |  | Shred |  | Verify     |    |            |      | +--------------+ |      | Service     | |
+| Validators |  Turbine  |  | Shred |  | Verify     |    |            |      | +--------------+ |      | Service     | |
 |            +------------->+ Fetch |  | Leader Sig |    +------+-----+      | | PoH Verify   | |      | (optional)  | |
 |            |           |  | Stage |  | Stage      |           ^            | | TX Sig Verify| |      +-------------+ |
 |            +------------->+       |  |            |           |            | |              | |                      |
-|            |   TVU     |  |       |  +--+---------+           |            | +-+------------+ |                      |
-+------------+   Forwards|  +-------+     ^                     |            |   |              |                      |
+|            |  Turbine  |  |       |  +--+---------+           |            | +-+------------+ |                      |
++------------+  Forwards |  +-------+     ^                     |            |   |              |                      |
                          |                |                     |            +------------------+                      |
                          |                |                     |                |                                     |
                          |                |                     |                |                                     |

+ 57 - 2
docs/src/validator/tvu.md

@@ -6,11 +6,66 @@ pagination_label: Validator's Transaction Validation Unit (TVU)
 ---
 
 TVU (Transaction Validation Unit) is the logic of the validator
-responsible for validating and propagating blocks and processing
-those blocks' transactions through the runtime.
+responsible for propagating blocks between validators and ensuring that
+those blocks' transactions reach the replay stage. Its principal external
+interface is the turbine protocol.
 
 ![TVU Block Diagram](/img/tvu.svg)
 
 ## Retransmit Stage
 
 ![Retransmit Block Diagram](/img/retransmit_stage.svg)
+
+## TVU sockets
+
+Externally, TVU UDP receiver appears to bind to one port, typically 8002 UDP.
+Internally, TVU is actually bound with multiple sockets to improve kernel's handling of the packet queues.
+
+> **NOTE:** TPU sockets use similar logic
+
+A node advertises one external ip/port for TVU while binding multiple sockets to that same port using SO_REUSEPORT:
+
+```rust
+let (tvu_port, tvu_sockets) = multi_bind_in_range_with_config(
+    bind_ip_addr,
+    port_range,
+    socket_config_reuseport,
+    num_tvu_sockets.get(),
+)
+.expect("tvu multi_bind");
+```
+
+ `multi_bind_in_range_with_config` sets `SO_REUSEPORT`. This means that other nodes only need to know about the one ip/port pair for TVU (similar principle applies in the case of TPU UDP sockets). The kernel distributes the incoming packets to all sockets bound to that port, and each socket can be serviced by a different thread.
+
+> **NOTE:** TVU QUIC socket does not use `SO_REUSEPORT`, but otherwise works similarly to UDP
+
+The TVU socket information is published via Gossip and is available in the `ContactInfo` struct.
+To set a TVU socket, the node calls `set_tvu(...)`. The `set_tvu()` method is created by the macro `set_socket!`. For example:
+
+```rust
+info.set_tvu(QUIC, (addr, tvu_quic_port)).unwrap();
+info.set_tvu(UDP, (addr, tvu_udp_port)).unwrap();
+```
+
+Under the hood, `set_tvu()` calls `set_socket()`.
+
+In the `ContactInfo` struct, all sockets are identified by a tag/key, e.g.:
+
+```rust
+const SOCKET_TAG_TVU: u8 = 10;       // For UDP
+const SOCKET_TAG_TVU_QUIC: u8 = 11;  // For QUIC
+```
+
+ * `set_socket()` creates a `SocketEntry` and stores that into `ContactInfo::sockets`
+ * `set_socket()` updates `ContactInfo::cache`
+
+```rust
+cache: [SocketAddr; SOCKET_CACHE_SIZE]
+```
+
+`cache` is purely for quick lookups and optimization; it is not serialized and sent to peer nodes.
+`SocketEntry` is serialized and sent to peer nodes within the gossip message type `CrdsData::ContactInfo`. Upon receiving the `ContactInfo`, the peer node calls the `get_socket!` macro to retrieve the TVU port associated with the node.
+For example, to retrieve the TVU ports of the remote node, the peer node calls:
+```rust
+get_socket!(tvu, SOCKET_TAG_TVU, SOCKET_TAG_TVU_QUIC);
+```