|author||Or Gerlitz <email@example.com>||2009-04-08 13:52:01 -0700|
|committer||Roland Dreier <firstname.lastname@example.org>||2009-04-08 13:52:01 -0700|
IPoIB: Document newish features
Update the documentation to include connected mode, stateless offloads and interrupt moderation, and add a reference to the connected mode RFC. Signed-off-by: Or Gerlitz <email@example.com> Signed-off-by: Roland Dreier <firstname.lastname@example.org>
Diffstat (limited to 'Documentation/infiniband/ipoib.txt')
1 files changed, 45 insertions, 0 deletions
diff --git a/Documentation/infiniband/ipoib.txt b/Documentation/infiniband/ipoib.txt
index 864ff328378..6d40f00b358 100644
@@ -24,6 +24,49 @@ Partitions and P_Keys
The P_Key for any interface is given by the "pkey" file, and the
main interface for a subinterface is in "parent."
+Datagram vs Connected modes
+ The IPoIB driver supports two modes of operation: datagram and
+ connected. The mode is set and read through an interface's
+ /sys/class/net/<intf name>/mode file.
+ In datagram mode, the IB UD (Unreliable Datagram) transport is used
+ and so the interface MTU has is equal to the IB L2 MTU minus the
+ IPoIB encapsulation header (4 bytes). For example, in a typical IB
+ fabric with a 2K MTU, the IPoIB MTU will be 2048 - 4 = 2044 bytes.
+ In connected mode, the IB RC (Reliable Connected) transport is used.
+ Connected mode is to takes advantage of the connected nature of the
+ IB transport and allows an MTU up to the maximal IP packet size of
+ 64K, which reduces the number of IP packets needed for handling
+ large UDP datagrams, TCP segments, etc and increases the performance
+ for large messages.
+ In connected mode, the interface's UD QP is still used for multicast
+ and communication with peers that don't support connected mode. In
+ this case, RX emulation of ICMP PMTU packets is used to cause the
+ networking stack to use the smaller UD MTU for these neighbours.
+ If the IB HW supports IPoIB stateless offloads, IPoIB advertises
+ TCP/IP checksum and/or Large Send (LSO) offloading capability to the
+ network stack.
+ Large Receive (LRO) offloading is also implemented and may be turned
+ on/off using ethtool calls. Currently LRO is supported only for
+ checksum offload capable devices.
+ Stateless offloads are supported only in datagram mode.
+ If the underlying IB device supports CQ event moderation, one can
+ use ethtool to set interrupt mitigation parameters and thus reduce
+ the overhead incurred by handling interrupts. The main code path of
+ IPoIB doesn't use events for TX completion signaling so only RX
+ moderation is supported.
By compiling the IPoIB driver with CONFIG_INFINIBAND_IPOIB_DEBUG set
@@ -55,3 +98,5 @@ References
IP over InfiniBand (IPoIB) Architecture (RFC 4392)
+ IP over InfiniBand: Connected Mode (RFC 4755)