From 6a3335b43342b42dd6c69b4bbbde15d622cb49ca Mon Sep 17 00:00:00 2001 From: Or Gerlitz Date: Wed, 8 Apr 2009 13:52:01 -0700 Subject: IPoIB: Document newish features Update the documentation to include connected mode, stateless offloads and interrupt moderation, and add a reference to the connected mode RFC. Signed-off-by: Or Gerlitz Signed-off-by: Roland Dreier --- Documentation/infiniband/ipoib.txt | 45 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 45 insertions(+) (limited to 'Documentation/infiniband') diff --git a/Documentation/infiniband/ipoib.txt b/Documentation/infiniband/ipoib.txt index 864ff328378..6d40f00b358 100644 --- a/Documentation/infiniband/ipoib.txt +++ b/Documentation/infiniband/ipoib.txt @@ -24,6 +24,49 @@ Partitions and P_Keys The P_Key for any interface is given by the "pkey" file, and the main interface for a subinterface is in "parent." +Datagram vs Connected modes + + The IPoIB driver supports two modes of operation: datagram and + connected. The mode is set and read through an interface's + /sys/class/net//mode file. + + In datagram mode, the IB UD (Unreliable Datagram) transport is used + and so the interface MTU has is equal to the IB L2 MTU minus the + IPoIB encapsulation header (4 bytes). For example, in a typical IB + fabric with a 2K MTU, the IPoIB MTU will be 2048 - 4 = 2044 bytes. + + In connected mode, the IB RC (Reliable Connected) transport is used. + Connected mode is to takes advantage of the connected nature of the + IB transport and allows an MTU up to the maximal IP packet size of + 64K, which reduces the number of IP packets needed for handling + large UDP datagrams, TCP segments, etc and increases the performance + for large messages. + + In connected mode, the interface's UD QP is still used for multicast + and communication with peers that don't support connected mode. In + this case, RX emulation of ICMP PMTU packets is used to cause the + networking stack to use the smaller UD MTU for these neighbours. + +Stateless offloads + + If the IB HW supports IPoIB stateless offloads, IPoIB advertises + TCP/IP checksum and/or Large Send (LSO) offloading capability to the + network stack. + + Large Receive (LRO) offloading is also implemented and may be turned + on/off using ethtool calls. Currently LRO is supported only for + checksum offload capable devices. + + Stateless offloads are supported only in datagram mode. + +Interrupt moderation + + If the underlying IB device supports CQ event moderation, one can + use ethtool to set interrupt mitigation parameters and thus reduce + the overhead incurred by handling interrupts. The main code path of + IPoIB doesn't use events for TX completion signaling so only RX + moderation is supported. + Debugging Information By compiling the IPoIB driver with CONFIG_INFINIBAND_IPOIB_DEBUG set @@ -55,3 +98,5 @@ References http://ietf.org/rfc/rfc4391.txt IP over InfiniBand (IPoIB) Architecture (RFC 4392) http://ietf.org/rfc/rfc4392.txt + IP over InfiniBand: Connected Mode (RFC 4755) + http://ietf.org/rfc/rfc4755.txt -- cgit v1.2.3