checksum-offloads.rst

  1.. SPDX-License-Identifier: GPL-2.0
  2
  3=================
  4Checksum Offloads
  5=================
  6
  7
  8Introduction
  9============
 10
 11This document describes a set of techniques in the Linux networking stack to
 12take advantage of checksum offload capabilities of various NICs.
 13
 14The following technologies are described:
 15
 16* TX Checksum Offload
 17* LCO: Local Checksum Offload
 18* RCO: Remote Checksum Offload
 19
 20Things that should be documented here but aren't yet:
 21
 22* RX Checksum Offload
 23* CHECKSUM_UNNECESSARY conversion
 24
 25
 26TX Checksum Offload
 27===================
 28
 29The interface for offloading a transmit checksum to a device is explained in
 30detail in comments near the top of include/linux/skbuff.h.
 31
 32In brief, it allows to request the device fill in a single ones-complement
 33checksum defined by the sk_buff fields skb->csum_start and skb->csum_offset.
 34The device should compute the 16-bit ones-complement checksum (i.e. the
 35'IP-style' checksum) from csum_start to the end of the packet, and fill in the
 36result at (csum_start + csum_offset).
 37
 38Because csum_offset cannot be negative, this ensures that the previous value of
 39the checksum field is included in the checksum computation, thus it can be used
 40to supply any needed corrections to the checksum (such as the sum of the
 41pseudo-header for UDP or TCP).
 42
 43This interface only allows a single checksum to be offloaded.  Where
 44encapsulation is used, the packet may have multiple checksum fields in
 45different header layers, and the rest will have to be handled by another
 46mechanism such as LCO or RCO.
 47
 48CRC32c can also be offloaded using this interface, by means of filling
 49skb->csum_start and skb->csum_offset as described above, and setting
 50skb->csum_not_inet: see skbuff.h comment (section 'D') for more details.
 51
 52No offloading of the IP header checksum is performed; it is always done in
 53software.  This is OK because when we build the IP header, we obviously have it
 54in cache, so summing it isn't expensive.  It's also rather short.
 55
 56The requirements for GSO are more complicated, because when segmenting an
 57encapsulated packet both the inner and outer checksums may need to be edited or
 58recomputed for each resulting segment.  See the skbuff.h comment (section 'E')
 59for more details.
 60
 61A driver declares its offload capabilities in netdev->hw_features; see
 62Documentation/networking/netdev-features.rst for more.  Note that a device
 63which only advertises NETIF_F_IP[V6]_CSUM must still obey the csum_start and
 64csum_offset given in the SKB; if it tries to deduce these itself in hardware
 65(as some NICs do) the driver should check that the values in the SKB match
 66those which the hardware will deduce, and if not, fall back to checksumming in
 67software instead (with skb_csum_hwoffload_help() or one of the
 68skb_checksum_help() / skb_crc32c_csum_help functions, as mentioned in
 69include/linux/skbuff.h).
 70
 71The stack should, for the most part, assume that checksum offload is supported
 72by the underlying device.  The only place that should check is
 73validate_xmit_skb(), and the functions it calls directly or indirectly.  That
 74function compares the offload features requested by the SKB (which may include
 75other offloads besides TX Checksum Offload) and, if they are not supported or
 76enabled on the device (determined by netdev->features), performs the
 77corresponding offload in software.  In the case of TX Checksum Offload, that
 78means calling skb_csum_hwoffload_help(skb, features).
 79
 80
 81LCO: Local Checksum Offload
 82===========================
 83
 84LCO is a technique for efficiently computing the outer checksum of an
 85encapsulated datagram when the inner checksum is due to be offloaded.
 86
 87The ones-complement sum of a correctly checksummed TCP or UDP packet is equal
 88to the complement of the sum of the pseudo header, because everything else gets
 89'cancelled out' by the checksum field.  This is because the sum was
 90complemented before being written to the checksum field.
 91
 92More generally, this holds in any case where the 'IP-style' ones complement
 93checksum is used, and thus any checksum that TX Checksum Offload supports.
 94
 95That is, if we have set up TX Checksum Offload with a start/offset pair, we
 96know that after the device has filled in that checksum, the ones complement sum
 97from csum_start to the end of the packet will be equal to the complement of
 98whatever value we put in the checksum field beforehand.  This allows us to
 99compute the outer checksum without looking at the payload: we simply stop
100summing when we get to csum_start, then add the complement of the 16-bit word
101at (csum_start + csum_offset).
102
103Then, when the true inner checksum is filled in (either by hardware or by
104skb_checksum_help()), the outer checksum will become correct by virtue of the
105arithmetic.
106
107LCO is performed by the stack when constructing an outer UDP header for an
108encapsulation such as VXLAN or GENEVE, in udp_set_csum().  Similarly for the
109IPv6 equivalents, in udp6_set_csum().
110
111It is also performed when constructing an IPv4 GRE header, in
112net/ipv4/ip_gre.c:build_header().  It is *not* currently performed when
113constructing an IPv6 GRE header; the GRE checksum is computed over the whole
114packet in net/ipv6/ip6_gre.c:ip6gre_xmit2(), but it should be possible to use
115LCO here as IPv6 GRE still uses an IP-style checksum.
116
117All of the LCO implementations use a helper function lco_csum(), in
118include/linux/skbuff.h.
119
120LCO can safely be used for nested encapsulations; in this case, the outer
121encapsulation layer will sum over both its own header and the 'middle' header.
122This does mean that the 'middle' header will get summed multiple times, but
123there doesn't seem to be a way to avoid that without incurring bigger costs
124(e.g. in SKB bloat).
125
126
127RCO: Remote Checksum Offload
128============================
129
130RCO is a technique for eliding the inner checksum of an encapsulated datagram,
131allowing the outer checksum to be offloaded.  It does, however, involve a
132change to the encapsulation protocols, which the receiver must also support.
133For this reason, it is disabled by default.
134
135RCO is detailed in the following Internet-Drafts:
136
137* https://tools.ietf.org/html/draft-herbert-remotecsumoffload-00
138* https://tools.ietf.org/html/draft-herbert-vxlan-rco-00
139
140In Linux, RCO is implemented individually in each encapsulation protocol, and
141most tunnel types have flags controlling its use.  For instance, VXLAN has the
142flag VXLAN_F_REMCSUM_TX (per struct vxlan_rdst) to indicate that RCO should be
143used when transmitting to a given remote destination.