smbdirect.rst

  1.. SPDX-License-Identifier: GPL-2.0
  2
  3===========================
  4SMB Direct - SMB3 over RDMA
  5===========================
  6
  7This document describes how to set up the Linux SMB client and server to
  8use RDMA.
  9
 10Overview
 11========
 12The Linux SMB kernel client supports SMB Direct, which is a transport
 13scheme for SMB3 that uses RDMA (Remote Direct Memory Access) to provide
 14high throughput and low latencies by bypassing the traditional TCP/IP
 15stack.
 16SMB Direct on the Linux SMB client can be tested against KSMBD - a
 17kernel-space SMB server.
 18
 19Installation
 20=============
 21- Install an RDMA device. As long as the RDMA device driver is supported
 22  by the kernel, it should work. This includes both software emulators (soft
 23  RoCE, soft iWARP) and hardware devices (InfiniBand, RoCE, iWARP).
 24
 25- Install a kernel with SMB Direct support. The first kernel release to
 26  support SMB Direct on both the client and server side is 5.15. Therefore,
 27  a distribution compatible with kernel 5.15 or later is required.
 28
 29- Install cifs-utils, which provides the `mount.cifs` command to mount SMB
 30  shares.
 31
 32- Configure the RDMA stack
 33
 34  Make sure that your kernel configuration has RDMA support enabled. Under
 35  Device Drivers -> Infiniband support, update the kernel configuration to
 36  enable Infiniband support.
 37
 38  Enable the appropriate IB HCA support or iWARP adapter support,
 39  depending on your hardware.
 40
 41  If you are using InfiniBand, enable IP-over-InfiniBand support.
 42
 43  For soft RDMA, enable either the soft iWARP (`RDMA _SIW`) or soft RoCE
 44  (`RDMA_RXE`) module. Install the `iproute2` package and use the
 45  `rdma link add` command to load the module and create an
 46  RDMA interface.
 47
 48  e.g. if your local ethernet interface is `eth0`, you can use:
 49
 50    .. code-block:: bash
 51
 52        sudo rdma link add siw0 type siw netdev eth0
 53
 54- Enable SMB Direct support for both the server and the client in the kernel
 55  configuration.
 56
 57    Server Setup
 58
 59    .. code-block:: text
 60
 61        Network File Systems  --->
 62            <M> SMB3 server support
 63                [*] Support for SMB Direct protocol
 64
 65    Client Setup
 66
 67    .. code-block:: text
 68
 69        Network File Systems  --->
 70            <M> SMB3 and CIFS support (advanced network filesystem)
 71                [*] SMB Direct support
 72
 73- Build and install the kernel. SMB Direct support will be enabled in the
 74  cifs.ko and ksmbd.ko modules.
 75
 76Setup and Usage
 77================
 78
 79- Set up and start a KSMBD server as described in the `KSMBD documentation
 80  <https://www.kernel.org/doc/Documentation/filesystems/smb/ksmbd.rst>`_.
 81  Also add the "server multi channel support = yes" parameter to ksmbd.conf.
 82
 83- On the client, mount the share with `rdma` mount option to use SMB Direct
 84  (specify a SMB version 3.0 or higher using `vers`).
 85
 86  For example:
 87
 88    .. code-block:: bash
 89
 90        mount -t cifs //server/share /mnt/point -o vers=3.1.1,rdma
 91
 92- To verify that the mount is using SMB Direct, you can check dmesg for the
 93  following log line after mounting:
 94
 95    .. code-block:: text
 96
 97        CIFS: VFS: RDMA transport established
 98
 99  Or, verify `rdma` mount option for the share in `/proc/mounts`:
100
101    .. code-block:: bash
102
103        cat /proc/mounts | grep cifs