aoe.rst

  1Introduction
  2============
  3
  4ATA over Ethernet is a network protocol that provides simple access to
  5block storage on the LAN.
  6
  7  http://support.coraid.com/documents/AoEr11.txt
  8
  9The EtherDrive (R) HOWTO for 2.6 and 3.x kernels is found at ...
 10
 11  http://support.coraid.com/support/linux/EtherDrive-2.6-HOWTO.html
 12
 13It has many tips and hints!  Please see, especially, recommended
 14tunings for virtual memory:
 15
 16  http://support.coraid.com/support/linux/EtherDrive-2.6-HOWTO-5.html#ss5.19
 17
 18The aoetools are userland programs that are designed to work with this
 19driver.  The aoetools are on sourceforge.
 20
 21  http://aoetools.sourceforge.net/
 22
 23The scripts in this Documentation/admin-guide/aoe directory are intended to
 24document the use of the driver and are not necessary if you install
 25the aoetools.
 26
 27
 28Creating Device Nodes
 29=====================
 30
 31  Users of udev should find the block device nodes created
 32  automatically, but to create all the necessary device nodes, use the
 33  udev configuration rules provided in udev.txt (in this directory).
 34
 35  There is a udev-install.sh script that shows how to install these
 36  rules on your system.
 37
 38  There is also an autoload script that shows how to edit
 39  /etc/modprobe.d/aoe.conf to ensure that the aoe module is loaded when
 40  necessary.  Preloading the aoe module is preferable to autoloading,
 41  however, because AoE discovery takes a few seconds.  It can be
 42  confusing when an AoE device is not present the first time the a
 43  command is run but appears a second later.
 44
 45Using Device Nodes
 46==================
 47
 48  "cat /dev/etherd/err" blocks, waiting for error diagnostic output,
 49  like any retransmitted packets.
 50
 51  "echo eth2 eth4 > /dev/etherd/interfaces" tells the aoe driver to
 52  limit ATA over Ethernet traffic to eth2 and eth4.  AoE traffic from
 53  untrusted networks should be ignored as a matter of security.  See
 54  also the aoe_iflist driver option described below.
 55
 56  "echo > /dev/etherd/discover" tells the driver to find out what AoE
 57  devices are available.
 58
 59  In the future these character devices may disappear and be replaced
 60  by sysfs counterparts.  Using the commands in aoetools insulates
 61  users from these implementation details.
 62
 63  The block devices are named like this::
 64
 65	e{shelf}.{slot}
 66	e{shelf}.{slot}p{part}
 67
 68  ... so that "e0.2" is the third blade from the left (slot 2) in the
 69  first shelf (shelf address zero).  That's the whole disk.  The first
 70  partition on that disk would be "e0.2p1".
 71
 72Using sysfs
 73===========
 74
 75  Each aoe block device in /sys/block has the extra attributes of
 76  state, mac, and netif.  The state attribute is "up" when the device
 77  is ready for I/O and "down" if detected but unusable.  The
 78  "down,closewait" state shows that the device is still open and
 79  cannot come up again until it has been closed.
 80
 81  The mac attribute is the ethernet address of the remote AoE device.
 82  The netif attribute is the network interface on the localhost
 83  through which we are communicating with the remote AoE device.
 84
 85  There is a script in this directory that formats this information in
 86  a convenient way.  Users with aoetools should use the aoe-stat
 87  command::
 88
 89    root@makki root# sh Documentation/admin-guide/aoe/status.sh
 90       e10.0            eth3              up
 91       e10.1            eth3              up
 92       e10.2            eth3              up
 93       e10.3            eth3              up
 94       e10.4            eth3              up
 95       e10.5            eth3              up
 96       e10.6            eth3              up
 97       e10.7            eth3              up
 98       e10.8            eth3              up
 99       e10.9            eth3              up
100        e4.0            eth1              up
101        e4.1            eth1              up
102        e4.2            eth1              up
103        e4.3            eth1              up
104        e4.4            eth1              up
105        e4.5            eth1              up
106        e4.6            eth1              up
107        e4.7            eth1              up
108        e4.8            eth1              up
109        e4.9            eth1              up
110
111  Use /sys/module/aoe/parameters/aoe_iflist (or better, the driver
112  option discussed below) instead of /dev/etherd/interfaces to limit
113  AoE traffic to the network interfaces in the given
114  whitespace-separated list.  Unlike the old character device, the
115  sysfs entry can be read from as well as written to.
116
117  It's helpful to trigger discovery after setting the list of allowed
118  interfaces.  The aoetools package provides an aoe-discover script
119  for this purpose.  You can also directly use the
120  /dev/etherd/discover special file described above.
121
122Driver Options
123==============
124
125  There is a boot option for the built-in aoe driver and a
126  corresponding module parameter, aoe_iflist.  Without this option,
127  all network interfaces may be used for ATA over Ethernet.  Here is a
128  usage example for the module parameter::
129
130    modprobe aoe_iflist="eth1 eth3"
131
132  The aoe_deadsecs module parameter determines the maximum number of
133  seconds that the driver will wait for an AoE device to provide a
134  response to an AoE command.  After aoe_deadsecs seconds have
135  elapsed, the AoE device will be marked as "down".  A value of zero
136  is supported for testing purposes and makes the aoe driver keep
137  trying AoE commands forever.
138
139  The aoe_maxout module parameter has a default of 128.  This is the
140  maximum number of unresponded packets that will be sent to an AoE
141  target at one time.
142
143  The aoe_dyndevs module parameter defaults to 1, meaning that the
144  driver will assign a block device minor number to a discovered AoE
145  target based on the order of its discovery.  With dynamic minor
146  device numbers in use, a greater range of AoE shelf and slot
147  addresses can be supported.  Users with udev will never have to
148  think about minor numbers.  Using aoe_dyndevs=0 allows device nodes
149  to be pre-created using a static minor-number scheme with the
150  aoe-mkshelf script in the aoetools.