I have previously blogged about Ethernet bonding on Red Hat Enterprise Linux. Now I have a need to do the same thing on Debian Etch – to have multiple Ethernet links for redundancy so that if one breaks the system keeps working.
The first thing to do on Debian is to install the package ifenslave-2.6 which provides the utility to manage the bond device. Then create the file /etc/modprobe.d/aliases-bond with the following contents for a network that has 10.0.0.1 as either a reliable host or important router. Note that this will use ARP to ping the router every 2000ms, you could use a lower value for a faster failover or a higher value
alias bond0 bonding
options bond0 mode=1 arp_interval=2000 arp_ip_target=10.0.0.1
If you want to monitor link status then you can use the following options line instead, however I couldn’t test this because the MII link monitoring doesn’t seem to work correctly on my hardware (there are many Ethernet devices that don’t work well in this regard):
options bond0 mode=0 miimon=100
Then edit the file /etc/network/interfaces and inset something like the following (as a replacement for the configuration of eth0 that you might currently be using). Note that XX:XX:XX:XX:XX:XX must be replaced by the hardware address of one of the interfaces that are being bonded or by a locally administered address (see this Wikipedia page for details). If you don’t specify the Ethernet address then it will default to the address of the first interface that is enslaved. This might not sound like a problem, however if the machine boots and a hardware failure is experienced which makes the primary Ethernet device not visible to the OS (IE the PCI card is dead but not killing the machine) then the hardware address of the bond would change, this might cause problems with other parts of your network infrastructure.
auto bond0
iface bond0 inet static
pre-up modprobe bond0
hwaddress ether XX:XX:XX:XX:XX:XX
address 10.0.0.199
netmask 255.255.255.0
gateway 10.0.0.1
up ifenslave bond0 eth0 eth1
down ifenslave -d bond0 eth0 eth1
There is some special support for bonding in the Debian ifup and ifdown utilities. The following will give the same result as the above in /etc/network/interfaces:
auto bond0
iface bond0 inet static
pre-up modprobe bond0
hwaddress ether 00:02:55:E1:36:32
address 10.0.0.199
netmask 255.255.255.0
gateway 10.0.0.1
slaves eth0 eth1
The special file /proc/net/bonding/bond0 can be used to view the current configuration of the bond0 device.
In theory it should be possible to use bonding on a workstation with DHCP, but in my brief attempts I have not got it working – any comments from people who have this working would be appreciated. The first pre-requisite of doing so is to use either MII monitoring or broadcast (mode 3), I experimented with using options bond0 mode=3 in /etc/modprobe.d/aliases-bond but found that it took too long to get the bond working and dhclient timed out.
Thanks for the howtoforge.com article and the linuxhorizon.ro article that helped me discover some aspects of this.
Update: Thanks to Guus Sliepen on the debian-devel mailing list for giving an example of the slaves directive as part of an example of bridging and bonding in response to this question.
You wrote
Note that XX:XX:XX:XX:XX:XX must be replaced by the hardware address of one of the interfaces that are being bonded
and also wondered about failing PCI cards. But that does leave a question: will the kernel verify that you used one of mac address from one of the cards? If so, won’t it also fail when the PCI card that has that particular address fails?
You can use any MAC address you desire. The only important thing is that it must be unique in your hardware segment. So if you use the physical address of a device that is installed in the machine you are safe, as if you use an address of a network card that is destroyed (if you replace a failed network device you don’t need to change the MAC).
Another option that I noted is locally administered addresses – in which case the onus is on you as the sys-admin to make sure you don’t duplicate an address.
Ok, makes sense. I remembered something being discussed on netdev about verifying addresses but it was probably in a different context then.
Use these stanzas to get the interface up so it can ask for a dhcp request before it’s technically “ready”
pre-up ifconfig bond0 up;/sbin/ifenslave bond0 eth1 eth0
down /sbin/ifenslave –detach bond0 eth1 eth0
Does this ifenslave method require any support from the ethernet switch you use? If so, what do you need to check your switch supports?
[…] This entry was posted on Tuesday, August 14th, 2007 at 7:00 pm, for similar articles see the category Xen, Networking, Debian. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site. « Ethernet Bonding on Debian Etch […]
Geoff: I believe that hubs never support STP, all good switches support it, and I’m not sure about low-end switches. In all those cases things should just work, links are put in backup mode by the devices that implement STP without regard as to what exactly is between them.
But I am not an expert on this.
Thanks for the article is excelent, keep working.
A little question: can you used two or more bond in the same machine, I am having troubles with this, bond0 works ok, but bond1 doesn´t work neither start.
Any light, link or example to make work this??? I read almost any material on the web, but nothing. I am using Debian Etch, I make work bond0 from different ways, but I am having this:
SIOCSIFADDR: No such device
bond1: ERROR while getting interface flags: No such device
SIOCSIFNETMASK: No such device
bond1: ERROR while getting interface flags: No such device
Failed to bring up bond1.
mogojojo: I am not aware of any reason for it to not work. However I don’t have access to suitable hardware to test more than one bond. I can get two switches, and two target machines for ARP pings, but I can’t get a machine with 4 Ethernet ports.
If you happen to know of a way of getting some 2-port PCI (not PCI-e) Ethernet cards cheaply in Australia then I would be very interested.
etbe: I did it, only compile the new kernel 2.6.22 with the defaults options and every works, now I am using bond0 bond1 and bond2.
I am using a virtual machine of VMware (excelent software), then I can to create the Ethernet cards, I don´t have a multi NIC Ethernet Cards sorry.
I am working in my laptop ;)
The work now is to review the default kernel 2.6.18-686-5, to determine why don´t work and if need a compilation with any option changed.
Thanks a lot for the answer, I hope this post help to any guy with the same problem.
mogojojo : If it isn’t a big secret, how did you make bond1 working ? I’m having the same problem. Just compiled 2.6.22 kernel , but it didn’t help. bond0 works fine , but bond1 shows:
bond1: ERROR while getting interface flags: No such device
SIOCSIFNETMASK: No such device
bond1: ERROR while getting interface flags: No such device
Failed to bring up bond1.
All four real interfaces are working.
Thanks for any advise, on how to force bond1 to exist.
stalin: no problem, I use ifenslave-2.6, to compile the kernel as you know and put this:
alias bond0 bonding
options bond0 mode=0 arp_interval=2000 arp_ip_target=192.168.1.1 miimon=200 maxbonds=3 (I guess the big secret is this: maxbonds…)
alias bond1 bonding
options bond1 mode=1 arp_interval=2000 arp_ip_target=192.168.1.1 miimon=200
alias bond2 bonding
options bond2 mode1 arp_interval=2000 arp_ip_target=192.168.1.1 miimon=200
in this file /etc/modprobe.d/aliases-bond if does not exist then you need to create one.
after run as root update-modules.
You must modify /etc/network/interfaces, this the necessary about bonding:
auto bond0
iface bond0 inet static
pre-up modprobe bond0
hwaddress ether XX:XX:XX:XX:XX:XX (put yours)
address 192.168.1.107
netmask 255.255.255.0
gateway 192.168.1.1
up ifenslave-2.6 bond eth1 eth2 eth3
down ifenslave-2.6 bond eth1 eth2 eth3
to bond1 an other only change the numbers.
Please tell me if this help you.
Is it possible to add a second IP address to the same bonding interface (and how modify /etc/network/interfaces) ?
Thanks
Jacques: If you use the ip command, like “ip addr add dev bond0 10.11.12.13/24 broadcast +” then it should work. As far having it be a first-class object with the kernel using it for arp tests, I don’t think that is possible.
I preferred manage it by the “ifconfig bond0 add 10.11.12.13 netmask 255.255.255.0” command which is working.
According to my tests, the corresponding /etc/network/interfaces looks like:
auto bond0
iface bond0 inet static
pre-up modprobe bond0
hwaddress ether 00:02:55:E1:36:32
address 10.11.12.12
netmask 255.255.255.0
gateway 10.11.12.1
slaves eth0 eth1
auto bond0:0
iface bond0:0 inet static
address 10.11.12.13
netmask 255.255.255.0