When installing Xen servers one issue that arises is how to assign MAC addresses. The Wikipedia page about MAC addresses [1] shows that all addresses that have the second least significant bit of the most significant byte set to 1 are “locally administered”. In practice people just use addresses starting with 02: for this purpose although any number congruent to two mod four used in the first octet would give the same result. I prefer to use 02: because it’s best known and therefore casual observers will be more likely to realise what is happening.
Now if you have a Xen bridge that is private to one Dom0 (for communication between Xen DomU’s on the same host) or on a private network (a switch that connects servers owned by one organisation and not connected to machines owned by others) then it’s easy to just pick MAC addresses starting with 02: or 00:16:3e: (the range assigned to the Xen project). But if Xen servers run by other people are likely to be on the same network then there is a problem.
Currently I’m setting up some Xen servers that have public and private networks. The private network will either be a local bridge (that doesn’t permit sending data out any Ethernet ports) or a bridge to an Ethernet port that is connected to a private switch, for that I am using MAC addresses starting with 02:. As far as I am aware there is no issue with machine A having a particular MAC address on one VLAN while machine B has the same MAC address on another VLAN.
My strategy for dealing with the MAC addresses for the public network at the moment is to copy MAC addresses from machines that will never be in the same network. For example if I use the MAC addresses from Ethernet cards in a P3 desktop system running as a router in a small company in Australia then I can safely use them in a Xen server in a co-location center in the US (there’s no chance of someone taking the PCI ethernet cards from the machine in Australia and sending them to the US – and no-one sells servers that can use such cards anyway). Note that I only do this when I have root on the machine in question and where there is no doubt about who runs the machine, so there should not be any risk.
Of course if someone from the ISP analyses the MAC addresses on their network it will look like they have some very old machines in their server room. ;)
I wonder if there are any protocols that do anything nasty with MAC addresses. I know that IPv6 addresses can be based on the MAC address, but as long as the separate networks have separate IPv6 ranges that shouldn’t be a problem. I’m certainly not going to try bridging networks between Australia and the US!
Another possible way of solving this issue would be to have the people who run a server room assign and manage MAC addresses. One way of doing this would be to specify a mapping of IP addresses to MAC addresses, EG you could have the first two bytes be 02:00: and the next four be the same as the IPv4 address assigned to the DomU in question. In the vast majority of server rooms I’ve encountered the number of public IP addresses has been greater than or equal to the number of MAC addresses with the only exception being corporate server rooms where everything runs on private IP address space (but there’s nothing wrong with 02:00:0a: as the prefix for a MAC address).
I also wonder if anyone else is thinking about the potential for MAC collisions. I’ve got Xen servers in a couple of server rooms, I told the relevant people in writing of my precise plans (and was assigned extra IP addresses for all the DomUs) but never had anyone mention any scheme for assigning MAC addresses.
“As far as I am aware there is no issue with machine A having a particular MAC address on one VLAN while machine B has the same MAC address on another VLAN.”
Some switches have a global MAC table rather than one per vlan, so if there is one of those on the same L2 segment you’ll get problems.
Cisco switches have a table per-vlan, but some other brands don’t – some of the older HP switches have only a single table (http://forums12.itrc.hp.com/service/forums/questionanswer.do?admit=109447627+1217942413573+28353475&threadId=914092). I don’t know if its an issue with current switches.
This isn’t a switch bug, since the original STP algorithm doesn’t know or care about VLANs.
Since you have to trust that noone will ever plug in one of those switches to the same L2 network, its probably best to ensure a unique MAC. It’ll work perfectly right up until some switch fails and STP changes to run slightly differently and its a real pain to debug.
“As far as I am aware there is no issue with machine A having a particular MAC address on one VLAN while machine B has the same MAC address on another VLAN.”
Some older switches, such as the HP ProCurve 4000/8000 series, have only one MAC address table across all VLANs. They’re old, but these switches have lifetime warranty so they might be hanging around in places you don’t expect.
At Faelix we use two schemes: something very like your 02:00:w:x:y:z (where w.x.y.z is the first IP address we allocated to the Xen instance), but often we change the second octet to give us a clue as to the “home server” the domU is meant to be running on.
I’ve heard of people using MAC addresses beginning 4x:xx:xx, but right now I can’t find a reference which explains why.
I’m using 00:ff:00:00:00:00 for my kvm machine using tap0, 00:ff:00:00:00:01 for my kvm machine using tap1, etc. There’s no specific reason I’m doing this (I should actually use something starting with 01 I think) other than that it’s pretty obvious those are assigned to virtual machines and assigned by hand. As Marek points out issues can occur with conflicting mac addresses and I would personally try to avoid duplicate mac addresses, it shouldn’t really be necessary.
Marek: Thanks for that information. I guess it’s a good thing that my duplicate addresses are on different continents! ;)
Starting a MAC address with 4X where X is congruent to 2 mod 4 will work just as well as 02 in strict technical terms, but will be less commonly used by other people which would be an advantage.
Michael: I believe that there is plenty of MAC address space that has not yet been assigned to vendors (I’m pretty sure that there have not been 2^24 address blocks assigned to vendors). Also you could do something like identify a MAC address range that was only used for ISA cards and operate on the assumption that such an old machine will never be plugged into your network. There are lots of ways of doing this. But it seems that variations on my idea (of reusing MAC addresses that you personally control) is best.
Another possibility is to buy a heap of cheap old ethernet cards (maybe 10base2), copy the MAC addresses and then destroy them.
I don’t know much about MAC addresses, but I did come across this handy util called easymac.sh which is commonly used with VM Ware machines:
http://www.easyvmx.com/software/easymac.sh
I’ve actually been wondering if it’s definately sure that there’s no 2 equal MAC addresses in hardware. I guess if you really want to have a reserved MAC range you should buy it, buying old ethernet cards is a decent idea, but seems like a lot of work… Actually I’ve already thrown some ethernet cards away because they weren’t working anymore, but I guess I could say I’m the owner of the MAC addresses that the cards used.
1) buy a manufacturer’s block, a MAC range
2) sell MACs guaranteed to be unique to the public for far less than the cost of a throw-away network card (and doing this is more ecologically sound too)
3) …
4) profit!
The size of the market is quite likely to be small, and there is no differentiating factor between you and a competitor… so this business model is not guaranteed to turn a profit for you ;-)
Albert: That URL doesn’t work for me.
Michael: There are rumors of hardware vendors stuffing up in this regard.
Marek: That is a really good idea. It would not be difficult for a company like Red Hat to buy such a block and then assign 50 addresses to each RHEL-AS customer and 4 addresses to every RHEL-ES customer (RHEL-ES limits you to running four RHEL DomUs). It wouldn’t be directly profitable for them, but it would make life easier for their customers. Especially if they integrated MAC management into the RHEL management system (RHN).