Link Aggregation Question

[Dave Hagan]

Does Link Aggregation need to be configured during initial setup or can it be enabled later on? Are there any pitfalls?

[jensonyu]

I set it up with a Nortel 425 switch. Seems working fine. Tested by pulling the cable from en1, after 30 sec pause, the connection is back on using the en0 link. However, if I pull the en0 cable, the connection cannot recover itself by just using the en1 link.

Ran ifconfig and found that under bond0 — the aggregated interface, it’s using en0’s MAC address, instead of creating a new one. I think that’s why pulling out the en1 link, the connection still works; but disconnecting en0, the connection will stop permanently. Any suggestions?

[dragonmac]

👿
Ok Finally someone who has the same exact problem and has seen the light at the end of the tunnel. A couple of things I’d like to know if we have them in common. Is yours an Intel system with two built-in en-X’s? If not what hardware do you have? What OS 10.4.x,6,7, or 8?
As to my problem it is a brand new Mac Pro Intel running 10.4.8 Server now. I noticed the same thing with the MAC addresses. I heard from some PPC users that had some problems with Link Aggregation in 10.4.8 but where able to fix but it seems Intel users still can’t get it to work. Spoke to Apple who said it’s not really covered under Applecare but I argued since I installed a clean 10.4.7 Original DVD-OS on a separate partition and it still won’t work Plus it is part of the GUI now so it should be covered.
My behavior is identical to what you described en0 works fine both in or solo but en1 won’t work in Link Aggregation. I think it’s the MAC address too.
Anyone know how to spoof the MAC address the right way via CLI so we can try it and not have to wait on apple patch?

[dragonmac]

Ok Problem is for the most part confirmed. Any Guru’s out there that could help with this would be great. I basiclly need a way in CLI at this point to set the MAC address properly for my bond0. I’m gonna post 2 ifconfig -b bond0 reports. One is of a working Link the second is mine. Just notice the ether address on his and mine and the MAC addresses listed for en0 & en1. You should see the problem and hoppfully someone could help me.

TNSteve Posts: 2
Registered: Jul 30, 2006 Re: Link Aggregation, 10.4.8 Server, D-Link problems.
Posted: Dec 20, 2006 11:09 AM

Our Xserv is connected to a netgear gs748T. I do not have trunking enabled on the switch. en0 and en1 are aggregated to bond0 in network setup.

xserv1:~ admin$ sudo ifconfig -b bond0
bond0: flags=8843 mtu 1500
inet6 fe80::20d:93ff:fe9e:9a41%bond0 prefixlen 64 scopeid 0x7
inet 192.168.0.1 netmask 0xffffff00 broadcast 192.168.0.255
ether 00:0d:93:9e:9a:41
media: autoselect (1000baseT ) status: active
supported media: autoselect
bond key: 0x0001 interfaces: en0 (selected) en1 (selected)
bond interface: en0 priority: 0x8000 state: 0x3d partner system: 0x8000,00:0d:93:9e:9a:41 key: 0x0001 port: 0x0005 priority: 0x8000 state: 0x3d
bond interface: en1 priority: 0x8000 state: 0x3d partner system: 0x8000,00:0d:93:9e:9a:41 key: 0x0001 port: 0x0004 priority: 0x8000 state: 0x3d

xserv1:~ admin$ sudo networksetup -listBonds
interface name: bond0
{
user-defined-name: Bond_NICS
devices: en0, en1
}

Here is an article that you might find of interest.
https://www.afp548.com/article.php?story=20060724001126511

G5 PowerMac Xserv Mac OS X (10.4.8)

Michael O Posts: 20
From: NYC, USA
Registered: Jul 16, 2003
Re: Link Aggregation, 10.4.8 Server, D-Link problems.
Posted: Dec 20, 2006 1:14 PM in response to: TNSteve

Thank You so much steve because you have confirmed my problem. It is the MAC address setup on the Mac Pro via network setup. Here is my ifconfig.

osxserver:~ root# ifconfig -b bond0
bond0: flags=8843 mtu 1500
inet6 fe80::217:f2ff:fe00:1012%bond0 prefixlen 64 scopeid 0x7
inet 192.30.40.2 netmask 0xffffff00 broadcast 192.30.40.255
ether 00:17:f2:00:10:12
media: autoselect (1000baseT ) status: active
supported media: autoselect
bond key: 0x0001 interfaces: en0 (selected) en1 (unselected)
bond interface: en0 priority: 0x8000 state: 0xbd partner system: 0x0001,00:0f:3d:f8:12:8f key: 0x0013 port: 0x0013 priority: 0x0001 state: 0x37
bond interface: en1 priority: 0x8000 state: 0x05 partner system: 0x0001,00:0f:3d:f8:12:8f key: 0x0013 port: 0x0014 priority: 0x0001 state: 0x37

See my “ether” address it DOES NOT match the Mac addresses assigned to the Bonded en0 & en1 as yours does. Apple does it again, sometimes I love them then hate them, lol.

osxserver:~ root# networksetup -listBonds
interface name: bond0
{
user-defined-name: bond0
devices: en0, en1
}

Please does anyone know what CLI commands will fix this?

Mac Pro 2 x 2.66Ghz intel 2006 Mac OS X (10.4.8) ATTO SCSI320 2CH, Highpoint 2322 SATA 2 RAID

[pingu]

Sorry to hijack a little…

Can anybody give any hints on how the switch end should be set up for HP Procurve switches.

We have choices of ‘Trunk’ or ‘LACP’ in ‘HP speak’. As far as I can tell from the HP manual, LACP is proper 802.3ad and can be configured as either static or dynamic. Trunk is used for non-standard protocol trunks. HP recommend using Trunk when the host is using an unknown trunking protocol.

So it seems like LACP is best option, but Static or Dynamic???

Cheers
Dan

[dragonmac]

http://docs.info.apple.com/article.html?artnum=304120

There where a few people on the Apple Forums that have Pro curves and what was need to set it up.

also here s a link here that may explain it better.

https://www.afp548.com/article.php?story=20060724001126511

Just so you know there are other articles posted in the support site for Apple about Link Aggregation. Apple does not support “Static” or “Trunk” setups as far as I’ve read in my research. Only LACP dynamic is supported I believe. You should realy search the Apple Forums cause I know I saw people taking about HP Procurve setups with this. Hope this help you pingu. 😉

[peterthorn]

A little late, but…
I had also problems with a procurve, but figured it out – read more here [url]forums1.itrc.hp.com/service/ forums/questionanswer.do?threadId=1059592[/url]

Peter

[jpbuse]

[QUOTE][u]Quote by: pingu[/u][p]Sorry to hijack a little…

Can anybody give any hints on how the switch end should be set up for HP Procurve switches.

We have choices of ‘Trunk’ or ‘LACP’ in ‘HP speak’. As far as I can tell from the HP manual, LACP is proper 802.3ad and can be configured as either static or dynamic. Trunk is used for non-standard protocol trunks. HP recommend using Trunk when the host is using an unknown trunking protocol.

So it seems like LACP is best option, but Static or Dynamic???

Cheers
Dan[/p][/QUOTE]

Dan – You just need to enable LACP for the ports you will be connecting your machine to. That was I needed to do on my 4108gl switch. -Jason

[pingu]

Thanks JP and Peter, that should give me enough to be getting on with…

Dan

[Flash]

Glad to see so many are diving into link aggregation. I’ve been using it successfully since Xserve G5’s first hit the scene, initially with Small Tree’s multi-port cards and software. I would like to caution you on a few poorly documented side effects.

1. Link negotiation at startup is very slow (at least with my Cisco switches), so if your server is an OD master or replica, the default replication at startup will fail and you’ll see errors like “Password service not found” in replication logs until a running replication occurs.

2. Further, if you happen to restart more than one replica and/or OD master at the same time, I’ve often seen the authserver-replicas files become corrupted on one or more of the restarted servers. This can cause the dreaded continuous replication phenomenon along with repeated replication errors “merging conflicting replicas lists”. Demoting to standalone, then back to replica has been my only brute-force fix for this. If the box is also a BDC, do your homework before demoting – there can be unexpected anomalies.

3. As we all know, OSX server is very dependent upon DNS. It’s always best practice, but particularly with LACP, make sure you’re running DNS on the box. If you don’t, the slow negotiation at startup will cause non-localhost DNS lookups to fail and the box will revert to its “server.local” hostname, rather than the desired “server.domain.com”. I’ve seen this result in malfunctioning of homedir automounts where automount path becomes “Network/Servers/server.local/Volumes…” rather than that defined in LDAP, “Network/Servers/server.domain.com/Volumes…”.

4. Lastly, realize that LACP does NOT support load balancing in the true sense. Cisco docs indicate that the second bonded port won’t even get used until the first port is saturated or experiencing some threshold of latency. Look at your switch port stats, you’ll see what I’m talking about.

All that aside, I highly recommend LACP, throughput and latency in high traffic environments will be noticeably improved.

[Flash]

My Cisco engineer has just further enlightened me on my load balancing statement above. LACP can load balance in some sense. The bonded ports can be configured to round-robin load balance by host or client MAC or IP address. The default on Cisco switches is to load balance by client MAC address. What this means is that each new client connection is assigned to one of the bonded ports for the duration of the connection. So it load balances by client connections, NOT by bandwidth consumed, which would be more ideal. Nonetheless, I stand corrected, LACP does have load balancing capability by client connection.

[dragonmac]

Well unfortunately I have to wait on Apple to fix my Mac Pro problem (see post above). Still no Link Aggregation and no one can figure out a workaround but it might be hardware so. Apple is aware of the problem.

[Flash]

[QUOTE][u]Quote by: dragonmac[/u][p]Well unfortunately I have to wait on Apple to fix my Mac Pro problem (see post above). Still no Link Aggregation and no one can figure out a workaround but it might be hardware so. Apple is aware of the problem.[/p][/QUOTE]

I believe Small Tree may have your solution, though not for free.

http://www.small-tree.com/solutions/linkaggregation.htm

[Author]

Posts