10.3.9 Network/Netinfo periodically dropping out

Hi,

I’m having a problem with an Xserve. The Xserve in question runs part of a reasonably high traffic website. It provides a copy of apache which both handles static file requests and redirects dynamic requests through mod_rewrite and mod_proxy to a cluster of servers which handle those requests. The system serves an average of 350 requests/sec, at 36.8 mbps. Load average on the box usually oscilates between 10 and 100, becoming sluggish around 140 or so.

Recently (in the last 48 hours or so), this system has started dropping off the network for periods of time. Running a constant ping from another box on the subnet shows response time suddenly rising to 30,000+ ms. ssh sessions, etc may time out and the box becomes generally unresponsive for an extended period of time.

Corresponding in the system.log when this happens is the line: localhost DirectoryService[220]: NetInfo connection failed for server 127.0.0.1/local. I don’t know if this is a cause or a symptom of another issue.

From what I can tell, 2 explanations are left to me.

a) Lookupd/Netinfod are hanging up on some lookup and the kernel backs up handling packets untill they stop spinning.
b) Some kernel network resource is getting used up, and the system stops responding untill it can become free again.

One recent observation is that the system will go longer between hanging up after this message appears in the console log: resize: max chain len 10, new table size 32768.

a) Has anyone seen similair issues with their 10.3.9+ system just falling off the network in periods of high use?
b) Does anyone know what the resize message is?

Thanks,

~a