AFP548 Changing the world one server at a time.

Tiger Server & AD Integration – Apple File Service Crashes !?!

Home › Forums › AFP548 Community › Open Mike › Tiger Server & AD Integration – Apple File Service Crashes !?!

Viewing 7 posts - 1 through 7 (of 7 total)
  • Author
    Posts
  • #362006
    jdyck
    Participant

    Hello all – I’m desperately hoping someone can help point me in the right direction to solve this problem.

    I’ve installed a nice new complete fresh Tiger server onto an X-Server dual G5 machine. It has never been used before, but we are trying to move a bunch of user folders off an old Windows server.

    After installing I’ve installed all updates (10.4.1, QTime, Security, etc). I’ve verified that the DNS has been correctly entered by going into terminal and doing an ‘nslookup’ and verifiying both the forward and reverse lookups. so far so good.

    I then go into Directory Access and bind the Server into our Active Directory system and verify that AD is also now a part of the authentication path (Tiger seems to do this automatically – much better than Panther). The computer tells me if I wish to also join a kerberos domain to do that in Open Directory. i then go back to Terminal and use ‘dscl localhost’ and am able to successfully browse the AD user list and read records. Seems my binding to AD is good.

    I then go into Server Admin -> Open Directory and click the Join Kerberos button and am able to successfully join the Windows/AD Kerberos domain. I am able to test this by logging into a client as a user who’s Windows profile has been set with a home folder on the Tiger server. Running through SMB this is great.

    However, most of our users aren’t actually binding to AD at this time, so I setup an AFP/SMB sharepoint that is setup with ACL’s to allow specific AD groups to acces them. Here’s where I begin to get a problem…

    I am able to successfully go to any client machine and do the “Connect to Server” and point to the Tiger server and login via AFP. However, after a few logins, this dies. After this, I am able to bring up the login screen and enter my credentials, but then it brings up an “Authenticating” box with the barbershop poles and just sits there and does nothing.

    If I then try to stop the service it sits there forever as well – a quick look at Activety Monitor shows the AppleFileService process taking ~98% of the CPU (thankfully there are two so the X-Serve still runs ok). Thus I am unable to stop the AFP service via Server Admin.

    If I completely restart the Server I am again able to make a few logins before the fault repeats exactly the same.

    If I unbind from Active Directory and just setup user accounts on the Tiger Server, the AFP service appears to be rock solid (at least with only two different people actually logging on at the same time for testing purposes), so I’m starting to suspect that the fault only occurs when bound to AD.

    I’ve just reinstalled the Server from scratch, rebound to AD but NOT joined the Kerberos domain just as another test – now if I log onto a client bound to AD I get my home folder via SMB, which is good. But if I try to Connect to Server to the afp share it is unable to find my credentials. Once again, if I try to stop the AFP service the AppleFileService process shoots up to ~99.8% cpu usage and hangs.

    I’ve checked the AFP access log, which shows the successful connections, but has no record of the failed attempts. I’ve also checked the error log, which has no information at all (AFP logging is turned on). I tried going into slapd in diagnostic mode but if fails – I assume because my actual ldap process is on Active Directory, not Tiger.

    Am I missing something obvious?🙄 is there some log file somewhere I can look at? Actually, I just checked the console and system logs and they both have dozens of entrees regarding failed automounts on the server…

    Console log makes a lot of references to automount failures to some of the shares I’ve setup, which makes me wonder WHY Tiger server is attempting to automount these share – they weren’t added to an OD ldap automount share and I’m logged in to the Tiger server as a local admin, so no network account…

    System log references a lot of nfs server automount -fstab errors, which also makes me wonder as we don’t have any linux/unix servers and nfs is NOT running at all.

    Sorry for the long post – figure it’s better to have lots of detailed information than not enough.

    I must say at this point I’m a bit dissapointed in OS X server… We bought this X-Serve and are trying to implement it because our district has moved to a majority of OS X client machines and we felt that OS X Server would provide better service than our Windows servers. My experience with 10.2, 10.3 and 10.4 server is that the Apple Reality distortion field is in full effect – so much is promised and it’s made to look so easy, but when you actually go to do it you run along really well then suddenly BAMM run into a brick wall of a show-stopping bug… As much as I’m a huge fan of Apple’s computers and OS X client, at this point Windows servers have been, dare I say, easier to setup and more robust (at lease in Active Directory). For the most part I’ve never felt like I was treading on thin ice like I often do with OS X server… And I’ve even attended the Sys Admin of OS X Server course, so I’m not flying totally blind here… (end frustrated rant – sorry)

    For reference, here are some of the lines from the system & console logs – for security I’ve replaced the dns name of my server with fully.qualified.dns.name – my boss would kill me if I posted all our info on the internet 😉.

    ie: from System Log:

    Jun 16 13:24:28 cosmos automount[3940]: Can’t mount fully.qualified.dns.name:/bbaker$ on /private/Network/Servers/fully.qualified.dns.name/bbaker$: Permission denied (13)
    Jun 16 13:24:28 cosmos automount[3940]: Attempt to mount /automount/Servers/fully.qualified.dns.name/bbaker$ returned 13 (Permission denied)
    Jun 16 13:27:30 cosmos automount[3963]: Can’t mount fully.qualified.dns.name:/bbaker$ on /private/Network/Servers/fully.qualified.dns.name/bbaker$: Permission denied (13)
    Jun 16 13:27:30 cosmos automount[3963]: Attempt to mount /automount/Servers/fully.qualified.dns.name/bbaker$ returned 13 (Permission denied)
    Jun 16 13:30:48 cosmos automount[3990]: Can’t mount fully.qualified.dns.name:/bbaker$ on /private/Network/Servers/fully.qualified.dns.name/bbaker$: Permission denied (13)
    Jun 16 13:30:48 cosmos automount[3990]: Attempt to mount /automount/Servers/fully.qualified.dns.name/bbaker$ returned 13 (Permission denied)
    Jun 16 13:34:23 cosmos automount[4018]: Can’t mount fully.qualified.dns.name:/bbaker$ on /private/Network/Servers/fully.qualified.dns.name/bbaker$: Permission denied (13)
    Jun 16 13:34:23 cosmos automount[4018]: Attempt to mount /automount/Servers/fully.qualified.dns.name/bbaker$ returned 13 (Permission denied)
    Jun 16 13:38:13 cosmos automount[4046]: Can’t mount fully.qualified.dns.name:/bbaker$ on /private/Network/Servers/fully.qualified.dns.name/bbaker$: Permission denied (13)
    Jun 16 13:38:13 cosmos automount[4046]: Attempt to mount /automount/Servers/fully.qualified.dns.name/bbaker$ returned 13 (Permission denied)
    Jun 16 13:42:15 cosmos automount[4078]: Can’t mount fully.qualified.dns.name:/bbaker$ on /private/Network/Servers/fully.qualified.dns.name/bbaker$: Operation timed out (60)
    Jun 16 13:42:15 cosmos automount[4078]: Attempt to mount /automount/Servers/fully.qualified.dns.name/bbaker$ returned 60 (Operation timed out)
    Jun 16 13:46:17 cosmos automount[4111]: Can’t mount fully.qualified.dns.name:/bbaker$ on /private/Network/Servers/fully.qualified.dns.name/bbaker$: Operation timed out (60)
    Jun 16 13:46:17 cosmos automount[4111]: Attempt to mount /automount/Servers/fully.qualified.dns.name/bbaker$ returned 60 (Operation timed out)

    And from Console.log

    Jun 16 13:42:27 cosmos kernel[0]: nfs server automount -fstab [209]: not responding
    Jun 16 13:42:58 cosmos kernel[0]: nfs server automount -fstab [209]: not responding
    Jun 16 13:43:19 cosmos kernel[0]: smb_iod_recvall: drop resp: mid 2, cmd 115
    Jun 16 13:43:29 cosmos kernel[0]: nfs server automount -fstab [209]: not responding
    Jun 16 13:44:00 cosmos kernel[0]: nfs server automount -fstab [209]: not responding
    Jun 16 13:44:16 cosmos automount[4083]: Can’t mount fully.qualified.dns.namea:/TRC_Share on /private/Network/Servers/fully.qualified.dns.name/TRC_Share: Operation timed out (60)
    Jun 16 13:44:16 cosmos automount[4083]: Attempt to mount /automount/Servers/fully.qualified.dns.name/TRC_Share returned 60 (Operation timed out)
    Jun 16 13:44:31 cosmos kernel[0]: nfs server automount -fstab [209]: not responding
    Jun 16 13:45:02 cosmos kernel[0]: nfs server automount -fstab [209]: not responding
    Jun 16 13:45:22 cosmos kernel[0]: smb_iod_recvall: drop resp: mid 2, cmd 115
    Jun 16 13:45:33 cosmos kernel[0]: nfs server automount -fstab [209]: not responding
    Jun 16 13:46:04 cosmos kernel[0]: nfs server automount -fstab [209]: not responding
    Jun 16 13:46:17 cosmos automount[4111]: Can’t mount fully.qualified.dns.name:/bbaker$ on /private/Network/Servers/fully.qualified.dns.name/bbaker$: Operation timed out (60)
    Jun 16 13:46:17 cosmos automount[4111]: Attempt to mount /automount/Servers/fully.qualified.dns.name/bbaker$ returned 60 (Operation timed out)
    Jun 16 13:46:35 cosmos kernel[0]: nfs server automount -fstab [209]: not responding
    Jun 16 13:46:52 cosmos kernel[0]: AFP_VFS afpfs_mount: /Volumes/jldyck, pid 4133
    Jun 16 13:46:52 cosmos KernelEventAgent[42]: tid 00000000 type ‘nfs’, mounted on ‘/automount/Servers’, from ‘automount -fstab [209]’, not responding
    Jun 16 13:47:06 cosmos kernel[0]: nfs server automount -fstab [209]: not responding
    Jun 16 13:47:26 cosmos kernel[0]: smb_iod_recvall: drop resp: mid 2, cmd 115
    Jun 16 13:47:37 cosmos kernel[0]: nfs server automount -fstab [209]: not responding
    Jun 16 13:48:08 cosmos kernel[0]: nfs server automount -fstab [209]: not responding
    Jun 16 13:48:18 cosmos automount[4116]: Can’t mount fully.qualified.dns.name:/TRC_Share on /private/Network/Servers/fully.qualified.dns.name/TRC_Share: Operation timed out (60)
    Jun 16 13:48:18 cosmos automount[4116]: Attempt to mount /automount/Servers/fully.qualified.dns.name/TRC_Share returned 60 (Operation timed out)
    Jun 16 13:48:39 cosmos kernel[0]: nfs server automount -fstab [209]: not responding
    🙄

    #362170
    jdyck
    Participant

    Since nobody seems to have a solution to this problem, can anybody else at least confirm that they are having the same problem? Anyone?

    #362171
    Zeheeba
    Participant

    Hello Jeff,

    Since there have been no responses, I might as well throw some thoughts into the mix.

    From what I see, you are receiving a “Permission denied (13)” error 5 times. Is your AD setup to deny access, or lock accounts after a certain amount of failed logins, etc? Often the count is 5 when this setup is enabled. If so, perhaps the 5 failed attempts to mount or access this directory are disabling any further attemps, giving you the “Operation timed out (60)” errors. You mentioned getting the spinning beach ball when trying to log in. This could be explained by a timeout as well. Either you are not being permitted to authenticate, or perhaps your communications are becoming disrupted somehow.

    Just a thought. Another thing to look into is the time frame your kerberos tickets remain valid. If the time is really short, perhaps the ticket is expiring and therefore giving you the errors. I believe you can check the Kerberos ticket times via /System/Library/CoreServices/Kerberos.app. Then again, the tickets may be renewed automatically after you try to use this access again, I’m not sure, but its an idea. : )

    Hope some of this may help or shed some light on your issues.

    Regards,
    Z

    #363566
    jcjamesx
    Participant

    Has anybody figured anything out on this?

    Our server is connected to an AD, but is not bound to it, but we are having the exact same problems. For an hour or 2, people can make connections through AFP to the server. Then the server will just hang and nobody can make a connection.

    We get the exact same System log, with all of the Can’t mount / Permission denied errors. We get those right after bootup. But like I said, we can have 60 connections at a time, but then it will just hang and we have to restart the server 6 or 7 times a day. I haven’t tried quitting AFP once it appears to hang, but usually I can’t ARD into the machine at that point.

    #363571
    superrcat
    Participant

    This is a known issue. It should be addressed in the next update.

    #364404
    Anonymous
    Guest

    I just wrote this to get around manually restarting the service after every AppleFileServer hangup.
    Start it with “nohup afs-restarter.sh >/dev/null &”
    Awfully wrong way to work around any bug. Is running in production but I cannot confirm that it works until the AFS happens to break again.

    #!/bin/sh
    # afs-restarter: daemon to restart AFS when it goes crazy
    
    REPORT_TO="${REPORT_TO:-your@email}"
    
    mail_report() {
        sendmail "$REPORT_TO" <<EOF
    From: "`basename "$0"`" <root@`hostname`>
    To: "$REPORT_TO"
    Date: `date`
    Subject: AppleFileServer restarted
    
    $0 $* on `hostname` just noticed that AppleFileServer was running wild,
    killed and started it.
    
    Please check that afp://fileserver works now.
    EOF
    }
    
    # just test mailing
    if [ "$1" == "--testmail" ]; then
        mail_report
        exit 0
    fi
    
    cd / # directory to start AppleFileServer in
    
    # loop infinitely (until sleep halted)
    while sleep 5; do
    
        # is system load over 1?
        if ! uptime | grep -q 'load averages: 0'; then
            # has AppleFileServer gone wild?
            if ps uxw | grep -q ' 99\.[0-9] .* AppleFileServer$'; then
    
                # take down AppleFileServer
                for SIG in TERM KILL KILL; do
                    PIDS=`ps uxw | sed -ne 's/^root *\([0-9]*\) .* AppleFileServer$/\1/p'`
                    [ -z "$PIDS" ] && break
                    kill -s $SIG $PIDS
                    sleep 5
                done
    
                # did killing succeed?
                if [ -z "$PIDS" ]; then
                    # yes, start the server
                    AppleFileServer
                    mail_report
                fi
            fi # gone wild
        fi # system load
    done
    
    
    
    #364420
    Anonymous
    Guest

    Looks like either your DNS is not setup correctly. Or your AD admins have not allowed you to bind correctly or with insufficient privileges.

Viewing 7 posts - 1 through 7 (of 7 total)
  • You must be logged in to reply to this topic.