[morgant]

We’re running into this issue as well on Mac OS X 10.5.5 Server (soon to be 10.5.8 Server). There seems to be a good description of locking down the ports [URL=http://www.macosxhints.com/article.php?story=20100219094419683]over at Mac OS X Hints[/URL]. It certainly works over HTTP, but what are the pros & cons of HTTP? It seems most are still using NFS.

[morgant]

I’ve run into the [url=https://www.afp548.com/comment.php?mode=display&format=threaded&order=ASC&pid=6809]same issue[/url] with the [url=https://www.afp548.com/article.php?story=20050622155757690]server admin method of backing up OD[/url] and have had to resort to using [url=https://www.afp548.com/forum/viewtopic.php?showtopic=17562#17871]slapconfig and expect[/url].

Has no-one really solved the issue with the script? How are others automating the backup of Open Directory data?

[morgant]

For the record, my [i]expect[/i] script is as follows (based on the suggestion [url=https://www.afp548.com/comment.php?mode=view&cid=6405]here[/url]):

[code]#!/usr/bin/expect -f

set date [timestamp -format “%Y-%m-%d”]
set archive_path “path/to/you/backup/dir”
set archive_password “password”
set archive_name “opendirectory_backup”

spawn /usr/sbin/slapconfig -backupdb $archive_path/$archive_name-$date
expect “Enter archive password”
send “$archive_password\r”
expect eof[/code]

Hope that helps others since I’ve not seen one like this actually posted anywhere. I really do wish the [url=https://www.afp548.com/article.php?story=20050622155757690]serveradmin method[/url] worked for me though.

[b]Edit:[/b] Of course, you really want to run put this file in root’s crontab so it runs nightly and also change the permissions to r/w/x for root-only as you’ll be storing the password right in the file.

[morgant]

I’ve already stated that this seems to be an AFP issue to me, but I’ve now migrated that file server to a new Xserve Xeon running Mac OS X 10.4.10 server and the issue still persists. [s]I’d assume it’s an Open Directory issue at this point, but I deleted the user and recreated her from scratch.[/s]

Oops, I wasn’t paying attention to the fact that the 10.3.9 server was the one that worked. [i](And, no, I didn’t just migrate the whole server because of that, it was in desperate need of an upgrade and had been locking up due to I/O errors recently.)[/i]

Does anyone have any suggestions?

[morgant]

Hmmm, after a very a quick inspection, I am seeing some replication errors. I’ll have to dig into this and see what was actually going on on that date.

[morgant]

And apparently this is one of those issues that a reboot will solve. I hate those, I like answers!

[morgant]

[QUOTE][u]Quote by: fherbert[/u][p]Make sure your security policies match on the client and server. Since you are using trusted binding, I think you will find the client will not try to authenticate using clear text passwords, but the server may not be configured for this (or it might be the other way round). [/p][/QUOTE]

Okay, I don’t quite understand where the security policies are on the client (no, I hadn’t been checking system.log on the client either), but this did get me pointed in the right direction. I had clear-text passwords disabled, but was not using SSL. When enabling SSL (using just the default self-assigned certificate), but not requiring all packets to be encrypted or to block man-in-the-middle attacks, I was able to successfully bind the clients to the domain.

I’ll tighten up security as I go, but for now I’m happy to have it working correctly. Why would this function this way? Is SSL required when not sending clear-text passwords (and not noted anywhere)?

[morgant]

I’m having a similar problem with a Tiger Server Open Directory Master that had its configuration imported from a Panther Server OD Master (not upgraded, but imported). There were, for some reason, no network views in the 10.3 OD Master, so there are still none in the 10.4 OD Master (don’t know how much of a difference this makes).

Basically the Tiger OD Master has: lookupd, LDAP, password server, and Kerberos all running; SSL disabled; directory binding enabled (but not required); clear text passwords disabled; and forward and reverse DNS are working and correct. Now, there are a couple other Xserves manually configured (server DNS address & search base manually entered) to get authentication (and Address Book, although only used for testing ldap requests) from the Open Directory master… this has worked correctly since day one.

I can use Directory Access on workstations to do an add new LDAP server and bind successfully with the directory administrator’s account (it creates the computer record in Open Directory). However, you can’t search for users in the domain using Address Book and the login window declares that “Network Accounts Unavailable”. When you try to unbind the workstation from the Open Directory domain using the same directory administrator’s account, it fails stating that it couldn’t communicate with the Open Directory server.

The odd part is that I can take any workstation, and manually add the LDAP server (and search base) without binding, manually add it to the search path for authentication and the address book and it all works (mostly) correctly: searching for users in Address Book is immediately functional and the login window immediately states that “Network Accounts [are] Available”.

I’ve been reading through the User Management Admin and Open Directory documentation, but don’t quite understand why this is functioning in this way. Is it something residual from the import from the 10.3 OD Master? Is it because I’m not using LDAP or Kerberos?

Any assistance would be greatly appreciated as I’d like to get this corrected before I start rolling out any number of workstations using Open Directory authentication (primarily local home directories, but a few are planned to be network home directories).

[morgant]

My backup drive up and failed on me, but the backups still took just as long after replacing it with a fresh (zeroed & barren) drive, so it wasn’t disk related. However, I did finally switch to using ‘mailbfr’ in my backup script and although the first backup took the usual extended period of time (as expected), the subsequent rsyncs have only been taking about 40 minutes.
A [i]much[/i] better way to go.
I still could probably move the mail store and database to a separate volume and gain a little more performance (both during regular usage and during backup, but then I’d be rolling my own rsync backup script entirely).

[morgant]

I just wanted to revisit this as I just had a [url=https://www.afp548.com/forum/viewtopic.php?showtopic=12669]failure[/url] which could have potentially needed my mail backups (luckily I was able to just rebuild the mailboxes.db database and didn’t have to roll back to previous backup).
Having done more testing, it appears that it’s mainly an issue with the sheer number of files being backed up. The /var/spool/imap directory is 12GB and contains so many small files that it takes the machine approx. 40 mins to run ‘du’ on it.
Currently the mail store is on the boot volume (which is a software-mirrored RAID), and the server only has 1.5GB of RAM. So, the disk probably has higher I/O needs anyway. The mail service data is backed up in full every morning to a separate Apple Drive Module (this is all in a dual 1.33GHz G4 Xserver running Mac OS X Server 10.4.6) and this now takes about 4 hours.
This week I’ll be either modifying ‘mailbak.sh’ to use ‘rsync’ instead of ‘ditto’ or switching my script to wrap ‘mailbfr’ instead, so that I don’t have to complete a full backup every morning (of course, I’ll be rolling the backup during the compression process, after the mail service is restarted). This should improve the backup time significantly, but judging by how long it takes to run ‘du’, it’ll still be a long process.
I’m also considering installing moving the mail store and database over to a separate volume to try to reduce the amount of I/O on the drive holding it. I don’t know how much of a difference this’ll make.
I’m open for other suggestions and constructive criticism. Anybody have any experiences which might help point me in the direction of further improvements?
I’ve got a dual 2.3GHz G5 Xserve with an Apple Hardware RAID card lined up to take over mail server duty if needed, but honestly this dual 1.33GHz G4 Xserve has more than enough processing power to handle the load on it (it only rarely breaks 1.5 load average and usually only breaks 2.5 load average if it’s bzip-ing the previous night’s backups when everyone logs in in the morning.)

[morgant]

Josh,
Thanks for the correction, as usual. Exactly the info I was looking for.
Of course, the mail was recovered and the mail service has been back up for a few days now, I’m considering this issue closed (the other errors I was seeing are not necessarily related to the database having been corrupted).
The SQUAT errors went away once the server caught up on trying to re-index all of mailboxes. I have noticed that SQUAT indexing is turned off on all the mailboxes (atleast in SirAdmin.app, but I should probably verify in cyradm), so I’ll give sysctl a try and start re-enabling SQUAT indexing on the mailboxes (a few at a time).
Thanks again!

[morgant]

Of course, mailbfr saved the day and was able to reconstruct the database, but I’m still getting errors such as:
[code]Jun 19 18:02:51 postoffice imaps[2158]: SQUAT failed
Jun 19 18:02:51 postoffice imaps[2158]: SQUAT failed to open index file[/code]
According to [url=http://www.irbs.net/internet/info-cyrus/0502/0268.html]this[/url], that means it’s running out of file descriptors (or too many files are open). I’ve not seen those before, is that likely just because everyone’s mail clients are trying to re-sync to the server?
[b]UPDATE:[/b] According to “Re: cyrus connection timeout” [url=http://blog.gmane.org/gmane.mail.imap.cyrus/day=20051011]here[/url], these SQUAT errors may not be important.
Can you even up the number of open files in Mac OS X Server? I have a vague recollection that it’s hard-coded into Darwin at compile-time, so you’d have to rebuild the kernel (obviously not an option).

[morgant]

I’m currently trying to reconstruct the database using mailbfr (my mail server still takes a good 4 hours to back up its 12+GB of data, including about 30 minutes to run ‘du’ alone, so it’s been chugging away at this for a while now).
Having had some time to dig through my logs, specifically mail.log, mailaccess.log, and system.log, I’m still finding these (in system.log):
[code]Jun 19 08:05:24 postoffice lmtpunix[7621]: DBERROR: opening /var/imap/deliver.db: Cannot allocate memory
Jun 19 08:05:24 postoffice lmtpunix[7621]: DBERROR: opening /var/imap/deliver.db: cyrusdb error
Jun 19 08:05:24 postoffice lmtpunix[7621]: FATAL: lmtpd: unable to init duplicate delivery database
Jun 19 08:05:24 postoffice master[21962]: service lmtpunix pid 7621 in READY state: terminated abnormally
Jun 19 08:05:24 postoffice lmtpunix[7622]: DBERROR db4: Logging region out of memory; you may need to increase its size
Jun 19 08:05:24 postoffice lmtpunix[7622]: DBERROR: opening /var/imap/deliver.db: Cannot allocate memory
Jun 19 08:05:24 postoffice lmtpunix[7622]: DBERROR: opening /var/imap/deliver.db: cyrusdb error
Jun 19 08:05:24 postoffice lmtpunix[7622]: FATAL: lmtpd: unable to init duplicate delivery database
Jun 19 08:05:24 postoffice master[21962]: service lmtpunix pid 7622 in READY state: terminated abnormally
Jun 19 08:05:24 postoffice lmtpunix[7623]: DBERROR db4: Logging region out of memory; you may need to increase its size[/code]
Part of the reason I upgraded from 10.4.2 to 10.4.6 on Friday was in hopes of getting rid of this. For the last two weeks I’ve had the delivery queue balloon to 700-1000+ messages with my logs filled with these. On one occasion I discovered that one user alone had somehow spawned 100 IMAP processes (I keep a limit of 128).
Also, it appears that I was getting frequent crashes of lmtpd this morning and imapd crashed at one point as well (I’m assuming that’s when the mailboxes.db database got corrupted).
Has anyone else seen these errors? The mail server only has 1.5GB of RAM, so I’ll likely upgrade that tomorrow, but I would think that the mail server could function on 1.5GB or less of RAM (albeit slowly when under stress).

[morgant]

No responses yet?

Are very few of you running mail services under Tiger Server? Are you not running custom backup scripts? Are you using a higher end backup solution (Retrospect Server, BRU, etc.)?

Just looking for ideas here, so feel free to pipe up even if you’re shutting down your mail server on Sunday mornings and using Carbon Copy Cloner or SuperDuper.

[morgant]

Exactly what I was looking for. Thanks.

[Author]

Posts