More efficient Mail backup scripts for Tiger server?

[morgant]

I’ve written a script that handles performing nightly backups of my mail server, rolling the backups, compressing them, etc. Currently it calls mailbak.sh (with a few minor fixes) to do the actual stopping of the mail services and performing a full backup (to a staging directory on a second volume) at o’dark hundred (mailbak.sh syntax: “mailback -b path_to_backup_directory -p“).
Unfortunately, it’s currently taking about 2¹/₂ hours to complete.Resulting nightly backup archives (compressed w/bz2) are 5+GB.
I’ve used mailbfr a couple of times for moving mail configurations (another question regarding that shortly in another thread) and it’s seemed reasonable and maybe even a little faster. My assumption is that I could then also let mailbfr do its incremental backups to the same staging directory to improve the backup speed, correct?
What are others using and what are your experiences? Would switching my script over to use mailbfr be a better choice? Has anyone run into issues with letting mailbfr do its incremental backups for long periods of time (i.e. do you have to blow away the staging directory once a week/month/etc to make sure it’s reliable)?
Thanks for any feedback.

[morgant]

No responses yet?

Are very few of you running mail services under Tiger Server? Are you not running custom backup scripts? Are you using a higher end backup solution (Retrospect Server, BRU, etc.)?

Just looking for ideas here, so feel free to pipe up even if you’re shutting down your mail server on Sunday mornings and using Carbon Copy Cloner or SuperDuper.

[morgant]

I just wanted to revisit this as I just had a [url=https://www.afp548.com/forum/viewtopic.php?showtopic=12669]failure[/url] which could have potentially needed my mail backups (luckily I was able to just rebuild the mailboxes.db database and didn’t have to roll back to previous backup).
Having done more testing, it appears that it’s mainly an issue with the sheer number of files being backed up. The /var/spool/imap directory is 12GB and contains so many small files that it takes the machine approx. 40 mins to run ‘du’ on it.
Currently the mail store is on the boot volume (which is a software-mirrored RAID), and the server only has 1.5GB of RAM. So, the disk probably has higher I/O needs anyway. The mail service data is backed up in full every morning to a separate Apple Drive Module (this is all in a dual 1.33GHz G4 Xserver running Mac OS X Server 10.4.6) and this now takes about 4 hours.
This week I’ll be either modifying ‘mailbak.sh’ to use ‘rsync’ instead of ‘ditto’ or switching my script to wrap ‘mailbfr’ instead, so that I don’t have to complete a full backup every morning (of course, I’ll be rolling the backup during the compression process, after the mail service is restarted). This should improve the backup time significantly, but judging by how long it takes to run ‘du’, it’ll still be a long process.
I’m also considering installing moving the mail store and database over to a separate volume to try to reduce the amount of I/O on the drive holding it. I don’t know how much of a difference this’ll make.
I’m open for other suggestions and constructive criticism. Anybody have any experiences which might help point me in the direction of further improvements?
I’ve got a dual 2.3GHz G5 Xserve with an Apple Hardware RAID card lined up to take over mail server duty if needed, but honestly this dual 1.33GHz G4 Xserve has more than enough processing power to handle the load on it (it only rarely breaks 1.5 load average and usually only breaks 2.5 load average if it’s bzip-ing the previous night’s backups when everyone logs in in the morning.)

[morgant]

My backup drive up and failed on me, but the backups still took just as long after replacing it with a fresh (zeroed & barren) drive, so it wasn’t disk related. However, I did finally switch to using ‘mailbfr’ in my backup script and although the first backup took the usual extended period of time (as expected), the subsequent rsyncs have only been taking about 40 minutes.
A [i]much[/i] better way to go.
I still could probably move the mail store and database to a separate volume and gain a little more performance (both during regular usage and during backup, but then I’d be rolling my own rsync backup script entirely).

[peet1]

[QUOTE][u]Quote by: morgant[/u]I still could probably move the mail store and database to a separate volume and gain a little more performance (both during regular usage and during backup, but then I’d be rolling my own rsync backup script entirely).[/QUOTE]

mailbfr seems to read the cyrus config files to find the mail store. I’ve never had an issue running mailbfr on servers with the mailstore on a seperate volume. In fact, I’d be a bit more than a bit frightened to run my mail store/db on the boot volume. I’ve had to roll back to previous OS releases a few to many times to ever trust keeping ANY data on a server boot volume. Just my 2 cents.

peet

[Author]

Posts