Panther Server: Berkeley DB problems and converting to Cyrus skiplist DB format

[BryanDH]

Hi:

It seems a lot of DB corruption occurs when running Cyrus using the Berkeley DB format for the mailbox DB’s. I’ve run into situations where I have a server running normally, then after a restart of the mail service, I have all sorts of DB4 errors and panic situations. I decided to investigate how a server running fine for months can be brought to its knees just by restarting the mail services.

Yesterday I had this situation where the mail services failed to start due to DB4 errors.

According to the Cyrus documentation, Berkeley DB is a bit unstable:

“…it has proved to be somewhat unstable and prone to locking problems” ( http://acs-wiki.andrew.cmu.edu/twiki/bin/view/Cyrus/WhatDatabaseBackend )

On various mailing lists it seems that people recommend not using the DB4 format at all, and to use the Cyrus proprietary “skiplist” format, as it is less prone to corruption and performs on par with Berkeley DB.

After searching more on this I found an interesting script on Apple’s open source site:

http://www.opensource.apple.com/darwinsource/Current/CyrusIMAP-156.3/SetupExtras/upgradedb

This is the script that runs to “upgrade” the mailbox DB’s when you perform an upgrade to Tiger Server from Panther. This DB upgrade is in fact just a conversion from Berkeley DB format to Cyrus skiplist. I decided to modify the script to take out the Tiger-specific parts, and make it perform the conversion on a Panther mail server install.

After modifying the script, the conversion process went like this:

1. Stop all mail (serveradmin stop mail)

2. Make a backup of your /var/imap directory in case you need to roll it back ( I just tarred it up)

3. Run script (I pasted it below) You may need to modify the directory paths if you’ve customized anything:

#!/bin/sh
#
# Converts cyrus databases to skiplist format
# For Panther Server ONLY
# Modified from Apple’s “upgrade” script that
# is used when an upgrade to Tiger is performed
#

BACKUP=”1 2 3 4 5 6 7 8 9″
USER_DIR=”a b c d e f g h i j k l m n o p q r s t u v w x y z”
CY_PATH=”/usr/bin/cyrus/bin”
IMAPD_CONF=”/etc/imapd.conf”
IMAPD_CONF_TMP=”/etc/imapd.conf.upgrade.tmp”

###################
# Is there an imap config file

if [ ! -e “$IMAPD_CONF” ] ; then
echo “Unable to upgrade mail database due to missing config file: $IMAPD_CONF”
exit 0
fi

###################
# Get the path to the mail database and verify that it exists

DB_PATH=”`/usr/bin/grep “configdirectory” “$IMAPD_CONF” | sed ‘s/^.*://’ | sed -e ‘s/^ *//’`”

if [ ! -d “$DB_PATH” ] ; then
echo “Mail database path: $DB_PATH does not exist”
exit 0
fi

###################
# Upgrade mail databases

cd “$DB_PATH”

###################
# Delete old mail db files

if [ -d “$DB_PATH/db” ] ; then
cd “$DB_PATH/db”
/bin/rm -rf *
fi

###################
# Delete backups

for X in $BACKUP
do
if [ -d “$DB_PATH/db.backup$X” ] ; then
cd “$DB_PATH/db.backup$X”
for backup_file in *
do
/bin/rm “$backup_file”
done
fi
done

###################
# Delete deliver and tls session database files

if [ -e “$DB_PATH/deliver.db” ] ; then
/bin/rm “$DB_PATH/deliver.db”
fi

if [ -e “$DB_PATH/tls_sessions.db” ] ; then
/bin/rm “$DB_PATH/tls_sessions.db”
fi

###################
# Upgrade mailboxes.db if it exists

# First check for mailboxes.db
# If it exists set mboxlist_db key to berkeley in imapd.conf
# Then convert the mailboxes.db file form Berkeley DB to test
# Remove the mboxlist_db key from imapd.conf
# Import mailboxes into new mailboxes.db as skiplist
if [ -e “$DB_PATH/mailboxes.db” ] ; then

###################
# Set mboxlist_db key to berkeley in imapd.conf
if /usr/bin/grep “mboxlist_db” “$IMAPD_CONF” > /dev/null ; then
sed -e ‘/mboxlist_db/d’ “$IMAPD_CONF” > “$IMAPD_CONF_TMP”
if [ -e “$IMAPD_CONF_TMP” ] ; then
/bin/rm “$IMAPD_CONF”
/bin/mv “$IMAPD_CONF_TMP” “$IMAPD_CONF”
fi
fi

if ! /usr/bin/grep “mboxlist_db” “$IMAPD_CONF” > /dev/null ; then
echo “mboxlist_db: berkeley” >> “$IMAPD_CONF”
fi

###################
# Create skipstamp so that cyrus doesn’t complain
/usr/bin/sudo -u cyrus touch “$DB_PATH/db/skipstamp”

###################
# Convert mailboxes.db to text file
/usr/bin/sudo -u cyrus “$CY_PATH/ctl_mboxlist” -d > “$DB_PATH/mailboxes.txt”
/usr/bin/sudo -u cyrus /bin/mv “$DB_PATH/mailboxes.db” “$DB_PATH/mailboxes.db.old”

###################
# Remove newly created db files
if [ -d “$DB_PATH/db” ] ; then
cd “$DB_PATH/db”
/bin/rm -rf *
fi

###################
# Create skipstamp so that cyrus doesn’t complain
/usr/bin/sudo -u cyrus touch “$DB_PATH/db/skipstamp”

###################
# Remove mboxlist_db key from config file
sed -e ‘/mboxlist_db/d’ “$IMAPD_CONF” > “$IMAPD_CONF_TMP”
if [ -e “$IMAPD_CONF_TMP” ] ; then
/bin/rm “$IMAPD_CONF”
/bin/mv “$IMAPD_CONF_TMP” “$IMAPD_CONF”
fi

###################
# Set default mboxlist_db key to skiplist
echo “mboxlist_db: skiplist” >> “$IMAPD_CONF”

###################
# Import user mailboxes to new skiplist format
if [ -e “$DB_PATH/mailboxes.txt” ] ; then
/usr/bin/sudo -u cyrus “$CY_PATH/ctl_mboxlist” -u < "$DB_PATH/mailboxes.txt"
/bin/rm “$DB_PATH/mailboxes.txt”
fi
fi

###################
# Convert individual user seen.db files to skiplist
# Go to each user directory from a – z and look for .seen files and convert

for dir in $USER_DIR
do
if [ -d “$DB_PATH/user/$dir” ] ; then
cd “$DB_PATH/user/$dir”
WORKING_DIR=`pwd`
for file in `find . -name \*.seen`
do
/bin/mv “$WORKING_DIR/$file” “$WORKING_DIR/$file.old”
/usr/bin/sudo -u cyrus “$CY_PATH/cvt_cyrusdb” “$WORKING_DIR/$file.old” flat “$WORKING_DIR/$file” skiplist >/dev/null
done
fi
done

###################
# Remove seenstate_db key from config file
if [ -e “$IMAPD_CONF” ] ; then
if /usr/bin/grep “seenstate_db” “$IMAPD_CONF” > /dev/null ; then
sed -e ‘/seenstate_db/d’ “$IMAPD_CONF” > “$IMAPD_CONF_TMP”
if [ -e “$IMAPD_CONF_TMP” ] ; then
/bin/rm “$IMAPD_CONF”
/bin/mv “$IMAPD_CONF_TMP” “$IMAPD_CONF”
fi
fi

if ! /usr/bin/grep “seenstate_db” “$IMAPD_CONF” > /dev/null ; then
echo “seenstate_db: skiplist” >> “$IMAPD_CONF”
fi
fi

###################
# Add duplicate_db and tlscache_db keys to config file, setting their
# values for skiplist Added by Bryan Hill 10-4-2005
# When cyrus is restarted after this script is run, new deliver.db and
# tls_sessions.db will be created, using skiplist format

if ! /usr/bin/grep “duplicate_db” “$IMAPD_CONF” > /dev/null ; then
echo “duplicate_db: skiplist” >> “$IMAPD_CONF”
fi

if ! /usr/bin/grep “tlscache_db” “$IMAPD_CONF” > /dev/null ; then
echo “tlscache_db: skiplist” >> “$IMAPD_CONF”
fi

#End script

4. Restart mail services (serveradmin start mail) Watch your
/var/log/system.log to see if you are getting any errors.

For me, my mail server came back up fully functional with no DB errors. There should be no DB4 errors at this point, becuase DB4 isn’t even in the picture now.

I also run the sieve installation that is posted on this site. It is not affected by this DB conversion.

Hope this may help some of you,
Bryan

[Author]

Posts