Mail migration 10.4 to 10.6 with cyrus2dovecot

[mosen]

Hi guys,

I’m planning to migrate my mail from an old 10.4 tiger server box to a new snow leopard server. I’ve read that the best
script to convert cyrus to dovecot mailboxes is cyrus2dovecot.

The problem now is that cyrus stores each dir under the short username and dovecot seems to store each mailbox under a directory named after the
OD GeneratedUID attribute. I was wondering if anyone has already written a shell script to do a straight conversion from cyrus boxes to dovecot, which automatically
looks up the GeneratedUID and places the mailboxes into the correct folders?

Also, if there is another preferred method let me know.

[mosen]

Ok im not that patient 🙂 so I will write a shell script which employs cyrus2dovecot to convert cyrus mail to dovecot with Open Directory lookup of GUID.
Will post here when i have something to test out.

[mosen]

My mail migration was successful and the whole place is running on snow leopard server w/dovecot.

The steps we took were:

1. To avoid a massive amount of downtime when copying mail from server1 to server2 we used a modified mailBfr script (so that it would accept remote hosts) as a cron job every night, to ensure that when the migration happened, the mail spool would already exist on the target server (server2).
2. I ran a script which ran cyrus2dovecot.pl. This is explained by the cyrus2dovecot documentation.. it basically loops through the user directories and then launches cyrus2dovecot to convert each. The output of this went to a directory called dovecot_temp
3. I made up a script which loops through each user directory in dovecot_temp and mv (renames) each directory to the associated GeneratedUID attribute of the relevant open directory user. Basically it uses ldapsearch to get the GeneratedUID and then renames the folder. It also corrects the permissions to the ones dovecot expects. Note: like me, you might have accounts that were deleted, but their mail spool remains. Just letting you know.
4. The GUID folders were moved into the storage folder for dovecot, for me this is on an external raid volume. Mail is started and hey presto.. I had working user accounts (not often that anything goes this smoothly.. but i did do a fair bit of testing 🙂

If you need detail on any of this, go ahead and ask.. I’ll try to help as much as i can.

mo.

[mosen]

There has been a few E-mails directed my way regarding the script source and
more of the process. So I will try to reiterate and remember as much as I can.

[b]10.4 or 10.5 cyrus imap to 10.6 dovecot migration[/b]

1. Back up the cyrus mail store, configuration, seen files etc. SERIOUSLY (you can use the nice mailBfr package to do this from [url]http://osx.topicdesk.com/[/url]. Personally I modified the mailBfr script to allow remote rsync locations so that there was less downtime on the switch from server1 to server2. (There’s still downtime though).

2. Install cyrus2dovecot. I believe there is a .pkg now 🙂

3. Create a script (I may supply one if i can figure out how 🙂 ) to convert the cyrus mail store + seen + subscription files to dovecot format. I used an example from the cyrus2dovecot documentation page: [url]http://www.cyrus2dovecot.sw.fu-berlin.de/documentation.html[/url]. I pretty much took the example script verbatim and adjusted it for my mail spool locations. I also made the output directory something like dovecot_temp, because there is another step before the mail will be recognised by dovecot in snow leopard.

4. Run the script. It will convert the cyrus store to a dovecot store. It preserves flags pretty well, i.e you won’t be left with inboxes full of unread mail etc. You do need to specify the quota again. For more information on the caveats of conversion, consult the documentation for cyrus2dovecot.

Now that you have a freshly converted dovecot mail store, snow leopard will be able to read the mail.. but it will expect the folder to be a different name than before. Instead of using the short username “jbloggs”, it will look up the GeneratedUID attribute of the user record from Open Directory (local or not). So we need a way to rename each user “jbloggs” to the GeneratedUID – something like E1126AC8-EAEF-11D8-855B-000A95C4234E/

I created a script for this purpose. It uses dscl /Search to use the available bound directories to look up the user account. Your setup may be different / mileage may vary / etc.
Please forgive me if you live in shell scripting, or you have a shebang tattoo, or if you thought the bourne identity was about shell process IDs, I only touch on bash scripting occasionally!

[b]guidrename.sh[/b]
[code]
#!/bin/bash

# Directory containing Maildirs output by cyrus2dovecot.pl
SOURCE_PARTITION_DIR=/Volumes/Mail/dovecot_temp
DEST_PARTITION_DIR=/Volumes/Mail/dovecot_guid

# Iterate through source directory with mailboxes and rename them
function process_directories ()
{
SOURCE_DIR_LIST=”$(ls -1 $SOURCE_PARTITION_DIR)”

# Only break on newlines or line breaks, to avoid interpreting dirs with spaces.
IFS=`echo -en “\n\b”`

for d in `ls -1 $SOURCE_PARTITION_DIR`; do

DS_RECORD=`dscl “/Search” -read “Users/”$d GeneratedUID`

if [ $? -eq 56 ]; then
echo “Could not find GeneratedUID for user ‘$d’, skipping..”
else
echo -n “$d -> ”
GUID=`echo -n $DS_RECORD |sed -e “s/[^:]*: //g”`
echo $GUID
mv “$SOURCE_PARTITION_DIR/$d/Maildir” “$DEST_PARTITION_DIR/$GUID”
chown -R $d “$DEST_PARTITION_DIR/$GUID”
chgrp -R mail “$DEST_PARTITION_DIR/$GUID”
chmod 0700 “$DEST_PARTITION_DIR/$GUID”
fi
done
}

process_directories
[/code]

[i]
Note: My source dir is $d/Maildir because in the conversion step i used a destination of $d/Maildir. I thought this was necessary at the time.. turns out you can put the maildir storage inside the user folder itself and it doesn’t matter. If your conversion script is a copy of the cyrus2dovecot one you will want to include the sub-directory Maildir.[/i]

5. Run the rename script against the already converted dovecot mail. You should end up with a folder full of GUID folders. If you deleted a user from open directory in 10.5 or 10.4 – it removed the user but not the mail. This is why you will see an error trying to rename that users folder – the user does not exist anymore but the mail does. If you want to rescue the persons mail, make a user for them or import the mail some other way. I decided to back up the old people on tape and then delete them.

Congratulations, your mail store folder should be ready for dovecot/snow leopard server. You should relocate it to its final place, and then using Server Admin, configure the mail store location accordingly. Start up the mail service and test a few accounts.

mo.

[Author]

Posts