Upgrading from Mac OS X Server 10.2.8 to 10.3.3
17 March 2004—updated 19 March 2004
This article is more of a personal journey than a technical article; however, I hope that admins out there who have been eyeing Panther Server with lust in their hearts will find some of my experiences helpful.
Note: I started this article right after upgrading from 10.2.8, but due to disastrous problems with a client’s 10.3.2 Server, got sidetracked. Now that 10.3.3 is out, I’m hoping to be able to finish before any new disasters pop up.
The Server in question is a trusty dual-processor 450 MHz G4 with 896 MB of RAM. The Server offers Web, Mail, FTP, AFP, and occasionally Windows services. Prior to the acquisition of a test server in 2003, the Server was also the target of a fair number of upgrades from Apple’s defaults. Among the services upgraded were mail (replacing Apple Mail Service with Postfix and UW IMAP/POP), PHP, and MySQL. Fink was added as well, and a number of basic UNIX tools were improved over time.
It is to Apple’s great credit that they ship Mac OS X Server 10.3.3 with reasonably recent versions of major software, including a LOT of underlying libraries and packages that weren’t available for Mac OS X 10.2. In some cases, this meant I didn’t need to do anything after upgrading. In others—mainly mail-related for now—the Apple-supplied software still doesn’t function to the level that I need.
Before you leap…
Every software manufacturer tells you to backup before installing. So am I :-). If you have a backup solution in place now, use it and make a complete backup of your system. Don’t have a backup solution? Shame on you! Put this at the top of your “to-do” list.
At the very least, you should backup your User home folders; your Apache configuration; your MySQL databases (if any); and, if you use Postfix already, your Postfix config files. If you have turned PHP on—and especially if you replaced Apple’s seriously-outdated version—back up your php.ini file.
Next, if you have any customers who rely on your server, make sure to tell them in advance (give them a few days!) that you will be down for an upgrade. If possible, set up a backup MX host for e-mail and failover hosts for your Web sites. The Web sites will probably be less of a problem, since there is less to worry about on that front.
On the Day of the Deed
When Upgrade Day arrives, marshall the backup media and approach the Server with confidence. This won’t hurt much, really!
Make sure any file sharing users are disconnected, then shut down mail service and let the queues drain. Go ahead and insert CD one and start the upgrade process. Remember to install XCode when you are done with the CDs—XCode is required for UNIX tool installations! I’d recommend going ahead and telling the Server Setup tool which services you plan to use and have them turn on automatically—then turn them off the first thing you do upon logging in to your snazzy new 10.3 Server.
Don’t run Software Update until you have installed XCode. Once the developer tools are in place, you can go ahead and run Software Update to get the updates Apple has released to date.
Fixing the Wobbly Bits
Once you’re all caught up with the Software Updates, you’ll want to go fix any broken bits. The following list elaborates on the basic fixes I had to deal with:
PHP: If you upgraded Mac OS X Server’s PHP installation under Mac OS X Server 10.2.x, chances are Server Admin will have a malformed entry in httpd.conf. The easiest fix is to grab a copy of Marc Liyange’s PHP distribution. Marc’s version has virtually every PHP add-in out there, and is head and shoulders above Apple’s distribution—even though the default distribution in Mac OS X Server 10.3 is quite recent and far more functional than the version available in Mac OS X Server 10.2. If you don’t absolutely need the wide breadth of functions that Marc’s module provides, however, stick with Apple’s. While two of the Servers I have upgraded work just fine with Marc’s module, my main Server’s Apache goes into a panicky tailspin whenever I try to use it.
Also, if you just purchased or downloaded Zend’s Server product, BEWARE! Their installation script will completely hose your httpd.conf file, erasing all of your virtual hosts and mucking around with other settings. Undoing their mess took more hours than I care to remember.
Mailman: If you installed Mailman under Mac OS X Server 10.2, it’s possible that your installation directory wasn’t /var/mailman and that your lists won’t be immediately recognized in Server Admin. The simplest fix to symlink your existing Mailman installation to /var/mailman, then run /var/mailman/bin/check_perms -f a few times, until no problems are reported. Now, if you did install Mailman a while ago, chances are the uid for your Mailman won’t match that of the one Apple installs, so you may have to do some recursive chowning as well. Don’t fret, it won’t take long and once you are done Server Admin will pick up all your lists (though you will still want to use the Web interface to really control the lists; Server Admin’s GUI is as barebones as they come).
Note: When you apply the Mac OS X Server 10.3.2 -> Mac OS X Server 10.3.3 update, Apple shuts down Mailman but doesn’t properly restart it. To get Mailman running again, issue the command /var/mailman/bin/mailmanctl -s restart as root. From that point on, Mailman will run normally.
Retrospect: Retrospect isn’t part of Mac OS X Server, but (for better or worse) it is the ‘best’ backup solution available for Mac OS X, despite its idiosyncracies. Restrospect 5.0 is extremely unreliable under Mac OS X 10.3, so it’s worth your while to fork over the big bucks to Dantz to upgrade to Retrospect 6.0—especially if you have to restore any files from your 10.2 Server installation!
MySQL: This isn’t a wobbly bit; Apple’s default distribtuion in Mac OS X Server 10.3 is 4.0.16; as of writing, MySQL AB’s official release is version 4.0.18. Not a big deal, I suspect (though I am hardly a MySQL guru, so I could be wrong). In any event, if you’re not satisfied with Apple’s release version, MySQL AB claims that 4.0.18 will install on Mac OS X “Jaguar” or higher. (Note: Mac OS X Server 10.3.3 updates MySQL to 4.0.18. Bless you, Apple.)
Note: if you had upgraded MySQL under 10.2, chances are your MySQL databases aren’t where MySQL expects them to be (i.e., they will be in /usr/local/mysql/data, instead of /var/mysql). Make sure to check the locations, and symlink as needed.
Users: For some reason, Apple decided to stop allowing hyphens in usernames (“short names”) in Mac OS X Server 10.3. If you have established usernames with dashes, you will have to make new ones with ‘master’ usernames without hyphens, and give them secondary usernames with them. Why this change? Beats me!
Wailing and Gnashing—Over Mail
One of the biggest complaints I had about Mac OS X Server was Apple’s implementation of Mail Service. Apple Mail Service was … well, barely passably usable if you had a very basic setup. That’s why one of the first things I did when I switched from Linux to Mac OS X Server was replace Apple Mail Service with a stable, powerful Open Source e-mail server: Postfix.
Now, I’m not saying that my rhapsodising about Postfix on this site influenced Apple, but with Mac OS X Server 10.3, they got even more Open Source religion and replaced Apple Mail Service … with Postfix! For IMAP (and POP3) services, they decided to use Cyrus (more on that later). Unfortunately, what we got is lacking.
Apple includes Postfix 2.0.10 with Mac OS X Server 10.3. The current official release is 2.0.19; while normally minor version numbers would mean very little difference, with Postfix there are significant changes. One of these important changes is that as of 2.0.13-20030704, Postfix permits the use of CIDR network/netmask map types in access map files. In case this doesn’t get you excited, it means that you can finally reject spam using netmasks (e.g., 200.0.0.0/8 will cover the entire LACNIC range) instead of being limited to /24, /16, and /8s. Trust me, this enhancement alone makes blocking spam a LOT easier.
The problem? If you were already using Postfix more recent than 2.0.10 when you upgrade Mac OS X Server, you immediately lose all access to CIDR mapping. And it’s silent—if Postfix sees CIDR entries in your access maps, it will just ignore those lines.
Bizarrely, though, Apple includes support for CIDR mapping in the Server Admin GUI! You can enter CIDR maps in the GUI and they will be saved to /etc/postfix/smtpdreject, the one and only file the GUI uses for IP-based rejections. What will make you want to rip out your hair is that if you make changes to that file through the command line, they will never get picked up. The GUI won’t show the new lines, and Postfix doesn’t seem to be able to wotk with them, either. This completely breaks the way Postfix is supposed to work. The only way to use CIDR maps is to use the GUI.
So why not use the GUI? Because Apple barely even touches the surface of Postfix‘ power through the GUI. And if you make changes to main.cf and then use the GUI, you’re likely to find yourself with a radically different file after you click “Save.”
What you should watch out for
Procmail. If you want to use procmail for pre-sorting e-mail, or using advanced spam and virus detection, you’ll want to set mailbox_transport = procmail in main.cf, and include a new procmail transport line in master.cf. Unfortunately, if you make any configuration changes in the GUI and click “Save,” Apple will overwrite your line and replace it with mailbox_transport = cyrus. To overcome this unfortunate decision, you will need to be vigilant (or don’t use the GUI).
Deprecation. Apple allows you to specify DNSBL hosts to query against in the GUI. Unfortunately, they use the deprecated “maps_rbl_domains” setting to specify the servers, so a warning will get logged when the checks occur.
Minimal Spam Control. Making matters worse (prepare to rip out your hair here), if you make changes to the DNSBL hosts using the GUI, Apple will wipe out your smtpd_recipient_restrictions and replace it with their simplistic version. Yes, this means that you will lose all of the fine-tuned, non-lookup-based anti-spam controls that Postfix offers—unless you don’t use the GUI. Apple’s GUI doesn’t provide any means for filtering on domains, addresses, header matches, or body matches. Luckily, header and body matches are controlled using settings that the GUI doesn’t overwrite, so while you will lose most of Postfix’ tools when you click the “Save” button, you’ll still have header and body checks (if you use them—both operate on outgoing mail as well as incoming, so you may want to consider their ultimate utility. Here, I mainly use body checks to stop the piles of Windows worms and virii by raw base-64 text matching.)
Disconnection. Although Server Admin does seem to reflect some settings from main.cf in the GUI, it doesn’t respect or reflect the contents of /etc/postfix/smtpdreject or show any changes you manually make to maps_rbl_domains (including if you remove the directive altogether!). What does this mean? It means that you can’t trust what the GUI says or what main.cf says.
Suspicious Changes. Apple includes a couple of directives in main.cf that aren’t in the official Postfix lexicon. This implies that Apple has changed the source they used to build Postfix, but do not include anywhere in the Server distribution. The disconnection between the GUI and main.cf backs this suspicion up as well. As yet, I am not even sure if it is now possible to build Postfix from official source and have it work with Mac OS X Server. Although it should, I am not enough of an SSL/OpenDirectory person to know if it really will, and haven’t had time to build a newer distribution on my test Server. If anyone has experience with this situation, please let me know!
Cyrus
Apple decided to use the Cyrus IMAP/POP3 suite with Mac OS X Server 10.3. Cyrus is a robust system used by organizations with tens of thousands of users. That is good. Unfortunately, Apple doesn’t appear to have quite finished Cyrus’ setup. The cyradm tool, the ‘standard’ way of controlling Cyrus, isn’t enabled. SIEVE, the built-in server-side filtering system, isn’t working. And at least on my Servers, you can’t actually use the /usr/bin/cyrus/bin/reconstruct tool as outlined in the Mail Service guide. Apple has chosen a few settings in imapd.conf that may cause confusion, as well (for example, the default mailbox delimiter is ‘.’ while Apple chose ‘/’—perfectly within spec, but not what most administrators will expect).
Going Forward
Despite my criticisms above, I’m generally pleased with Mac OS X Server 10.3. It is definitely an improvement over Mac OS X Server 10.2, and shows that Apple’s committment to Open Source software isn’t just smoke and mirrors. I like that the bash shell is now available, and that a lot of work has gone into providing ways to administer the server using only the command line. I love having color-xterm Terminals. Hopefully, as Mac OS X Server matures through 10.3.4 and on to 10.4.0, Apple will continue to update the software they supply; perhaps someday, we’ll see services like Postfix and Cyrus as up to date and complete as MySQL!
Maybe they’ll even give us manuals some day.