Understanding InstallESD.dmg, Recovery HD, and Lion Internet Recovery

If you’ve done any deployment work with OS X Lion, InstallESD.dmg is probably not a new concept for you. But just in case you haven’t, or just in case you need a quick refresher, with the release of OS X Lion, Apple completely changed its distribution method for Mac OS X. OS X Lion is only available via an Internet download through the Mac App Store. And although downloaded as an application from the Mac App Store, Install Mac OS X Lion.app, contained within it is the disk image that makes the OS X Lion installer magic happen: InstallESD.dmg.

InstallESD.dmg does more than just handle the initial Lion installation. Initially contained within it is also the disk image that forms the basis of Recovery HD. Because InstallESD.dmg is available via the web, users are able to reinstall Lion through Recovery HD or Lion Internet Recovery in worst-case scenarios without needing physical restore media.

Let’s take a deeper look at InstallESD.dmg, Recovery HD, and Lion Internet Recovery to see what they are, what they mean for you as a Mac Admin, and how they’re all related.

InstallESD.dmg

After downloading Lion from the Mac App Store, Install Mac OS X Lion.app is placed in the /Applications directory. InstallESD.dmg is located within the application’s Contents/SharedSupport directories.

mikes-imac:Contents mike$ cd /Applications/Install\ Mac\ OS\ X\ Lion.app/Contents/SharedSupport/
mikes-imac:SharedSupport mike$ ls

InstallESD.dmg

       OSInstall.mpkg

InstallESD.dmg is the new-age Mac OS X retail DVD. As a Mac admin, you can use it to create an external bootable Lion install disk, build a NetInstall or NetRestore set, have it act as the Install DVD for an InstaDMG workflow, and more. A consumer installing Lion would have several pre-installation steps handled by the Install Mac OS X Lion application. After those completed, the installer would prepare InstallESD.dmg to be mounted as the boot volume. The computer would then restart and proceed with and finish the installation.

mikes-imac:SharedSupport mike$ hdiutil attach InstallESD.dmg
mikes-imac:SharedSupport mike$ cd /Volumes/Mac\ OS\ X\ Install\ ESD/
mikes-imac:Mac OS X Install ESD mike$ ls
BaseSystem.chunklist        MacOSX_Media_Background.png kernelcache

BaseSystem.dmg

           Packages            mach_kernel

Install Mac OS X Lion.app

    System              private
Library             boot.efi            usr

Looking inside of InstallESD.dmg, take note of two important files: Install Mac OS X Lion.app and BaseSystem.dmg. Yes, that is another Install Mac OS X Lion.app. That installer runs when booted from InstallESD.dmg. It is what actually installs Lion to the hard drive. BaseSystem.dmg is copied to the Recovery HD partition after it’s created during the Lion installation process. Let’s take a look at Recovery HD.

Recovery HD

As a Mac admin, you can rejoice in the fact that the days of keeping stacks of hardware-specific restore discs to reinstall Mac OS X are gone. Recovery HD is created during the Lion installation process as a very small hidden partition on the primary boot drive.

mikes-imac:~ mike$ diskutil list
/dev/disk0
   #:                         TYPE NAME            SIZE       IDENTIFIER
   0:        GUID_partition_scheme                *1.0 TB     disk0
   1:                          EFI                 209.7 MB   disk0s1
   2:                    Apple_HFS Macintosh HD    999.3 GB   disk0s2

   3:                   Apple_Boot Recovery HD     650.0 MB   disk0s3

As mentioned earlier, BaseSystem.dmg can be found on Recovery HD, as shown below. Newer Mac models that support diskless Apple Hardware Test may also have a disk image for it contained within a hidden .diagnostics folder inside of the com.apple.recovery.boot folder.

mikes-imac:~ mike$ diskutil mount /dev/disk0s3
Volume Recovery HD on /dev/disk0s3 mounted
mikes-imac:~ mike$ cd /Volumes/Recovery\ HD/com.apple.recovery.boot/
mikes-imac:com.apple.recovery.boot mike$ ls
BaseSystem.chunklist    PlatformSupport.plist   boot.efi        kernelcache

BaseSystem.dmg

       SystemVersion.plist

   com.apple.Boot.plist

When booted into Recovery HD, BaseSystem.dmg is mounted as the boot volume with the volume name “Mac OS X Base System”. This is shown in the com.apple.Boot.plist file (“rp” stands for root path).

mikes-imac:com.apple.recovery.boot mike$ /usr/libexec/plistbuddy ./com.apple.Boot.plist -c "print:Kernel\ Flags"
rp=file:///com.apple.recovery.boot/BaseSystem.dmg

The contents of BaseSystem.dmg are shown below.

mikes-imac:Mac OS X Install ESD mike$ hdiutil attach BaseSystem.dmg
mikes-imac:Mac OS X Install ESD mike$ cd /Volumes/Mac\ OS\ X\ Base\ System/
mikes-imac:Mac OS X Base System mike$ ls
Applications    Library     Volumes     dev
private     tmp     var

       Install Mac OS X Lion.app

System        bin     etc     sbin                usr

In case you’re wondering, yes that is yet another Install Mac OS X Lion.app. That one runs when choosing to reinstall Lion when booted into Recovery HD. If you’re super clever, you might wish to customize the “Mac OS X Utilities” screen that is displayed when booted into Recovery HD (like the folks at Google have done). I’ll leave that to the true tinkerers, but taking a peek at the following application should get you started:

/Volumes/Mac\ OS\ X\ Base\ System/System/Installation/CDIS/Mac\ OS\ X\ Utilities.app

Because of Recovery HD’s small size, there is not enough room to store a copy of InstallESD.dmg within it. But thanks to its Internet availability, that’s not a problem. After authenticating with Apple and the iTunes store, the installer downloads a fresh copy of InstallESD.dmg disguised as a package. After download, the installer prepares InstallESD.dmg to be mounted as the boot volume, restarts the computer, and then continues with and finishes the installation.

It’s important to note that DHCP must be available for either a Recovery HD reinstallation of Lion, or a Lion Internet Recovery boot to function. Using Wireshark, I observed the process of reinstalling Lion through Recovery HD. There are far too many steps involved to list them all, but below are the ones I found to be most important. Note that I’m sure it’s possible that the random package names shown below can change, but they did remain consistent throughout my tests. It’s also safe to assume that the mirror names will vary by location, as they are Akamai mirrors. None of the following is officially documented by Apple; the implementation and details are subject to change at any time.

• After getting a DHCP address, osrecovery.apple.com is where it all begins. HTTP GET and POST requests are made to give the Mac a valid session cookie for the process. Some information about the Mac is sent back to Apple, including a model identifier and what appears to be a modified or encrypted version of the serial number.

• An HTTP GET request is made to a1166.phobos.apple.com for mzm.hgbvjzlz.pfpkg. This compressed package is what is responsible for performing the pre-installation checks to make sure the Mac meets the minimum system requirements for Lion. One of the more interesting pieces inside this package, and what allows Lion to be installed in a virtual machine, is this:

  function isSupportedPlatform(){

        if( isVirtualMachine() ){
                return true;
        }

• An HTTP GET request is made to swscan.apple.com for Apple’s software update catalog.

• Various types of requests are made to iTunes (usually to ax.init.itunes.apple.com) to authenticate the user and ensure that Lion has been previously purchased. If it has not been previously purchased, the user won’t be eligible to reinstall through Recovery HD or Lion Internet Recovery.

• An HTTP GET request is made to a774.phobos.apple.com for MacOS_10_7_IncompatibleAppList.pkg. This package checks the system for any applications that are incompatible with Lion and moves them to an Incompatible Software folder during the installation. See http://support.apple.com/kb/HT3258 for more information.

• Finally, an HTTP GET request is made to a1166.phobos.apple.com for mzm.ajzbytae.pkg. I was unable to open this package after it downloaded, but can safely assume it is InstallESD.dmg. It is a 4.18 GB “package” and downloads during the installation step where the installer downloads “additional components” just before restarting.

I encourage you to packet capture the process yourself, whether it be through using a VM at home and sniffing your LAN traffic, or by using port mirroring on a switch. It’s interesting to observe just how many steps are involved behind the scenes. But what is even more interesting, is how Apple has created what appears to be globally available NetBoot over the Internet with Lion Internet Recovery.

Lion Internet Recovery

If for any reason Recovery HD isn’t available or otherwise becomes corrupted, Apple created Lion Internet Recovery as a last resort for users. It’s an EFI firmware function available on most Macs shipped in 2010 or later. It enables users to boot over the Internet to Apple’s servers which will eventually present Recovery HD through a downloaded copy of a BaseSystem.dmg equivalent.

Here’s a riddle: What looks like NetBoot, smells like NetBoot, even tastes a bit like NetBoot, but isn’t actually NetBoot? Lion Internet Recovery. Or rather, it’s not traditional NetBoot with BSDP, TFTP, etc. Almost all Lion Internet Recovery traffic is HTTP traffic.

Just like with Recovery HD, there are many steps involved in booting to Lion Internet Recovery, but below are the most important. None of the following is officially documented by Apple; the implementation and details are subject to change at any time.

• After getting a DHCP address, the process again begins with osrecovery.apple.com. An HTTP GET request with an HTTP User-Agent header of “InternetRecovery” is made to osrecovery.apple.com

• osrecovery.apple.com responds with an HTTP OK giving the Mac a valid session cookie

• An HTTP POST request is made to osrecovery.apple.com/InstallationPayload/RecoveryImage. Some information about the Mac is sent back to Apple, including a model identifier and what appears to be a modified or encrypted version of the serial number, just like with a Recovery HD boot.

• osrecovery.apple.com/InstallationPayload/RecoveryImage replies with an HTTP OK passing additional information back to the Mac, such as where to actually download the Recovery Image from which it will boot.

• An HTTP GET request is made to oscdn.apple.com for RecoveryImage.chunklist. From what I can tell, this file is a checksum of sorts for the recovery image. The actual request for the image, detailed in the next step, uses HTTP range headers. I assume that comparing what has already been downloaded to the contents of the chunklist allows the Mac to only download the parts of the image it requires. This was most likely implemented to handle any network disruptions that might occur during a Lion Internet Recovery boot.

• Finally, an HTTP GET request is made to oscdn.apple.com for the RecoveryImage itself, appropriately named RecoveryImage.dmg. With Lion Internet Recovery, RecoveryImage.dmg is synonymous with BaseSystem.dmg.

Once booted to Lion Internet Recovery, the steps the installer performs to reinstall Lion are identical to those mentioned earlier that occur during a standard Recovery HD initiated reinstallation.

Lion Internet Recovery is an impressive feat of engineering. Again, I would strongly encourage you to packet capture the process to see how it works for yourself. If you’d like to see the full packet capture output from my Lion Internet Recovery boot, it is available here.

Putting It All Together

InstallESD.dmg, contained within the Mac App Store downloadable installer, is at the heart of Recovery HD and Lion Internet Recovery. Recovery HD is there when your users need it, and Lion Internet Recovery is there as a last resort just in case Recovery HD is not. Both rely on BaseSystem.dmg (or RecoveryImage.dmg in the case of Lion Internet Recovery) to provide them with a minimal interface with a few utilities and the ability to reinstall Lion. When choosing to reinstall Lion, either from Recovery HD or Lion Internet Recovery, the installer downloads the latest copy of InstallESD.dmg from the Internet in order to proceed with and finish the reinstallation.

Hopefully after reading this article, the Lion installation and recovery processes have become clearer. Understanding the relationships between them and InstallESD.dmg is critical for deploying Lion and understanding the recovery options available to your users. Hopefully they won’t need to use Recovery HD nor Lion Internet Recovery nearly as many times as I needed to in order to write this article. Fortunately for me, my ISP doesn’t have a bandwidth cap.