Flying Racoons: IPSec, Mac OS X Server 10.2, and you 
22 August 2002
Part One
So, Vapor isn’t enough for you: you want an always-on secure, transparent to the user, and cutting-edge VPN solution for your machines. Not a problem! Mac OS X 10.2, both client and Server, has the potential do all of that for you with a minimum of effort.
Jaguar has the Kame IPv6 stack built-in. Whoa—already a few new terms in one sentence, so let’s dig into what this means. First off, the Kame project is based out of Japan, and is working on creating the best IPv6 implementation out there. IPv6 is the next generation of IP addressing. Currently we use IPv4 and that gets us address like 17.56.3.5. This is great, but the Internet is running out of numbers. Each IPv4 address has four numbers that go from 0 to 255 for a possible total of 4,294,967,296 addresses. At first this is a lot, but once you remove all of the broadcast addresses, the private addresses (like 10.x.x.x), and then consider all of the assigned but unused addresses out there and things dry up pretty quick—trust me on this one.
To get around this situation, the IPv6 standard was developed. Instead of four 8-bit numbers, you now have eight 16-bit numbers. The end result is a possible total of 2128 total addresses—3.402823669209385 x 1038 of them. This should be enough to last us a few more years. Using IPv6, your refrigerator, toaster, and microwave can all have their own addresses and you don’t have to worry about some poor kid in another country growing up without one for himself. In fact, the hope is that every person will get their own network. So instead of just getting one static IP from your DSL provider, you’ll get a whole subnet that you can do with as you wish.
The numbers look funny since they are written in hexadecimal and separated by colons. For example, 1C02:5:D250:0:0:1A:F2:75E is an IPv6 address. Suddenly, a working DNS server becomes a lot more important to navigating the Internet. There are very few people actually using these addresses just yet, but network hardware manufacturers and operating system vendors are working hard to make sure that everything will work as advertised when IPv6 becomes widespread.
While this is cool and all, what does it mean to you? Glad you asked. First off, you should keep the basic idea of IPv6 in the back of your head since it will eventually become the norm. Secondly, you can now drop the term at UNIX dinner parties and prove your level of geekiness. Finally, you can celebrate because in addition to the larger address space, IPv6 brings important new features to our current IP implementations. Of most interest to us when building VPNs is IPSec, or Internet Protocol Security.
IPSec has a few different parts. First of all, it can prove that a packet came from the person that you think it did and that it hasn’t been altered in transit. This is called the AH or Authentication Header. You create it by calculating an encrypted checksum of the packet’s contents. The sender creates this using a “pre-shared secret” or password that both the send and the receiver know. The receiver then checks the authentication header, determines its validity and then accepts or denies the packet. This does not encrypt the packet, it just proves that it came from the sender and hasn’t been tampered with.
Packet encryption is done by another part of the IPSec architecture. ESP, or Encapsulating Security Payload, uses high-security symmetric algorithms to encrypt the data. Ideally, this encryption uses passwords, or keys, that are randomly generated by the two machines and changed as often as every few minutes or less. This ensures that even if a cracker was able to break the encryption only a small part of your conversation would be available to them. You can use either AH and ESP by themselves, or most commonly, together, with either IPv4 or IPv6-style addresses on compliant operating systems.
Once the packets are secure you can use IPSec in either a simple one-machine to one-machine setup, transport mode, or a network to network setup, tunnel mode. The first is just a secure connection between two hosts, much like a simple SSH connection, except that all traffic between the two parties is secure. The second method allows you to tunnel all traffic from your network to another network—a branch office, for example—through a secure tunnel that is created between two IPSec gateways.
The fact that the IPSec implementation in Mac OS X is based on the Kame stack is important because a number of different firewall and VPN appliance manufacturers use the Kame stack on their devices. With a bit of luck, we should be able to get Mac OS X to connect to them without having to install extra software.
This has been an incredibly simplistic overview of IPSec, and IPv6 for that matter. For more information the first place you should start is the Kame Project’s Web site. If you are interested in learning even more there are a number of books on the subject available. Also, keep in mind that many of the flavors of BSD use the Kame stack, so IPSec and IPv6 tutorials for those operating systems may very well also apply to Mac OS X.
In part two we’ll show you how to create a simple machine to machine transporting IPSec VPN, and you’ll understand the title of this series.