Setting up the PPTP Server on Mac OS X Server 10.2

—by Joel Rennich, mactroll@afp548.com

15 October 2002—Updated 5 June 2003

A couple of words of warning before getting into this. While this feature was included by Apple, they have never listed it as part of the feature set. Most likely this means that something is broken with it. In my limited testing I have not necessarily seen that to be the case. However, we are not responsible for what happens to your machine after you engage this.

Included with Mac OS X Server 10.2 is a PPTP VPN Server. This is the most common Microsoft VPN. It is also the most problematic. However since most Windows systems of recent vintage have a client for this and now all Mac OS X 10.2 machines do it may be of use to people.

All of the software is included in the standard install, but Apple has made no mention of its presence. They did make a man page for it and include the some config files. One can only wonder why they didn’t complete their work. Hopefully, it is because they were devoting all available resources to IPSec, a VPN solution with more options and better security.

Either way, the setup is pretty straightforward.

First you’ll need to edit /etc/hostconfig to include:

VPNSERVER=-YES- 
VPN_ARGS=" " 
IPFORWARDING=-YES-

Now edit /etc/ppp/pptp_address by adding valid IP numbers for your internal network. For instance, if all of your internal devices are on the 10.0.1.x subnet than add a few empty addresses here. These will be given out to the client machines when they successfully connect.

If you don’t have any available address you can set up a NAT subnet on your server to give you more address that will work. Take a look at our article on NAT on Mac OS X Server 10.2 for more info on this.

Next comment out the first line of /etc/ppp/pptp_service by putting a “#” in front of “nodetach". Many thanks to Michael Bartosh for this bit. On older Mac OS X Server versions (I believe prior to 10.2.3), this file is /etc/ppp/pptp_configuration.

You’ll also want to list some DNS servers in this file. Find one or two that you want to use and list them like this.

ms-dns 10.0.1.10 
ms-dns 10.0.1.45

You should finish off the file with your Server’s IP address followed by a colon. This prevents a common error where pppd can’t determine your local ip address.

Now you’ll want to create a list of usernames and passwords used for your PPTP clients. I would suggest that these are either generic or at least different from your other passwords to provide the highest degree of security. A brief aside here: don’t treat having a VPN as an excuse to remove, or trivialize, your current forms of security on your network. A VPN is a great way to allow your remote users into your system, but it should not be used as a get into the server free card. Good security, like a good relationship, has many layers.

Apple didn’t include this file so you’ll have to make it.

sudo touch /etc/ppp/chap-secrets 
sudo chmod 600 /etc/ppp/chap-secrets

In this file each entry has four parts: the username, the server’s name, the password, and the IP address associated with the user. To make things easier, use the “*” wildcard for both the server name and the IP address. For example:

mactroll * 
secretpassword *

This would allow mactroll to login with “secretpassword” as the password and be assigned the next IP address available.

Now for the final set of turning on the VPN server:

sudo SystemStarter start VPNService 

Set up this way we have had numerous reports of the PPTP server working with OS X, Windows 98 through XP, and Linux systems. So, chances are good it will work for you too. To set up the client on any Mac OS X 10.2 machine, use Internet Connect to add a new PPTP VPN using your Server’s IP address and the username and password that you have configured.

If that doesn’t work here are a few things I have found in troubleshooting:

1. Try using the “Automatic” network location. PPTP requires another interface, the PPTP tunnel, to be set up. On my TiBook running Mac OS X 10.2.1 it wouldn’t do this unless I was on “Automatic.” After it was created I could rename the configuration and go from there. Annoying.
2. Make sure your network devices support PPTP passthrough. Some don't—for example, the old graphite AirPort basestations. Also you can try leaving port 1723 open on your firewall since that is used by the PPTP connection.
3. Check the logs. All errors should be logged to /var/log/system.log