Leopard/ Snow Leopard Software Only IPSec VPN Server

[cralhr]

Hello,
I recently decided to install OS X Server (Leopard) on a MacMini on for a small startup that I am a part of. We previously used a Linux server for all of are needs, including IPSec VPN server.

On Linux, I was able to use the guide here: http://www.netbsd.org/docs/network/ipsec/rasvpn.html with some minor tweaks to create a Cisco client compatible IPSec VPN server.

I want to try to recreate that setup on the Mac Mini, but haven’t been able to succeed so far. I can’t start racoon directly (without vpnd) as I get the following error: ERROR: failed to initialize isakmp.

I don’t want to use L2TP as some clients block that type of VPN on their network (but allow IPSec) and I’d like to let the group use the profiles I already pushed out for Cisco client on the iPhones and iPads for the previous server.

Any tips on whether an IPSec only VPN is possible on OS X?

[cralhr]

I’ve been able to get the whole thing almost entirely configured, but I may have run into a serious dead end. In order to provide support for Cisco VPN clients, I need to use XAuth. For my test, I’ve decided to go with the less secure xauth_psk_server.

The only mode-cfg auth_source option supported by the racoon build provided in OS X is system. OS X doesn’t use the standard /etc/passwd database that racoon will probably look for when system is set, so no matter what I do, I can’t get past the XAuth step of the connection.

A few questions come to mind:
1. Is my assumption about where racoon on OS X is looking for the system auth_source valid?
2. If so, is there any way to get a new user into there (vs the local OpenDirectory) so that I can get past that authentication step?
3. If not, what are the groups thoughts on trying to rebuild racoon from the Darwin source with pam support.
The idea here is that with pam.d support, I could create a pam config file for racoon and use the auth_source pam option to point racoon to pam for the xauth step. Hopefully, this would allow me to plug into the OpenDirectory and take advantage of the users already setup on the server.

Here’s the racoon.conf for the curious:

path pre_shared_key “/etc/racoon/psk.txt”;

listen {
# adminsock disabled;
}

remote anonymous {
exchange_mode aggressive;
proposal_check obey;
generate_policy on; # automatically generate IPsec policies
nat_traversal force; # always use NAT-T
ike_frag on; # use IKE fragmentation
proposal {
encryption_algorithm aes;
hash_algorithm md5;
authentication_method xauth_psk_server;
dh_group 2;
}
}
mode_cfg {
network4 10.22.24.2; # 1st address of VPN IPv4 pool
pool_size 252; # size of the VPN IP pool: 253 addresses
auth_source system; # validate logins against /etc/passwd
dns4 10.11.12.1; # IPv4 DNS server
wins4 10.11.12.1; # IPv4 WINS server
}

sainfo anonymous {
encryption_algorithm aes, 3des;
authentication_algorithm hmac_md5, hmac_sha1;
compression_algorithm deflate;
}

[Author]

Posts