
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi All,
After various attempts I managed to get Apple Mac's native VPN client to connect to a Netvanta 3120. Apple are using their own fork of racoon to manage IKE negotiation, but it will not work out of the box by just using the Network Preferences VPN GUI, without creating a separate configuration file for it. I tried this on an Apple MacBook Pro running OSX El Capitan v.10.11.6. Other recent OSX versions should work too.
Some of my findings and conclusions:
- The native Apple Mac 'Cisco IPSec' VPN client requires XAUTH. Attempting to connect without XAUTH is a hit and miss affair for IKE Phase 1. Even if Phase 1 completes, IPSec Phase 2 always fails.
- The Apple Mac client asks Netvanta for MODE_CONFIG data. Even if you modify its racoon.conf file by setting 'mode_cfg off;', this client setting appears to have been hard coded by Apple and will still ask for MODE_CONFIG information from the router.
- The Apple Mac's Network Preferences GUI does not provide sufficient settings to allow you to configure a connection with the Netvanta. You will have to create a separate racoon configuration file with your settings and add an include directive in Apple's default /etc/racoon/racoon.conf file, to make sure the racoon client reads your modified configuration and executes it.
- You will also have to create an ipsec-tools.conf file with the required SA selectors and run this file manually as a script from a terminal, because Apple's racoon client will not pick it up and use it.
- When it connects, racoon by default sets up a full VPN tunnel, with all and any connections from the MackBook directed through the tunnel to Netvanta. Unless you configure the Netvanta's firewall to forward VPN packets out through its WAN port, you will only be able to connect to PCs within Netvanta's LAN. In the example configuration below I offer a solution for creating a split VPN tunnel, so connections to the Internet from the MacBook do not go through the VPN tunnel, but via the local router.
- The whole process of setting up the MacBook and getting it to connect is a bit of a chore, so 3rd party VPN clients may be an easier bet, if you do not have the patience to get this going.
STEPS TO GET A VPN CONNECTION GOING BETWEEN APPLE MAC AND NETVANTA
The main steps to get a VPN connection going are as follows:
- Configure your Netvanta's VPN.
- Configure Apple Macs' 'Cisco IPSec' VPN client GUI.
- Run a script to set up Security Policies on the Apple Mac.
- Connect using the Apple VPN GUI.
- Ping the Netvanta to confirm connectivity.
- Set up routes to implement a split VPN tunnel (optional).
NETWORK TOPOLOGY
MacBook===========Router=======(INTERNET)=======Netvanta-----------LAN_SUBNET
192.168.1.8 RST.UVW.XY.Z ABC.DEF.GH.IJK 10.10.10.0/24
1. NETVANTA CONFIGURATION
The configuration below shows only the VPN and XAUTH specific settings:
!
service password-encryption
!
username "admin" password encrypted "secret_admin_passwd"
username "macbookpro" password encrypted "xauth_macbookpro_passwd"
!
!
ip crypto
!
crypto ike client configuration pool Netvanta_VPN_modconfig
ip-range 172.16.5.1 172.16.5.254
dns-server 10.10.10.1
!
crypto ike policy 100
no initiate
respond main
local-id address ABC.DEF.GH.IJK
peer any
client authentication server list LoginUseLocalUsers
client configuration pool Netvanta_VPN_modconfig
attribute 3
encryption aes-256-cbc
authentication rsa-sig
group 5
lifetime 7080
!
crypto ike remote-id fqdn macbook_VPN ike-policy 100 crypto map VPN 10
!
ip crypto ipsec transform-set esp-aes-256-cbc-esp-sha-hmac esp-aes-256-cbc esp-sha-hmac
mode tunnel
!
ip crypto map VPN 10 ipsec-ike
description VPN_IPSec_Access
match address ip VPN-10-vpn-selectors
set transform-set esp-aes-256-cbc-esp-sha-hmac
set security-association lifetime seconds 3600
set pfs group5
ike-policy 100
mobile
!
crypto ca profile "VPN SSL CA"
ip-address ABC.DEF.GH.IJK
subject-name "CN=3120_VPN; OU=VPN Gateway; O=VPN; C=US; ST=TN"
fqdn 3120_VPN
!
crypto ca certificate chain "VPN SSL CA"
[Certificate details follow ... ]
!
!
ip access-list extended VPN-10-vpn-selectors
permit ip 10.10.10.0 0.0.0.255 172.16.5.0 0.0.0.255
deny ip any any log
!
!
ip policy-class Private
allow list VPN-10-vpn-selectors stateless
[snip ...]
!
!
ip policy-class Public
allow reverse list VPN-10-vpn-selectors stateless
[snip ...]
NOTE: I created the SSL CA, Netvanta and MacBook client certificates using OpenSSL. It is important to add to the Netvanta certificate the IP address and/or its FQDN in the subjectAltName field, because Apple Mac reads those to determine the remote peer. The Apple Mac Client SSL Certificate should also contain its FQDN in the subjectAltName field.
2. CONFIGURE APPLE MAC's VPN CLIENT
2.1 Network Preferences GUI
To configure the Apple Mac client, go to its Network Preferences GUI and create a new service, selecting from the drop down options:
Interface: VPN
VPN Type: Cisco IPSec
Server Address: ABC.DEF.GH.IJK (use your Netvanta's public IP address)
Account Name: macbookpro (use your Netvanta's XAUTH details here)
Password: xauth_macbookpro_passwd
2.2 Authentication Settings
Select your client SSL certificate, after you import it into the "System" keychain, using MacBook's 'Keychain Access' application. Configure your private key preferences, using the 'Keychain Access' GUI, so that it can accessed by /usr/sbin/racoon. Also make sure you import the CA Certificate and edit its Trust settings to mark it as trusted.
If for some reason you do not want to use certificates, the proposed set up should also work with PSK, but SSL certificates are arguably more secure and in VPN Main Mode allow you to use FQDN, or USER_FQDN, etc., as the client's peer ID, rather than its public IP address; which on a laptop is likely to change regularly.
2.3 Racoon Configuration Files
Configure Apple Mac's racoon configuration next. Open a terminal and type:
sudo mkdir /etc/racoon/remote
to create a new directory for storing your own racoon configuration. Then type:
sudo cat /var/run/racoon/ABC.DEF.GH.IJK.conf > /etc/racoon/remote/ABC.DEF.GH.IJK.conf
but do not run this syntax yet. Start the VPN connection for racoon to generate the dynamic content of file /var/run/racoon/ABC.DEF.GH.IJK.conf and while it is trying to connect (you only have a few seconds for this) press the enter key to run the above syntax already entered into the command line, to capture this file's content. Then go to /etc/racoon/remote/ABC.DEF.GH.IJK.conf and edit it as is appropriate for your connection. I give a working example below, which I arrived at after some trial and error:
remote ABC.DEF.GH.IJK {
doi ipsec_doi;
situation identity_only;
exchange_mode main;
verify_identifier on;
peers_identifier address ABC.DEF.GH.IJK;
my_identifier fqdn "macbook_VPN";
certificate_type x509 in_keychain "...Long_string_for_SSL_cert...";
verify_cert on;
certificate_verification sec_framework use_peers_identifier;
#local_address 192.168.1.8; # [NOTE 1]
nonce_size 16;
dpd_delay 20;
dpd_retry 5;
dpd_maxfail 5;
dpd_algorithm dpd_blackhole_detect;
initial_contact on;
support_proxy on;
proposal_check obey;
xauth_login "macbookpro";
nat_traversal force; # [NOTE 2]
ike_frag on;
esp_frag 1280; # [NOTE 3]
mode_cfg on;
proposal {
authentication_method xauth_rsa_client;
hash_algorithm sha1;
encryption_algorithm aes 256;
lifetime time 7080 sec;
dh_group 5;
}
}
sainfo anonymous
{
pfs_group 5;
lifetime time 3600 sec;
encryption_algorithm aes 256;
authentication_algorithm hmac_sha1;
compression_algorithm deflate ; # [NOTE 4]
}
NOTES:
- Comment this out if you connect from different locations at a time.
- I had to set his as "force" because the MacBook is behind NAT.
- This seems to be the default fragment size for Apple Mac and can't be changed. The Apple Mac logs complain that esp_frag is not built in Apple's OS kernel.
- I suspect that Apple's racoon does not perform compression, but the connection works all the same.
Following your edits save any changes and check the file for syntax errors:
sudo /usr/sbin/racoon -v -C -f /etc/racoon/remote/ABC.DEF.GH.IJK.conf
If it returns you to a prompt there are no errors in the file, otherwise it will complain that there was an error and it couldn't parse the configuration file. Fix your error and check again the syntax before you move on to the next step.
For this file to be read by racoon you will need to add an include directive in Apple Mac's default configuration file. Edit /etc/racoon/racoon.conf by commenting out the last line:
#include "/var/run/racoon/*.conf"
and adding:
include "/etc/racoon/remote/ABC.DEF.GH.IJK.conf"
2.4 Set Password for XAUTH
Configure the password for XAUTH. This you can do from the Network Preferences GUI as mentioned above, but I also added it in /etc/racoon/psk.txt:
# IPv4/v6 addresses
# 10.160.94.3 asecretkeygoeshere
# 172.16.5.133 asecretkeygoeshere
# 3ffe:501:410:ffff:200:86ff:fe05:80fa asecretkeygoeshere
# 3ffe:501:410:ffff:210:4bff:fea2:8baa asecretkeygoeshere
macbookpro xauth_macbookpro_passwd
# USER_FQDN
# macuser@localhost somethingsecret
# FQDN
# kame hoge
3. SET UP SECURITY POLICIES
You will need to set up Security Policies, but Apple Mac's racoon version does not seem to load them from the default /etc/ipsec-tools.conf file. So you need to create this file, make it executable and run it yourself from a terminal. I created and used /etc/ipsec-tools.conf which is the file racoon in Linux is using for the same purpose, but you can name it and save as and where it suits you:
#!/usr/sbin/setkey -f
flush;
spdflush;
# SYNTAX
# spdadd <client_tunnel_subnet> <Netvanta_LAN_subnet> <protocol> -P out ipsec esp/tunnel/<Client_LAN_IP>-<Netvanta_public_IP>/require;
spdadd 172.16.5.0/24 10.10.10.0/24 any -P out ipsec esp/tunnel/192.168.1.8-ABC.DEF.GH.IJK/require;
spdadd 10.10.10.0/24 172.16.5.0/24 any -P in ipsec esp/tunnel/ABC.DEF.GH.IJK-192.168.1.8/require;
spdadd 172.16.5.0/24 172.16.5.0/24 any -P out none;
It is best to run this file as a script each time from a terminal before you try to connect, to flush any old security policies out of the kernel. Make it executable:
sudo chmod -v u+x,g+x /etc/racoon/ipsec-tools.conf
and when you need to connect, first run it to set up security policies:
sudo /etc/racoon/ipsec-tools.conf
Check that security policies have been loaded as intended:
sudo setkey -DP
4. CONNECT USING THE GUI
Then you can run 'tail -f /var/log/system.log' in a terminal on Apple Mac, or use the Console application to see what is happening as you try to connect. The Apple Mac racoon client will ping Netvanta once Phase 1 is complete to try to set up an IPSec tunnel. In a few seconds you should be able to see in Apple Mac's log the tunnel being established, after the mode_config settings are received from the Netvanta:
nesessionmanager[531]: IPSec Phase 2 established.
configd[48]: network changed: v4(utun0+:172.16.5.1, en0) DNS* Proxy SMB
5. CONFIRM CONNECTIVITY
You should now be able to ping the Netvanta and receive a response. Here is where the fun starts. The connection is quite erratic with a lot of retransmissions. This is because the Apple Mac seems to be setting new separate routes in its route table and then it tries to negotiate a new IKE connection, for each IP address any applications running on it may require. The Netvanta will reject these new tunnel requests, because there are no SA selectors that match the various addresses Apple Mac may want to connect to. All these attempts to connect to the Internet via the Netvanta generate a lot of messages like these on the Netvanta:
CRYPTO_IPSEC AdSendPktWithSecurity : No Matching SPDPolicy found
CRYPTO_IPSEC AdSendPktWithSecurity : No Matching SPDPolicy found
CRYPTO_IPSEC AdSendPktWithSecurity : No Matching SPDPolicy found
[snip ...]
You can check that ISAKMP SAs and IPSec SAs have been established on the Netvanta:
Netvanta-3120# show crypto ike sa
Using 1 SAs out of 20
Peak concurrent SAs: 13
IKE Security Associations:
Peer IP Address: RST.UVW.XY.Z
Mode-config Address: 172.16.5.1
Local IP Address: ABC.DEF.GH.IJK
Remote ID: macbook_VPN
XAUTH Username: macbookpro
Lifetime: 6754
Status: UP (SA_MATURE)
IKE Policy: 100
NAT-traversal: V2
Detected NAT / Behind NAT: Yes / Yes
Dead Peer Detection: Yes
Netvanta-3120# show ip crypto ipsec sa
2 current IPv4 IPsec SAs on default VRF
2 current IPv4 + IPv6 IPsec SAs on all VRFs (4 peak of 40 max)
IPsec Security Associations:
Peer IP Address: ABC.DEF.GH.IJK
Mode-config Address: 172.16.5.1
Remote ID: macbook_VPN
Crypto Map: VPN 10
Direction: Inbound
Encapsulation: ESP
SPI: 0xD6E6196E (3605404014)
RX Bytes: 51027
Selectors: Src:172.16.5.0/255.255.255.0 Port:ANY Proto:ALL IP
Dst:10.10.10.0/255.255.255.0 Port:ANY Proto:ALL IP
Hard Lifetime: 3280
Soft Lifetime: 0
Out-of-Sequence Errors: 0
Peer IP Address: RST.UVW.XY.Z
Mode-config Address: 172.16.5.1
Remote ID: macbook_VPN
Crypto Map: VPN 10
Direction: Outbound
Encapsulation: ESP
SPI: 0x0DF3B1F0 (234074608)
TX Bytes: 10802
Selectors: Src:10.10.10.0/255.255.255.0 Port:ANY Proto:ALL IP
Dst:172.16.5.0/255.255.255.0 Port:ANY Proto:ALL IP
Hard Lifetime: 3280
Soft Lifetime: 3230
Egress MTU: 1419 bytes
6. SET UP A SPLIT VPN
Repeated Phase 1 negotiations for each Apple Mac Internet connection will create multiple IKE SAs, which will be thereafter deleted, since they won't match any selector on the Netvanta. I am not sure why Apple Mac's racoon tries to create new VPN associations for each separate outgoing connection, instead of using the first VPN tunnel which has been already established. Linux does not have such a problem, so it may be a bug with Apple's fork of racoon. The solution I came up with is to set up a split tunnel manually (the El Capitan GUI does not seem to offer this option). As soon as the VPN tunnel is established and you can ping the Netvanta, run from a terminal:
sudo route -nv add -net 10.10.10.0/24 -interface utun0
sudo route change default 192.168.1.254
The first line above ensures any communication with the LAN behind the Netvanta always connects via the Apple Mac VPN tunnel, while the second line allows all other traffic to be routed directly via the local router out to the Internet. Adjust these to match your Netvanta's LAN subnet and your local router's IP address accordingly.
Once this split routing is set up the continuous renegotiations of new IKE SAs stop and the tunnel remains connected as expected. I have not tried reconfiguring the Netvanta firewall to forward packets received from the Apple Mac to the Internet, to see what effect this may have on the continuous tunnel negotiations.
Can you see anything in the above I should revisit, or configure differently? Any suggestions for improving it?
Accepted Solutions

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Not the Solution
- Report Inappropriate Content
Thanks for putting this together mick. Looks like a lot of work and will definitely help people down the line when they run into this setup. As a user of OSX I always appreciate documentation like this one.

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Not the Solution
- Report Inappropriate Content
Thanks for putting this together mick. Looks like a lot of work and will definitely help people down the line when they run into this setup. As a user of OSX I always appreciate documentation like this one.

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Accept as Solution
- Report Inappropriate Content
Re: Apple MacBook Pro 'Cisco IPSec' native VPN client
Good info! Adtran's VPN Products are rock solid. It's a shame they do not seem to care about clients for mac and mobile devices...

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Accept as Solution
- Report Inappropriate Content
Re: Apple MacBook Pro 'Cisco IPSec' native VPN client
Thanks guys, it did take me some time tweaking settings and debugging to get this going, but don't let my long post put you off using the native Apple VPN client. Once you have configured the router and Apple's VPN client settings using my suggestions above, all it takes to connect is 3 or 4 simple steps:
- Find out your client's LAN IP address and default gateway - use the Network Preferences GUI, or may be easier to run 'ifconfig' in a terminal.
- Update /etc/ipsec-tools.conf and run it to set up Security Policies on the client.
- Connect using the GUI by right-clicking the VPN icon in the desktop tool tray.
- Split the VPN tunnel to separate your encrypted connection with the LAN behind the Netvanta, from all other Internet traffic.
That's only a couple more steps than using a third party VPN client and if you configure your Netvanta to forward traffic from the tunnel to the Internet you don't even need to split the VPN tunnel. Once established I have found the VPN connection to be very reliable even over flaky WiFi. Expiring SAs are renewed at regular intervals without any drop outs.
Regarding Adtran providing their own client applications for VPN connectivity on different platforms, I understand it may be convenient for some users, but I suggest we view this in context. In some sense it is like asking Adtran to provide their own internet browser application, just because their routers also offer a HTTPS admin GUI. IKEv1 and IPSec are not Adtran's exclusive protocols and there are different platform specific VPN clients available. Personally, I would find it more useful if Adtran invested in IKEv2 based VPN firmware, which mobile platforms have moved to and also stronger ciphers and algorithms.