Showing results for 
Show  only  | Search instead for 
Did you mean: 

Apple MacBook Pro 'Cisco IPSec' native VPN client

Jump to solution

Hi All,

After various attempts I managed to get Apple Mac's native VPN client to connect to a Netvanta 3120.  Apple are using their own fork of racoon to manage IKE negotiation, but it will not work out of the box by just using the Network Preferences VPN GUI, without creating a separate configuration file for it.  I tried this on an Apple MacBook Pro running OSX El Capitan v.10.11.6.  Other recent OSX versions should work too.

Some of my findings and conclusions:

  • The native Apple Mac 'Cisco IPSec' VPN client requires XAUTH.  Attempting to connect without XAUTH is a hit and miss affair for IKE Phase 1.  Even if Phase 1 completes, IPSec Phase 2 always fails.
  • The Apple Mac client asks Netvanta for MODE_CONFIG data.  Even if you modify its racoon.conf file by setting 'mode_cfg off;', this client setting appears to have been hard coded by Apple and will still ask for MODE_CONFIG information from the router.
  • The Apple Mac's Network Preferences GUI does not provide sufficient settings to allow you to configure a connection with the Netvanta.  You will have to create a separate racoon configuration file with your settings and add an include directive in Apple's default /etc/racoon/racoon.conf file, to make sure the racoon client reads your modified configuration and executes it.
  • You will also have to create an ipsec-tools.conf file with the required SA selectors and run this file manually as a script from a terminal, because Apple's racoon client will not pick it up and use it.
  • When it connects, racoon by default sets up a full VPN tunnel, with all and any connections from the MackBook directed through the tunnel to Netvanta.  Unless you configure the Netvanta's firewall to forward VPN packets out through its WAN port, you will only be able to connect to PCs within Netvanta's LAN.  In the example configuration below I offer a solution for creating a split VPN tunnel, so connections to the Internet from the MacBook do not go through the VPN tunnel, but via the local router.
  • The whole process of setting up the MacBook and getting it to connect is a bit of a chore, so 3rd party VPN clients may be an easier bet, if you do not have the patience to get this going.


The main steps to get a VPN connection going are as follows:

  1. Configure your Netvanta's VPN.
  2. Configure Apple Macs' 'Cisco IPSec' VPN client GUI.
  3. Run a script to set up Security Policies on the Apple Mac.
  4. Connect using the Apple VPN GUI.
  5. Ping the Netvanta to confirm connectivity.
  6. Set up routes to implement a split VPN tunnel (optional).


   MacBook===========Router=======(INTERNET)=======Netvanta-----------LAN_SUBNET                RST.UVW.XY.Z                                  ABC.DEF.GH.IJK


The configuration below shows only the VPN and XAUTH specific settings:


service password-encryption


username "admin" password encrypted "secret_admin_passwd"

username "macbookpro" password encrypted "xauth_macbookpro_passwd"



ip crypto


crypto ike client configuration pool Netvanta_VPN_modconfig




crypto ike policy 100

  no initiate

  respond main

  local-id address ABC.DEF.GH.IJK

  peer any

  client authentication server list LoginUseLocalUsers

  client configuration pool Netvanta_VPN_modconfig

  attribute 3

    encryption aes-256-cbc

    authentication rsa-sig

    group 5

    lifetime 7080


crypto ike remote-id fqdn macbook_VPN ike-policy 100 crypto map VPN 10


ip crypto ipsec transform-set esp-aes-256-cbc-esp-sha-hmac esp-aes-256-cbc esp-sha-hmac

  mode tunnel


ip crypto map VPN 10 ipsec-ike

  description VPN_IPSec_Access

  match address ip VPN-10-vpn-selectors

  set transform-set esp-aes-256-cbc-esp-sha-hmac

  set security-association lifetime seconds 3600

  set pfs group5

  ike-policy 100



crypto ca profile "VPN SSL CA"

  ip-address ABC.DEF.GH.IJK

  subject-name "CN=3120_VPN; OU=VPN Gateway; O=VPN; C=US; ST=TN"

  fqdn 3120_VPN


crypto ca certificate chain "VPN SSL CA"

[Certificate details follow ... ]



ip access-list extended VPN-10-vpn-selectors

  permit ip  

  deny   ip any  any     log



ip policy-class Private

  allow list VPN-10-vpn-selectors stateless

[snip ...]



ip policy-class Public

  allow reverse list VPN-10-vpn-selectors stateless

[snip ...]

NOTE: I created the SSL CA, Netvanta and MacBook client certificates using OpenSSL.  It is important to add to the Netvanta certificate the IP address and/or its FQDN in the subjectAltName field, because Apple Mac reads those to determine the remote peer.  The Apple Mac Client SSL Certificate should also contain its FQDN in the subjectAltName field.


2.1 Network Preferences GUI

To configure the Apple Mac client, go to its Network Preferences GUI and create a new service, selecting from the drop down options:

Interface: VPN

VPN Type: Cisco IPSec

Server Address: ABC.DEF.GH.IJK (use your Netvanta's public IP address)

Account Name:   macbookpro     (use your Netvanta's XAUTH details here)

Password:       xauth_macbookpro_passwd

2.2 Authentication Settings

Select your client SSL certificate, after you import it into the "System" keychain, using MacBook's 'Keychain Access' application.  Configure your private key preferences, using the 'Keychain Access' GUI, so that it can accessed by /usr/sbin/racoon.  Also make sure you import the CA Certificate and edit its Trust settings to mark it as trusted.

If for some reason you do not want to use certificates, the proposed set up should also work with PSK, but SSL certificates are arguably more secure and in VPN Main Mode allow you to use FQDN, or USER_FQDN, etc., as the client's peer ID, rather than its public IP address;  which on a laptop is likely to change regularly.

2.3 Racoon Configuration Files

Configure Apple Mac's racoon configuration next.  Open a terminal and type:

sudo mkdir /etc/racoon/remote

to create a new directory for storing your own racoon configuration.  Then type:

sudo cat /var/run/racoon/ABC.DEF.GH.IJK.conf > /etc/racoon/remote/ABC.DEF.GH.IJK.conf

but do not run this syntax yet.  Start the VPN connection for racoon to generate the dynamic content of file /var/run/racoon/ABC.DEF.GH.IJK.conf and while it is trying to connect (you only have a few seconds for this) press the enter key to run the above syntax already entered into the command line, to capture this file's content.  Then go to /etc/racoon/remote/ABC.DEF.GH.IJK.conf and edit it as is appropriate for your connection.  I give a working example below, which I arrived at after some trial and error:

remote ABC.DEF.GH.IJK {

   doi ipsec_doi;

   situation identity_only;

   exchange_mode main;

   verify_identifier on;

   peers_identifier address ABC.DEF.GH.IJK;

   my_identifier fqdn "macbook_VPN";

   certificate_type x509 in_keychain "...Long_string_for_SSL_cert...";

   verify_cert on;

   certificate_verification sec_framework use_peers_identifier;

   #local_address;  # [NOTE 1]

   nonce_size 16;

   dpd_delay 20;

   dpd_retry 5;

   dpd_maxfail 5;

   dpd_algorithm dpd_blackhole_detect;

   initial_contact on;

   support_proxy on;

   proposal_check obey;

   xauth_login "macbookpro";

   nat_traversal force;  # [NOTE 2]

   ike_frag on;

   esp_frag 1280;  # [NOTE 3]

   mode_cfg on;

   proposal {

      authentication_method xauth_rsa_client;

      hash_algorithm sha1;

      encryption_algorithm aes 256;

      lifetime time 7080 sec;

      dh_group 5;



sainfo anonymous


       pfs_group 5;

       lifetime time 3600 sec;

       encryption_algorithm aes 256;

       authentication_algorithm hmac_sha1;

       compression_algorithm deflate ;  # [NOTE 4]



  1. Comment this out if you connect from different locations at a time.
  2. I had to set his as "force" because the MacBook is behind NAT.
  3. This seems to be the default fragment size for Apple Mac and can't be changed.  The Apple Mac logs complain that esp_frag is not built in Apple's OS kernel.
  4. I suspect that Apple's racoon does not perform compression, but the connection works all the same.

Following your edits save any changes and check the file for syntax errors:

sudo /usr/sbin/racoon -v -C -f /etc/racoon/remote/ABC.DEF.GH.IJK.conf

If it returns you to a prompt there are no errors in the file, otherwise it will complain that there was an error and it couldn't parse the configuration file.  Fix your error and check again the syntax before you move on to the next step.

For this file to be read by racoon you will need to add an include directive in Apple Mac's default configuration file.  Edit /etc/racoon/racoon.conf by commenting out the last line:

#include "/var/run/racoon/*.conf"

and adding:

include "/etc/racoon/remote/ABC.DEF.GH.IJK.conf"

2.4 Set Password for XAUTH

Configure the password for XAUTH.  This you can do from the Network Preferences GUI as mentioned above, but I also added it in /etc/racoon/psk.txt:

# IPv4/v6 addresses

#   asecretkeygoeshere

#  asecretkeygoeshere

# 3ffe:501:410:ffff:200:86ff:fe05:80fa  asecretkeygoeshere

# 3ffe:501:410:ffff:210:4bff:fea2:8baa  asecretkeygoeshere

macbookpro xauth_macbookpro_passwd


# macuser@localhost     somethingsecret


# kame          hoge


You will need to set up Security Policies, but Apple Mac's racoon version does not seem to load them from the default /etc/ipsec-tools.conf file.  So you need to create this file, make it executable and run it yourself from a terminal.  I created and used /etc/ipsec-tools.conf which is the file racoon in Linux is using for the same purpose, but you can name it and save as and where it suits you:

#!/usr/sbin/setkey -f




# spdadd <client_tunnel_subnet> <Netvanta_LAN_subnet> <protocol> -P out ipsec esp/tunnel/<Client_LAN_IP>-<Netvanta_public_IP>/require;

spdadd any -P out ipsec esp/tunnel/;

spdadd any -P in ipsec esp/tunnel/ABC.DEF.GH.IJK-;

spdadd any -P out none;

It is best to run this file as a script each time from a terminal before you try to connect, to flush any old security policies out of the kernel.  Make it executable:

sudo chmod -v u+x,g+x /etc/racoon/ipsec-tools.conf

and when you need to connect, first run it to set up security policies:

sudo /etc/racoon/ipsec-tools.conf

Check that security policies have been loaded as intended:

sudo setkey -DP


Then you can run 'tail -f /var/log/system.log' in a terminal on Apple Mac, or use the Console application to see what is happening as you try to connect.  The Apple Mac racoon client will ping Netvanta once Phase 1 is complete to try to set up an IPSec tunnel.  In a few seconds you should be able to see in Apple Mac's log the tunnel being established, after the mode_config settings are received from the Netvanta:

nesessionmanager[531]: IPSec Phase 2 established.

configd[48]: network changed: v4(utun0+:, en0) DNS* Proxy SMB


You should now be able to ping the Netvanta and receive a response.  Here is where the fun starts.  The connection is quite erratic with a lot of retransmissions.  This is because the Apple Mac seems to be setting new separate routes in its route table and then it tries to negotiate a new IKE connection, for each IP address any applications running on it may require.  The Netvanta will reject these new tunnel requests, because there are no SA selectors that match the various addresses Apple Mac may want to connect to.  All these attempts to connect to the Internet via the Netvanta generate a lot of messages like these on the Netvanta:

CRYPTO_IPSEC AdSendPktWithSecurity : No Matching SPDPolicy found 

CRYPTO_IPSEC AdSendPktWithSecurity : No Matching SPDPolicy found 

CRYPTO_IPSEC AdSendPktWithSecurity : No Matching SPDPolicy found 

[snip ...]

You can check that ISAKMP SAs and IPSec SAs have been established on the Netvanta:

Netvanta-3120# show crypto ike sa

Using 1 SAs out of 20

Peak concurrent SAs: 13

IKE Security Associations:

Peer IP Address: RST.UVW.XY.Z

  Mode-config Address:

  Local IP Address: ABC.DEF.GH.IJK

  Remote ID: macbook_VPN

  XAUTH Username: macbookpro

  Lifetime: 6754

  Status: UP (SA_MATURE)

  IKE Policy: 100

  NAT-traversal: V2

  Detected NAT / Behind NAT: Yes / Yes

  Dead Peer Detection: Yes

Netvanta-3120# show ip crypto ipsec sa

2 current IPv4 IPsec SAs on default VRF

2 current IPv4 + IPv6 IPsec SAs on all VRFs (4 peak of 40 max)

IPsec Security Associations:

Peer IP Address: ABC.DEF.GH.IJK

  Mode-config Address:

  Remote ID: macbook_VPN

  Crypto Map: VPN 10

  Direction: Inbound

  Encapsulation: ESP

  SPI: 0xD6E6196E (3605404014)

  RX Bytes: 51027

  Selectors: Src:  Port:ANY  Proto:ALL IP

             Dst:  Port:ANY  Proto:ALL IP

  Hard Lifetime: 3280

  Soft Lifetime: 0

  Out-of-Sequence Errors: 0

Peer IP Address: RST.UVW.XY.Z

  Mode-config Address:

  Remote ID: macbook_VPN

  Crypto Map: VPN 10

  Direction: Outbound

  Encapsulation: ESP

  SPI: 0x0DF3B1F0 (234074608)

  TX Bytes: 10802

  Selectors: Src:  Port:ANY  Proto:ALL IP

             Dst:  Port:ANY  Proto:ALL IP

  Hard Lifetime: 3280

  Soft Lifetime: 3230

  Egress MTU: 1419 bytes


Repeated Phase 1 negotiations for each Apple Mac Internet connection will create multiple IKE SAs, which will be thereafter deleted, since they won't match any selector on the Netvanta.  I am not sure why Apple Mac's racoon tries to create new VPN associations for each separate outgoing connection, instead of using the first VPN tunnel which has been already established.  Linux does not have such a problem, so it may be a bug with Apple's fork of racoon.  The solution I came up with is to set up a split tunnel manually (the El Capitan GUI does not seem to offer this option).  As soon as the VPN tunnel is established and you can ping the Netvanta, run from a terminal:

sudo route -nv add -net -interface utun0

sudo route change default

The first line above ensures any communication with the LAN behind the Netvanta always connects via the Apple Mac VPN tunnel, while the second line allows all other traffic to be routed directly via the local router out to the Internet.  Adjust these to match your Netvanta's LAN subnet and your local router's IP address accordingly.

Once this split routing is set up the continuous renegotiations of new IKE SAs stop and the tunnel remains connected as expected.  I have not tried reconfiguring the Netvanta firewall to forward packets received from the Apple Mac to the Internet, to see what effect this may have on the continuous tunnel negotiations.

Can you see anything in the above I should revisit, or configure differently?  Any suggestions for improving it?

Labels (3)
0 Kudos
1 Solution

Accepted Solutions
Not applicable

Re: Apple MacBook Pro 'Cisco IPSec' native VPN client

Jump to solution

Thanks for putting this together mick.  Looks like a lot of work and will definitely help people down the line when they run into this setup.  As a user of OSX I always appreciate documentation like this one.

View solution in original post

0 Kudos
3 Replies
Not applicable

Re: Apple MacBook Pro 'Cisco IPSec' native VPN client

Jump to solution

Thanks for putting this together mick.  Looks like a lot of work and will definitely help people down the line when they run into this setup.  As a user of OSX I always appreciate documentation like this one.

0 Kudos
New Contributor

Re: Apple MacBook Pro 'Cisco IPSec' native VPN client

Jump to solution

Good info! Adtran's VPN Products are rock solid. It's a shame they do not seem to care about clients for mac and mobile devices...

Re: Apple MacBook Pro 'Cisco IPSec' native VPN client

Jump to solution

Thanks guys, it did take me some time tweaking settings and debugging to get this going, but don't let my long post put you off using the native Apple VPN client.  Once you have configured the router and Apple's VPN client settings using my suggestions above, all it takes to connect is 3 or 4 simple steps:

  1. Find out your client's LAN IP address and default gateway - use the Network Preferences GUI, or may be easier to run 'ifconfig' in a terminal.
  2. Update /etc/ipsec-tools.conf and run it to set up Security Policies on the client.
  3. Connect using the GUI by right-clicking the VPN icon in the desktop tool tray.
  4. Split the VPN tunnel to separate your encrypted connection with the LAN behind the Netvanta, from all other Internet traffic.

That's only a couple more steps than using a third party VPN client and if you configure your Netvanta to forward traffic from the tunnel to the Internet you don't even need to split the VPN tunnel.  Once established I have found the VPN connection to be very reliable even over flaky WiFi.  Expiring SAs are renewed at regular intervals without any drop outs.

Regarding Adtran providing their own client applications for VPN connectivity on different platforms, I understand it may be convenient for some users, but I suggest we view this in context.  In some sense it is like asking Adtran to provide their own internet browser application, just because their routers also offer a HTTPS admin GUI.  IKEv1 and IPSec are not Adtran's exclusive protocols and there are different platform specific VPN clients available.  Personally, I would find it more useful if Adtran invested in IKEv2 based VPN firmware, which mobile platforms have moved to and also stronger ciphers and algorithms.