I'm attempting to use Shrew VPN(2.2.2) client to connect to a NetVanta 3120 but it hangs on "bringing up tunnel." It looks like I'm receiving "CRYPTO_IKE.MODE_CONFIG ModeCfgProcess: ModeCfgAllocateResources Failed" Any ideas?
This looks like a remote access VPN, where you assign a private address from a pool to the client.
The address pool you are using is part of a subnet bound to an interface.
Try configuring a new subnet for the remote access clients not bound to an interface.
Not clear on what you want me to do. Doesn't the VPN client need to be on the same subnet as the internal network?
No. It should be on a separate subnet than your connected interface. Otherwise there's an IP conflict between the tunnel source and tunnel destination. So use something like 10.100.0.1 through 10.100.0.254 as an example assuming it isn't used elsewhere. Keep your DNS and WINS servers in the configuration as they are. Clients will reach them over the VPN.
The last screenshot from your router shows that the IPSec tunnel is not yet established. So something is causing phase 2 of the VPN to fail. What does the log on either side of the tunnel show?
If you followed the configuration instructions on the shrew.net page the connection ought to succeed. Connect to the router with SSH or Telnet and run a debug session while the Shrew client attempts to connect. The debug command to run is: 'debug crypto ike'
Then search through the stream of debug messages to find confirmation of the following:
1. A message from CRYPTO_IKE.NEGOTIATION which will say the 'aggressive mode is complete'.
2. A message from CRYPTO_IKE, which will say the XAuthentication .has succeeded: CRYPTO_IKE.XAUTH EDCallBackFun: Xauth succeeded
3. A message from CRYPTO_IKE confirming the Quick Mode is starting: CRYPTO_IKE.NEGOTIATION peer AA.BBB.CCC.DD: Received first message of quick mode (where AA.BBB.CCC.DD is the Internet IP address of the Shrew client PC).
4. A message from CRYPTO_IKE confirming the Quick Mode has completed: CRYPTO_IKE.NEGOTIATION peer AA.BBB.CCC.DD: Quick mode completed
Until step 4 above is completed the IPSec tunnel is not yet up and no packets will flow. To get the tunnel established you may need to ping the server from the client PC, or a routable device behind the Netvanta, once or twice. If you never arrive at step 4, then you will need to retrace your steps for any typos or configuration errors. If you do arrive at step 4 but still cannot access the server, then you should check the server configuration and logs to confirm if any packets arrive there from the client PC.
Please report back with your results if you get stuck.