GRE Tunnels Fail Over IPSec with Failover Setup

[vmirinav]

Dear All,

I am trying to setup GRE over IPsec with Failover on Two Routers. I have the config for one router below(the other router config is a mirror of this one). I think all my settings are correct however the GRE tunnels fail for some reason.

I would be very grateful if anyone could help me out.

Warm Regards,

Vito

!

!

! ADTRAN OS version 18.02.01.00.E

! Boot ROM version 14.04.00

! Platform: NetVanta 3120, part number 1700600L2

! Serial number LBADTN1313AA927

!

!

hostname "xxx"

enable password xxx

!

!

ip subnet-zero

ip classless

ip routing

ip domain-proxy

ip name-server 8.8.8.8 8.8.4.4

!

!

no auto-config

!

event-history on

no logging forwarding

logging forwarding priority-level info

no logging email

!

no service password-encryption

!

username "admin" password "xxx"

!

!

ip firewall

no ip firewall alg msn

no ip firewall alg mszone

no ip firewall alg h323

!

!

!

!

!

!

!

no dot11ap access-point-control

!

!

!

probe VPN200Primary icmp-echo

  destination xx.xxx.62.200

  source-address xx.xxx.173.10

  period 3

  timeout 500

  tolerance consecutive fail 3 pass 40

  no shutdown

!

track "VPN200Primary"

  snmp trap state-change

  test if probe VPN200Primary

  no shutdown

!

!

!

!

ip dhcp-server pool "Private"

  network 10.10.20.0 255.255.255.0

  dns-server 10.10.20.1

  netbios-node-type h-node

  default-router 10.10.20.1

!

!

!

ip crypto

!

crypto ike policy 100

  initiate main

  respond main

  peer xx.xxx.62.200

  attribute 1

    encryption 3des

    hash md5

    authentication pre-share

!

crypto ike policy 101

  initiate main

  respond main

  peer xx.xxx.173.14

  attribute 1

    encryption 3des

    hash md5

    authentication pre-share

!

crypto ike remote-id address xx.xxx.62.200 preshared-key xxxx ike-policy 100 crypto map VPN 10 no-mode-config no-xauth

crypto ike remote-id address xx.xxx.173.14 preshared-key xxxx ike-policy 101 crypto map VPN 20 no-mode-config no-xauth

!

crypto ipsec transform-set esp-3des-esp-md5-hmac esp-3des esp-md5-hmac

  mode tunnel

!

crypto map VPN 10 ipsec-ike

  description GRE Tunner Peer

  match address VPN-Selectors

  set peer xx.xxx.62.200

  set transform-set esp-3des-esp-md5-hmac

  ike-policy 100

crypto map VPN 20 ipsec-ike

  description GRE Tunner Peer Failover

  match address VPN-Selectors-Failover

  set peer xx.xxx.173.14

  set transform-set esp-3des-esp-md5-hmac

  ike-policy 101

!

!

!

!

vlan 1

  name "Default"

!

vlan 301

  name "Failover"

!

!

interface eth 0/1

  description TowerStream

  ip address  xx.xxx.173.10  255.255.255.192

  ip access-policy Public

  crypto map VPN

  no shutdown

  no lldp send-and-receive

!

!

interface switchport 0/1

  no shutdown

  switchport access vlan 301

!

interface switchport 0/2

  no shutdown

!

interface switchport 0/3

  no shutdown

!

interface switchport 0/4

  no shutdown

!

!

!

interface vlan 1

  ip address  10.10.20.1  255.255.255.0

  ip access-policy Private

  no shutdown

!

interface vlan 301

  ip address  xx.xxx.62.209  255.255.255.224

  ip mtu 1500

  ip access-policy Failover

  media-gateway ip primary

  no awcp

  no shutdown

!

interface modem 0/1

  shutdown

!

!

interface tunnel 1

  ip address  172.16.0.2  255.255.255.252

  ip mtu 1400

  ip access-policy tunnel

  tunnel mode gre

  tunnel source xx.xxx.173.10

  tunnel destination xx.xxx.62.200

  keepalive

  no shutdown

!

!

interface tunnel 2

  ip address  172.16.1.2  255.255.255.252

  ip mtu 1400

  ip access-policy tunnel-failover

  tunnel mode gre

  tunnel source xx.xxx.62.209

  tunnel destination xx.xxx.173.14

  keepalive

  no shutdown

!

!

!

!

!

ip access-list extended AdminAccess

  remark Public Admin Access

  permit tcp any  any eq ssh 

  permit tcp any  any eq https 

!

ip access-list extended AdminAccessFailover

  remark Public Admin Access Failover

  permit tcp any  any eq ssh 

  permit tcp any  any eq https 

!

ip access-list extended nat

  remark NAT to the Internet

  permit ip any  any     log

!

ip access-list extended self

  remark Traffic to NetVanta

  permit ip any  any     log

!

ip access-list extended tunnel

  remark Traffic to GRE Tunnel

  permit ip any  any   

!

ip access-list extended VPN-Selectors

  remark GRE Tunnel Selectors

  ! Implicit permit (only for empty ACLs)

!

ip access-list extended VPN-Selectors-Failover

  remark GRE Tunnel Selectors Failover

  ! Implicit permit (only for empty ACLs)

!

!

!

no ip policy-class Failover rpf-check

ip policy-class Failover

  allow list AdminAccessFailover

!

no ip policy-class Private rpf-check

ip policy-class Private

  allow list tunnel policy tunnel

  allow list self self

  nat source list nat interface eth 0/1 overload policy Public

  allow list tunnel-failover policy tunnel

  nat source list nat interface vlan 301 overload policy Failover

!

no ip policy-class Public rpf-check

ip policy-class Public

  allow reverse list VPN-Selectors stateless

  allow list AdminAccess self

!

ip policy-class tunnel

  allow list self self

  allow list tunnel policy Private

!

ip policy-class tunnel-failover

  allow list self self

  allow list tunn policy Failover

!

!

ip route 0.0.0.0 0.0.0.0 xx.xxx.173.1 track VPN200Primary

ip route 0.0.0.0 0.0.0.0 xx.xxx.62.193 100

ip route 10.10.10.0 255.255.255.0 tunnel 1 track VPN200Primary

ip route 10.10.10.0 255.255.255.0 tunnel 2

ip route xx.xxx.62.200 255.255.255.255 xx.xxx.173.1

!

no tftp server

no tftp server overwrite

ip http server

no ip http secure-server

no snmp agent

no ip ftp server

ip ftp server default-filesystem flash

no ip scp server

no ip sntp server

!

!

!

!

!

!

!

!

!

ip sip udp 5060

ip sip tcp 5060

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

ip rtp quality-monitoring

ip rtp quality-monitoring udp

ip rtp quality-monitoring sip

!

line con 0

  login

!

line telnet 0 4

  login local-userlist

  password password

  shutdown

line ssh 0 4

  login local-userlist

  no shutdown

!

!

ntp server time.inscitek.net version 3 prefer

!

!

!

!

!

end

nnels

[Anonymous]

Vito:

Thank you for asking this question in the support community.  Can you confirm that the track is in a passing state and the proper default route is in the route table?  I noticed you have the probe configured to pass after 40 consecutive successful pings (of 3 seconds between pings); therefore, in your case, the default route might be removed via the track.  When the default route is correct, and the probe/track are passing, then I suggest you debug the IPSec tunnel to verify that it is negotiating properly, then make sure the GRE traffic is being routed properly. Furthermore, there are a few things that I recommend you correct.   

The ACL referenced in the "tunnel-failover" policy-class is "tunn" instead of "tunnel."   

!

ip policy-class tunnel-failover

  allow list self self

  allow list tunn policy Failover

!

There is not administrative distance on the backup tunnel route:

!

ip route 0.0.0.0 0.0.0.0 xx.xxx.173.1 track VPN200Primary

ip route 0.0.0.0 0.0.0.0 xx.xxx.62.193 100

ip route 10.10.10.0 255.255.255.0 tunnel 1 track VPN200Primary

ip route 10.10.10.0 255.255.255.0 tunnel 2 <admin distance>

I hope that makes sense, but please do not hesitate to reply with any additional questions or information.  I will be happy to help in any way I can.

Levi

[Anonymous]

:

I went ahead and flagged the "Correct Answer" on this post to make it more visible and help other members of the community find solutions more easily. If you don't feel like the answer I marked was correct, feel free to come back to this post and unmark it and select another in its place with the applicable buttons.  If you still need assistance, we would be more than happy to continue working with you on this - just let us know in a reply.

Thanks,

Levi

[vmirinav]

Its not really resolved yet. I have been on with support trying to solve this problem for over 1week plus now.

I may have a routing loop somwhere.

[Anonymous]

:

Please, update this forum thread when appropriate.

Levi

[vmirinav]

Yep will advice when I have something more detailed.

[vmirinav]

I had to remove the following line from the config as per manual:https://supportforums.adtran.com/docs/DOC-2310

Routing Settings

The firewall has been setup to take its cue from the routing engine, so a properly

functioning routing table is critical. If the routing table is not setup correctly, especially in

the case of funneling all Internet traffic through the GRE to a central location, recursive

routing errors may occur.

The first step to avoid routing errors is to create a static route to the GRE tunnel peer.

This will force the router to always use this path when accessing the GRE tunnel peer. If

this route was not entered, and the default route was pointing through the GRE tunnel, the

only way the router could get to the GRE tunnel peer would be to traverse the GRE

tunnel, which results in a recursive routing error. Using the example configuration, the

route would be configured in this manner in the command line:

ip route <GRE Tunnel Peer IP> 255.255.255.255 <Internet Gateway>

ip route 65.162.109.201 255.255.255.255 208.61.209.254

------------------------------------------------------------------------------------------

When I had a route simular as above on my routers the Tunnel 1 was going down all the time when there was traffic send to it.

Also the SSH connection to the Primary was going down as well and I had to constantly reconnect. Maybe there was a routing loop somewhere.

At this point I am pretty much stuck.

Now both tunnels are down

Here are two config files:

!

!

! ADTRAN OS version 18.02.01.00.E

! Boot ROM version 14.04.00

! Platform: NetVanta 3120, part number 1700600L2

! Serial number LBADTN1313AA924

!

!

hostname "xxxx"

enable password encrypted xxx

!

!

ip subnet-zero

ip classless

ip routing

ip domain-proxy

ip name-server 8.8.8.8 8.8.4.4

!

!

no auto-config

!

event-history on

no logging forwarding

logging forwarding priority-level info

no logging email

!

service password-encryption

!

username "admin" password encrypted "xxxx"

!

!

ip firewall

no ip firewall alg msn

no ip firewall alg mszone

no ip firewall alg h323

!

!

!

!

!

!

!

no dot11ap access-point-control

!

!

!

probe VPNxx.xxx.173.1 icmp-echo

  destination 8.8.8.8

  source-address xx.xxx.62.212

  period 3

  timeout 500

  tolerance consecutive fail 3 pass 4

  no shutdown

!

track "VPNxx.xxx.173.1"

  snmp trap state-change

  test if probe VPNxx.xxx.173.1

  no shutdown

!

!

!

!

ip dhcp-server pool "Private"

  network 10.10.10.0 255.255.255.0

  dns-server 10.10.10.1

  netbios-node-type h-node

  default-router 10.10.10.1

!

!

!

ip crypto

ip crypto fast-failover

!

crypto ike policy 100

  initiate main

  respond main

  peer xx.xxx.173.10

  attribute 1

    encryption 3des

    hash md5

    authentication pre-share

!

crypto ike policy 101

  initiate main

  respond main

  peer xx.xxx.62.209

  attribute 1

    encryption 3des

    hash md5

    authentication pre-share

!

crypto ike remote-id address xx.xxx.62.209 preshared-key xxxx ike-policy 101 crypto map VPN 20 no-mode-config no-xauth

crypto ike remote-id address xx.xxx.173.10 preshared-key xxxx ike-policy 100 crypto map VPN 10 no-mode-config no-xauth

!

crypto ipsec transform-set esp-3des-esp-md5-hmac esp-3des esp-md5-hmac

  mode tunnel

!

crypto map VPN 10 ipsec-ike

  description GRE Tunner Peer

  match address VPN-Selectors

  set peer xx.xxx.173.10

  set transform-set esp-3des-esp-md5-hmac

  ike-policy 100

crypto map VPN 20 ipsec-ike

  description GRE Tunner Peer

  match address VPN-Selectors-Failover

  set peer xx.xx.62.209

  set transform-set esp-3des-esp-md5-hmac

  ike-policy 101

!

!

!

!

vlan 1

  name "Default"

!

vlan 301

  name "Failover"

!

!

interface eth 0/1

  description TimeWarner

  ip address  xx.xx.62.212  255.255.255.224

  ip access-policy Public

  crypto map VPN

  no shutdown

  no lldp send-and-receive

!

!

interface switchport 0/1

  no shutdown

  switchport access vlan 301

!

interface switchport 0/2

  no shutdown

!

interface switchport 0/3

  no shutdown

!

interface switchport 0/4

  no shutdown

!

!

!

interface vlan 1

  ip address  10.10.10.1  255.255.255.0

  ip access-policy Private

  no shutdown

!

interface vlan 301

  ip address  xx.xxx.173.14  255.255.255.192

  ip access-policy Failover

  media-gateway ip primary

  no awcp

  no shutdown

!

interface modem 0/1

  shutdown

!

!

interface tunnel 1

  ip address  172.16.0.1  255.255.255.252

  ip mtu 1400

  ip access-policy tunnel

  tunnel mode gre

  tunnel source xx.xx.62.212

  tunnel destination xx.xxx.173.10

  keepalive

  no shutdown

!

!

interface tunnel 2

  ip address  172.16.1.1  255.255.255.0

  ip mtu 1400

  ip access-policy tunnel-failover

  tunnel mode gre

  tunnel source xx.xxx.173.14

  tunnel destination xx.xxx.62.209

  keepalive

  no shutdown

!

!

!

!

!

ip access-list extended AdminAccess

  remark Public Admin Access

  permit tcp any  any eq ssh

  permit tcp any  any eq https

!

ip access-list extended AdminAccessFailover

  remark Public Admin Access Failover

  permit tcp any  any eq ssh

  permit tcp any  any eq https

!

ip access-list extended nat

  remark NAT to the Internet

  permit ip any  any     log

!

ip access-list extended self

  remark Traffic to NetVanta

  permit ip any  any     log

!

ip access-list extended tunnel

  remark Traffic to GRE Tunnel

  permit ip any  any  

!

ip access-list extended VPN-Selectors

  remark GRE Tunnel Selectors

  permit gre host xx.xx.62.212  host xx.xxx.173.10     log

!

ip access-list extended VPN-Selectors-Failover

  remark GRE Tunnel Selectors Failover

  permit gre host xx.xxx.173.14  host xx.xx.62.209  

!

!

!

no ip policy-class Failover rpf-check

ip policy-class Failover

  allow list AdminAccessFailover

  allow reverse list VPN-Selectors-Failover stateless

!

no ip policy-class Private rpf-check

ip policy-class Private

  allow list tunnel policy tunnel

  allow list self self

  nat source list nat interface eth 0/1 overload policy Public

  nat source list nat interface vlan 301 overload policy Failover

  allow list tunnel policy tunnel-failover

!

no ip policy-class Public rpf-check

ip policy-class Public

  allow reverse list VPN-Selectors stateless

  allow list AdminAccess self

!

no ip policy-class tunnel rpf-check

ip policy-class tunnel

  allow list self self

  allow list tunnel policy Private

!

no ip policy-class tunnel-failover rpf-check

ip policy-class tunnel-failover

  allow list self self

  allow list tunnel policy Failover

!

!

ip route 0.0.0.0 0.0.0.0 xx.xx.62.193 track VPNxx.xxx.173.1

ip route 0.0.0.0 0.0.0.0 xx.xxx.173.1 100

ip route 8.8.8.8 255.255.255.255 xx.xx.62.193

ip route 10.10.20.0 255.255.255.0 tunnel 1 track VPNxx.xxx.173.1

ip route 10.10.20.0 255.255.255.0 tunnel 2 100

!

no tftp server

no tftp server overwrite

ip http server

no ip http secure-server

no snmp agent

no ip ftp server

ip ftp server default-filesystem flash

no ip scp server

no ip sntp server

!

!

!

!

!

!

!

!

!

ip sip udp 5060

ip sip tcp 5060

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

ip rtp quality-monitoring

ip rtp quality-monitoring udp

ip rtp quality-monitoring sip

!

line con 0

  login

!

line telnet 0 4

  login local-userlist

  password encrypted xxx

  shutdown

line ssh 0 4

  login local-userlist

  no shutdown

!

!

ntp server time.inscitek.net version 3 prefer

!

!

!

!

!

end

!

!

! ADTRAN OS version 18.02.01.00.E

! Boot ROM version 14.04.00

! Platform: NetVanta 3120, part number 1700600L2

! Serial number LBADTN1313AA927

!

!

hostname "xxx"

enable password encrypted xxxx

!

clock timezone -5-Eastern-Time

!

ip subnet-zero

ip classless

ip routing

ip domain-proxy

ip name-server 8.8.8.8 8.8.4.4

!

!

no auto-config

!

event-history on

no logging forwarding

logging forwarding priority-level info

no logging email

!

service password-encryption

!

username "admin" password encrypted "xxxx"

!

!

ip firewall

no ip firewall alg msn

no ip firewall alg mszone

no ip firewall alg h323

!

!

!

!

!

!

!

no dot11ap access-point-control

!

!

!

probe VPN200Primary icmp-echo

  destination 8.8.8.8

  source-address xx.xxx.173.10

  period 3

  timeout 500

  tolerance consecutive fail 3 pass 4

  no shutdown

!

track "VPN200Primary"

  snmp trap state-change

  test if probe VPN200Primary

  no shutdown

!

!

!

!

ip dhcp-server pool "Private"

  network 10.10.20.0 255.255.255.0

  dns-server 10.10.20.1

  netbios-node-type h-node

  default-router 10.10.20.1

!

!

!

ip crypto

ip crypto fast-failover

!

crypto ike policy 100

  initiate main

  respond main

  peer xx.xx.62.212

  attribute 1

    encryption 3des

    hash md5

    authentication pre-share

!

crypto ike policy 101

  initiate main

  respond main

  peer xx.xxx.173.14

  attribute 1

    encryption 3des

    hash md5

    authentication pre-share

!

crypto ike remote-id address xx.xx.62.212 preshared-key xxxx ike-policy 100 crypto map VPN 10 no-mode-config no-xauth

crypto ike remote-id address xx.xxx.173.14 preshared-key xxxx ike-policy 101 crypto map VPN 20 no-mode-config no-xauth

!

crypto ipsec transform-set esp-3des-esp-md5-hmac esp-3des esp-md5-hmac

  mode tunnel

!

crypto map VPN 10 ipsec-ike

  description GRE Tunner Peer

  match address VPN-Selectors

  set peer xx.xx.62.212

  set transform-set esp-3des-esp-md5-hmac

  ike-policy 100

crypto map VPN 20 ipsec-ike

  description GRE Tunner Peer Failover

  match address VPN-Selectors-Failover

  set peer xx.xxx.173.14

  set transform-set esp-3des-esp-md5-hmac

  ike-policy 101

!

!

!

!

vlan 1

  name "Default"

!

vlan 301

  name "Failover"

!

!

interface eth 0/1

  description TowerStream

  ip address  xx.xxx.173.10  255.255.255.192

  ip access-policy Public

  crypto map VPN

  no shutdown

  no lldp send-and-receive

!

!

interface switchport 0/1

  no shutdown

  switchport access vlan 301

!

interface switchport 0/2

  no shutdown

!

interface switchport 0/3

  no shutdown

!

interface switchport 0/4

  no shutdown

!

!

!

interface vlan 1

  ip address  10.10.20.1  255.255.255.0

  ip access-policy Private

  no shutdown

!

interface vlan 301

  ip address  xx.xx.62.209  255.255.255.224

  ip access-policy Failover

  media-gateway ip primary

  no awcp

  no shutdown

!

interface modem 0/1

  shutdown

!

!

interface tunnel 1

  ip address  172.16.0.2  255.255.255.252

  ip mtu 1400

  ip access-policy tunnel

  tunnel mode gre

  tunnel source xx.xxx.173.10

  tunnel destination xx.xx.62.212

  keepalive

  no shutdown

!

!

interface tunnel 2

  ip address  172.16.1.2  255.255.255.0

  ip mtu 1400

  ip access-policy tunnel-failover

  tunnel mode gre

  tunnel source xx.xx.62.209

  tunnel destination xx.xxx.173.14

  keepalive

  no shutdown

!

!

!

!

!

ip access-list extended AdminAccess

  remark Public Admin Access

  permit tcp any  any eq ssh

  permit tcp any  any eq https

!

ip access-list extended AdminAccessFailover

  remark Public Admin Access Failover

  permit tcp any  any eq ssh

  permit tcp any  any eq https

!

ip access-list extended nat

  remark NAT to the Internet

  permit ip any  any     log

!

ip access-list extended self

  remark Traffic to NetVanta

  permit ip any  any     log

!

ip access-list extended tunnel

  remark Traffic to GRE Tunnel

  permit ip any  any  

!

ip access-list extended VPN-Selectors

  remark GRE Tunnel Selectors

  permit gre host xx.xxx.173.10  host xx.xx.62.212  

!

ip access-list extended VPN-Selectors-Failover

  remark GRE Tunnel Selectors Failover

  permit gre host xx.xx.62.209  host xx.xxx.173.14  

!

!

!

no ip policy-class Failover rpf-check

ip policy-class Failover

  allow list AdminAccessFailover

  allow reverse list VPN-Selectors-Failover stateless

!

no ip policy-class Private rpf-check

ip policy-class Private

  allow list tunnel policy tunnel

  allow list self self

  nat source list nat interface eth 0/1 overload policy Public

  nat source list nat interface vlan 301 overload policy Failover

  allow list tunnel policy tunnel-failover

!

no ip policy-class Public rpf-check

ip policy-class Public

  allow reverse list VPN-Selectors stateless

  allow list AdminAccess self

!

ip policy-class tunnel

  allow list self self

  allow list tunnel policy Private

!

ip policy-class tunnel-failover

  allow list self self

  allow list tunnel policy Failover

!

!

ip route 0.0.0.0 0.0.0.0 xx.xxx.173.1 track VPN200Primary

ip route 0.0.0.0 0.0.0.0 xx.xx.62.193 100

ip route 8.8.8.8 255.255.255.255 xx.xxx.173.1

ip route 10.10.10.0 255.255.255.0 tunnel 1 track VPN200Primary

ip route 10.10.10.0 255.255.255.0 tunnel 2 100

!

no tftp server

no tftp server overwrite

ip http server

no ip http secure-server

no snmp agent

no ip ftp server

ip ftp server default-filesystem flash

no ip scp server

no ip sntp server

!

!

!

!

!

!

!

!

!

ip sip udp 5060

ip sip tcp 5060

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

ip rtp quality-monitoring

ip rtp quality-monitoring udp

ip rtp quality-monitoring sip

!

line con 0

  login

!

line telnet 0 4

  login local-userlist

  password encrypted xxxx

  shutdown

line ssh 0 4

  login local-userlist

  no shutdown

!

!

!

!

!

!

!

end

[vmirinav]

This is still being worked on. I will write something up when positive results are available.

[vmirinav · Accepted solution]

We were able to resolve this after Adtran was involved. We had to purchase 5 PSV's to get Adtran engineers to look and resolve the issues. Now everything works.

If anyone is interested to implement the scenario where crisscross connections fail-over correctly allot of work is involved and I suggest purchasing Adtran PSV's work work with an engineer.