Accessing router via secondary WAN interface after using PBRs

[Anonymous]

I have a NV3140 router installed where I have two different ISP connections. Both are Ethernet handoff connections. The router is currently configured with the primary WAN (Cable Connection) in GigEth 1 and the secondary WAN in GigEth 2 (Bonded T's with Eth Handoff). GigEth 3 is the LAN connection.

The client has a remote application that they access in a hosted data center (static IP address). I configured a PBR to send only traffic destined to this remote data center location to go out GigEth2. It is working fine.

The problem I'm encountering is that I'm unable to ping or ssh into the router via the secondary WAN connection. My hunch is that the traffic is coming into the secondary WAN connection but returning through the primary WAN connection and therefore never getting to me. I added the no ip policy-class WAN2 rpt-check and still no response.

I need to be able to setup a remote ping to the secondary WAN interface for monitoring purposes. Also, today they had an extended outage and I couldn't remote into the router even though the secondary WAN remained up. I've run into this issue previously with dual WAN connections and need to figure out how to avoid it. We will ultimately be implementing a full autofailover between the two connections, but at a later date.

Thank you,

Chad

[jayh · Accepted solution]

This is a little tricky because you're manipulating traffic to the router itself rather than through it.

Assume for this example that your secondary WAN interface is 172.16.1.1/30 and its gateway is 172.16.1.2.

First create an access-list that captures traffic destined to the secondary WAN interface itself.

ip access-list extended backup-gw-list

  permit ip any host 172.16.1.1

Next create a route-map that matches traffic to that list and sets the next-hop as its gateway. All other traffic should follow default routing.

route-map local-map permit 10

   match ip address backup-gw-list

   set ip next-hop 172.16.1.2

route-map local-map permit 20

Now set local policy routing to honor that route-map for traffic destined to the router itself.

ip local policy route-map local-map

For a backup-only scenario where you only need to access the secondary gateway if the primary is down, you can build a probe-and-track on the primary gateway, set your default route to track it, and put in a floating static default to the secondary gateway. Using the route-map should allow you to reach the unit via the secondary at all times regardless of the status of the primary.

If you're doing this remotely, you might want to do a "reload in 15" first in case something gets fat-fingered and you lock yourself out before writing to memory. When all is good, do a "reload cancel" and then write mem.

[Anonymous]

jayh,

I was following you until you got to:

Now set local policy routing to honor that route-map for traffic destined to the router itself.

ip local policy route-map local-map

Can you elaborate on that last part some more?jayh​

Thank you,

Chad

[Anonymous]

In addition to the original request, we went ahead and got the failover working fine between the two ISPs using track/probe system. One additional thing is that we have a site-to-site VPN (IPSec) going to the DC which is terminating on the primary ISP interface. Any recommendations on how to configure it so I can get the IPSec VPN to failover to the other connection?

I can establish two separate IPSec VPNs (one terminating on interface A and the second terminating on the Interface B), but not sure how to handle the following:

1. Can you have the same crypto VPN statement on multiple WAN interfaces?

2. How would one handle the VPN IP Selectors to failover over during the failed state?

Thank you,

Chad

[jayh]

tincg_cw wrote:

jayh,

I was following you until you got to:

Now set local policy routing to honor that route-map for traffic destined to the router itself.

ip local policy route-map local-map

Can you elaborate on that last part some more?jayh

Sure. Policy routing by default manipulates traffic going through a router. Based on the route-map applied to the ingress interface, we can set the exit interface, change the next-hop, tweak BGP communities, etc. for packets transiting the router to their destination. In your case, you want to manipulate traffic that terminates locally on the router itself such as ssh connections and pings terminating locally on the backup interface of the router, not traversing it to some other destination.

The router has a default route to the ISP gateway on its primary interface that is used as long as the track/probe is up.

When we send traffic to the backup interface such as a ping or ssh session, it enters the backup interface, is processed by the router, and then follows the default route back out the primary. The primary ISP sees a packet originating from the IP of the backup interface. The ISP doesn't expect this traffic and drops it. Search BCP 38 for an explanation as to why this is done.

Therefore, we need a policy to match the destination of the backup interface and set the next hop to the backup ISP's gateway. But, this policy isn't for traffic through the router. It's for administrative traffic to the router. This policy therefore isn't applied to the interface. We want the policy to only affect traffic local to and processed by the router. This is defined by the syntax ip local policy. Because the policy isn't applied to an ingress interface, it doesn't affect traffic transiting the router. However, local traffic destined for the router's backup IP is inspected by the local policy, matches the ACL, and is directed by policy back out to the backup ISP rather than out the default route. This happens regardless of whether the primary ISP is up or down. The backup ISP expects this traffic and happily forwards it to its destination.

tl;dr: Local policy routing is used to set policy for traffic destined for the router, not transiting it.

[jayh]

tincg_cw wrote:

In addition to the original request, we went ahead and got the failover working fine between the two ISPs using track/probe system. One additional thing is that we have a site-to-site VPN (IPSec) going to the DC which is terminating on the primary ISP interface. Any recommendations on how to configure it so I can get the IPSec VPN to failover to the other connection?

Try this: Configuring Redundant VPN Tunnel Failover in AOS

If the data center end isn't Adtran gear, you'll need to consult that vendor's documentation for the other side.

[Anonymous]

Jayh,

Thank you very much for the additional information. This helps tremendously. I’ll work on implementing over the next few days and report back when I have it successfully working.

Thank you,

Chad