Static Route vs Network Monitor

[moulegend]

We had a network outage today where are primary service provider went down.  I configured our network here with a floating static route based on the fact that were using a PPP interface.  When everything went down, nothing transitioned over to our backup line.  Should I follow the instructions to setup up a WAN failover using Network Monitor in my 5305?  I need to resolve this issue as we were down for almost an hour and I was caught off guard because my floating static route did not failover.

Any help you can provide is greatly appreciated.

James

[Anonymous]

If you have another WAN connection, then I would definitely suggest configuring it for that.  Network monitoring is the way to go for that.  You can get away with a secondary route table entry with a higher cost than the default.  That will work if the PPP interface goes down, but network monitoring works better.

Check out this document by Adtran: https://supportforums.adtran.com/docs/DOC-2266

.

[Anonymous]

This one is better: https://supportforums.adtran.com/docs/DOC-2299

[jayh]

moulegend wrote:

We had a network outage today where are primary service provider went down.  I configured our network here with a floating static route based on the fact that were using a PPP interface.  When everything went down, nothing transitioned over to our backup line.  Should I follow the instructions to setup up a WAN failover using Network Monitor in my 5305?  I need to resolve this issue as we were down for almost an hour and I was caught off guard because my floating static route did not failover.

In almost all cases a PPP interface will logically go down if there's a physical issue with the underlying transport, such as a bad T1 cable span.  A simple floating static route to the backup handles these cases just fine.  There are rare failure modes where the underlying layer 2 link will stay up but the link doesn't pass traffic.  It sounds like this is what bit you.

You'll still need the floating static for the backup route, but you'll want a probe and track on the primary.  Typically you'll probe the gateway with an ICMP ping and track that probe.  Then your main default router will have the track statement added after the gateway, like:

ip route 0.0.0.0 0.0.0.0 ww.xx.yy.zz track primary-track

Ethernet interfaces to a DSL or cable modem almost always will stay up as long as the cable to the local modem is good and a probe-and-track is essential here.

I typically like to set the timers so that the backup route cuts over relatively quickly, around ten seconds, but it takes a significantly longer uptime on the primary before the unit switches back.  This way the unit will stay on the backup link if the primary is intermittent or flaky.

Sample code:

probe primary-probe icmp-echo

  destination ww.xx.yy.zz  ! Primary gateway or other remote test IP

  source-address ww.xx.yy.xx  !  IP address of primary interface

  period 3

  tolerance consecutive fail 3 pass 40  ! Cutover in 9-12 seconds, back in 2 minutes.

  no shutdown

!

!

track primary-track

  test if probe primary-probe

  no shutdown

!

ip route 0.0.0.0 0.0.0.0 ww.xx.yy.zz track primary-track

ip route 0.0.0.0 0.0.0.0 aa.bb.cc.dd 100 ! "100" is distance - make it greater than 1 and less than 255

[moulegend]

Just some background info, we have a T3 line from Verizon that is dedicated and hooked up through the use of PPP.  We have a secondary backup line that is direct fiber.  The secondary line was handed off through an Ethernet connection which I have plugged into the 2nd eth port on the back of my 5305.  Yesterday when we began experiencing issues, I physically unplugged the T3 connection and forced it down and it appeared the switch began to start but never transpired.  I kept getting a DNS issue but I'm assuming I have the failover setup incorrectly using the static route IP address of my secondary connection.

[Anonymous]

Sounds like something is missing in your configuration.

Ideally the T3 w/PPP should make the primary route 0.0.0.0 /0 in your configuration with an administrative distance that is higher than the primary, it should take over as the default gateway.

Usually that interface is in a different policy class with the appropriate NAT statements in your firewall to support it.

Are you able to ping an outside address such as 8.8.8.8 when you have the T3 connection disconnected? It may just be a simple DNS issue that needs to be addressed.

[jayh]

It would help to see the configuration (stripped of passwords, etc.)

Physically unplugging the T3 should cause the primary route to be withdrawn and fail over to the floating static.

[moulegend]

Correct, it is set up that way, I’ll have to see if I can ping 8.8.8.8 when its disconnected.

V/R,

James W. Mou

IT & Personnel Security Manager

GCC Technologies, LLC

Message was edited by: levi (Removed Contact Information)

[moulegend]

Please see the attached file. I exported from the GUI. Not sure why the failover test is still in there, I removed that from the GUI. Additionally, the route map is consistent with what I see in the GUI, the .1 address does not show.

James

[Anonymous]

I would get rid of the following route statements in the config:

ip route 0.0.0.0 0.0.0.0 204.111.217.226 10 (That is the interface address and not the gateway

The following should dynamically appear in the route table as connected routes so, they do not need to be included in the startup-config:

ip route 192.168.0.0 255.255.255.0 192.168.0.1

ip route 192.168.1.0 255.255.255.0 192.168.1.1

ip route 204.111.217.224 255.255.255.248 204.111.217.226 10

ip route 204.111.217.224 255.255.255.248 204.111.217.225 10

ip route 204.111.217.224 255.255.255.248 204.111.217.1 10

ip route 204.111.217.226 255.255.255.255 204.111.217.1 10

I would consider making a probe & track configuration vs. route maps. Less CPU overhead and easier to keep track of.

Network Monitoring

=================================

Probe wan1 icmp-echo

Destination 8.8.8.8 (a known good IP address that returns pings that is beyond your primary WAN subnet.)

period 2

tolerance consecutive fail 2 pass 2

no shutdown

track wan1

test if probe wan1

no shutdown

ip route 8.8.8.8 255.255.255.255 157.130.42.85

ip route 8.8.8.8 255.255.255.255 null 0 10 (this makes the previous route statement the only way to ping 8.8.8.8)

(without the null statement, the probe will be able to ping 8.8.8.8 via. The secondary wan address and it will switch back to the primary with a false positive. )

ip route 0.0.0.0 0.0.0.0 157.130.42.85 track wan1 (only good when wan1 is in a passed state)

ip route 0.0.0.0 0.0.0.0 204.111.217.1 10 (only works when the primary route is unavailable)

Ip firewall fast-nat-failover

Interface ETH 0/2 should be assigned to a policy-class just like PPP 1 is. Ideally a separate policy-class such as Public2

Then issue a second NAT statement in the Private security zone that NAT’s to the ETH 0/2 interface or IP address overload policy Public2

That should do it.

[moulegend]

I’ve attached the updated config file, does that look about right?

James

[Anonymous]

- Are you still in need of assistance regarding this configuration? I took a look at the configuration and noticed that you had added the network monitoring portion below the configuration file. If you plan on uploading the configuration in that format, it would cause some conflicts in the settings applied.

Please let us know if you are still in need of any assistance and we will be more than happy to help.

Thanks,

Noor

[moulegend]

Good Afternoon Noor,

I've attached our updated config, please let me know if you see any issues with it now.

Thanks,

James

[Anonymous · Accepted solution]

- I think you are almost there with the failover. I do have a couple of suggestions that should help the transition be seamless:

1. In the Private Security Zone, I would move the "Traffic to NetVanta" rule so that is above the "NAT wizard-ics" rule.

2. In the CLI, I would disable load sharing. From enable mode, the command would be no ip load-sharing per-destination.

3. Assign the security zone "Public-Shentel" to interface eth 0/2. Currently, eth 0/2 does not have a security zone assigned to it and this will cause issues when traffic is NATted out that connection.

4. In the CLI, from enable mode, the following command should be entered: ip firewall fast-nat-failover

This should get failover to work should the T3 or PPP ever change its state to down. We would not need network monitoring in this case because if the T3 or PPP interface goes down, the route pointing out the PPP interface will automatically be removed. This is not available for those routes that point out an ethernet interface. The question is if this is sufficient. If there is an outage further down the line, then you will need to configure network monitoring because T3 or PPP could potentially remain in an up state while something further in the provider's network has an outage.

If you would like to configure network monitoring, I would recommend following this guide to get your probe and track set up: Configuring WAN Failover with Network Monitor in AOS

The firewall/security zone settings should be fine as long as you make the changes I suggested earlier in the post.

Please do not hesitate to let us know if you have any further questions.

Thanks,

Noor

[moulegend]

Thank you so much!

V/R,

James W. Mou

IT & Personnel Security Manager

tion

[Anonymous]

-

I went ahead and flagged the "Correct Answer" on this post to make it more visible and help other members of the community find solutions more easily. If you don't feel like the answer I marked was correct, feel free to come back to this post to unmark it and select another in its place with the applicable buttons.  If you have any additional information on this that others may benefit from, please come back to this post to provide an update.  If you still need assistance, we would be more than happy to continue working with you on this - just let us know in a reply.

Thanks,

Noor