Netvanta 3120 Multiple WAN Connections

[jcrabtreetol]

From what I have read the 3120 should support this.

Here is the configuration, have 3 T1's bonded to an ethernet hand off provided by the ISP. This connection is currently connected to the Eth0 on the Netvanta and is working fine.

The T1's are getting saturated. Have a 30/3 Fiber brought in with an ethernet hand-off.

Want to route all Web HTTP/HTTPS traffic over the Fiber and use the T1 as a failover in the event the Fiber goes down.
Want to keep my current Port Forwarding going to the server, going to also want the Fiber to act as a secondary connection for redundancy to the Servers (SMTP Services)

Have setup vlan 2 on Switchport 0/1 and configured an IP Interface to that VLAN, named Fiber. From the connectivity menu I can ping out the Fiber. However if I manually tell a PC to route out the Fiber it Fails. I already setup the ACL to allow all out on the Fiber Policy. I also cannot seem to ping the fiber externally even though the ACL permits so.

Attempted to follow the PDF for Dual WAN in the AOS, That does not really get into the specifics in this setup as I cannot assign an IP directly to the Switchport like an Ethernet port.

Have setup dual wan on a 3448 in the past and other routers. This client would like to try this on the current 3120 they have vs spending the money for the upgrade.

Below is the config (Scrubbed of some data or replaced with generic numbers for representation.)
Hoping the community can help as I am having a problem getting this to operate as intended.

! ADTRAN OS version 18.02.01.00.E

! Platform: NetVanta 3120, part number 1700601G2

!

hostname "ImaRouter"

enable password encrypted xxxxxxxxxxxxxxxxxxxxxxxxxx

!

clock timezone -5-Eastern-Time

!

ip subnet-zero

ip classless

ip default-gateway 184.0.0.1

ip routing

ip domain-proxy

ip name-server 8.8.8.8

!

ip local policy route-map Failover

!

no auto-config

!

event-history on

event-history priority debug

no logging forwarding

logging forwarding priority-level info

no logging email

!

service password-encryption

!

username "netadmin" password encrypted "xxxxxxxxxx"

!

!

ip firewall

no ip firewall alg msn

no ip firewall alg mszone

no ip firewall alg h323

!

no dot11ap access-point-control

!

track "Failover"

  no shutdown

!

ip crypto

!

crypto ike policy 101

  initiate main

  respond main

  attribute 1

    encryption 3des

    hash md5

    authentication pre-share

!

crypto ike policy 102

  initiate main

  respond main

  attribute 1

    encryption 3des

    hash md5

    authentication pre-share

!

VPN 1 - scrubbed

VPN 2 - scrubbed

!

crypto ipsec transform-set esp-3des-esp-md5-hmac esp-3des esp-md5-hmac

  mode tunnel

!

crypto map VPN 20 ipsec-ike

  description VPN1

  match address VPN-20-vpn-selectors

  set transform-set esp-3des-esp-md5-hmac

  set pfs group5

  ike-policy 101

crypto map VPN 30 ipsec-ike

  description VPN2

  match address VPN-30-vpn-selectors

  set transform-set esp-3des-esp-md5-hmac

  set pfs group5

  ike-policy 102

!

qos dscp-cos 0 8 16 24 32 40 48 56 to 0 1 2 3 4 5 6 7

!

!

!

!

vlan 1

  name "Default"

!

vlan 2

  name "Fiber"

!

!

interface eth 0/1

  ip address  184.0.0.2  255.255.255.248

  ip address  184.0.0.3  255.255.255.255  secondary

  ip access-policy Public

  crypto map VPN

  no rtp quality-monitoring

  no shutdown

  no lldp send-and-receive

!

!

interface switchport 0/1

  no shutdown

  switchport access vlan 2

!

interface switchport 0/2

  no shutdown

!

interface switchport 0/3

  no shutdown

!

interface switchport 0/4

  no shutdown

!

interface vlan 1

  ip address  192.168.1.1  255.255.255.0

  ip access-policy Private

  no shutdown

!

interface vlan 2

  description Fiber WAN

  ip address  71.0.0.2  255.255.255.252

  ip mtu 1500

  ip access-policy Public-Fiber

  no rtp quality-monitoring

  no awcp

  no shutdown

!

route-map Failover permit 1

  description "Failover"

  match ip address Failover

  set ip next-hop 71.0.0.1

  set interface null 0

!

ip access-list standard wizard-ics

  remark Internet Connection Sharing

  permit any

!

ip access-list extended Failover

  permit icmp any  hostname 4.2.2.2   

!

ip access-list extended self

  remark Traffic to NetVanta

  permit ip any  any     log

!

ip access-list extended VPN-20-vpn-selectors

!

ip access-list extended VPN-30-vpn-selectors

!

ip access-list extended web-acl-10

  remark PRTG Traffic Monitoring

  permit tcp any  host 184.0.0.2 eq 8080   log

!

ip access-list extended web-acl-14

  remark Many:1 Fiber

  permit ip any  any   

!

ip access-list extended web-acl-15

  remark Allow Ping

  permit icmp any  any  echo   log

!

ip access-list extended web-acl-4

  remark Server

  permit tcp any  host 184.0.0.2 eq smtp   log

  permit tcp any  host 184.0.0.2 eq www   log

  permit tcp any  host 184.0.0.2 eq https   log

  permit tcp any  host 184.0.0.2 eq 1723   log

!

ip access-list extended web-acl-5

  remark SERVER TS

  permit tcp any  host 184.0.0.3 eq 3389   log

!

ip access-list extended web-acl-6

  remark PhoneSystem

  permit tcp any  host 184.0.0.2 eq xxxxx   log

!

ip access-list extended web-acl-7

  remark PhoneSystem Voicemail

  permit tcp any  host 184.0.0.2 eq xxxx   log

!

ip access-list extended web-acl-9

  remark Block SMTP on workstations

  deny   tcp host 192.168.1.5  any    log

  permit tcp any  any eq smtp   log

!

ip access-list extended wizard-remote-access

  remark do not hand edit this ACL

  permit icmp any  any  echo   log

!

!

ip policy-class Private

  allow list VPN-30-vpn-selectors stateless

  allow list VPN-20-vpn-selectors stateless

  allow list self self

  discard list web-acl-9

  nat source list wizard-ics interface eth 0/1 overload

!

ip policy-class Public

  allow reverse list VPN-30-vpn-selectors stateless

  allow reverse list VPN-20-vpn-selectors stateless

  nat destination list web-acl-5 address 192.168.1.5

  allow list wizard-remote-access self

  nat destination list web-acl-10 address 192.168.1.6

  allow list web-acl-12 self

!

ip policy-class Public-Fiber

  nat source list web-acl-14 address 71.0.0.2 overload

  allow list web-acl-15 self

!

!

ip route 0.0.0.0 0.0.0.0 184.0.0.1

ip route 0.0.0.0 0.0.0.0 71.0.0.1

!

no tftp server

no tftp server overwrite

ip http server 80

ip http secure-server

snmp agent

no ip ftp server

ip ftp server default-filesystem flash

no ip scp server

no ip sntp server

!

snmp-server community public RO

snmp-server group Public v1

snmp-server group Public v2

!

ip sip udp 5060

ip sip tcp 5060

!

line con 0

  login

  password encrypted xxxxxxxxxxx

!

line telnet 0 4

  login local-userlist

  password encrypted xxxxxxxxxxxxxxx

  no shutdown

line ssh 0 4

  login local-userlist

  no shutdown

!

end

[Anonymous]

Hi jcrabtreetol:

I'm on my way out the door, but noticed a couple of things.  I would:

• Add a probe to ping 4.2.2.2
• Test if probe (above) in the track
• Add a nat overload statement to Private by way of vlan 2 (so you'll end up with two overload lines)
• Default routes
  • ip route 0.0.0.0 0.0.0.0 184.0.0.1 Failover
    • This makes the route valid only when Failover track = PASS
  • ip route 0.0.0.0 0.0.0.0 71.0.0.1 10
    • This makes the route at a higher distance/metric than the first route which is distance = 0

Generally, we use at least two probes.  If something happens to 4.2.2.2 (not impossible)--or your path to it--then you don't want the Internet connection to failover.  But the chances of two well-known hosts going down simultaneously are smaller.  Then make the track require that both probes are failed before the track state changes to fail, like this:

!

probe Internet1 icmp-echo

  destination 4.2.2.2

  source-address 173.161.18.9

  period 6

  tolerance rate fail 8 pass 8 of 10

  no shutdown

!

probe Internet2 icmp-echo

  destination 8.8.8.8

  source-address 173.161.18.9

  period 6

  tolerance rate fail 8 pass 8 of 10

  no shutdown

!

!

! * "or" means track PASS if either probe is in PASS state

!

track Internet

  test list or

    if probe Internet1

    if probe Internet2

  no shutdown

!

Note the tolerances.  You want to be sure when you failover and not that you're just seeing a blip on the radar or typical Internet congestion.  Resist the temptation to failover too quickly.  I think it's a good deed to avoid pinging public hosts too often.  Also, you may want to have your policy-sessions clear when the default route fails over:

!

ip firewall fast-nat-failover

ip firewall fast-allow-failover

!

For icing on the cake, have the router email you and your team when the track changes (requires an SMTP server to be available):

!

! * Enable event history, SMTP logging; account details

!

event-history on

no logging forwarding

logging forwarding priority-level info

logging email on

logging email priority-level fatal

logging email receiver-ip alerts.example.com port 30025 auth-username monitor-alert auth-password xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

logging email address-list noc-alert@example.com

logging email sender monitor-alert@example.com

!

!

!

! EDIT:  "do show interfaces" to match your actual interfaces

!

mail-client Internet-up

  subject Internet Up 🙂

  capture commands

    do show interfaces eth 0/2

    do show probe

    do show track

  do show ip route | include 0.0.0.0/0

  do show ip policy-stats

    do show event-history | exclude id=firewall

  do show version

    exit

  send trigger track Internet pass

  no shutdown

!

mail-client Internet-down

  subject Internet Down 😞

  capture commands

    do show interfaces eth 0/2

    do show probe

    do show track

  do show ip route | include 0.0.0.0/0

    do show ip policy-stats

    do show event-history | exclude id=firewall

  do show version

    exit

  send trigger track Internet fail

  no shutdown

!

Best,

CJ

[jcrabtreetol]

Yup I had that in my mind to do for the fail over.

However here is what I am seeing as a problem.

I setup a route in the route table say to 8.8.8.8 / 32 and I use the gateway of the Fiber 71.0.0.1 I cannot ping or do anything.

If I trace it from a workstation it just dies at the Netvanta. Leaving the same route in palce and do a trace from the connectivity it goes out VLAN 2 no problem and reaches the destination.

so likely the problem is that switch port is VLAN 2. How do I go about getting around that?

I did have btw the fail over in place when I first set it up and the T1 did have a problem and rolled over to the Fiber and all internet went offline. So at least I know the fail over was working. so now I just have to get this last part going correctly.

[Anonymous]

I missed that your Internet interfaces are in separate policy-classes.  Given that, maybe verify the routes as I suggested earlier and try this for your policy-classes.  No nat overload statement in Public-Fiber.

!

ip policy-class Private

  allow list VPN-30-vpn-selectors stateless

  allow list VPN-20-vpn-selectors stateless

  allow list self self

  discard list web-acl-9

  nat source list wizard-ics interface eth 0/1 overload policy Public

  nat source list wizard-ics interface vlan 2 overload policy Public-Fiber

!

ip policy-class Public

  allow reverse list VPN-30-vpn-selectors stateless

  allow reverse list VPN-20-vpn-selectors stateless

  nat destination list web-acl-5 address 192.168.1.5

  allow list wizard-remote-access self

  nat destination list web-acl-10 address 192.168.1.6

  allow list web-acl-12 self

!

ip policy-class Public-Fiber

  allow list web-acl-15 self

!

[jcrabtreetol]

So I can ping out if I create a route for the specific IP in the route table now.

How do I tell all traffic to use that route for all http/https traffic?

[Anonymous · Accepted solution]

Forgive me, I thought you were aiming for a simple, everything-goes failover.  If you want to force http/s traffic out your "secondary" Internet connection, then you should use a route-map.  This is called Policy Based Routing (PBR).  In a nutshell, you apply a route-map to an interface to analyze traffic at ingress.  The route-map looks for matching criteria.  This can be a variety of things, but an ACL is often best.  When matched, you can set the next-hop address or egress interface.  The route-map policy name is arbitrary.

!

ip local policy route-map Detour

!

interface vlan 1

  description LAN

  ip address  192.168.1.1  255.255.255.0

  ip policy route-map Detour

  ip access-policy Private

  no shutdown

!

!

route-map Detour permit 10

  match ip address out-ISP2

  set ip next-hop 71.0.0.1

!

!

ip access-list extended out-ISP2

  remark PBR for HTTP and HTTPS

  permit tcp 192.168.1.0 0.0.0.255 any eq www

  permit tcp 192.168.1.0 0.0.0.255 any eq https

!

That much should get PBR working, but if you want failover for this policy-routed traffic, then you should apply a track to your ACL permit lines.  Note that deny any is typically used at the end to keep the ACL from becoming "empty."  If the track fails, then it essentially negates those permit lines, leaving an empty ACL, which is equal to an implicit match all in AOS.  Adding deny any after your permit lines should keep the ACL from becoming empty, but I have experienced a problem in the R10.9 series firmware where that line goes missing.  I ended up adding a 'nonsense' permit line which achieves the same goal to get by.  Hope this isn't too confusing (and hopefully it'll be fixed soon)...

Here's how you might alter the ACLs above to include a track:

!

ip access-list extended out-ISP2

  remark PBR for HTTP and HTTPS

  permit tcp 192.168.1.0 0.0.0.255 any eq www  track Internet

  permit tcp 192.168.1.0 0.0.0.255 any eq https  track Internet

  deny any

   permit ip host 1.1.1.1  host 1.1.1.2 (forget this line if the deny any stays put for you)

!

[jcrabtreetol]

So I added the IP Policy and created the new Route Map.

Here is the interesting thing.

in our monitoring software all the machines now state their externail ip is the 71.0.0.2 however when you run a speed test it comes back as the T1 speed.

However the other problem is even the SMTP outbound is stating the messages are being received by the 71.0.0.2 and not the 184.0.0.2

So I thought what would happen is I chnaged from permit tcp to permit ip and change to ALL vs matching http/https

This however did make the PC speed test come back in the 20Mbps range.

How do I keep the SMTP and servers out of this new policy? I tried a deny in the route map and that did not change anything. (Deny is what I seen in the PBR documentation)

I also appear to be having some odd problems with the traffic coming over the VPN. it comes in over the interface 184.0.0.1. Though not as concerned with that at the moment until I get this part working I have implemented a work around for those remote offices.

Though it does appear to finally be getting someplace.

[Anonymous]

I get things mixed up sometimes--any chance you could provide a current config?

[jcrabtreetol]

Below is what is currently in use.

! ADTRAN OS version 18.02.01.00.E

! Boot ROM version 17.01.01.00

! Platform: NetVanta 3120, part number 1700601G2

!

hostname "IAMAROUTER"

enable password encrypted xxxxxxxxxxxxxxxxxxxxx

!

clock timezone -5-Eastern-Time

!

ip subnet-zero

ip classless

ip default-gateway 184.0.0.1

ip routing

ip domain-proxy

ip name-server 8.8.8.8

!

ip local policy route-map Detour

!

no auto-config

!

event-history on

event-history priority debug

no logging forwarding

logging forwarding priority-level info

no logging email

!

service password-encryption

!

username "admin" password encrypted "xxxxxxxxxxxxxxxxx"

!

!

ip firewall

ip firewall fast-nat-failover

ip firewall fast-allow-failover

no ip firewall alg msn

no ip firewall alg mszone

no ip firewall alg h323

!

no dot11ap access-point-control

!

track "Failover"

  snmp trap state-change

  no shutdown

!

ip crypto

!

crypto ike policy 101

  initiate main

  respond main

  local-id address 184.0.0.2

  peer 24.0.0.100

  attribute 1

    encryption 3des

    hash md5

    authentication pre-share

!

crypto ike policy 102

  initiate main

  respond main

  local-id address 184.0.0.2

  peer 70.0.0.100

  attribute 1

    encryption 3des

    hash md5

    authentication pre-share

!

crypto ike remote-id address 24.0.0.100 preshared-key xxxxx ike-policy 101 crypto map VPN 20 no-mode-config no-xauth

crypto ike remote-id address 70.0.0.100 preshared-key xxxxx ike-policy 102 crypto map VPN 30 no-mode-config no-xauth

!

crypto ipsec transform-set esp-3des-esp-md5-hmac esp-3des esp-md5-hmac

  mode tunnel

!

crypto map VPN 20 ipsec-ike

  description Third Location

  match address VPN-20-vpn-selectors

  set peer 24.0.0.100

  set transform-set esp-3des-esp-md5-hmac

  set pfs group1

  ike-policy 101

crypto map VPN 30 ipsec-ike

  description Second location

  match address VPN-30-vpn-selectors

  set peer 70.0.0.100

  set transform-set esp-3des-esp-md5-hmac

  set pfs group1

  ike-policy 102

!

qos dscp-cos 0 8 16 24 32 40 48 56 to 0 1 2 3 4 5 6 7

!

vlan 1

  name "Default"

!

vlan 2

  name "Fiber"

!

!

interface eth 0/1

  ip address  184.0.0.2  255.255.255.248

  ip address  184.0.0.3  255.255.255.255  secondary

  ip access-policy Public

  crypto map VPN

  no rtp quality-monitoring

  no shutdown

  no lldp send-and-receive

!

!

interface switchport 0/1

  spanning-tree edgeport

  no shutdown

  switchport access vlan 2

!

interface switchport 0/2

  no shutdown

!

interface switchport 0/3

  no shutdown

!

interface switchport 0/4

  no shutdown

!

!

interface vlan 1

  ip address  192.168.1.1  255.255.255.0

  ip policy route-map Detour

  ip access-policy Private

  no shutdown

!

interface vlan 2

  description Fiber WAN

  ip address  71.0.0.2  255.255.255.252

  ip access-policy Public-Fiber

  no rtp quality-monitoring

  no awcp

  no shutdown

!

!

!

route-map Failover permit 1

  description "Failover"

  match ip address Failover

  set ip next-hop 71.0.0.1

  set interface null 0

route-map Detour permit 10

  match ip address out-ISP2

  set ip next-hop 71.0.0.1

!

!

!

ip access-list standard wizard-ics

  remark Internet Connection Sharing

  permit any

!

!

ip access-list extended Failover

  permit icmp any  hostname 4.2.2.2

!

ip access-list extended out-ISP2

  permit tcp 192.168.17.0 0.0.0.255  any eq www

  permit tcp 192.168.17.0 0.0.0.255  any eq https

  permit tcp 192.168.18.0 0.0.0.255  any eq www

  permit tcp 192.168.18.0 0.0.0.255  any eq https

  permit tcp 192.168.1.0 0.0.0.255  any eq www

  permit tcp 192.168.1.0 0.0.0.255  any eq https

  permit ip host 192.168.1.27  any     log

  permit ip host 192.168.1.6  any     log

  permit tcp host 192.168.1.5  any

!

ip access-list extended self

  remark Traffic to NetVanta

  permit ip any  any     log

!

ip access-list extended VPN-20-vpn-selectors

  permit ip 192.168.1.0 0.0.0.255  192.168.18.0 0.0.0.255

!

ip access-list extended VPN-30-vpn-selectors

  permit ip 192.168.1.0 0.0.0.255  192.168.17.0 0.0.0.255

!

ip access-list extended web-acl-10

  remark PRTG Traffic Monitoring

  permit tcp any  host 184.0.0.2 eq 8080   log

!

ip access-list extended web-acl-12

  remark Nemsys Remote Router Access

  permit tcp host 71.0.0.206  any eq www   log

  permit tcp host 71.0.0.206  any eq telnet   log

  permit tcp host 71.0.0.206  any eq https   log

  permit icmp host 71.0.0.206  any  echo   log

!

ip access-list extended web-acl-14

  remark Many:1 Fiber

  permit ip any  any

!

ip access-list extended web-acl-15

  remark Allow Ping

  permit icmp any  any  echo   log

!

ip access-list extended web-acl-22

  remark External access

  permit tcp host 192.252.202.248  any eq www   log

  permit tcp host 192.252.202.248  any eq telnet   log

  permit tcp host 192.252.202.248  any eq ssh   log

  permit icmp host 192.252.202.248  any  echo   log

!

ip access-list extended web-acl-4

  remark SBS2011 Server

  permit tcp any  host 184.0.0.2 eq smtp   log

  permit tcp any  host 184.0.0.2 eq www   log

  permit tcp any  host 184.0.0.2 eq https   log

  permit tcp any  host 184.0.0.2 eq pop3   log

  permit tcp any  host 184.0.0.2 eq 1723   log

  permit tcp any  host 184.0.0.2 eq 4125   log

!

ip access-list extended web-acl-5

  remark Hyper-V Host

  permit tcp any  host 184.0.0.2 eq 3389   log

!

ip access-list extended web-acl-6

  remark PhoneSystem Admin

  permit tcp any  host 184.0.0.2 eq 35300   log

!

ip access-list extended web-acl-7

  remark PhoneSystem Voicemail

  permit tcp any  host 184.0.0.2 eq 10000   log

!

ip access-list extended web-acl-9

  remark Block SMTP on workstations

  deny   tcp host 192.168.1.4  any    log

  permit tcp any  any eq smtp   log

!

ip access-list extended wizard-remote-access

  remark do not hand edit this ACL

  permit icmp any  any  echo   log

!

!

ip policy-class Private

  allow list VPN-30-vpn-selectors stateless

  allow list VPN-20-vpn-selectors stateless

  allow list self self

  discard list web-acl-9

  nat source list wizard-ics interface eth 0/1 overload policy Public

  nat source list wizard-ics interface vlan 2 overload policy Public-Fiber

!

ip policy-class Public

  allow reverse list VPN-30-vpn-selectors stateless

  allow reverse list VPN-20-vpn-selectors stateless

  nat destination list web-acl-7 address 192.168.1.152

  nat destination list web-acl-4 address 192.168.1.4

  nat destination list web-acl-6 address 192.168.1.150

  nat destination list web-acl-5 address 192.168.1.5

  allow list wizard-remote-access self

  nat destination list web-acl-10 address 192.168.1.6

  allow list web-acl-12 self

  allow list web-acl-22 self

!

ip policy-class Public-Fiber

  allow list web-acl-15 self

!

!

ip route 0.0.0.0 0.0.0.0 184.0.0.1

ip route 0.0.0.0 0.0.0.0 71.0.0.1 10

!

no tftp server

no tftp server overwrite

ip http server

ip http secure-server

snmp agent

no ip ftp server

ip ftp server default-filesystem flash

no ip scp server

no ip sntp server

!

snmp-server community public RO

snmp-server group Public v1

snmp-server group Public v2

!

ip sip udp 5060

ip sip tcp 5060

!

line con 0

  login

  password encrypted xxxxxxxxxxxxxxxxxxxxxxxx

!

line telnet 0 4

  login local-userlist

  password encrypted xxxxxxxxxxxxxxxxxxxxxxxxx

  no shutdown

line ssh 0 4

  login local-userlist

  no shutdown

!

end

[jcrabtreetol]

I hate it when I come across a post that has the original problem but not a full outline of what corrected the problem. We got busy and I could not come back to post up. I also hit up Adtran Support also for some of the answers.

too much scrubbing to post the whole config again. but here were the areas that helped. and all answers were correct as they lead me into the direction.

So what was helpful is remembering the order of the rules, top down. Added a deny rule in the Out-ISP2 to the Routers LAN Interface. this allowed the Web interface to start working again.

Why I say that is helping is because in order to re-arrange the rules the GUI is nice to click the up/down arrow.

so here is what caused a number of problems. Having matching IPs in the detour group that overlap. once the match is met it stops processing the list. so start the layering up top.