complex network design, help needed please...

[Anonymous]

i have a rather large network spanned across a 25 acre facility that has several buildings, a large warehouse with multiple IDFs (to distribute PoE switching) and several outdoor equipment (wifi/cameras) on light poles. This network is is aggregated using two 1544F switches. the two 1544F switches are port-channeled on ports 27 and 28 using 2 x 2.5Gbps SFPs to provide a 5Gbps backplane and ports 25 and 26 are uplinked to two separate 1544 ethernet switches (so each 1544 switch is connected to both fiber switches to provide uplink redundancy). The LAN consists of several VLANs to segregate traffic for different business entities residing in the premises for both voice and data, guest wireless, security systems, HVAC, handheld barcode scanners etc. all VLANs point to the IP 10.10.xxx.1 as their default gateway

I only have 1 router (Adtran 5305) at the moment that provides internet access over DS3 MPLS (secondary 10Mbps circuit is due in next few months)

I have a design question. I understand that with one router and single internet service my options for L2/L3 resilience are limited but I want to design my network the right way so I am prepared to scale it when the time comes.

Challenge 1. If I configure 10.10.xx.1 addresses on sub-interfaces of Adtran 5305 that will be straight forward but in case if the switch to which the router is connected failed I will have to manually move the router cable from the broken switch to the working switch and it should be fine. downside is that router's inside interface will become very busy doing inter-vlan routing.

Q2 To keep the router's inside interface from being the bottleneck  I could make vlan interfaces on one of the switch as 10.10.xxx.1 if that switch failed I will have to manually create the vlan interfaces on 2nd 1544 switch and move the cable across can I avoid this by doing VRRP between the two 1544 switches? (i understand i will still have to physically move the router cable over)

i am open to all ideas/suggestions

many thanks

J2

[Anonymous · Accepted solution]

J2:

Thank you for asking this question in the Support Community.  When designing a network for redundancy and scalability there are a vast array of variables that should be taken into consideration in the design.  I will not go into how much redundancy and scalability you desire, because the options are nearly infinite.  Based on my understanding of your network, I have recommended a design that provides a moderate amount of both redundancy and scalability.  Further, this option will alleviate any manual manipulation, and should failover automatically.  I will not go into the technical configurations, but instead describe the general concepts.

Below are the concepts I would employ in the design:

• Enable Layer 3 switching on the core switches and have the Internet router simply route traffic to the Internet (Configuring Layer 3 Switching in AOS)
• Directly connect the two core switches to each other via a port channel for redundancy and failover
• Establish a VRRP instance on the two core switches for the "LAN" facing devices to use as the default gateway (Configuring VRRP in AOS)
• On the Internet router, connect the Ethernet 0/1 interface to Core_Switch1 and the Ethernet 0/2 interface to Core_Switch2; create two arbitrary subnets on the Internet router ( i.e. - eth 0/1 = 10.1.1.0 /24; eth 0/2 = 10.1.2.0 /24)  and the corresponding VLAN interfaces on the directly connected core switches
• Core_Switch1 will have a default route to the Ethernet 0/1 IP address on the Internet router; this default route will be tracked with a Network Monitor ping probe (Configuring Network Monitor in AOS), to remove this default route if the link between the Core_Switch1 and the Internet router fails; add a floating static route to Core_Switch2 that will be added when the ping probe fails
• Core_Switch2 will have a default route to the Ethernet 0/2 IP address on the Internet router; this default route will be tracked with a Network Monitor ping probe, to remove this default route if the Internet router fails; add a floating static route to the Backup router (not implemented yet in your design) that will be added when this ping probe fails (Note:  This is not discussed here, but you can add further redundancy if the Backup router has multiple Ethernet interfaces)

Failover scenarios:

1. If Core_Switch1 fails or the port/cable between Core_Switch1 and the Internet router fails - VRRP will transition to make Core_Switch2 the master; and Core_Switch2 will route traffic to the Internet router on Ethernet 0/2
2. If the Internet router fails - traffic will route from the core switches to the Backup router

I hope this makes sense, but this is just a suggestion that I believe provides a reasonable amount of automatic failover.  However, there are multiple ways to achieve this, and you have to determine how much configuration, failover, redundancy, and scalability you desire and chose the proper design based on those requirements.  Please, do not hesitate to reply with any questions or additional information.  I will be happy to assist you in any way I can.

Levi

[Anonymous]

J2:

Do you have any follow up questions on my suggestions?  Please, let me know if there is anything I can do to help.

Levi

[Anonymous]

Hi Levi,

thank you very much for the design recommendation. I followed it and have designed the network along your suggestions. the two 1544F switches have VRRP enabled on all VLAN interfaces. Most PoE switches are connected to both fiber switches (with some doing port trunking to just one fiber switch, I understand that in case of a fiber switch failure any access switch that is directly and only connected to the affected switch will go down) besides this the two 1534 gigabit access switches are uplinked to both 1544 switches with STP keeping one link down to prvent loops.

I tested the network by powering off one of the 1544 switches and immediately vrrp on the 2nd 1544 makes it the master and starts advertising the default gateway IP arp for all VLANs, hence the devices connected to the 1534 switches also continue to work. Only disappointment was to learn that 1534 only supports a max of 8 VLAN interfaces, I hope a future firmware upgrade will extend this a bit.

If I have more questions I will follow up on the portal again.

Many thanks

J2

[Anonymous]

Fascinating stuff--looks like a fun project!