- Subscribe to RSS Feed
- Bookmark
- Subscribe
- Printer Friendly Page
- Report Inappropriate Content
- Adtran Support Community
- :
- Documentation
- :
- ADTRAN Operating System - AOS
- :
- General
- :
- RapidRoute/ FFE in AOS
RapidRoute/ FFE in AOS
RapidRoute/ FFE in AOS
RapidRoute, or the fast forwarding engine (FFE), is used by ADTRAN routers to help optimize the packet-routing process for packets that go through repetitive sets of rules and procedures before being routed to their destination. After the first packet in a stream establishes a "flow", the following packets from that stream will match the flow and follow the same routing operations as the first packet without the extra processing decisions normally associated with a packet. This document covers FFE and when it should be used.
Sections Included In this Document
Hardware and Software Requirements
Configuration using the AOS Web Interface
Troubleshooting FFE and Using FFE Show Commands
Hardware and Software Requirements
FFE was introduced in AOS 13 and is included on all platforms capable of running that version that are noted in the AOS Feature Matrix .
As of R10.4.0, FFE is enabled on all supported IP interfaces by default.
RapidRoute is the fast forwarding engine (FFE) used by ADTRAN routers to help optimize the packet-routing process for packets that go through repetitive sets of rules and procedures before being routed to their destination. This packet processing architecture in NetVanta routers and IAD/IPBG units classifies packets into packet flows based upon:
- the IP protocol used by the packet
- the source and destination IP address
- protocol-specific information, such as source and destination TCP/UDP port numbers.
- Ingress interface
- IP Type of Service (ToS) bytes
Packet flows are defined as the unidirectional representation of a conversation between two IP hosts. Using this information, each configured ingress interface maintains a traffic flow table. The identifiers in this table are the same as those in the firewall association table. This allows a commonality between a flow entry and the firewall’s association selector. When packets are received, they can match a previous session and bypass extra unnecessary processing since the router can predict the final routing decision. This session-matching allows the router to process traffic more quickly and efficiently, because as each packet is classified, it is placed in a traffic flow of other packets with similar features. Each packet is classified only once, rather than classified every time it is used by an AOS feature, such as the firewall/NAT, VPN, or access-groups. RapidRoute is a beneficial routing enhancement, especially in instances where traffic must be prioritized, delivered on quality of service (QoS) requirements, or kept from monopolizing bandwidth. Using RapidRoute, especially in conjunction with the AOS firewall, can greatly improve performance of a NetVanta router or IAD/IBPG product.
Below is a piece of the AOS device’s configuration for the example below (note that some firewall commands changed in AOS version 18.03.01- The command below are taken from 18.03.01/R10 and up). In this example, the router allows SSH and other remote access to itself from the outside:
# show running-config
Lines Omitted for Brevity…
interface eth 0/1
ip access-policy Public
!
Lines Omitted for Brevity…
!
ip access-list extended ListOne
permit tcp any any eq 10000
!
ip access-list extended ListTwo
permit tcp any any eq 20000
!
ip access-list extended ListThree
permit tcp any any eq 30000
!
ip access-list extended RemoteAccess
permit tcp any any eq http
permit tcp any any eq https
permit tcp any any eq telnet
permit tcp any any eq ftp
permit tcp any any eq ssh
!
ip policy-class Public
allow list ListOne
allow list ListTwo
allow list ListThree
allow list RemoteAccess
Looking above, consider a SSH packet sourced from 10.0.0.1 destined for 192.168.1.1. The source port is TCP 5000 and the destination port is SSH’s port, TCP 22. This packet is received on the AOS device’s Ethernet 0/1 interface. When it is received and enters the firewall, the firewall will see that it entered the Public policy-class and it will try to match it to every entry inside of each access list existing in the policy-class rules (in a top-down fashion). The firewall will first check it against Access-Control List (ACL) “ListOne” and see that it does not match the packet’s port (SSH is TCP 22). It will do the same for ACL’s “ListTwo” and “ListThree”. Once it gets to ACL “RemoteAccess,” it will begin checking each of the rules inside that access list. After not matching the first four entries in the ACL, the packet will finally match the 5th rule, permit tcp any any eq ssh.
This can be a time consuming process if thousands of packets come over that same process (SSH will send several per second). FFE allows AOS to create a session using the following parameters in the packet:
- Source IP address
- Destination IP address
- Protocol
- Source port
- Destination port
- Ingress interface
- IP ToS byte
Then, the session will keep track of how the firewall and other processes treated this packet and get to the final routing decision- cutting out all of the lengthy tasks in the middle. In the example above, when the second SSH packet enters the firewall, it will match the FFE session in the FFE table and be immediately matched to the allow rule in the firewall and sent to the routing table accordingly. The session would look like the output below:
T Proto Source Destination Specific Age Used Drops Flags
* tcp 10.0.0.1 192.168.1.1 5000 22 11s 0 0 F
FFE works in the background on every flow in the unit and is essential to efficient operation in the unit. The above is only a short example of what it does, but provides context for the show commands below.
It is recommended to always have FFE enabled.
Configuration using the AOS Web Interface
Currently FFE can not be enabled or disabled using the AOS Web interface.
To enable RapidRoute on an interface, use the ip ffe command from the interface configuration mode prompt. This command should be applied to all active IP interfaces. For example:
(config)# interface eth 0/1
(config-int-eth 0/1)# ip ffe
(config)# interface ppp 1
(config-int-ppp 1)# ip ffe
(config)# interface vlan 10
(config-int-vlan10)# ip ffe
It is recommended to always have FFE enabled.
Troubleshooting and Using FFE Show Commands
For this section, it is important to understand two terms/phrases that are used throughout this document and in troubleshooting FFE and other problems where FFE is involved. These are “slow path” and “fast path”. The term “slow path” is the path a packet takes through access-lists, access-groups, etc. without ip ffe enabled or without matching an FFE flow. The “fast path” is the opposite – the path that a packet takes when it matches an FFE session and is routed accordingly. The distinction between the two is important when troubleshooting. A session is created when a packet comes into the router on an ingress interface and FFE sees it as a potential session. That first packet comes in before a session is established (it is used to create the session) so the first packet will always take the “slow path”. Every packet after that should take the “fast path,” speeding up processing of packets as they ingress on an interface.
FFE show commands will give information about all of the current traffic running through the AOS unit. Just like the show ip policy-sessions command, all of the sessions associated with each interface and their protocol, ports, and IP addresses will be cataloged in the output.. However, FFE can also show you how long the session has been in use, how many packets have hit and not hit the policy, and a flags field which we will discuss later.
Below is an example of the output from the show ip ffe command:
(config)#show ip ffe
Timeout TCP UDP ICMP AH ESP GRE Other
Age: 30m 0s 30m 0s 30m 0s 30m 0s 30m 0s 30m 0s 30m 0s
Inactive: 15s 15s 15s 15s 15s 15s 15s
Type: * valid, ! ineligible, - rejected
Flags: F firewall, N NAT, T altered ToS, D don't fragment, I IPSec
--------------------------------
Ingress: eth 0/1
3902006 hits, 963818 misses, 0 drops
T Proto Source Destination Specific Age Used Drops Flags
* tcp 64.74.103.129 209.168.144.146 443 2860 11s 0 0 FN
* tcp 64.74.103.134 209.168.144.146 443 1036 14s 0 0 FN
* tcp 216.115.218.197 209.168.144.146 443 2772 7m12s 72 0 FN
* tcp 64.74.103.161 209.168.144.146 443 1225 3s 0 0 FN
* tcp 64.74.103.148 209.168.144.146 443 1041 5s 0 0 FN
! tcp 76.164.174.115 209.168.144.146 37427 22 9s 44 0
* tcp 216.115.218.197 209.168.144.146 443 1054 1m39s 16 0 FN
Number of entries: 7 of 7 (4096 maximum)
Below, the show command output will be examined in sections:
----Section 1----
(config)#show ip ffe
Timeout TCP UDP ICMP AH ESP GRE Other
Age: 30m 0s 30m 0s 30m 0s 30m 0s 30m 0s 30m 0s 30m 0s
Inactive: 15s 15s 15s 15s 15s 15s 15s
----Section 2----
Type: * valid, ! ineligible, - rejected
Flags: F firewall, N NAT, T altered ToS, D don't fragment, I IPSec
----Section 3----
--------------------------------
Ingress: eth 0/1
3902006 hits, 963818 misses, 0 drops
T Proto Source Destination Specific Age Used Drops Flags
* tcp 64.74.103.129 209.168.144.146 443 2860 11s 0 0 FN
* tcp 64.74.103.134 209.168.144.146 443 1036 14s 0 0 FN
* tcp 216.115.218.197 209.168.144.146 443 2772 7m12s 72 0 FN
* tcp 64.74.103.161 209.168.144.146 443 1225 3s 0 0 FN
* tcp 64.74.103.148 209.168.144.146 443 1041 5s 0 0 FN
! tcp 76.164.174.115 209.168.144.146 37427 22 9s 44 0
* tcp 216.115.218.197 209.168.144.146 443 1054 1m39s 16 0 FN
Number of entries: 7 of 7 (4096 maximum)
----End----
Section 1, below, tells you what the limits are for each of the different protocols FFE looks for when creating sessions.
Timeout TCP UDP ICMP AH ESP GRE Other
Age: 30m 0s 30m 0s 30m 0s 30m 0s 30m 0s 30m 0s 30m 0s
Inactive: 15s 15s 15s 15s 15s 15s 15s
The “Age” (max timeout) section shows how long an entry will remain in the table, while the session is being used and active, before we refresh the session. For example, if you are running an FTP transfer, which uses TCP port 21, the first packet will take the “slow path”- but will establish an FFE session – and then all of the other packets in the stream will take the “fast path”. Once 30 minutes (default age timeout) has elapsed, the session will be eliminated. If the transfer is still running, the next packet after the 30 minute mark will create a new session (by taking the “slow path”). This is to keep infrequent sessions out of the FFE table which could diminish your routers functionality.
The “Inactive timeout”, which defaults to 15 seconds, is how long the FFE table will wait before eliminating an idle session. For example, if you ping something on your network, FFE would create a session for it. Commonly, a ping will only send five attempts so you do not want an idle session remaining in the FFE table for the full 30 minutes.
Both the “age” and “inactive” timeouts are configurable, shown in Table 1 at the end of the document. This gives the user choices for timeouts when they have many different protocols running through the unit. For example, if you mostly use ping as a troubleshooting tool, the inactive session time could be changed to one second so that if multiple pings are issued at once, the sessions will not be sitting in the active session table for the full 15 seconds. On the other hand, if you have a UDP program running, like a video streaming service, that usually lasts for more than an hour, you could set the age timeout to 3600 seconds.
Section 2 provides information about the types of entries and flags that you will see associated with the information in the FFE table:
Type: * valid, ! ineligible, - rejected
Flags: F firewall, N NAT, T altered ToS, D don't fragment, I IPSec
The “Type” field tells you what type of session it is. The first type, “valid,” is what most of the entries should be. This means that the session is eligible for FFE, and is being used currently. The “!”, for ineligible, means that the packet’s path is not a path that can be sped up by FFE and contributes to FFE misses – which will be explained shortly. The “–“ type, which means “rejected,” is used when the router rejected the session altogether. You will mostly see this when the firewall kills a session, drops a packet that violates a rule, or it considers to be an attack like a SYN flood, or a SMURF attack.
The second section gives you a key for the flags that are associated with each session. These let a user know what the session was created for and where it is being used. Here is a more in-depth explanation of each flag:
- F- Firewall : This means the session has a path through the firewall
- N- NAT : The packet’s session is being affected by NAT (remember this could be source or destination NAT/Port Forward)
- T – Altered ToS – The session is using Type of Service rules
- D – Don’t Fragment – A session where the router modifies this bit (a policy-based routing feature)
- I – IPSEC – The path is running through the IPSEC/VPN process.
Section 3 is the session table that FFE gives you:
Ingress: eth 0/1
3902006 hits, 963818 misses, 0 drops
T Proto Source Destination Specific Age Used Drops Flags
* tcp 64.74.103.129 209.168.144.146 443 2860 11s 0 0 FN
* tcp 64.74.103.134 209.168.144.146 443 1036 14s 0 0 FN
* tcp 216.115.218.197 209.168.144.146 443 2772 7m12s 72 0 FN
* tcp 64.74.103.161 209.168.144.146 443 1225 3s 0 0 FN
* tcp 64.74.103.148 209.168.144.146 443 1041 5s 0 0 FN
! tcp 76.164.174.115 209.168.144.146 37427 22 9s 44 0
* tcp 216.115.218.197 209.168.144.146 443 1054 1m39s 16 0 FN
Number of entries: 7 of 7 (4096 maximum)
The “Ingress:” shows the interface these sessions are for. Under that, it tells you how many packets have come in the listed interface and have been counted as “hits”, “misses” or “drops”. “Hits” are packets that came in and went through the “fast path” session that was already in the FFE table. The “misses” section shows you how many packets have come in on that interface but have not been able to route through an FFE session (note: there will always be misses because the first packet in a session will always take the “slow path,” as well as the fact that there are some sessions that FFE cannot summarize). We will fully touch on “misses” shortly. “Drops” are packets that hit the interface, but FFE knew that the firewall or some other process would drop these packets in the end, so FFE dropped these packets immediately. This could be based on a “rejected” session in the table for example. This way, a packet in a flow that will eventually be dropped anyway does not have to go through every single firewall rule. Do not confuse FFE drops with interface errors as they have no direct correlation.
In the session table you will see the type, the protocol of the flow, the source and destination IP addresses, and the section called “specific” which is the source and destination port. Following that you see the age of the session, how many times it was used and how many times packets were dropped from that session along with the flags to identify what processes that session is using.
The bottom of the session table for each interface tells you how many entries there are as well as the maximum entries allowed. By default, the maximum on an interface is 4096. This is configurable- please refer to Table 1 at the end of this document for the command.
The session table provides information about different flows that are running through your router at a particular point in time. For example, if you are troubleshooting a latency issue, you can check to make sure an FFE session is present in the table. You can also check how many total sessions you have.
Looking back at the example from the first section, How Does FFE Work? , when troubleshooting a problem with this configuration, this can be much quicker than looking at the show ip policy-sessions command. You can also use this when the firewall is off (and FFE is on) when you have no other way to see traffic flows as they progress through the box. This is useful when you are troubleshooting issues not associated with the firewall because it shows you packet paths without worrying about NAT addressing.
You can learn about the traffic on your network by examining your FFE misses. As mentioned before, misses are not necessarily bad. They indicate packets that are taking the slow path for some reason: whether they are not FFE eligible or they are the first packet in the stream, etc. Below are some other reasons that misses can occur:
With the firewall disabled here is a list of per-packet characteristics that cause FFE misses:
- IP header options present
- IP fragments
- TTL less than two (cannot be forwarded but might be locally destined for the router)
With the firewall enabled you have the same list as above plus the following:
- For TCP packets
- IP payload length does not have enough room to hold a complete TCP header (20 bytes)
- TCP header length is greater than the IP payload length
- No TCP flags are set
- TCP SYN, RST, URG, or FIN flags are set – commonly happens when session are being created or are ending.
- TCP sequence number is zero
- For UDP packets
- IP payload length does not have enough room to hold a complete UDP header (8 bytes)
- IP payload length does not have enough room for the entire UDP payload (variable)
Misses that exceed the number of hits on an interface could be an indication of a problem. For example, if there are a lot of fragmented packets or packets with a payload whose length is shorter than that of the packet’s header, these would all cause misses in FFE. They are also commonly (but not always) associated with viruses, malware, or poorly functioning NIC cards.
If you have a large amount of traffic running through a unit, then you will see a large amount of output when you use the show ip ffe command. When looking for specific data, it is important to use extensions of the show command to locate the exact criteria you are looking for.
You can use contact sensitive help with a “?”. By typing show ip ffe ? you get these options:
destination - Filter on destination IP
destination-port - Filter on destination port (TCP or UDP only)
details - Display detailed information
egress - Filter on egress interface
icmp-type - Filter on ICMP type (ICMP only)
ingress - Filter on ingress interface
protocol - Filter on protocol
source - Filter on source IP
source-port - Filter on source port (TCP or UDP only)
summary - Displays summary of FFE entries
type - Filter on type
| - Modify output
You can filter on destination or source IP and destination or source port-- for example, the command show ip ffe destination 192.168.1.1 will only show sessions that have a destination for the IP address 192.168.1.1. To achieve granular output, you can stack these commands: show ip ffe destination 192.168.1.1 source 10.10.10.1. This command shows sessions with a source IP address of 10.10.10.1 and a destination IP address of 192.168.1.1.
The “details” extension shows you extended output for a session (or all sessions if you do not use an extension to narrow it down) including where a packet’s NAT was destined for and the egress of that session.
# show ip ffe destination 192.168.1.1 details
T Proto Source Destination Specific Age Used Drops Flags
* tcp 172.16.0.1 192.168.1.1 80 60697 15s 1 0 FN
egress: vlan 1 (10.0.0.118)
NAT: dest(10.0.0.118)
This shows that this session is being port forwarded (NAT Destination) to 10.0.0.118.
The extension summary can be useful if you want to know how many sessions you have on each interface as well as hits, misses and drops:
# show ip ffe summary
Ingress MaxEntries Entries Hits Misses Drops
------- ---------- ---------- ---------- ---------- ----------
eth 0/1 4096 8 3906309 970928 0
vlan 1 4096 23 2449993 3223245 0
vlan 5 4096 0 0 0 0
vlan 10 4096 2 1302958 235403 0
---------- ---------- ---------- ---------- ----------
Global 16384 33 7659260 4429576 0
The extensions “ICMP-type” and “type” let you filter based on the ICMP protocol or the type of the entry respectively.
You can also use the “|” modify command to show output beginning at a certain section, or only including certain sections.
There are currently no debug commands associated with FFE.
For more information on access-lists, please see Configuring IPv4 Access Control Lists (ACLs) in AOS .
For more information on the AOS Firewall, please see Configuring the Firewall (IPv4) in AOS .
