This article aims to present best practice guidelines for all ADTRAN BlueSecure Controller (BSC) models and document steps that can be used to troubleshoot common BSC issues. Please note this product line has been discontinued as of 31 December 2014. The product line will no longer be supported after 31 December 2015. All documentation on the ADTRAN Support Community will remain available.
Sections in this document
Use the following suggestions to get the best performance out of the BSC platform. More detailed information, if available, can be found by clicking the entries.
Make sure the BSC is running the latest software and patches, and check that the BlueSecure Access Points (BSAPs) are also running the latest available software compatible with the BSC platform. ADTRAN recommends the following software for BSC deployments.
The easiest way to find software is by following the links below.
The BSC software can be confirmed under Maintenance > Upgrade. The BSC maintains two separate partitions. The current runtime (partition) and version are shown in bold.
The installed patches are listed under Maintenance > Patch. There are two sections displayed on this page. The left-hand section lists all installed patches. Clicking on a patch will display details about that patch on the right-hand side.
The BSAP software is located under Wireless > Firmware. A default and alternative firmware version is listed as shown in the image below. Any alternative version can be made the default by clicking the green check near the alternative firmware version.
Use the system summary (Status > Summary) to compare the Total Number of Users to the max supported for the model in question. Also compare the Total Number of Active Access Points to the max supported for the model in question. The following table is an abbreviation of the existing article How many concurrent authenticated users and BSAPs does each model BSC support?
Table 1. Max Supported Users and APs per BSC Model
|Model||Max Users||Max APS|
It is important that the number of users and APs be less than the allowed maximum for the model as described above. Reduce the number of APs or users as needed. Reducing the number of users on a wireless system can sometimes be difficult. Implementing a password protected SSID is one option (Wireless > SSID). Reducing the number of APs or implementing a client limit per AP is another option. Client limits are global settings per radio (Wireless > Global) or per AP (Wireless > AP). This is different than simply deleting obsolete user accounts, which should also be done as a part of regular maintenance.
Local and guest users can be configured to expire automatically. If the accounts are not setup to expire or perhaps just did not expire when they were supposed to, then delete all obsolete or unused accounts. This cleanup should also include MAC Device Authentication entries that are not being used. Local, guest, and MAC devices authentication are found under the section of the BSC. Click the trashcan icon under the Actions column to delete an entry. Alternatively check multiple entries and click the Delete button located under the list.
If replication is not being used, then make sure that replication is disabled. Leaving replication enabled can lead to high CPU utilization when there are no replication nodes available. This can lead to new problems or exacerbate existing ones. When replication is disabled, the Mobility MatriX > Replication Setup page could indicate as much in bold.
The BSC solution utilizes role-based access control to enforce firewall policies on clients connected to the managed side. Roles are found under Roles > Roles. These policies are processed in a top-down fashion until a match is found or no more policies are present. This provides two options for optimization: place common ports at the top of the policy list and keep the policies as concise as possible. For example, DNS, HTTP, and HTTPS are the primary protocols used for typical internet access. If a policy is related to these protocols, then it should likely be at the top of the policy list.
The server_black_list is a file (Web Logins > File Uploads) used in the Un-registered role (walled garden). When an Un-registered client is being redirected to a captive portal, if the URL they are trying to access contains a term in the black list, then that redirect attempt will be dropped. This action uses significantly less CPU cycles, and therefore is a good way to reduce CPU overhead associated with captive portal authentication. For example, mobile devices often try to access a site hosted by the vendor or provider, and this happens as soon as an internet connection is enabled. Adding the vendor name to the black list means that these initial, uncontrolled requests will not burden the CPU as much as if those requests were redirected to the BSC. This is because actions based on the server black list happen prior to the web server needing to process the traffic.
ADTRAN recommends that all BSCs run the latest available software as listed below.
The Software (BSC) section of the ADTRAN Support Community can be used to find the appropriate controller software. BSAP software is available by following the appropriate link under the Software (BSAP) section. BSC software and patches have two formats depending on the BSC model. The extension *.i386.bpf is used for the 2200/3200/5200 platforms while *.mips.bpf is used for the 600/1200 models. Follow the outline below after downloading the appropriate software.
Start by making sure to follow the best practices above. Try to eliminate unnecessarily high client counts as reducing the number of clients will alleviate most CPU and webserver related problems. Next, note the following information on the summary page.
Using the information above and any other problem description, proceed to the applicable sections below. If the steps in the other sections did not resolve the issue, then reboot the equipment under Maintenance > Restart Services > select Reboot > click Submit. If the administrative portal is not available, then use a console cable to access the serial console menu.
There are two primary causes of high CPU utilization: user count and logging. Make sure user and AP totals are within allowed range of the BSC model. Table 1 above provides these numbers or check How many concurrent authenticated users and BSAPs does each model BSC support?. If users are over 90% of the total allowed, it can still cause high CPU numbers. If open SSIDs are present (Wireless > SSID), then try switching to an authentication mechanism that still fits the needs of the application. ADTRAN recommends at least WPA2-PSK/AES.
If the user count is within the acceptable limits for the BSC model in question, then check logging levels and lower any debug-level settings under General > Logging. Try to identify any roles that could be trimmed. Follow the directions in the next section to eliminate some web server traffic, if applicable.
Redirecting traffic associated with captive portal authentication is one of the main culprits for high CPU utilization on the BSC platforms. When suspect traffic occurs, or a difference in performance is noted, go to General > Logging and set User Tracking to Debug level. Note that additional performance impacts may be felt while the logging levels are set like this. These logging settings should only be temporary. Give the BSC some time to build up log entries, then download the logs under Status > Logs and open them for review. The message column can be used to identify the undesired traffic. Look for redirect messages.
Items can be added to the blacklist file so that all traffic attempting to contact a URL containing the keyword in the blacklist will simply be dropped, not redirected if the user is in the Un-registered role. The black list file itself is found under Web Logins > File Uploads, however, the file may not be shown if it has been removed or does not exist. If the file exists, then download it and open it in a simple text editor. If the file does not exist, then create a new file in a text editor, and save it as server_black_list. If additional keywords are added to the blacklist, save the file with the name server_black_list. Upload the server_black_list file to the BSC under Web Logins > File Uploads. Restart the web server under Maintenance > Restart Services > Advanced.
Check the Log Space Usage (Current Partition) under System > Summary. If Log Space Usage (Current Partition) is over 90%, then consider purging logs under System > Logs. Reset or adjust logging levels under General > Logging. Disable all debug-level logging, and only enable debug-level logging when necessary. Monitoring processes requires the BSC Processes log level be set to at least Info level. Disable “Connection Tracking” or increase the interval (default: 120). Connection tracking sends a record of all network connections to syslog which can result in a large number of log messages and impact BSC performance. Only use if all network connection information needs to be logged for auditing purposes. Consider exporting logs to a remote syslog server. Generally avoid increasing the maximum number of log entries to store in the local database. If the max value is already set, then lower the number of logs to keep, or increase the number of log entries to delete when reaching the maximum.
If graphs do not appear properly or the report contains no data, ensure the Start Date and End Date are accurate. Make sure the end date is not before the start date and the years are correct. Verify there actually is data under Status > Logs. A lot of times that data is empty because the logs have already rolled off of the BSC. For example, a report is needed for the last 30 days of logs, but Status > Logs has the last 5 days. In this case, the BSC will display an error.
There are a few settings which can help improve responsiveness of the web server. The comma separated list of HTTP/proxy ports to monitor (General > HTTP) should only include port 80. Remove 443 if it is in the list. Make sure Denial-of-Service Evasion Options (General > HTTP) are set to a reasonable value. Use the recommended settings as guidelines.
These recommended values work for most deployments. Custom login forms with lots of images may take longer to load. In those cases, increase the seconds a client is allowed to hold the webserver to a maximum of 30 seconds. The other values should rarely increase.
Sometimes clients may not redirect to the Web Page served on the BSC. Before a client is authenticated they are placed in the Un-Registered role. By default the Un-Registered role only allows DNS outgoing therefore the browser is unable to check the validity of the certificate and doesn't redirect to the login page. Upon redirection, the client’s browser uses OCSPs and CRLs to check the validity of the Web Server certificate. The default behavior of many of the browsers today is if they cannot check the validity of the SSL certificate they consider it invalid. The unfortunate thing is the browser does not display a message or anything to indicate it could not validate the certificate, it simply just doesn't display a page or displays a generic page cannot be displayed message. To find the OCSPs and CRLs, go to Web Logins > SSL Certificate on the right hand side and see the properties of your certificate. There you should see the OCSP (Online Certificate Status Protocol) or CRL (Certificate Revocation List) URLs. You may see one or both depending on the certificate. The 6.5.1.03 release introduced Destination Hostnames to account for the multiple IP addresses that may resolve from a host name. Once you have the OSCP and CRL URLs, you need to create Destination Hostnames (User Roles > Destinations > Create Destination Hostname) for each URL. Next, navigate to User Roles > Roles and click to edit the Un-Registered Role and allow HTTP access to the Destination Hostnames of the OCSP and CRL URLs.