I recently wrote a guest diary on the SANS Internet Storm Center about using pfSense’s Traffic Shaper Limiters to address potential Bufferbloat issues in networks.
Please click here to read the full diary entry, and this diary entry has been briefly mentioned in the SANS Daily Network Security Podcast (Stormcast) for Tuesday, February 15th, 2021 over here. Alternatively, the full diary is reposted in full below.
In light of recent challenges brought about by COVID-19, the need for remote work, meetings, training and learning have increased exponentially. A secure and optimized network would certainly allow users to do their best, despite being unable to work from their usual offices. I was recently asked if the addition of an open-sourced firewall, such as pfSense, would introduce network latency, reduce network speeds and affect productivity. A concern brought up was the issue of Bufferbloat, which is briefly defined as high latency within the network when multiple types of network traffic [for example bulk traffic (think upload/download)] hinders time-sensitive traffic (think gaming, Zoom/Skype etc) [1]. This phenomenon is usually caused by a router lacking the ability to execute Smart Queue Management (SQM) for the network traffic. After some experiments and research, I would like to share that using pfSense’s Traffic Shaper Limiters could help optimize networks and address potential Bufferbloat issues. I also further discuss some settings that were tweaked and tested (vis-à-vis what is usually outlined) that ensures conformance to cybersecurity best practices while network performance is optimized.
Note: All networks are unique, and multiple factors can affect network performance. Processing power and availability of memory do affect the capability of a pfSense firewall, so please make sure to buy/build an appropriately sized firewall that matches the network bandwidth requirements. There are also alternative solutions that address Bufferbloat issues, such as using Ubiquiti products [2] or Untangle NG Firewall [3]. Last but not least, please remember to do a config backup on your pfSense firewall before changing anything, and test the changes made in a test network before deploying them into the production environment.
1) Assess network condition
Firstly, assess your network connection with your ISP. The website DSLReports (https://www.dslreports.com/speedtest) is a suitable to start with. Start the test by selecting the appropriate connection type (E.g. Gigabit/Fiber, Cable, DSL etc). Note down the final result, especially the grade for Bufferbloat.
2) Set up Download Limiter
Next, log in to the pfSense Web GUI, and navigate to the Limiters section. (Click on “Firewall” > “Traffic Shaper”). Following that, click on “Limiters”, and select the “New Limiter” button on the left side of the menu (Please refer to Figure 1 for the areas to select).
Input the following configurations (Please refer to Figure 2 for a screenshot of the configuration) for the new Limiter:
Enable – Enable limiter and its children (Check this box)
Name – WANDownload (Feel free to name it otherwise, no spaces allowed)
Bandwidth – Input the bandwidth for download as subscribed from your ISP, and select the unit (Bit/s, Kbit/s, Mbit/s). Leave the “Schedule” field unmodified. (In Figure 2, I used 1000 Mbit/s. Please adjust the values here according to your ISP subscription, or at least the guaranteed speed you are aware that the ISP is giving you.))
Mask – None
Description – WAN Download Speed (Feel free to name it otherwise, spaces allowed)
Queue Management Algorithm – CoDel
Scheduler – FQ_CODEL (Note: You can set the value of the “quantum” parameter to 300 if you want to give priority to Voice over IP (VoIP) traffic. In addition, if your pfSense firewall is not constrained by memory, you can change the value of the “limit” parameter to 20480, and the value of “flows” parameter to 65535. Figure 2 does not show this yet, but you will be able to see these fine-grained settings after you save the limiter.)
Queue Length – 1000 (for a connection with higher bandwidth, this value can be increased to 2000/5000/10000. Start off with 1000 first.)
ECN – Enable Explicit Congestion Notification (Check this box)
At the bottom of the page, click “Save”. Do not navigate away from the page just yet.
Note: You will get the following message “The traffic shaper configuration has been changed. The changes must be applied for them to take effect.”. This message can be ignored, as the changes can be applied later in one go once the download and upload limiters and queues are configured.
3) Set up Download Limiter Queue
After clicking the “Save” button, click on the “Add new Queue” button next to it (Please refer to Figure 3 for the illustration). Note: pfSense may not direct you to the Queue configuration page even after you select the “Add new Queue” button. You will notice this if you see the input for bandwidth (this is not needed for Queue configuration). In this case, select the limiter you have just created, e.g. “WANDownload”, on the left side of the menu. Scroll down, and click on the “Add new Queue” button again. You will then be redirected to the Queue configuration page.
Input the following configurations (Please refer to Figure 4 for a screenshot of the configuration):
Enable – Enable this queue (Check this box)
Name – WANDownloadQueue (Feel free to name it otherwise, spaces not allowed)
Mask – None
Description – WAN Download Queue (Feel free to name it otherwise, spaces allowed)
Queue Management Algorithm – CoDel
Queue Length – (Leave this blank)
ECN – Enable Explicit Congestion Notification (Check this box)
At the bottom of the page, click “Save”.
4) Set up Upload Limiter
It is now time to set up the Upload Limiter. Select “New Limiter” on the left side of the menu (Please refer to Figure 5 on how things should look like now).
Input the following configurations (Please refer to Figure 6 for a screenshot of the configuration):
Enable – Enable limiter and its children (Check this box)
Name – WANUpload (Feel free to name it otherwise, no spaces allowed)
Bandwidth – Input the bandwidth for upload as subscribed from your ISP, and select the unit (Bit/s, Kbit/s, Mbit/s). Leave the “Schedule” field unmodified. (In Figure 6, I used 500 Mbit/s. Please adjust the values here according to your ISP subscription, or at least the guaranteed speed you are aware that the ISP is giving you.)
Mask – None
Description – WAN Upload Speed (Feel free to name it otherwise, spaces allowed)
Queue Management Algorithm – CoDel
Scheduler – FQ_CODEL (Note: You can set the value of the “quantum” parameter to 300 if you want to give priority to Voice over IP (VoIP) traffic. In addition, if your pfSense firewall is not constrained by memory, you can change the value of the “limit” parameter to 20480, and the value of “flows” parameter to 65535. Further explanations available here [4].)
Queue Length – 1000 (for a connection with higher bandwidth, this value can be increased to 2000/5000/10000. Start off with 1000 first.)
ECN – Enable Explicit Congestion Notification (Check this box)
At the bottom of the page, click “Save”. Do not navigate away from the page just yet.
5) Set up Upload Limiter Queue
After clicking the “Save” button, click on the “Add new Queue” button next to it (Please refer to Figure 3 for the illustration). Note: pfSense may not direct you to the Queue configuration page even after you select the “Add new Queue” button. You will notice this if you see the input for bandwidth (this is not needed for Queue configuration). In this case, select the limiter you have just created, e.g. “WANUpload”, on the left side of the menu. Scroll down, and click on the “Add new Queue” button again. You will then be redirected to the Queue configuration page.
Input the following configurations (Please refer to Figure 7 for a screenshot of the configuration):
Enable – Enable this queue (Check this box)
Name – WANUploadQueue (Feel free to name it otherwise, spaces not allowed)
Mask – None
Description – WAN Upload Queue (Feel free to name it otherwise, spaces allowed)
Queue Management Algorithm – CoDel
Queue Length – (Leave this blank)
ECN – Enable Explicit Congestion Notification (Check this box)
At the bottom of the page, click “Save”.
It is now time to apply the changes made. Click on the “Apply Changes” button on the top of the page (Please refer to Figure 8). After applying the changes, you will see a message stating “The changes have been applied successfully.”.
6) Set up Floating Firewall Rules
Navigate to the pfSense Firewall Rules page (Click on “Firewall” > “Rules”). Following that, click on “Floating” (by default, you will be brought to the “WAN” rules page). Select the first “Add” button (With reference to Figure 9, the button highlighted by the red box and with the arrow pointing up).
Floating Rule #1 (Please refer to Figure 10 for a screenshot of the configuration):
• Action: Pass
• Quick: Tick Apply the action immediately on match.
• Interface: WAN
• Direction: out
• Address Family: IPv4
• Protocol: ICMP
• ICMP subtypes: Traceroute
• Source: any
• Destination: any
• Description: Traceroute routing workaround
• Advanced Options: Click on “Display Advanced”.
Scroll down.
• Gateway: Do not use Default. Select your firewall WAN gateway.
Click “Save”. You will see a message “The firewall rule configuration has been changed. The changes must be applied for them to take effect.” on the top of the GUI (with reference to Figure 11). This message can be ignored, as the changes can be applied later in one go once all the firewall rules have been configured.
This rule is required for users who experience their pfSense firewall not being able send out ICMP traceroute [5] when Traffic Shaper Limiters are used. I personally did not experience this when I tested my configuration, and thus disabled this Floating Rule.
Select the second “Add” button (With reference to Figure 11, the button highlighted by the red box and with the arrow pointing down).
Floating Rule #2 (Please refer to Figure 12 for a screenshot of the configuration):
• Action: Pass
• Quick: Tick Apply the action immediately on match.
• Interface: WAN
• Direction: out
• Address Family: IPv4
• Protocol: ICMP
• ICMP subtypes: Echo reply, Echo Request
• Source: any
• Destination: any
• Description: Limiter drop ping traffic under load workaround (Bug #9024)
• Advanced Options: Click on “Display Advanced”.
Scroll down.
• Gateway: Do not use Default. Select your firewall WAN gateway.
Click “Save”.
This rule is required for users who experience their pfSense firewall dropping ping traffic when it is under heavy load when Traffic Shaper Limiters are used [6]. I personally did not experience this when I tested my configuration, and thus disabled this Floating Rule. In addition, particular note has to be taken on the “Direction” option. While it is tempting to put “any” instead of “out”, “any” will allow the pfSense firewall to respond to incoming ping requests (something which many users may not want their firewalls to do so).
Next, we have to create the floating rules that will optimize the network traffic. There are some who suggest the usage of a single “Pass” action floating rule, and with the “Direction” option configured to “out”. This is not recommended. Firstly, according to Netgate documentation, Traffic Shaping related rules should utilize the “Match” action [7]. Secondly, by using only “out” for the “Direction” option, only outgoing traffic will be optimized and incoming traffic ignored. As such, 2 floating rules (one for incoming, and one for outgoing) should be created. They are as follows:
Select the second “Add” button (With reference to Figure 11, the button highlighted by the red box and with the arrow pointing down) again.
Floating Rule #3 (Incoming Traffic) (Please refer to Figure 13 for a screenshot of the configuration):
• Action: Match
• Interface: WAN
• Direction: in
• Address Family: IPv4 (Select IPv4+IPv6 if the network has IPv6 traffic)
• Protocol: Any
• Source: any
• Destination: any
• Description: WAN Incoming Traffic Queue
• Advanced Options: Click on “Display Advanced”.
Scroll down.
• Gateway: Do not use Default. Select your firewall WAN gateway.
• In / Out pipe: WANDownloadQueue / WANUploadQueue
Click “Save”.
Select the second “Add” button (With reference to Figure 11, the button highlighted by the red box and with the arrow pointing down) for the last time.
Floating Rule #4 (Outgoing Traffic) (Please refer to Figure 14 for a screenshot of the configuration):
• Action: Match
• Interface: WAN
• Direction: out
• Address Family: IPv4 (Select IPv4+IPv6 if the network has IPv6 traffic)
• Protocol: Any
• Source: any
• Destination: any
• Description: WAN Outgoing Traffic Queue
• Advanced Options: Click on “Display Advanced”.
Scroll down.
• Gateway: Do not use Default. Select your firewall WAN gateway.
• In / Out pipe: WANUploadQueue / WANDownloadQueue
Click “Save”.
Finally, you can go ahead to click the “Apply Changes” button at the top of the GUI to apply the created firewall rules (Please refer to Figure 15). After applying the changes, you will see a message “The changes have been applied successfully. The firewall rules are now reloading in the background. Monitor the filter reload progress.”.
7) Re-assess network condition
Finally, re-assess your network connection with your ISP at the DSLReports website. (https://www.dslreports.com/speedtest). Start the test by selecting the appropriate connection type (E.g. Gigabit/Fiber, Cable, DSL etc), and the grade for Bufferbloat should have improved with the implementation of Traffic Shaping Limiters and Queues.
That is all! I hope this guide has been useful in introducing network enhancements while maintaining the security of networks, especially for pfSense users. Do note that implementing Traffic Shaping may require a bit of further tweaking (e.g. Download and Upload Limiters, especially the parameter values under the Scheduler option highlighted in Steps 2 and 4 previously) due to various factors (e.g. ISP, geolocation, business requirements, equipment, etc). However, the steps outlined above should be enough to get you started on optimizing networks (especially home networks) that are increasingly being used for work, studies and personal entertainment.
[1] https://www.bufferbloat.net/
[2] https://help.ubnt.com/hc/en-us/articles/220716608-EdgeRouter-Advanced-queue-CLI-examples
[3] https://wiki.untangle.com/index.php/Bufferbloat
[4] https://forum.netgate.com/post/807490
[5] https://docs.netgate.com/pfsense/en/latest/troubleshooting/traceroute-output.html
[6] https://redmine.pfsense.org/issues/9024
[7] https://docs.netgate.com/pfsense/en/latest/firewall/floating-rules.html
Update: This post has been updated to address an issue reported by a reader where the order of the traffic flow in the floating firewall rule was configured in the wrong orientation.