Optimizing TCP Throughput on FreeBSD system

I stumbled across a rather dated document about optimizing TCP throughput on FreeBSD system; where its proof of theory was carried out using FreeBSD OS 4.0. Herewith the link: 

I would like to provide a quick reference to view some of these tunings and bring it to our awareness.

Quote from “Introduction”: This paper explores the end-system factors that can limit bandwidth for TCP on high-speed networks and the techniques to overcome those limitations… The limiting factor is not CPU processing power but the ability to move data through the host I/O system and memory.

The paper then proceeded to present four techniques: 1. Large MTU, 2. Interrupt coalescing, 3. Checksum offloading, 4. Zero-Copy (page remapping)

I checked on my FreeBSD 7.0 system and noted most of these options have been enabled by default, or included under sysctl, and herewith my findings:

1. For larger MTU size, use "ifconfig em0 mtu "

2. Interrupt coalescing. At first, I thought it would have been the "device polling" option used in the GENERIC kernel file, but it turned out to be always enabled and option is invisible under "sysctl -a"... http://freebsd.1045724.n5.nabble.com/Configuring-BCE-Interrupt-Coalescing-on-8-1-RELEASE-td3750002.html

3. Checksum Offloading (onto the interface), enabled by default

# ifconfig -m # to display the hardware capabilities, which are shown under options. # Use "-txcsum -rxcsum" options to switch off.

FreeBSD# ifconfig -m em0: flags=8843 metric 0 mtu 1500 options=19b

4. Zero copying, once again, another built in feature and the closest thing I can find are some vnodes parameters...

FreeBSD# sysctl -a | grep numvnodes vfs.numvnodes: 61486

FreeBSD# sysctl -a | grep maxvnodes kern.maxvnodes: 69071

[Output from top command]

Memory Usage…

last pid: 50644; load averages: 0.37, 0.21,0.09 up 1+23:21:49 12:11:30 64 processes: 1 running, 63 sleeping CPU states: 5.0% user, 0.0% nice, 11.0% system, 0.9% interrupt, 83.0% idle Mem: 30M Active, 471M Inact, 115M Wired, 192K Cache, 110M Buf, 366M Free Swap: 8192M Total, 8192M Free

My reference is nowhere near completion, but it would be a good start prior to reading up on this other TCP/IP optimization paper on FreeBSD system:

http://people.freebsd.org/~andre/Optimizing%20the%20FreeBSD%20IP%20and%20TCP%20Stack.pdf

Quick note on "Wall" command

On a FreeBSD system, if you would like to send messages to all users, then login as root and type:

#wall

-- Message

[Ctrl-D]

On Netgear WNR2000 Wifi Router (for Linksys ATA and Iomega NAS to work)

It has been almost 6 months since my last post... I believe I have collected sufficient information from using the Netgear WNR2000 wifi router for another post. By the way, this router is 3 years old and was released back in June 2008 (http://routers.productwiki.com/netgear-wnr2000/).

I purchased a Linksys ATA device and in the process of getting it work behind the Netgear router, there were several interesting findings with regards to the port-forwarding settings:

1. Using WNR2000 default settings, the Linksys ATA could only make out-going phone calls. Incoming phone calls resulted in a "person you are trying to call is unavailable..." message, despite successful SIP registration.

2. Netgear WNR2000 port forwarding settings only came into effect AFTER the router rebooted.

3. F.Y.I. I configured static IP address on the Linksys ATA device (which, by the way, only comes into effect after disabling DHCP on Linksys ATA)

The following Port Forwarding rules were referenced from another post, and the rule numbers on WNR2000 MATTERS. Have them configured the other way around and incoming calls would't work.

Rule 1: Permit UDP 5060, Linksys ATA static IP

Rule 2: Permit UDP 16384 - 16482, Linksys ATA static IP

I also own an Iomega NAS (with an Ethernet port). Interesting enough, everytime after I restarted my laptop, the NAS' share drives were no longer visible (The laptop connects to the same Netgear AP which the NAS is also connected to through a CAT5e). Out of interest, Wireshark capture indicated that when Iomega started up, it first multicasts MDNS queries (port 5353 rather than UDP 53 for standard DNS).

This was then followed by exchanging SMB messages (TCP 445) between the laptop and NAS. While I am no expert in SMB, I recall reading messages annoucing each of the shared directories.

As such, it would appear either the NAS stopped annoucing these messages after the TCP connection timed out (due to laptop restart), or the wireless router stopped new TCP sessions from being established.

The only workaround was to restart the wifi router. Afterwards, the NAS shared drives would magically become visible from my laptop again.

P.S. This morning the Netgear router finally gave in after several restarts. Both my laptop and iPhone had difficulty connecting to the AP. As a result, I have replaced this router with a TP-Link TL-WR740. Upgraded this box to the latest firmware and so far so good... the Linksys ATA was a matter of simply plug-and-play while the IP NAS has remained visible so far... perhaps I shall have more to add after another couple of months.

P.P.S. For convenience, the dialplan on the Linksys ATA has been simplified from
# Original dialplan, local calls only

(*xx|[3469]11|0|00|[2-9]xxxxxx|1xxx[2-9]xxxxxxS0|xxxxxxxxxxxx|<#0:>xx.<:@gw0>.)

To
# For dialing everything
(*xx|xxx.)

Utilizing Home VoIP Server from Remote (over internet)

Background

Once I had a chat with a colleague and he came up with this question, "How could I use my home's PSTN or VoIP line from a remote location?" For example, he were working overseas and wishes to contact his family at home (without using his company's mobile or the customer's phone), or the other way around. The most obvious answer would be to setup a SIP client on his laptop to register with a VoIP server located at home. 

To make it more interesting, the SIP client on his laptop would also be able to use his home's Asterisk server to dial out phone calls as well as receive them (virtual "call-forwarding", if you like).

First let's cover the easy part, setting up Asterisk server at his home to accept public client registration. This includes two basic operations: Registering the server's dynamic address using DynDNS and listening on port 5060 (UDP). Note: please take a look at a previous blog on setting up Asterisk on BSD to register with a public SIP server (and be assigned a public PSTN number):

http://homeopensource.blogspot.com/2010/03/setting-up-bsd-ata-box-with-asterisk.html

Configuration 

Firstly on the home's Asterisk server, setup a new SIP account by configuring the "sip.conf" file:

;XLite over Internet - Remote VoIP testing

[user-name]

type=friend

secret=secret

host=dynamic                   ; This device needs to register

nat=yes                        ; X-Lite is behind a NAT router

disallow=all

allow=ulaw

allow=alaw

Also, set Asterisk to listen on all interfaces, including internet-facing ones.

bindport=5060                   ; UDP Port to bind to (SIP standard port is 5060)

                                ; bindport is the local UDP port that Asterisk will listen on

bindaddr=0.0.0.0

On a personal note, I would prefer Asterisk to support the configuration option of setting "bindaddr" to a particular interface (e.g. em0) allowing for more user control. 

Add an extension for this SIP client locally by configuring the "extensions.conf" file:

[outbound-local]

exten => 12345678,1,Dial(SIP/user-name)

In this example, the extension given to SIP client is 12345678. "outbound-local" is binded to the locally attached PSTN phone, remember to include this under "default" context.

  

DynDNS - Remote Access over Internet

DynDNS (http://www.dyndns.com/) provides free reverse DNS service. The idea behind this service is that, you can basically access your home computer via the internet, by typing a domain name of your choice. Diving into more details... a dyndns client is installed on your home computer. Whenever the home computer receives a public ip address, it registers with its allocated URL on the DynDNS server. The DynDNS server then updates the DNS record and make it publicly available. So, when you access your home computer using its URL, DNS lookup returns your home computer's latest registered public IP.

It is a free service, as mentioned, as a matter of fact I have seen lots of DSL broadband routers providing this setting also... which opens up opportunities for other applications such as remote home monitoring, but I shall leave those for another blog. 

Configuration for Dynamic DNS.

There is a good tutorial on DynDNS page for setting up "ddclient", along with configuration step-guide (http://www.dyndns.com/support/kb/using_ddclient_with_dyndns_services.html)

Herewith my configuration: 

/usr/local/etc/ddclient.conf

# Basic configuration file for ddclient

#

# /etc/ddclient.conf

daemon=600

cache=/tmp/ddclient.cache

pid=/var/run/ddclient.pid

ssl=yes

use=web, web=checkip.dyndns.com/, web-skip='IP Address'

login=

password=

protocol=dyndns2

server=members.dyndns.org

wildcard=YES

custom=yes, example.com

XLite Configuration

XLite is a free Windows program. Setting it up as a VoIP client is quite straightforward. Just configure the username, password and domain (home Asterisk server's URL). Once the registration is successful, you are ready to go. 

Testing Scenarios

Moment of truth. My colleague and I have tried the following scenarios with their results attached. 

XLite calls homephone                           OK

XLite calls local number                         OK*

(caller is PSTN number, however after mobile denies incoming call, XLite is still ringing)

homephone calls XLite                          OK (extension 12345678)

Mobile calls XLite                                 Not OK.

(This is because the extension is not registered to the public PSTN network... )

In conclusion, this exercise shows that it is possible to setup a VoIP server at home and offloading most of its functions onto a light-weighted VoIP client registered over the public internet. To fully offload the SIP server's functions onto the VoIP client, the PSTN phone number must also be set for the VoIP client - effectively removing the home phone usage. 

Cost Comparison between EoP versus Wifi

Background/ Requirements
I have moved home recently. There are two TV sets connected to a Western Digital Live Media Player and XBMC respectively. Both requires wired Ethernet connection to a Netgear wireless router located in a separate room, in order to access the internet:

The TV connected to XBMC is sitting in a different room. I am reusing the previously built solution DD-WRT Wifi client http://homeopensource.blogspot.com/2010/03/converting-wifi-adapter-into-80211g.html

This setup has proven to be quite reliable, although its throughput is only around 5Mbps. For example, copying a 1.36GB file (DVD) from my laptop to the XBMC takes around 37minutes.

While this is ample bandwidth for Standard Definition video streaming (and of course internet radio stations ~128kbps), the video quality degrades significantly when, say, I am using my laptop to access the internet. 

I would like to calculate the cost of setting up internet connection for the WD Media Player, connected to the second TV through HDMI. It is intended to stream High-Def contents so the bandwidth required will be higher than 11g. Also, I would prefer my YouTube videos being unaffected by wireless internet connection.

Wireless Connection (G/N Dual-Band)

Initially, I considered using g/n dual-band setup with the Access Point supporting both G and N clients simultaneously. A forum post indicates this setup works quite well (http://forums.whirlpool.net.au/archive/1226195) and should be able to support up to 80Mbps throughput - more than sufficient for HD streaming. There is only the additional cost of purchasing a 11n client with a Ethernet port, but unfortunately, they are hard to come by. As most of the commercially available wireless routers' firmware only allows the device to be configured as an Access Point, if I were to go for a wireless router, then it must support DD-WRT.

Ethernet over Power (EoP) 

This device essentially transmits data signal across the residential circuit using a different frequency band to that of electrical, in another words it acts bit like a frequency modulator. Its operation is straightforward, there is no software driver nor any configuration required, simply plug-and-play. I have managed to find an extensive review on TP-Link TL-PA201 EoP adapters (http://www.tp-link.com/products/productDetails.asp?pmodel=TL-PA201), along with its throughput performance http://www.vitocassisi.com/archives/790/6, which is quite similar to that of 802.11n.

Price Point
I searched www.msy.com.au, and found a DLink DIR-600 WirelessN router selling for $50.00, whereas the TP-Link TL-PA201 EoP adapter pair is selling for $109.00 - roughly $60.00 difference. The wireless solution is still significantly cheaper than the EoP adapter pair.


Conclusion

This analysis is rather preliminary in terms of its application (streaming video to TV). Consider when additional users require internet access. Most importantly, the smartphones definitely cannot benefit from a pure Network over Power setup. In another words, NoP effectively hinders internet mobility. This means a wireless router is needed nonetheless. Whereas in a pure wireless environment, we would see a degrade in performance as the number of devices grow, which is bad news for bandwidth sensitive IPTV applications.  

In summary, EoP shows lots of potential (e.g. ease of use, high-throughput, operates independently to wireless internet and no new wires across home required), but it is the pricing and limited application at this stage, that hinders its popularity. In fact, I see these two technologies complimenting each other more than one is better than the other.

Note: 

For those who are interested, please take a look at the FAQ for EoP on Netgear's website, it contains lots of good information including the recommended (maximum) number of EoP adapters, expected throughput etc...