Saturday, June 4, 2011

Optimizing TCP Throughput on FreeBSD system

I stumbled across a rather dated document about optimizing TCP throughput on FreeBSD system; where its proof of theory was carried out using FreeBSD OS 4.0. Herewith the link
I would like to provide a quick reference to view some of these tunings and bring it to our awareness.
Quote from “Introduction”: This paper explores the end-system factors that can limit bandwidth for TCP on high-speed networks and the techniques to overcome those limitations… The limiting factor is not CPU processing power but the ability to move data through the host I/O system and memory.
The paper then proceeded to present four techniques: 1. Large MTU, 2. Interrupt coalescing, 3. Checksum offloading, 4. Zero-Copy (page remapping)
I checked on my FreeBSD 7.0 system and noted most of these options have been enabled by default, or included under sysctl, and herewith my findings:
1. For larger MTU size, use "ifconfig em0 mtu "
2. Interrupt coalescing. At first, I thought it would have been the "device polling" option used in the GENERIC kernel file, but it turned out to be always enabled and option is invisible under "sysctl -a"... http://freebsd.1045724.n5.nabble.com/Configuring-BCE-Interrupt-Coalescing-on-8-1-RELEASE-td3750002.html
3. Checksum Offloading (onto the interface), enabled by default
# ifconfig -m # to display the hardware capabilities, which are shown under options. # Use "-txcsum -rxcsum" options to switch off.
FreeBSD# ifconfig -m em0: flags=8843 metric 0 mtu 1500 options=19b
4. Zero copying, once again, another built in feature and the closest thing I can find are some vnodes parameters...
FreeBSD# sysctl -a | grep numvnodes vfs.numvnodes: 61486
FreeBSD# sysctl -a | grep maxvnodes kern.maxvnodes: 69071
[Output from top command]
Memory Usage…
last pid: 50644; load averages: 0.37, 0.21,0.09 up 1+23:21:49 12:11:30 64 processes: 1 running, 63 sleeping CPU states: 5.0% user, 0.0% nice, 11.0% system, 0.9% interrupt, 83.0% idle Mem: 30M Active, 471M Inact, 115M Wired, 192K Cache, 110M Buf, 366M Free Swap: 8192M Total, 8192M Free
My reference is nowhere near completion, but it would be a good start prior to reading up on this other TCP/IP optimization paper on FreeBSD system: