Tuning the kernel of a relatively less known operating system often creates this "wow" factor. You would be seen as a technical expert but also a real geek, and this comment actually came from one of my colleagues - a Network Consultant...
Fact is, the FreeBSD kernel (or any operating system) is essentially a resource manager. By resources I am referring to CPU cycles, hard-disk spaces, memory etc. There is a great introductory video posted on youtube about FreeBSD kernel internals. [http://www.youtube.com/watch?v=nwbqBdghh6E]
There are two main "tools" when it comes to configuring and tuning the kernel. The kernel's configuration file and "sysctl".
Firstly the "GENERIC" kernel configuration file is located under the "/usr/src/sys/i386/conf/" directory. This configuration file basically contains modules/devices being loaded into the system. It is good practice to copy this "GENERIC" file to another file named, say "MyKERNEL" for editing purposes.
Removing unnecessary modules reduces resource consumption, hence resulting in faster reboot time and smoother operations. Using my second-hand Compaq as an example, I have removed the following devices from my kernel configuration file:
#cpu I486_CPU
#cpu I586_CPU
#options SMP # Symmetric MultiProcessor Kernel
#device apic # I/O APIC
#device fdc # Floppy disk
#device atapifd # ATAPI floppy drives
#device atapist # ATAPI tape drives
and everything under...
# PCI Ethernet NICs.
# PCI Ethernet NICs that use the common MII bus controller code.
# USB Ethernet, requires miibus
# FireWire support
Also, this configuration file contains "options". These are some of the interesting ones:
makeoptions COPTFLAGS="-O2 -pipe -funroll-loops -ffast-math" #Optimize C compiler
options DEVICE_POLLING # Device Polling
options HZ=2000 # "top" shows reduced interrupts from NIC
options IPFIREWALL # NAT Service
options IPFIREWALL_VERBOSE # Syslog logging
options IPDIVERT # Socket Divert
There is a chapter on configuring BSD kernel in the FreeBSD handbook, [http://www.freebsd.org/doc/handbook/kernelconfig.html].
To compile and install the kernel, use the following commands:
# /usr/src
# make buildkernel KERNCONF=MyKERNEL && make installkernel KERNCONF=MyKERNEL
# shutdown -r now
... where "MyKERNEL" is the name of the kernel configuration text file you edit.
Speaking from experiences, these were the two biggest mistakes I made. I once tried to copy a FreeBSD 7.0 kernel configuration text file and tried compiling it on a FreeBSD 8.0 system. The result was lots of "parameters unknown/not found" error messages. Just stick with the GENERIC configuration file which comes with installation. In case if you have forgotten to install the source code, you can always use "sysinstall" to do a post-installation of all packages to re-add the source code. Another mistake I made was to install ports (and compile code) during kernel compilation. It happened when I was working till 1am during weekday and I wished to speed up the process. Patience is a virtual...
Secondly, "sysctl" command is also useful for tuning kernel parameters as recommended in the FreeBSD handbook.
[http://www.freebsd.org/doc/handbook/configtuning-kernel-limits.html]
I admit I have not tried all of these options, as my experience with "sysctl" evolves around tuning parameters at run-time e.g. limiting icmp responses (net.inet.icmp.icmplim) and switching off the "beep" sound (hw.syscons.bell). Of course, you can always save these parameters into the "/etc/sysctl.conf" file such that they remain constant after system restart.
I suppose the best way to become familiar with any system is to get your hands dirty, so to speak. So if you'd like to learn more, please give it a go and good luck with your Unix experience!
Friday, March 19, 2010
Tuesday, March 16, 2010
A$125 second hand Compaq and FreeBSD
Think about this, $125 isn't really that much when it comes to computer and its peripherals. A descend pair of noise-cancellation headphones probably costs a good portion of that (I have seen some nice Bose ones costing $600), and so does a branded wireless broadband router.
Well, one day after work I was on my way to MSY (I work around the city) and went passed this second hand computer store. I found a second-hand Compaq and it would have been at least 3/4 years old, look at its specifications.
CPU: Intel(R) Pentium(R) 4 CPU 2.40GHz (2392.04-MHz 686-class CPU)
real memory = 537395200 (512 MB)
fxp0: Intel 82801DB (ICH4) Pro/100 VM Ethernet
... and a 32G HD (ide cable) and a DVD rom.
While there maybe nothing impressive, it was exactly what I was after as I needed a "known-to-work" hardware for my FreeBSD server (text-based, not X Windows). I was not looking for performance so I didn't even bother checking the graphics card.
Why FreeBSD? Previous work experience. It is a lesser known, not-so-well hardware driver support, open source operating system but it is simply stable and secure.
Why Compaq? Also previous work experience... The Compaq/BSD combination is stable speaking from experiences and I wanted something that "works" - no more spending hours on downloading/testing different drivers. Also worth mentioning are the Dell PCs. The Dell/BSD also works quite well - I once converted a second hand Dell PC into a FreeBSD/Asterisk machine for interest and had it in comparison to a "Shuttle X" box. Let's say it was also why a phpBB/MySQL Trial Forum "mini" FreeBSD (7.0) server at work is on Dell.
It was around the same time when FreeBSD 8.0 was released. I read that there is a new routing architecture written by Bluecoat's developers which certainly sounds exciting. It was my OS of choice.
I chose standard installation and the steps have been captured as follows (Note you can also invoke the same menu using "sysinstall" after installation).
Filesystem Size Mounted on
/dev/ad0s1a 989M /
/dev/ad0s1b 2048MB SWAP
/dev/ad0s1d 989M /tmp
/dev/ad0s1e 17G /usr
/dev/ad0s1f 15G /var
I always start with the "/" partition because the installation begins at the out-most "ring" of the hard-drive and works its way inwards. The out-most rim is also where the boot-loader looks for the OS. BSD simply won't start up if it cannot be found. (From memory, it was the case with 6.0 and I am not sure whether this applies to 8.0). Also note SWAP is placed next for faster r/w.
Also speaking from experience, I originally allocated only 8G to /var and /usr respectively. This was proven to be insufficient soon as I added more applications onto the system. Both /var and /usr partitions consumed their 8GB within months so for this round I gave allocated the majority of HD space to these two partitions... right now, my gateway has been running Asterisk, PPPoE client, DHCP server etc for a couple of months. The /ussr consumption is stable around 6GB (/var is less than 200MB).
Standard boot-manager and select all packages so that the source code is included for kernel tuning. Also, ports collection is extremely important. The only post installation options I configured were enabling system as a gateway and setting the time.
There you go, herewith my FreeBSD installation steps. FreeBSD is actually very well documented, please take a look at http://www.freebsd.org/doc/handbook/install.html for more information.
Finally, I wish to acknowledge my former colleagues in HK who brought me into the wonderful/technical world of this relatively unknown giant of an OS.
Well, one day after work I was on my way to MSY (I work around the city) and went passed this second hand computer store. I found a second-hand Compaq and it would have been at least 3/4 years old, look at its specifications.
CPU: Intel(R) Pentium(R) 4 CPU 2.40GHz (2392.04-MHz 686-class CPU)
real memory = 537395200 (512 MB)
fxp0: Intel 82801DB (ICH4) Pro/100 VM Ethernet
... and a 32G HD (ide cable) and a DVD rom.
While there maybe nothing impressive, it was exactly what I was after as I needed a "known-to-work" hardware for my FreeBSD server (text-based, not X Windows). I was not looking for performance so I didn't even bother checking the graphics card.
Why FreeBSD? Previous work experience. It is a lesser known, not-so-well hardware driver support, open source operating system but it is simply stable and secure.
Why Compaq? Also previous work experience... The Compaq/BSD combination is stable speaking from experiences and I wanted something that "works" - no more spending hours on downloading/testing different drivers. Also worth mentioning are the Dell PCs. The Dell/BSD also works quite well - I once converted a second hand Dell PC into a FreeBSD/Asterisk machine for interest and had it in comparison to a "Shuttle X" box. Let's say it was also why a phpBB/MySQL Trial Forum "mini" FreeBSD (7.0) server at work is on Dell.
It was around the same time when FreeBSD 8.0 was released. I read that there is a new routing architecture written by Bluecoat's developers which certainly sounds exciting. It was my OS of choice.
I chose standard installation and the steps have been captured as follows (Note you can also invoke the same menu using "sysinstall" after installation).
Filesystem Size Mounted on
/dev/ad0s1a 989M /
/dev/ad0s1b 2048MB SWAP
/dev/ad0s1d 989M /tmp
/dev/ad0s1e 17G /usr
/dev/ad0s1f 15G /var
I always start with the "/" partition because the installation begins at the out-most "ring" of the hard-drive and works its way inwards. The out-most rim is also where the boot-loader looks for the OS. BSD simply won't start up if it cannot be found. (From memory, it was the case with 6.0 and I am not sure whether this applies to 8.0). Also note SWAP is placed next for faster r/w.
Also speaking from experience, I originally allocated only 8G to /var and /usr respectively. This was proven to be insufficient soon as I added more applications onto the system. Both /var and /usr partitions consumed their 8GB within months so for this round I gave allocated the majority of HD space to these two partitions... right now, my gateway has been running Asterisk, PPPoE client, DHCP server etc for a couple of months. The /ussr consumption is stable around 6GB (/var is less than 200MB).
Standard boot-manager and select all packages so that the source code is included for kernel tuning. Also, ports collection is extremely important. The only post installation options I configured were enabling system as a gateway and setting the time.
There you go, herewith my FreeBSD installation steps. FreeBSD is actually very well documented, please take a look at http://www.freebsd.org/doc/handbook/install.html for more information.
Finally, I wish to acknowledge my former colleagues in HK who brought me into the wonderful/technical world of this relatively unknown giant of an OS.
Monday, March 15, 2010
So, how did it all begin?
I have been using a Sony laptop running on Vista using Optus USB 3g modem for internet access over the past two years.
At the same time, I have had this "no-frill" Intel/ASUS (Dual Core, motherboard is ASUS P5KPL) PC for over a year, several wireless routers as well as this Panasonic cordless phone (with an FXO port) along with other peripherals, not to mention the Netbook which was basically not doing much other than collecting dust...
Quite a waste, I thought.
So in the beginning of this year, a "Home Media Center" idea came to mind, giving credit to a colleague with lots of experiences with MythTV/MeTV on Ubuntu.
I wanted to convert the Intel/ASUS P5KPL PC into something along this line. Initially I was aiming to build a DVB-T (digital TV) recorder which could also stream DVD and TV over WiFi to all computers at home. I also thought it could simply be done once I purchased a TV card, running on FreeBSD.
Why FreeBSD? Mostly due to previous work experiences and personally I felt more comfortable with this operating system in comparison to other linux/unix distros.
Unfortunately, getting the FreeBSD DVB-T driver to work turned out to be much more difficult than initially anticipated... I have purchased two DVB TV cards from MSY (KWorld TPC-160-T and Leadtek DVR3200H) but soon given up.
The challenges I encountered included KWorld DVB being recognized as an USB connector when I did a "pciconf -lv"; Also, the onboard NIC (Realtek) could not be detected and I had to use Realtek website's BSD's driver... which replaced the original "rl" source code required by cx88 driver to be compiled(?). The Leadtek card could be seen on but the cx88 was not loaded onto it (I am starting to feel this deserve another post with all the specs and such)...
Anyway, it was getting too much having stayed up till 2am for over two weeks while working full-time. I decided to this plan on hold.
During these attempts, the lack of bandwidth was also becoming apparent. Because I never connected my phone-line nor sign up for DSL service, the only way I could allow this PC to access the internet was to share the 3g modem's connection through Vista's internet connection sharing (ICS). It simply wasn't fast enough, nor the 3g data plan had enough data (Imagine doing a "portsnap fetch/extract/update" on FreeBSD would have taken me a good couple of hours and consumed a substantial portion of my monthly quota).
In the end, I spent A$125 at a second hand computer shop to purchase a Compaq P4 with 32G Hard-drive. It would soon become a FreeBSD internet access gateway and is still working well. Also, I decided to sign up for DSL (and to my joy iiNet also gave me a VoIP account along with other stuffs, but that's for a different post).
A second hand Compaq, ADSL2 DSL connection, and a bunch of other equipments.
At the same time, I have had this "no-frill" Intel/ASUS (Dual Core, motherboard is ASUS P5KPL) PC for over a year, several wireless routers as well as this Panasonic cordless phone (with an FXO port) along with other peripherals, not to mention the Netbook which was basically not doing much other than collecting dust...
Quite a waste, I thought.
So in the beginning of this year, a "Home Media Center" idea came to mind, giving credit to a colleague with lots of experiences with MythTV/MeTV on Ubuntu.
I wanted to convert the Intel/ASUS P5KPL PC into something along this line. Initially I was aiming to build a DVB-T (digital TV) recorder which could also stream DVD and TV over WiFi to all computers at home. I also thought it could simply be done once I purchased a TV card, running on FreeBSD.
Why FreeBSD? Mostly due to previous work experiences and personally I felt more comfortable with this operating system in comparison to other linux/unix distros.
Unfortunately, getting the FreeBSD DVB-T driver to work turned out to be much more difficult than initially anticipated... I have purchased two DVB TV cards from MSY (KWorld TPC-160-T and Leadtek DVR3200H) but soon given up.
The challenges I encountered included KWorld DVB being recognized as an USB connector when I did a "pciconf -lv"; Also, the onboard NIC (Realtek) could not be detected and I had to use Realtek website's BSD's driver... which replaced the original "rl" source code required by cx88 driver to be compiled(?). The Leadtek card could be seen on but the cx88 was not loaded onto it (I am starting to feel this deserve another post with all the specs and such)...
Anyway, it was getting too much having stayed up till 2am for over two weeks while working full-time. I decided to this plan on hold.
During these attempts, the lack of bandwidth was also becoming apparent. Because I never connected my phone-line nor sign up for DSL service, the only way I could allow this PC to access the internet was to share the 3g modem's connection through Vista's internet connection sharing (ICS). It simply wasn't fast enough, nor the 3g data plan had enough data (Imagine doing a "portsnap fetch/extract/update" on FreeBSD would have taken me a good couple of hours and consumed a substantial portion of my monthly quota).
In the end, I spent A$125 at a second hand computer shop to purchase a Compaq P4 with 32G Hard-drive. It would soon become a FreeBSD internet access gateway and is still working well. Also, I decided to sign up for DSL (and to my joy iiNet also gave me a VoIP account along with other stuffs, but that's for a different post).
A second hand Compaq, ADSL2 DSL connection, and a bunch of other equipments.
Subscribe to:
Posts (Atom)