≡ Menu

OpenBSD

Linux and other Unix-like operating systems use the term "swap" to describe both the act of moving memory pages between RAM and disk, and the region of a disk the pages are stored on. It is common to use a whole partition of a hard disk for swapping. However, with the 2.6 Linux kernel, swap files are just as fast as swap partitions. Now, many admins (both Windows and Linux/UNIX) follow an old rule of thumb that your swap partition should be twice the size of your main system RAM. Let us say I've 32GB RAM, should I set swap space to 64 GB? Is 64 GB of swap space really required? How big should your Linux / UNIX swap space be?

Old dumb memory managers

I think the '2x swap space' rule came from Old Solaris and Windows admins. Also, earlier memory mangers were very badly designed. There were not very smart. Today, we have very smart and intelligent memory manager for both Linux and UNIX.

Nonsense rule: Twice the size of your main system RAM for Servers

According to OpenBSD FAQ:

Many people follow an old rule of thumb that your swap partition should be twice the size of your main system RAM. This rule is nonsense. On a modern system, that's a LOT of swap, most people prefer that their systems never swap. You don't want your system to ever run out of RAM+swap, but you usually would rather have enough RAM in the system so it doesn't need to swap.

Select right size for your setup

Here is my rule for normal server (Web / Mail etc):

  1. Swap space == Equal RAM size (if RAM < 2GB)
  2. Swap space == 2GB size (if RAM > 2GB)

My friend who is a true Oracle GURU recommends something as follows for heavy duty Oracle server with fast storage such as RAID 10:

  1. Swap space == Equal RAM size (if RAM < 8GB)
  2. Swap space == 0.50 times the size of RAM (if RAM > 8GB)

Red Hat Recommendation

Red hat recommends setting as follows for RHEL 5:

The reality is the amount of swap space a system needs is not really a function of the amount of RAM it has but rather the memory workload that is running on that system. A Red Hat Enterprise Linux 5 system will run just fine with no swap space at all as long as the sum of anonymous memory and system V shared memory is less than about 3/4 the amount of RAM. In this case the system will simply lock the anonymous and system V shared memory into RAM and use the remaining RAM for caching file system data so when memory is exhausted the kernel only reclaims pagecache memory.

Considering that 1) At installation time when configuring the swap space there is no easy way to predetermine the memory a workload will require, and 2) The more RAM a system has the less swap space it typically needs, a better swap space

  1. Systems with 4GB of ram or less require a minimum of 2GB of swap space
  2. Systems with 4GB to 16GB of ram require a minimum of 4GB of swap space
  3. Systems with 16GB to 64GB of ram require a minimum of 8GB of swap space
  4. Systems with 64GB to 256GB of ram require a minimum of 16GB of swap space

Swap will just keep running servers...

Swap space will just keep operation running for a while on heavy duty servers by swapping process. You can always find out swap space utilization using any one of the following command:
cat /proc/swaps
swapon -s
free -m
top

See how to find out disk I/O and related information under Linux. In the end, you need to add more RAM, adjust software (like controlling Apache workers or using lighttpd web server to save RAM) or use some sort of load balancing.

Also, refer Linux kernel documentation for /proc/sys/vm/swappiness. With this you can fine tune swap space.

A note about Desktop and Laptop

If you are going to suspend to disk, then you need swap space more than actual RAM. For example, my laptop has 1GB RAM and swap is setup to 2GB. This only applies to Laptop or desktop but not to servers.

Kernel hackers need more swap space

If you are a kernel hacker (debugging and fixing kernel issues) and generating core dumps, you need twice the RAM swap space.

Conclusion

If Linux kernel is going to use more than 2GiB swap space at a time, all users will feel the heat. Either, you get more RAM (recommend) and move to faster storage to improve disk I/O. There are no rules, each setup and configuration is unique. Adjust values as per your requirements. Select amount of swap that is right for you.

What do you think? Please add your thoughts about 'swap space' in the comments below.

pssh: Run Command On Multiple SSH Servers

I've already written about tentakel tool and shell script hack to run a single command on multiple Linux / UNIX / BSD server. This is useful to save time and run UNIX commands on multiple machines. Linux.com has published an article about a new and better tool called pssh:

If you want to increase your productivity with SSH, you can try a tool that lets you run commands on more than one remote machine at the same time. Parallel ssh, Cluster SSH, and ClusterIt let you specify commands in a single terminal window and send them to a collection of remote machines where they can be executed.

Download of the day: OpenBSD 4.4 CD ISO Images

OpenBSD 4.4 has been released and available for download (jump to download link ) from official project website. OpenBSD is often the first to add new security tools to make it harder to break, developers have also carefully read through the programming code to check for mistakes more than once. OpenBSD is also used for protecting networking using its pf firewall.

What's new in OpenBSD 4.4

  • A new tool sysmerge(8), derived from the old mergemaster port, makes it easier to merge configuration files changes during an upgrade.
  • Fully support OpenBSD inside extended partitions on i386 and amd64.
  • During installation 'dhcp' is now the initial default answer during network configuration.
    OpenSSH 5.1
  • New experimental fingerprint ASCII art visualisation system for easier verification of remote keys. Added chroot(2) support for sshd(8).
  • Added an extended test mode (-T) to sshd(8).
  • Make ssh(1) support negation of groups in a "Match group" block.
  • Over 5205 ports, minor robustness improvements in package tools.
  • See all new features here.


OpenBSD 4.4 Download

You can grab OpenBSD ISO and boot files from the FTP server or mirror machines.

OpenBSD download: OpenBSD 4.4 CD ISO download

OpenBSD 4.4 Download Mirrors

A Redundant Array of Independent Drives (or Disks), also known as Redundant Array of Inexpensive Drives (or Disks) (RAID) is an term for data storage schemes that divide and/or replicate data among multiple hard drives. RAID can be designed to provide increased data reliability or increased I/O performance, though one goal may compromise the other. There are 10 RAID level. But which one is recommended for data safety and performance considering that hard drives are commodity priced?

I did some research in last few months and based upon my experince I started to use RAID10 for both Vmware / XEN Virtualization and database servers. A few MS-Exchange and Oracle admins also recommended RAID 10 for both safety and performance over RAID 5.

Quick RAID 10 overview (raid 10 explained)

RAID 10 = Combining features of RAID 0 + RAID 1. It provides optimization for fault tolerance.

RAID 0 helps to increase performance by striping volume data across multiple disk drives.

RAID 1 provides disk mirroring which duplicates your data.

In some cases, RAID 10 offers faster data reads and writes than RAID 5 because it does not need to manage parity.

Fig.01: Raid 10 in action

Fig.01: Raid 10 in action

RAID 5 vs RAID 10

From Art S. Kagel research findings:

If a drive costs $1000US (and most are far less expensive than that) then switching from a 4 pair RAID10 array to a 5 drive RAID5 array will save 3 drives or $3000US. What is the cost of overtime, wear and tear on the technicians, DBAs, managers, and customers of even a recovery scare? What is the cost of reduced performance and possibly reduced customer satisfaction? Finally what is the cost of lost business if data is unrecoverable? I maintain that the drives are FAR cheaper! Hence my mantra:
NO RAID5! NO RAID5! NO RAID5! NO RAID5! NO RAID5! NO RAID5! NO RAID5!

Is RAID 5 Really a Bargain?

Cary Millsap, manager of Hotsos LLC and the editor of Hotsos Journal found the following facts - Is RAID 5 Really a Bargain?":

  • RAID 5 costs more for write-intensive applications than RAID 1.
  • RAID 5 is less outage resilient than RAID 1.
  • RAID 5 suffers massive performance degradation during partial outage.
  • RAID 5 is less architecturally flexible than RAID 1.
  • Correcting RAID 5 performance problems can be very expensive.

My practical experience with RAID arrays configuration

To make picture clear, I'm putting RAID 10 vs RAID 5 configuration for high-load database, Vmware / Xen servers, mail servers, MS - Exchange mail server etc:

RAID LevelTotal array capacityFault toleranceRead speedWrite speed
RAID-10
500GB x 4 disks
1000 GB1 disk4X2X
RAID-5
500GB x 3 disks
1000 GB1 disk2XSpeed of a RAID 5 depends upon the controller implementation

You can clearly see RAID 10 outperforms RAID 5 at fraction of cost in terms of read and write operations.

A note about backup

Any RAID level will not protect you from multiple disk failures. While one disk is off line for any reason, your disk array is not fully redundant. Therefore, old good tape backups are always recommended.

Please add your thoughts and experience in the comments below.

Further readings:

This is an user contributed article.

PuTTY is a terminal emulator application which can act as a client for the SSH, Telnet, rlogin, and raw TCP computing protocols. You can use putty for remote login or to control your router connected via serial devices.

By default PuTTY stores the session information in the registry on Windows machine. If you have several PuTTY sessions stored in one laptop and would like to transfer those sessions to another laptop, you need to transfer HKEY_CURRENT_USER\Software\SimonTatham registry key and value as explained below:

Export the PuTTY registry key on source windows machine

Click on Start -> Run -> and enter the following regedit command in the run dialog box, which will place the PuTTY registry key and value on your desktop in the putty-registry.reg file. Please note that the name of the registry key (Simon Thatham) is the author of PuTTY.

regedit /e "%userprofile%\desktop\putty-registry.reg" HKEY_CURRENT_USER\Software\Simontatham

You can also launch the registry and interactively export the registry key value as shown below. Click on Start -> Run -> regedit -> Click File menu -> Click Export menu-item -> Enter HKEY_CURRENT_USER\Software\Simontatham in the "Selected branch" -> Save the putty-registry.reg to your desktop.

Import the PuTTY registry key on destination windows machine

Transfer the putty-registry.reg to the destination Windows machine. Right click on the .reg file and select Merge as shown below. This will display a confirmation message: Are you sure you want to add the information in putty-registry.reg to registry?. Click on 'Yes' to accept this message.

Launch the putty to verify the new sessions are transferred successfully. The registry key merge will not delete the previous PuTTY sessions. Instead, it will merge the entries to the existing PuTTY sessions on the destination windows machine.

You can also import the registry key and value interactively: Click on Start -> Run -> regedit -> Click File menu -> Click Import menu-item ->select the putty-registry.reg -> click on Import, to import the PuTTY sessions to the destination windows machine.

Turbocharge PuTTY with 12 Powerful Add-Ons explains about some modified versions of the PuTTY that stores the session information in a file instead of Windows registry.

I was a big fan of OpenDNS dns service, but recently I found few bad things about their offerings. I strongly recommend to stay away from OpenDNS service.

All your search queries belongs to OpenDNS

OpenDNS redirects all your Google search queries though their servers. They captures your search query data and they forwards to real google.com domain. Here is a quick DNS lookup:
$ host www.google.co.in 208.67.220.220
Sample output:

Using domain server:
Name: 208.67.220.220
Address: 208.67.220.220#53
Aliases:
www.google.co.in is an alias for www.google.com.
www.google.com is an alias for google.navigation.opendns.com.
google.navigation.opendns.com has address 208.67.219.230
google.navigation.opendns.com has address 208.67.219.231

They may also do same for your email and other search engine.

Update: Dave has pointed out the reason why OpenDNS forwards google through their server. You can also turn on or off this feature from OpenDNS control panel.

OpenDNS is bad for server

Don't use them on your colocated server or vps server. They redirect web browser users or scripts accessing nonexistent domains to a page containing sponsored search results, ads, and a search form. The DNS protocol requires that a query for a nonexistent domain must return the "NXDOMAIN" error response. Here is a sample output:
$ host abcabcxyzxyz.com 208.67.220.220
Sample output:

Using domain server:
Name: 208.67.220.220
Address: 208.67.220.220#53
Aliases:
abcabcxyzxyz.com has address 208.67.219.132
Host abcabcxyzxyz.com not found: 3(NXDOMAIN)

This encourages spam as you will not able to filter out spam queries using their dns servers.

OpenDNS caching sucks

I contacted their support about my problem but never got any reply. Their server always returns two IP address for my nameserver:
$ host ns2.nixcraft.net 208.67.220.220
Sample output:

Using domain server:
Name: 208.67.220.220
Address: 208.67.220.220#53
Aliases:
ns2.nixcraft.net has address 74.86.48.98
ns2.nixcraft.net has address 74.86.48.98

I don't have 2 IP address for ns2.nixcraft.net.

I strongly recommend running your own dns cache server along with your ISP forwarding nameservers.

Thanks to ricko for pointing out OpenDNS issue in a chat room and elsewhere on the Internet.

Update: Fri Nov 5, 2010 by Vivek: OpenDNS no longer redirects Google search queries though their servers:

$ host www.google.co.in 208.67.220.220
Using domain server:
Name: 208.67.220.220
Address: 208.67.220.220#53
Aliases:
www.google.co.in is an alias for www.google.com.
www.google.com is an alias for www.l.google.com.
www.l.google.com has address 173.194.33.104

Updated for accuracy.

shoes – A cross-platform Windowing Applikit

Shoes is a very informal graphics and windowing toolkit. It's for making regular old apps that run on Windows, Mac OS X and Linux. It's a blend of my favorite things from the Web, some Ruby style, and a sprinkling of cross-platform widgets. Shoes uses Ruby as its interface language.

It borrows a few things I like from the web:
=> Hyperlinks and URLs within Shoes and to the web.
=> Simple text layout -- though Shoes eschews floats.
=> Images and colors in the layout and in the background.
=> Margin and padding.
=> Resizable layouts.

How do I install shoes ?

First, get source code, enter:
$ cd /tmp
$ wget http://shoooes.net/dist/shoes-0.r925.tar.gz

Untar it:
$ tar -zxvf shoes-0.r925.tar.gz
$ cd shoes-0.r925
Now install required stuff under Debian / Ubuntu Linux:
$ sudo apt-get install libcairo2-dev libpixman-1-dev libpango1.0-dev libungif4-dev libjpeg62-dev libgtk2.0-dev vlc libvlc0-dev libsqlite3-dev libcurl4-openssl-dev ruby1.8-dev rake
If you are using RHEL / CentOS / Fedora Linux, enter:
# yum install giflib-devel cairo-devel libpixman-devel pango-devel libjpeg-devel gtk2-devel sqlite-devel vlc-devel libcurl-devel ruby-devel
Install it:
$ make
$ sudo make install

Sample output:

build options: shoes raisins (0.r925) [i486-linux]
CC       = cc
RUBY     = /usr
OPTIONS  =
installing executable file to /usr/local/bin
installing libraries to /usr/local/lib/shoes

See READM for for more information.

Hello World application

Sample hello.rb

 
Shoes.app {
  para strong("Hello, "), " world!"
}

OR

Shoes.app (:width => 200, :height => 50, :title => "Hi, Guest!") {
   para strong("Hello, "), " world!"
   @buttonQuit = button "Exit"
   @buttonQuit.click { exit() }
}

Run it as follows:
$ /usr/local/bin/shoes hello.rb
Another example:

Shoes.app {
  name = ask("Please, enter your name:")
  para "Hello, ", name
}

You can include images from web or create a simple links / urls:

 
Shoes.app (:title => "My App" ) {
  image "http://theos.in/wp-content/uploads/2008/06/honda-fcx-clarity-car-photo.jpg"
  para( link("Info").click{ alert ("The FCX Clarity, which runs on hydrogen and electricity, emits only water and none of the noxious fumes believed to induce global warming.") })
  para( link("Exit").click{ exit() } )
}

Sample output:

Loading images from the web

Fig.01: Loading images from the web

Sample applications

You can find sample application in /tmp/shoes-0.r925/samples/ directory. Here is animated clock program:

#
# Shoes Clock by Thomas Bell
# posted to the Shoes mailing list on 04 Dec 2007
#
Shoes.app :height => 260, :width => 250 do
  @radius, @centerx, @centery = 90, 126, 140
  animate(8) do
    @time = Time.now
    clear do
      draw_background
      stack do
        background black
        para @time.strftime("%a"),
          span(@time.strftime(" %b %d, %Y "), :stroke => "#ccc"),
          strong(@time.strftime("%I:%M"), :stroke => white),
          @time.strftime(".%S"), :align => "center", :stroke => "#666",
            :margin => 4
      end
      clock_hand @time.sec + (@time.usec * 0.000001),2,30,red
      clock_hand @time.min + (@time.sec / 60.0),5
      clock_hand @time.hour + (@time.min / 60.0),8,6
    end
  end
  def draw_background
    background rgb(230, 240, 200)
 
    fill white
    stroke black
    strokewidth 4
    oval @centerx - 102, @centery - 102, 204, 204
 
    fill black
    nostroke
    oval @centerx - 5, @centery - 5, 10, 10
 
    stroke black
    strokewidth 1
    line(@centerx, @centery - 102, @centerx, @centery - 95)
    line(@centerx - 102, @centery, @centerx - 95, @centery)
    line(@centerx + 95, @centery, @centerx + 102, @centery)
    line(@centerx, @centery + 95, @centerx, @centery + 102)
  end
  def clock_hand(time, sw, unit=30, color=black)
    radius_local = unit == 30 ? @radius : @radius - 15
    _x = radius_local * Math.sin( time * Math::PI / unit )
    _y = radius_local * Math.cos( time * Math::PI / unit )
    stroke color
    strokewidth sw
    line(@centerx, @centery, @centerx + _x, @centery - _y)
  end
end
Fig. 02: Animated clock

Fig. 02: Animated clock

Shoes manual

The manual can be launched by typing the following command
$ shoes -m

Further readings: