≡ Menu

sysctl conf file

One of my client has server node located at north America, Asia and Europe data centers. All servers are connected using 1000Mbps links. They transfers lots of data between all nodes over ssh session using scp / sftp. However, performance was horrible. After some research I came across High Performance SSH/SCP - HPN-SSH patch for OpenSSH:

SCP and the underlying SSH2 protocol implementation in OpenSSH is network performance limited by statically defined internal flow control buffers. These buffers often end up acting as a bottleneck for network throughput of SCP, especially on long and high bandwith network links.

Modifying the ssh code to allow the buffers to be defined at run time eliminates this bottleneck. We have created a patch that will remove the bottlenecks in OpenSSH and is fully interoperable with other servers and clients. In addition HPN clients will be able to download faster from non HPN servers, and HPN servers will be able to receive uploads faster from non HPN clients. However, the host receiving the data must have a properly tuned TCP/IP stack.

The amount of improvement any specific user will see is dependent on a number of issues. Transfer rates cannot exceed the capacity of the network nor the throughput of the I/O subsystem including the disk and memory speed. The improvement will also be highly influenced by the capacity of the processor to perform the encryption and decryption. Less computational expensive ciphers will often provide better throughput than more complex ciphers.

You can download HPN-SSH patch here. This patch improved our performance. You also need to tweak Linux TCP/IP networking settings. Here is my sysctl.conf file ( read this TCP tunning Linux guide for detailed explanation) :
# optimization start
# increase TCP max buffer size setable using setsockopt()
net.ipv4.tcp_rmem = 4096 87380 8388608
net.ipv4.tcp_wmem = 4096 87380 8388608
# increase Linux auto tuning TCP buffer limits
# min, default, and max number of bytes to use
# set max to at least 4MB, or higher if you use very high BDP paths
net.core.rmem_max = 8388608
net.core.wmem_max = 8388608
net.core.netdev_max_backlog = 5000
net.ipv4.tcp_window_scaling = 1
# optimization end

Howto Reboot or halt Linux system in emergency

Linux kernel includes magic system request keys. It was originally developed for kernel hackers. However, you can use this hack to reboot, shutdown or halt computer safely (remember safe reboot/shutdown == flush filesystem buffers and unmount file system and then reboot so that data loss can be avoided).

This is quite useful when Linux based system is not available after boot or after a X server crashed ( svgalib program crashes) or no display on screen. Sysrq key combo forces the kernel to respond it regardless of whatever else it is doing, unless it is completely locked up (dead).

Using further extension to iptables called ipt_sysrq (new iptables target), which allows you to do the same as the magic sysrq key on a keyboard does, but over the network. So if your network server is not responding you can still reboot it. Please note that Magic SysRq support need to be compiled in your kernel. You need to say "yes" to 'Magic SysRq key (CONFIG_MAGIC_SYSRQ)' when configuring the kernel. I'm assuming that you have Magic SysRq key' support is compiled in your kernel.

Enable sysrq keys

By default it is not enabled on many Linux distributions. Add or modify following line (as soon as new Linux system installed) /etc/sysctl.conf:
# vi /etc/sysctl.conf
Append following config directive:
Save and close the file. Reload settings:
# sysctl -p

Save and close the file and reboot system to take effect

How do I use the magic SysRq keys in emergency?

You need to use following key combination in order to reboot/halt/sync file system etc:

The 'SysRq' key is also known as the 'Print Screen' key. COMMAND-KEY can be any one of the following (all keys need to hit simultaneously) :

  • 'b' : Will immediately reboot the system without syncing or unmounting your disks.
  • 'o' : Will shutdown your system off (if configured and supported).
  • 's': Will attempt to sync all mounted filesystems.
  • 'u' : Will attempt to remount all mounted filesystems read-only.
  • 'e' : Send a SIGTERM to all processes, except for init.
  • 'h': Show help, indeed this the one you need to remember.

So whey you need to tell your Linux computer to reboot or when your X server is crashed or you don't see anything going across the screen then just press:

ALT+SysRQ+s : (Press and hold down ALT, then SysRQ (Print Screen) key and press 's') -Will try to syn all mounted system

ALT+SysRQ+r : (Press and hold down ALT, then SysRQ (Print Screen) key and press 'r') -Will reboot the system.

If you wish to shutdown the system instead of reboot then press following key combination:

ipt_sysrq is a new iptables target that allows you to do the same as the magic sysrq key on a keyboard does, but over the network. Sometimes a remote server hangs and only responds to icmp echo request (ping). Every administrator of such machine is very unhappy because (s)he must go there and press the reset button. It takes a long time and it's inconvenient. So use the Network Magic SysRq and you will be able to do more than just pressing a reset button. You can remotely sync disks, remount them read-only, then do a reboot. And everything comfortably and only in a few seconds. Please see Marek Zelem page to enableIP Tables network magic SysRq function.

For more information read official Documentation for sysrq.c version 1.15 stored in /usr/src/linux/Documentation/sysrq.txt and read man page of sysctl, sysctl.conf.