Unplanned downtime may be the result of a software bug, human error, equipment failure, power failure, and much more. Last week was a bad one. We faced three different downtime:

  • First, there was a fiber cut for one of our data center resulting into routing anomalies due BGP reroute. Traffic was rerouted but updating those BGP tables took some time to update.
  • Someone from networking team failed to follow proper maintenance procedures for network device resulted into 55 minutes downtime.
  • One of our SAN hardware failure – Many internal UNIX / Linux web applications use SAN to store data including file server, tracking apps, R&D apps, IT help desk, LAN and WAN servers failed. This one lasted for 12 hrs. It was stared around midnight. The vendor replaced entire SAN hardware. Now we have dual stacked SAN as a backup device for internal usage.

[continue reading…]

Like most sys admin, I’m lazy. I try to automate almost all things in order to save time. Inexperienced sys admin and help desk staff working under me finds all these tools useful. It saves their time and avoids security issues. Automation allows help desk staff to do things that they don’t have enough direct system knowledge to do themselves. However, selecting correct tool and applying correct methodology is very important.
[continue reading…]

Result Last Poll: Your Remote Maintenance Services:

I’m not at all surprised with results. SSH is the most popular for managing server remotely. May be should donate some money to project, to show your support.

New Poll: Your SysAdmin Programming Language

A major class of scripting languages has grown out of the automation of job control under Linux and UNIX, which relates to starting and controlling the behavior of system programs. Personally I prefer to use perl and shell scripting. These days I’m also trying out python. Please vote your choice of scripting Language:


Maintenance services that give direct access to servers and devices for administrative purposes. Please vote your common remote maintenance services such as SSH, X Windows, VNC, and Microsoft Terminal Services etc:

Note: There is a poll embedded within this post at left sidebar, please visit the site to participate in this post’s poll. The poll is located on left sidebar.

In last months reader poll I asked about Firewall on dedicated UNIX / Linux box.

Do we really need a firewall?

Personally, I install firewall on all boxes to filter out unwanted junk and IPs; even if box is only running public service such as a web server. The overall idea is to limit access and reduce liability on my part if serer got rooted (read as compromised). Remember, bad boys never play by the rules.

Our last poll asked readers of this blog the question:

How do you keep remote users out in emergency?

Thanks to everyone, total 789 people responded to the poll. Results are as follows:

  • 47% admin prefer to take down eth0 using ifconfog eth0 down
  • 29% admin prefer to deny access to all using iptables
  • 14% admin prefer to use old good /etc/nologin file using touch /etc/nologin
  • 10% admin prefer to turn off both ftpd & sshd service

(click to enlarge image)

New poll: The best way to Learn Linux…

I am interested to see, how all of you started to learn Linux. What is the best way to learn Linux? Did you enrolled in class or just grabbed Linux and start using everyday. Sure search engine, online forums, chat room and Linux documentation projects helps but just imagine somebody want to learn Linux and have no idea where to begin…

Hope this poll will throw out some light 😀

Our last poll asked readers of this blog the question:

Which Webserver do you use?

Thanks to everyone, total 435 people responded to the poll. Results are as follows:

  • 66% users are using Apache v2.2.x web server
  • 11% uses are using Apache v1.3x web server
  • 8% users are using Microsoft IIS v5/6 web server
  • 6% users are using lighttpd web server

I guess apache rules when it comes to Web server.

New poll: How do you keep remote users out in emergency?

Last week I had some problem with 3 servers. Since I was patching system and backing up stuff I wanted to keep my remote user out for at least for 60-90 minutes. As usual I dropped the remote connections using /etc/nologin and turning off both ftpd/sshd.

Now I am wondering — how you guys keep remote users out in emergency? Let me know if you are using some other nifty technique.