Loading ...Monitor and restart Apache or lighttpd webserver when daemon is killed
When you cannot monitor your server for service availability, it is better to take help of automated monitor and restart utility. Last 4 days I was away from my server as I was enjoying my vacation. During this time due to load my lighttpd webserver died but it was restarted automatically within 2 minutes. I had utility configured for monitoring services on a Linux system called monit. It offers all features you ever needed for system monitoring and perform error recovery for UNIX like system.
Before monit I had my own shell and perl script for monitoring service. If service failed script will try to restart service and send an automated email to me. However monit is a superior solution.
monit is a utility for managing and monitoring processes, files, directories and devices on a Unix system. Monit conducts automatic maintenance and repair and can execute meaningful causal actions in error situations. For example, monit can start a process if it does not run, restart a process if it does not respond and stop a process if it uses to much resources. You may use monit to monitor files, directories and devices for changes, such as timestamps changes, checksum changes or size changes.
You may also use monit to monitor files, directories and devices on localhost. Monit can monitor these items for changes, such as timestamps changes, checksum changes or size changes. This is also useful for security reasons you can monitor the md5 checksum of files that should not change.
Personally I always install and configure monit on all boxes which are under my control.
Install monit under Debian or Ubuntu Linux
Use apt-get command to install monit
# apt-get install monitOR$ sudo apt-get install monit
Install monit under Red Hat enterprise Linux (source code installation)
Many distributions include monit. However monit is not included in official Red hat enterprise Linux. Just download monit source code from official web site using wget command:
# cd /optUntar monit
# wget http://www.tildeslash.com/monit/dist/monit-4.8.2.tar.gz
# tar -zxvf monit-4.8.2.tar.gz
# cd monit-4.8.2
Configure and compile monit:
# ./configure
# make
Install monit
# make install
Copy monit configuration file:
# cp monitrc /etc/monitrc
By default monit is located at /usr/local/bin/monit
How do I Configure monit?
monitrc is name of monit configuration file and it is by default located at /etc/monitrc location. However each distribution places file in different location: .
=> Source code installation : /etc/monitrc
=> Debian / Unentu Linux installation : /etc/monit/monitrc
Open monit configuration file and setup values as follows:
# vi /etc/monitrc
a) Run it as daemon and check the services (such as web, mysql, sshd) at 2-minute
intervals.
set daemon 120
b) Set syslog logging with the 'daemon' facility:
set logfile syslog facility log_daemon
c) Set mail server name to send email alert
set mailserver mail.cyberciti.biz
Set email format such as from email
set mail-format { from: alert@nixcraft.in
subject: $SERVICE $EVENT at $DATE
message: Monit $ACTION $SERVICE at $DATE on $HOST: $DESCRIPTION.
}
d) Now most important part, restart lighttpd or apache web server if failed or killed by Linux kernel due to any causes:
check process lighttpd with pidfile /var/run/lighttpd.pid
group lighttpd
start program = "/etc/init.d/lighttpd start"
stop program = "/etc/init.d/lighttpd stop"
if failed host 75.126.43.232 port 80
protocol http then restart
if 5 restarts within 5 cycles then timeout
Where,
- check process lighttpd with pidfile /var/run/lighttpd.pid : You are specifying lighttpd pid file and daemon name
- group lighttpd: Specify group name, which is allowed or used to start/restart lighttpd
- start program = "/etc/init.d/lighttpd start" : Command to start lighttpd server
- stop program = "/etc/init.d/lighttpd stop" : Command to stop lighttpd server
- if failed host 127.0.0.1 port 80 : Server IP address and port number (80)
- protocol http then restart : If above IP and port failed restart the webserver
- if 5 restarts within 5 cycles then timeout : Try to restart 5 times; if monit cannot restart webserver 5 times; just time out to avoid race condition.
Here is my mysql server restart configuration directives:
check process mysqld with pidfile /var/run/mysqld/mysqld.pid
group database
start program = "/etc/init.d/mysqld start"
stop program = "/etc/init.d/mysqld stop"
if failed host 127.0.0.1 port 3306 then restart
if 5 restarts within 5 cycles then timeout
Here is my sshd server configuration directives:
check process sshd with pidfile /var/run/sshd.pid
start program "/etc/init.d/sshd start"
stop program "/etc/init.d/sshd stop"
if failed host 127.0.0.1 port 22 protocol ssh then restart
if 5 restarts within 5 cycles then timeout
Here is my Apache serverrestart configuration directives:
check process httpd with pidfile /var/run/httpd.pid
group apache
start program = "/etc/init.d/httpd start"
stop program = "/etc/init.d/httpd stop"
if failed host 127.0.0.1 port 80
protocol http then restart
if 5 restarts within 5 cycles then timeout
Replace IP address 127.0.0.1 with your actual IP address. If you are using Debian just start monit:
# /etc/init.d/monit start
If you are using Red Hat Enterprise Linux, start monit from /etc/inittab file:
Open /etc/inittab file:
# vi /etc/inittab
Append following line:
mo:2345:respawn:/usr/local/bin/monit -Ic /etc/monitrc
Now start monit:
# init -qOR
# telinit -q
You can verify that monit is started from /var/log/message log file:
# tail -f /var/log/messageOutput:
Nov 21 04:39:21 server monit[8759]: Starting monit daemon Nov 21 04:39:21 server monit[8759]: Monit started
If lighttpd died, you will see something as follows in log file:
Nov 21 04:45:13 server monit[8759]: 'lighttpd' process is not running Nov 21 04:45:13 server monit[8759]: 'lighttpd' trying to restart Nov 21 04:45:13 server monit[8759]: 'lighttpd' start: /etc/init.d/lighttpd
You may use monit to monitor daemon processes or similar programs running on localhost or started from /etc/init.d/ location such as
=> Apache Web Server
=> SSH Server
=> Postfix/Sendmail MTA
=> MySQL etc
Further readings
- monit man page
- monit official web site and documenation
Want to stay up to date with the latest Linux tips, news and announcements? Subscribe to our free e-mail newsletter or full RSS feed to get all updates.
You can Email this page to a friend.
You may also be interested in...
- BSD start services
- FreeBSD: How To Start / Restart / Stop Network and Routing Service
- nixCraft FAQ roundup
- New 19″ Viewsonic Widescreen LCD Monitor
- nixCraft FAQ Roundup
Discussion on This Article:
Leave a Reply
We encourage your comments, and suggestions. But please stay on topic, be polite, and avoid spam. Thank you very much for stopping by our site!
Tags: error_recovery, Linux, monit, monitoring_services, Networking, service_availability, superior_solution, UNIX, web_servers ~ Last updated on: June 20, 2007
never took the time to read through the documentation. But after reading this article I say it sounds easy enough.
I will give it a try and probably abandon my custom setup. Thanks!
Your untar command is uncorrect. It should be:
tar -zxvf monit-4.8.2.tar.gz
Matthew, Thanks for heads up!
It was a typo…
I think “inittab -q” should actually be “telinit q”.
Thanks.
Jeff,
Yup, it was typo. You need to use init -q or telinit -q
Appreciate your post.
Does it work with Suse linux?
Brian,
Yes it does works with Sues Linux and all other UNIX like oses.
how do you configure monit to monitor apache’s load such that if reaches 5.00 and above it will automatically restart apache?
thanks in advance
Another option, which should be a bit simpler, is to use a cron script to check the health of Apache and MySQL.
Thanx for this tutorial. was very helpful and easy to understand.
But i noticed one flaw/bug with monit:
Assume that you have monit setup to monitor apache and mysql. And the section of apache comes first in the monit.conf file followed by the mysql section.
If only apache down, monit then ONLY restarts apache (since its first in the monit.conf file). But if ONLY mysql is down(apache still running), then monit first shut’s down and restart apache and the starts mysql. So in this way i cannot make monit ONLY restart the daemon that is down.
Any work around for this ??
thecoolone,
I’ve not noticed such a bug. Try to use latest version.
any body tell me how can i get site domin name and password please brothers please am comming newly in hacking so please brothers help me am your brother please help me
This document is simple and it is easy to set up. One question though.
Can monit be set up in such way: if it can not restart a service, then send an alert to another email address, which is a pager? The line I am looking at is:
if 5 restarts within 5 cycles then timeout
Thanks for help in advance.
Ira
This is default. Monit will raise an email alert when event is timeout.
You can also try as follows:
check process httpd with pidfile /var/run/httpd.pidalert pager@service.airtel.in only on { timeout, nonexist }
monit man page has all the example, please refer the same.
Thanks so much. Read through man page more.
Ira
I have a ” something.x86 ” file that I’m running as a executable
But I can’t find any “something.pid” anywhere
How do I check if ” something.x86 ” is running and how do I set it up to restart if it’s not
Best regards
jon ivar
us ps -aux command:
ps -aux | grep -i 'something'2nd column shows PID. Generally /var/run/something stores the PID. If it is your own custom-made program, consider writing sys v init script; so that you can integrate your program with monit.
HTH
thanks vivek
Hi all,
How to monitor bind named server?
Is it possible to monitor this kind of server.
TQ
Very useful article. I was up and running on SUSE in only a few minutes.
I completely forgot about monit and was looking for a solution to restart daemons just to find your article and realize I already know about it.
Monit is no enterprise grade solution though, I think the latest nagios is quite nice and allows daemon restarts, custom scripts, etc. Another one I’ve seen around is zabbix. Maybe someone will find these useful.
what advantages does the monit packag have compared to adding a 2 minute crontab entry that checks and restarts the daemon and sends the email alert
Reliability
Advanced configuration option
Manage process, files, directory
md5 security and checksum
Read monit docs for complete feature list. I don’t think so shell script offers all these features.
Thanks for a great article. Just what I was looking for. FYI, Monit 4.10.1 is out now.
I have installed monit on my server. But here a problem happened, such if I wants to run a process,such as “/root/appname -m ‘pram1′”, it wokrs, but if I write it in a executable Shell file, it cannot be executed. Why?
It’s me, the last problem is resolved, it has something to do with working path. But here other question.: why I cann’t excute a shell files, the content like this:
1 cd /root/test
2 ./appname -m “param1″
Hi.
My lighty instance only return 400 to monit. It is really strange since my app server with almost the same monit setup work flawlessly.
Any ideas ?
lighttpd-1.4.19 (compiled)
monit-4.8.1 (ubuntu-hardy)