Nagios: Monitor your remote Linux host using NRPE

Nagios Remote Plugin Executor (NRPE) is a Nagios agent that allows you to execute Nagios plugins remotely. This allows you to monitor remote resources like disk usage, CPU load, etc. Nagios periodically polls the agent on the remote system using the check_nrpe plugin. We need do install and configure NRPE on both Nagios server and the remote hosts which we are going to monitor. It's a good idea to start with remote hosts then proceed to the Nagios server.

Remote Linux host side:

1. Install PERL

[remotehost]#  yum -y install perl

2. Create Nagios user

[remotehost]# useradd nagios
[remotehost]# passwd nagios

3. Download and install Nagios Plugin

[remotehost]# wget http://prdownloads.sourceforge.net/sourceforge/nagiosplug/nagios-plugins-1.4.16.tar.gz
[remotehost]# tar zxvf nagios-plugins-1.4.16.tar.gz
[remotehost]# cd nagios-plugins-1.4.16
[remotehost]# export LDFLAGS=-ldl
[remotehost]# ./configure --with-nagios-user=nagios --with-nagios-group=nagios --enable-redhat-pthread-workaround --enable-libtap --enable-extra-opts --enable-perl-modules
### When you compile with  --enable-perl-modules you may get some error, see notes bottom of the page###
[remotehost]# make
[remotehost]# make install

[remotehost]# chown nagios.nagios /usr/local/nagios
[remotehost]# chown -R nagios.nagios /usr/local/nagios/libexec/

4. Download and Install NRPE

NRPE runs as a xinetd service, so make sure that xinetd is already installed and running on your system.

[remotehost]# yum -y install xinetd

You can manually install xinetd from http://www.xinetd.org/xinetd-2.3.15.tar.gz

[remotehost]# wget  http://downloads.sourceforge.net/project/nagios/nrpe-2.x/nrpe-2.14/nrpe-2.14.tar.gz?r=&ts=1363788540&use_mirror=hivelocity

[remotehost]# tar xvfz nrpe-2.14.tar.gz
[remotehost]# cd nrpe-2.14
[remotehost]#./configure --enable-command-args
[remotehost]# make all
[remotehost]# make install-plugin
[remotehost]# make install-daemon
[remotehost]# make install-daemon-config
[remotehost]# make install-xinetd

5. Set dont_blame_nrpe on nrpe.cfg

Open the file /usr/local/nagios/etc/nrpe.cfg on the remote host and set the dont_blame_nrpe directive to 1
[remotehost]# vim /usr/local/nagios/etc/nrpe.cfg
dont_blame_nrpe=1

6. Add Nagios server ip address to the file /etc/xinetd.d/nrpe

Add ip address of the Nagios server to the only_from directive to the file /etc/xinetd.d/nrpe. There is a space after the 127.0.0.1 and the nagios server ip address(in this example, nagios server ip address is 192.168.0.12)

[remotehost]# vim /etc/xinetd.d/nrpe

# description: NRPE (Nagios Remote Plugin Executor)
service nrpe
{
        flags                = REUSE
        socket_type      = stream
        port                   = 5666
        wait                   = no
        user                   = nagios
        group                = nagios
        server                = /usr/local/nagios/bin/nrpe
        server_args        = -c /usr/local/nagios/etc/nrpe.cfg --inetd
        log_on_failure    += USERID
        disable              = no
        only_from         = 127.0.0.1 192.168.0.12
}

7. Setup NRPE to run as daemon

Add the following at the end of the file /etc/services

[remotehost]# vim /etc/services
nrpe 5666/tcp # NRPE

8. Restart xinetd service and test to see NRPE is running

[remotehost]# service xinetd restart

[remotehost]# netstat -at | grep nrpe
tcp        0      0 *:nrpe                      *:*                         LISTEN

Run this command to make sure that NRPE is functioning properly, which outputs NRPE version

[remotehost]# /usr/local/nagios/libexec/check_nrpe -H localhost
NRPE v2.13

9. Define commands in /usr/local/nagios/etc/nrpe.cfg

The nrpe.cfg file located on the remote host contains the commands that are needed to check the services on the remote host. By default the nrpe.cfg comes with few standard check commands like check_users, check_load etc.

command[check_users]=/usr/local/nagios/libexec/check_users -w 5 -c 10
command[check_load]=/usr/local/nagios/libexec/check_load -w 15,10,5 -c 30,25,20

If you want to check more services then you need to define the commands in nrpe.cfg

10. Add new rules in iptables 

Add new rules in iptables to allow incoming connections for NRPE or just stop iptables initially to test the setup. Later on we can add rules to allow incoming connection for NRPE

[remotehost]# service iptables stop

Nagios server side:

I assume you already configured Nagios on the Nagios monitoring server. If not follow the instructions here.

1. Download and install NRPE on the nagios monitoring server

[nagios-server]# wget  http://downloads.sourceforge.net/project/nagios/nrpe-2.x/nrpe-2.14/nrpe-2.14.tar.gz?r=&ts=1363788540&use_mirror=hivelocity
[nagios-server]# tar xvfz nrpe-2.14.tar.gz
[nagios-server]# cd nrpe-2.14
[nagios-server]# ./configure --enable-command-args
[nagios-server]# make all
[nagios-server]# make install-plugin

2. Test to see Nagios monitoring server can talk to the remote host.

[nagios-server]# /usr/local/nagios/libexec/check_nrpe -H 192.168.0.11
NRPE v2.14

Here 192.168.0.11 is the IP address of the remote host, where Nagios plugins and NRPE were installed.
If you get the version of NRPE as a output of the command tells that you are good to go ahead.

3. Define check_nrpe command

You need to define check_nrpe command in one of the Nagios object configuration files
(/usr/local/nagios/etc/objects/commands.cfg) in order to use the check_nrpe plugin. For that, open the
command.cfg file and add the following command definition.

[nagios-server]# vim /usr/local/nagios/etc/objects/commands.cfg
# check_nrpe command definition
define command{
command_name check_nrpe
command_line $USER1$/check_nrpe -H $HOSTADDRESS$ -t 30 -c $ARG1$
}

4. Create host and service definition for remotehosts

Create a new configuration file /usr/local/nagios/etc/objects/remotehosts.cfg to define the host and service definition for this particular remotehost. It's good idea to make a copy of localhost.cfg, rename it to remotehosts.cfg and modify to meet your requirement. Define this remotehosts.cfg file to the Nagios configuration file nagios.cfg

[nagios-server]# cp /usr/local/nagios/etc/objects/localhost.cfg /usr/local/nagios/etc/objects/remotehosts.cfg

[nagios-server]# vim /usr/local/nagios/etc/nagios.cfg
# Definitions for monitoring the remote (Linux) host

cfg_file=/usr/local/nagios/etc/objects/remotehosts.cfg

[nagios-server]# vim /usr/local/nagios/etc/objects/remotehosts.cfg

i) Define Hosts to monitor

# Define multiple remote hosts

define host{
        use Linux-server        ; Name of host template to use
        host_name                 remotehost1
        alias                           Remote Host1
        address                     192.168.0.10
        }

define host{
        use Linux-server        ; Name of host template to use
        host_name                 remotehost2
        alias                           Remote Host2
        address                     192.168.0.11
        }

ii) Define Host group

# Define an optional hostgroup for remote Linux hosts

define hostgroup{
        hostgroup_name  linux-servers ; The name of the hostgroup
        alias                     Linux Servers ; Long name of the group
        members              remotehost1, remotehost2     ; Comma separated list of hosts that belong to this group
        }

You can replace remotehost1, remorehost2 with the corresponding hostname of your remotehosts.

iii) Define Services

# Define a service to check the disk space of the root partition

define service{
        use                                   local-service         ; Name of service template to use
        #host_name                      remotehost
hostgroup_name               linux-servers
        service_description           Root Partition
        check_command              check_nrpe!check_disk!20%!10%!/
        }

# Define a service to check the number of currently logged in

define service{
        use                                   local-service         ; Name of service template to use
        #host_name                      remotehost
hostgroup_name               linux-servers
        service_description           Current Users
        check_command              check_nrpe!check_users!20!50
        }

# Define a service to check the number of currently running procs

define service{
        use                                    local-service         ; Name of service template to use
        #host_name                       remotehost
hostgroup_name                linux-servers
        service_description            Total Processes
        check_command               check_nrpe!check_procs!250!400!RSZDT
        }

# Define a service to check the load on the local machine.

define service{
        use                                   local-service         ; Name of service template to use
        #host_name                      remotehost
hostgroup_name               linux-servers
        service_description           Current Load
        check_command              check_nrpe!check_load!5.0,4.0,3.0!10.0,6.0,4.0
        }

5. Restart the Nagios and Apache services

Restart the nagios and Apache and login to http://<your.nagios.server.ip>/nagios to verify the status of the remote Linux hosts that was added to nagios for monitoring.

[nagios-server]# service nagios restart
[nagios-server]# service httpd restart

Note 1:

When you compile with  --enable-perl-modules you may get error like this,

make[3]: Entering directory `/root/Downloads/nagios-plugins-1.4.16/perlmods/Test-Simple-0.70'
make[3]: *** No rule to make target `install'.  Stop.
make[3]: Leaving directory `/root/Downloads/nagios-plugins-1.4.16/perlmods/Test-Simple-0.70'
Can't run make install at ../tools/build_perl_modules line 85.

Solution:

Go inside the extracted nagios-plugins that is being built. Build the Test-Simple-0.70 package manually then make install the nagios-plugins again

[remotehost]# cd nagios-plugins-1.4.16/perlmods/Test-Simple-0.70
[remotehost]# perl Makefile.PL
[remotehost]# make
[remotehost]# make install

# go back to nagios-plugins-1.4.16 directory to make install

[remotehost]# cd ../..
[remotehost]# make install

Note 2: 
When you compile, sometimes it hangs and displays the message “checking for redhat spopen problem…”. To solve this issue, compile with --enable-redhat-pthread-workaround

Note 3:
To enable support for command arguments in the NRPE daemon, you need to compile with –enable-command-args