Monitoring remote systems: Using GTalk notifications

Since a few weeks ago I am using a Raspberry Pi as bittorrent client instead of my old, big and noisy Pentium III 1Ghz

{% img center /img/DSC_0321_sm.jpg 'admin' %}

I am pretty happy with the change (besides a few issues with the USB/Network controller). I managed to get rid of a big PC case, there are no fans spinning anymore (noiseless), and the power consumption is unbeatable, and I only paid around $40. Win-Win situation, isn't it?.

But this post is not about my rasPi (ok, maybe just a little bit), but about how to received XMMP notification from a remote system. In this case, I was interested in sending a brief report of my VPS status (CPU,MEM,HD, etc) every 12 hours to my Gtalk account, so I can see how it is doing.

Most of the alternatives are based on sending emails, but I wasn't very keen on setting up a MTA only to send a few reports every X hours, so I decided to use XMMP notifications instead.

Googling a little bit I found this:

apt-cache show sendxmpp
Package: sendxmpp
Priority: optional
Section: net
Installed-Size: 88
Maintainer: Guus Sliepen <guus@debian.org>
Architecture: all
Version: 1.20-1
Depends: perl, libnet-xmpp-perl
Filename: pool/main/s/sendxmpp/sendxmpp_1.20-1_all.deb
Size: 15026
MD5sum: 2a3fcdc995442ebd1f7d8a336fc88164
SHA1: 90dd5cc21c9b97872dec5a55ce9d55ea7d728d65
SHA256: eb8a2fd09ffc65c04cc54a8a718faf550b5864a3135a9d045d2657cd43ed0449
Description: commandline XMPP (jabber) utility
 sendxmpp is a perl script to send XMPP (jabber) messages, similar to what
 mail(1) does for mail. XMPP is an open, non-proprietary protocol for instant
 messaging. See www.jabber.org for more information.
Tag: implemented-in::perl, interface::commandline, protocol::jabber, role::program, scope::utility

Just what I needed.

PROs: - Easy to use and integrate in simple scripts. - Perl script available in Debian repositories. - Lightweight. There are no daemons running. The script is executed whenever is needed. - Secure. Using SSL.

CONS: - Only available to GTalk users (obviously). - Limited size of the message. (just because of the limited size of the screen of your mobile device) - In order to avoid SPAM, the account where you are sending the message from, has to be in the contact list of the receiver.

As, it was working like a charm in my Raspberry Pi, I decided to do the same in my VPS. Let me describe in a few steps how to configure it:

Installation? Debian's way.

 apt-get install sendxmpp
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following extra packages will be installed:
  libauthen-sasl-perl libdigest-sha1-perl libio-socket-ssl-perl libnet-libidn-perl libnet-ssleay-perl libnet-xmpp-perl
  libxml-stream-perl
Suggested packages:
  libdigest-hmac-perl libgssapi-perl libio-socket-inet6-perl libnet-dns-perl
The following NEW packages will be installed:
  libauthen-sasl-perl libdigest-sha1-perl libio-socket-ssl-perl libnet-libidn-perl libnet-ssleay-perl libnet-xmpp-perl
  libxml-stream-perl sendxmpp
0 upgraded, 8 newly installed, 0 to remove and 0 not upgraded.
Need to get 581 kB of archives.
After this operation, 2,543 kB of additional disk space will be used.
Do you want to continue [Y/n]? Y

Pretty easy configuration. Under your home directory, create a .sendxmpprc file

# cat .sendxmpprc
<your_email>@gmail.com <your_password>

Let's test it. First sending a message from the standard input:

echo "hello there" | /usr/bin/sendxmpp  -t -u <your_email> -o gmail.com <recipient Google ID> 

Notification received:

{% img center /img/gtalk1.jpg %}

And now, sending the text from a file:

echo "this is a message send from a text file" > text.txt
/usr/bin/sendxmpp -t -u <your_email> -o gmail.com <recipient Google ID> -m text.txt

Notification received:

{% img center /img/gtalk2.jpg %}

Working!

Integrating it in a simple bash script:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
#!/bin/bash

REPORT="/tmp/.report"

[  -f $REPORT ] && rm -f $REPORT
echo  "VPS report at `date`:" >> $REPORT
echo  "Machine stats : `uptime`" >> $REPORT
echo  "Memory stats : `free -m`" >> $REPORT
echo  "Hard Disk Space: `df -h`" >> $REPORT
echo "" >> $REPORT


#Sending the report to GTALK
/usr/bin/sendxmpp -t -u <your_email> -o gmail.com <recipient Google ID> -m $REPORT 

And setting up the task in cron:

crontab -e
57 17 * * * sh /scripts/send_report.sh

The notification received in my Gtalk client:

{% img center /img/gtalk3.jpg %}

This is just an example, but I am pretty sure you could find a lot of applications to this basic notification system

Enjoy!

Cacti monitoring over SSH

Recently I had to configure Cacti in order to monitor several blades in a cluster (nothing fancy, just CPU load, memory usage, load average and disk space) but this time NET-SNMP wasn't an option. I knew that I could use server scripts to access the remote blades using SSH, so after googling a while, I found this proejct from the Percona guys:

Percona Monitoring Plugins for Cacti

What can I say... Amazing. Everything working in a few minutes.

Although all the information about Cacti and Percona plugins can be found in their web sites, I decided to put it all together and take a few screenshots, so if you happen to do something similar, this is what I did:

Let's assume that you have a fresh a Debian box (In my case, I created a Debian virtual machine, running testing for this)

1) Install cacti (and its dependences) using aptitude

aptitude install cacti
The following NEW packages will be installed:
  apache2-mpm-prefork{a} apache2-utils{a} apache2.2-bin{a}
  apache2.2-common{a} cacti dbconfig-common{a} libaio1{a}
  libapache2-mod-php5{a} libaprutil1-ldap{a} libcairo2{a} libdatrie1{a}
  libdbd-mysql-perl{a} libdbi-perl{a} libdbi0{a} libfontenc1{a}
  libhtml-template-perl{a} libmysqlclient16{a} libnet-daemon-perl{a}
  libonig2{a} libpango1.0-0{a} libpango1.0-common{a} libphp-adodb{a}
  libpixman-1-0{a} libplrpc-perl{a} libpng12-0{a} libqdbm14{a} librrd4{a}
  libthai-data{a} libthai0{a} libx11-6{a} libx11-data{a} libxau6{a}
  libxcb-render-util0{a} libxcb-render0{a} libxcb1{a} libxdmcp6{a}
  libxfont1{a} libxft2{a} libxrender1{a} mysql-client-5.1{a}
  mysql-client-5.5{ab} mysql-common{a} mysql-server{a} mysql-server-5.5{ab}
  mysql-server-core-5.5{ab} php5-cli{a} php5-common{a} php5-mysql{a}
  php5-snmp{a} php5-suhosin{a} rrdtool{a} x-ttcidfont-conf{a} x11-common{a}
  xfonts-encodings{a} xfonts-utils{a}

2) Setup your timezone in php.ini

There are three different "php.ini" files in Debian:

/etc/php5/cli/php.ini
/etc/php5/apache2/php.ini
/etc/php.ini

Cacti is using PHP over the CLI, so it should be enough if you update "/etc/php5/cli/php.ini", but I would update all of them. Add this line to the file:

date.timezone = "<time zone>" 

For example:

date.timezone = "Asia/Tokyo"

The list of timezones is available here

3) Apache+MySQL+PHP+Cacti working out of the box with just a few commands

At this point you should be able to login in cacti on your apache webserver (http://VM_IP/cacti)

The default user/password is admin/admin. (the system will ask you to change it after the first login)

I won't explain Cacti configuration, because they already have very good documentation in their web site

4) Setting up the SSH authentication

This chapter is pretty well explained in Percona web site.

Basically we have to:
- Set up an SSH keypair for SSH authentication.
- Create a user on the remote server you want to graph.
- Install the public key into that user.s authorized_keys file.

Check the detailed information here Percona: SSH-Based Templates

5) Installing "Percona Monitoring Plugins for Cacti"

Download the plugins from Percona website: Percona - Download percona-monitoring-plugins

wget http://www.percona.com/redir/downloads/percona-monitoring-plugins/percona-monitoring-plugins-1.0.1.tar.gz

Extract the file, and copy ss_get_by_ssh.php to Cacti script folder:

tar zxvf percona-monitoring-plugins-1.0.1.tar.gz
cp percona-monitoring-plugins-1.0.1/cacti/scripts/ss_get_by_ssh.php /usr/share/cacti/site/scripts/

Edit /usr/share/cacti/site/scripts/ss_get_by_ssh.php with your remote user name and the path to the SSH identity from step 4.

$ssh_user   = 'cacti';                          # SSH username
$ssh_iden   = '-i /var/www/cacti/.ssh/id_rsa';  # SSH identity

6) Importing the template

Inside the tar file you will find different templates:

/percona-monitoring-plugins-1.0.1/cacti/templates# ls -lrt
total 1644
-rw-r--r-- 1 501 root 169078 Jun 15 04:44 cacti_host_template_percona_openvz_server_ht_0.8.6i-sver1.0.1.xml
-rw-r--r-- 1 501 root  41926 Jun 15 04:44 cacti_host_template_percona_nginx_server_ht_0.8.6i-sver1.0.1.xml
-rw-r--r-- 1 501 root 843345 Jun 15 04:44 cacti_host_template_percona_mysql_server_ht_0.8.6i-sver1.0.1.xml
-rw-r--r-- 1 501 root  89553 Jun 15 04:44 cacti_host_template_percona_mongodb_server_ht_0.8.6i-sver1.0.1.xml
-rw-r--r-- 1 501 root  75491 Jun 15 04:44 cacti_host_template_percona_memcached_server_ht_0.8.6i-sver1.0.1.xml
-rw-r--r-- 1 501 root  53416 Jun 15 04:44 cacti_host_template_percona_jmx_server_ht_0.8.6i-sver1.0.1.xml
-rw-r--r-- 1 501 root 249175 Jun 15 04:44 cacti_host_template_percona_gnu_linux_server_ht_0.8.6i-sver1.0.1.xml
-rw-r--r-- 1 501 root  73879 Jun 15 04:44 cacti_host_template_percona_apache_server_ht_0.8.6i-sver1.0.1.xml
-rw-r--r-- 1 501 root  37198 Jun 15 04:44 cacti_host_template_percona_redis_server_ht_0.8.6i-sver1.0.1.xml

In our case, we are only interested in cacti_host_template_percona_gnu_linux_server_ht_0.8.6i-sver1.0.1.xml

Console -> Import Templates -> Import template:

7) Create a new device

Under Console -> Devices, add a new device:

And complete the information (name, IP)

8) Modify the data template and the data source (I will use "Percona Disk Space DT" as example)

Check the Percona Data Templates availables:

Console -> Data Templates

Click in the name (Percona Disk Space DT) and let's edit the template. Go to the bottom of the page, and check "Custom Data":

Add Percona Disk Space DT as new data source for your host. Under Console -> Data Sources, click new:

Set the Remote IP and the partition (or volumen) to monitor

9) Create the graph

We have already created the data source for our remote server, so now, we just have to create a new graph.

Under Console -> Graph Management, add a new graph:

And select our data source (Percona Disk Space DT) for the device:

10) Result

At this point, the graph should be created, and after a few minutes, you should see something like this:

If something goes wrong, take a look at /var/log/cacti/cacti.log. There should be a line similar to this one:

<DATE> - CMDPHP: Poller[0] Host[3] DS[25] CMD: /usr/bin/php -q /usr/share/cacti/site/scripts/ss_get_by_ssh.php --host <REMOTE_IP> --type proc_stat --items gg,gh,gi,gj,gk,gl,gm,gn,go, output: gg:50994 gh:0 gi:76580 gj:56353167 gk:1617 gl:0 gm:0 gn:2135 go:-1

New house and new engine

So, here we go again...

It's been a while since I moved this blog to blogger, and I think it's about time to move it back to a hosting I can manage.

New house

So I decided to move bynario.com to a VPS... There are very good deals outside, and you can get your own VPS for 10 euros per month. Obviously, the specs of the virtual machine you can get for that money won't allow you to run a site like Slashdot, but I don't expect too many visits.

I have to say that Blogger is a great platform. It allows you to use your own domain name, it's free, very reliable and really easy to use.

Actually, too easy. I am not very interested in blogging "per se" (don't have too much to say), so I see this more like a tool to stay tuned with new technologies, and who knows, someone could even find something interesting here.

New engine

Instead of using any of the popular blog engines available (usually based on Apache+PHP+MySQL), like Wordpress, Drupal, etc, I decided to give Octopress a try. You can find all the information about what it is and how to install it in Octopress website.

I still have to migrate all the post from Blogger (thanks to this post seems to be really easy) and customize the whole site a little bit. I don't know if this will take me hours, days, weeks or even years ;)

See you around...

Interesting talks (I)

Sunday again...

Here you have a couple of interesting talks I found in Youtube today.

John Ousterhout - RAMCloud: Scalable High-Performance Storage Entirely in DRAM

https://www.youtube.com/watch?v=lcUvU3b5co8

Apache Hadoop - Petabytes and Terawatts

https://www.youtube.com/watch?v=SS27F-hYWfU

You can enjoy more videos in the LinkedIn Tech Talks channel in Youtube (LinkedInTechTalks)

New gadget: Super compact wireless router

I am staying a few weeks in Japan, and it seems that here, most of the hotels still offer wired LAN as the only way to connect to the internet, instead of WiFi (I'm sorry, are you from the past?).

I usually travel with my smartphone and tablet (and a laptop of course), and they don.t have RJ45 connector, so I .need. a wireless network for them.

Last week, I spent a few hours walking around in Akihabara, the electronic district here in Tokyo, and I bought this cheap (around 24Eur) AP/wireless router: LAN-W300N/PR5 (by ELECON Logitec).

{% img center http://4.bp.blogspot.com/-35mvVxjrUwQ/T8CzVJDHRnI/AAAAAAAANzk/7Mqbvyc7w2U/s320/DSC_0167%5B1%5D 320 300 'Nice box. Pretty small' %}

Inside of the package you will find the power adapter (micro USB 5V/0.6A), a LAN cable (really short), and instructions (in a perfect Japanese):

{% img center http://2.bp.blogspot.com/-LI3SLjTILlw/T8CzGuGEIrI/AAAAAAAANzM/nN9r_v9lB7c/s320/DSC_0170%5B1%5D 320 300 %}

The device is really small, I would say it is the size of a old cassette tape. It has 4 LAN 10/100 ports, 1 LAN port for WAN and 2 USB ports (mini and micro USB) for the power supply:

{% img center http://3.bp.blogspot.com/-cwcArvxhERI/T8CzCorXsEI/AAAAAAAANzE/GExGOToIrGE/s1600/DSC_0171%5B1%5D 320 300 %}

The device is working just out of the box, acting as AP and as neutral router. This is the default configuration:

External network: DHCP client Internal network: LAN: 192.168.2.1/24 WLAN:
SSID: logitecuser Encryption: WPA2/PSK (You will find the password written under your device.)

Unfortunately, the router utility setup is completely in Japanese, which makes any change a little bit more difficult (but Google Translator does the job).

{% img center http://1.bp.blogspot.com/-GBxvqRC1NkA/T8oaCzDNi3I/AAAAAAAAN7s/LU1RJn1_BkY/s320/logitec_homepage.JPG %}

You can access the web setup using admin/admin

I have to say that this gadget is not meant to be a .top performer.. Actually, I had a few issues using Skype video-call, so keep that in mind before buying it. It is very cheap, and I would say that it is enough for web surfing and connecting your smartphone, tablet, microwave (-: when you are traveling, but not for being used on a regular basis.

Conclusions: PROS: Cheap, really portable, works with a USB (5V), 4 10/100 ports, and oh yes, cheap! CONS: Configuration utility in Japanese, average performance, not easy to buy outside Japan.

Sayonara!