JGuru
When you need a guru
  • Home
  • About
  • Services
  • Blog
  • Projects
    • Liferay Backup
    • Photo Carousel Portlet
    • Social Comments Portlet
    • Liferay Yubikey OTP Login
  • Contact
October 17 2013

Monitoring and Graphing Liferay with MRTG

Mika Koivisto Howto jmx, jmx4perl, jolokia, liferay, monitoring, mrtg 0

MRTG (The Multi Router Traffic Grapher) is usually used to monitor SNMP enabled network devices and draw graphs of how much traffic has passed through each interface. It can also be used to graph any two values (in/out) and I use it for graphing cpu usage, loadavg, iowait, used memory, disk space and temperature sensor values that I can read through SNMP. Liferay however doesn’t support SNMP so I developed a perl script that can read JMX MBean values using JMX4Perl and Jolokia. I’m going to assume you have JMX4Perl and Jolokia setup already the way I describe it in my earlier post: Monitoring Liferay with Nagios, Jolokia and JMX4Perl. You should also note that MRTG won’t send you any alerts so it’s a good idea to setup Nagios to do just that.

Now you might wonder why would you need MRTG if you already have Nagios. Nagios operates on the present value although there’s a add-on nagios grapher that can create graphs like MRTG does but I like MRTG more because you can see all the graphs on one page. Being able to see a full overview of the system is very important when trying to identify performance problems. This is also why you want to get more information out of the application, Liferay in this case. My script will help in reading connection pool and thread pool utilization as well as heap usage. Those are essential when doing troubleshooting.

First you’ll need to install and setup MRTG. I’m not going to go into details on that because it dependes on your system and the internet is full of guides to do it. Once you have it done you’ll need to download my mrtg-jmx4perl.pl script which is available in my github repository. For rest of this post I’m going to assume it’s located in /usr/local/bin/mrtg-jmx4perl.pl but it’s up to you where you put it. Just adjust the script path accordingly.

Monitoring c3p0 connection pool

Getting the values for c3p0 is a little bit tricky because it will generate a unique mbean name based on the identity token it generates for the connection pool every time the server is started. Because of this my script assumes you only have one c3p0 connection pool if you have multiple pool you’ll need to add additional logic in to the mrtg-jmx4perl to find the correct mbean. Notice that this is the case if you configure Liferay to use connection pool from portal-ext.properties instead of using a JNDI resource. We can read the mbean for c3p0 by using mbean name as “com.mchange.v2.c3p0:type=*,*” and the attributes we are most interested are numConnectionsAllUsers and numBusyConnectionsAllUsers. Below is a sample mrtg configuration snippet.

Target[dbpool]: `/usr/local/bin/mrtg-jmx4perl.pl --server=servername --mbean="com.mchange.v2.c3p0:type=*,*" --attribute="numConnectionsAllUsers numBusyConnectionsAllUsers"`
MaxBytes[dbpool]: 20
Title[dbpool]: DB Pool
PageTop[dbpool]: <h1>DB Pool</h1>
WithPeak[dbpool]: dwmy
Unscaled[dbpool]: dwmy
Options[dbpool]: growright,unknaszero,nopercent,gauge
YLegend[dbpool]: Connections
ShortLegend[dbpool]: 
LegendI[dbpool]: Connections
LegendO[dbpool]: Busy Connections
Legend1[dbpool]: Connections
Legend2[dbpool]: Busy Connections
Legend3[dbpool]: Peak Connections
Legend4[dbpool]: Peak Busy Connections

Here’s a daily graph from one of my Liferay portal servers.

mrtg - db pool connections

Monitoring Tomcat AJP Thread Pool

This one is pretty easy because the the mbean name is static but it does vary depending on do tomcat version and connector you are using. In Tomcat 7 with native library the name for ajp thread pool is Catalina:type=ThreadPool,name=”ajp-apr-8009″. Without native library it would be ajp-bio-8009. In tomcat 6 my ajp pool mbean name is Catalina:type=ThreadPool,name=jk-8009. Notice the lack of double quotes in the name. You can easily check the name using jconsole. So for this one the config looks like:

Target[ajp-threadpool]: `/usr/local/bin/mrtg-jmx4perl.pl --server=servername --mbean="Catalina:type=ThreadPool,name=\"ajp-apr-8009\"" --attribute="currentThreadCount currentThreadsBusy"`
MaxBytes[ajp-threadpool]: 50
Title[ajp-threadpool]: AJP Thread Pool
PageTop[ajp-threadpool]: <h1>AJP Thread Pool</h1>
WithPeak[ajp-threadpool]: dwmy
#Unscaled[ajp-threadpool]: dwmy
Options[ajp-threadpool]:  growright,unknaszero,nopercent,gauge
YLegend[ajp-threadpool]: Threads
ShortLegend[ajp-threadpool]: 
LegendI[ajp-threadpool]: Threads
LegendO[ajp-threadpool]: Busy Threads
Legend1[ajp-threadpool]: Threads
Legend2[ajp-threadpool]: Busy Threads
Legend3[ajp-threadpool]: Peak Threads
Legend4[ajp-threadpool]: Peak Busy Threads

Here’s a daily graph of a thread pool.

mrtg - thread pool

Monitoring Heap Usage

The last one we are going to monitor is Java Heap usage. It can be read from java.lang:type=Memory using attribute HeapMemoryUsage and path used. Now we are reading only one value.

Target[heap]: `/usr/local/bin/mrtg-jmx4perl.pl --server=servername --mbean="java.lang:type=Memory" --attribute="HeapMemoryUsage" --path="used"`
MaxBytes[heap]: 1296302080
Title[heap]: Heap
PageTop[heap]: <h1>Heap</h1>
WithPeak[heap]: dwmy
Unscaled[heap]: dwmy
Options[heap]:  growright,unknaszero,nopercent,gauge,noo
YLegend[heap]: bytes
ShortLegend[heap]: 
kilo[heap]: 1024
LegendI[heap]: Used
Legend1[heap]: Used
Legend3[heap]: Peak Used

Here’s a daily graph of heap memory usage.

mrtg - heap usage

You can download the full sample-mrtg.cfg from github.

That’s how easy it is to monitor and graph Liferay or pretty much any Java webapp using MRTG. You could easily use this to monitor ehcache utilization or anything else that’s accessible via JMX.

Share this:

  • Tweet
  • Share on Tumblr
Liferay Maven Support in Liferay 6.1 GA3 New Site New Focus

Related Posts

cyber_security

Howto

How to get Tomcat to see HTTPS when it’s terminated elsewhere

nginx

Howto

Creating a custom Nginx build for Ubuntu/Debian

getting-git-right

Howto

Moving a project from one git repository to another while retaining it’s history

Tags

activemq apache backup c3p0 certificate connection pool debian debugging deployment git heap hook httpd https installer java jms jmx jmx4perl jndi jolokia jvm liferay mariadb maven memory monitoring mrtg mysql nagios newrelic nginx ngx_cache_purge pagespeed performance repository salesforce saml security spring ssl struts tls tomcat ubuntu

Categories

  • General
  • Howto
  • Liferay

Recent Posts

  • cyber_securityHow to get Tomcat to see HTTPS when it’s terminated elsewhere
  • failTop 3 reasons why Liferay projects fail
  • FIDO-Yubico-YubiKeys-2-444x296Liferay Yubikey OTP Login
  • nginxCreating a custom Nginx build for Ubuntu/Debian
  • getting-git-rightMoving a project from one git repository to another while retaining it’s history

Archives

  • October 2017
  • February 2017
  • January 2017
  • July 2016
  • June 2016
  • May 2016
  • August 2015
  • April 2015
  • January 2015
  • October 2013
  • June 2013
  • August 2012
  • July 2012
  • June 2012
  • May 2012
  • March 2012
  • February 2012
  • January 2012
  • January 2011
  • September 2010
  • August 2010
  • December 2009
  • October 2008
  • December 2007

Recent Posts

  • cyber_securityHow to get Tomcat to see HTTPS when it’s terminated elsewhere
  • failTop 3 reasons why Liferay projects fail
  • FIDO-Yubico-YubiKeys-2-444x296Liferay Yubikey OTP Login
  • nginxCreating a custom Nginx build for Ubuntu/Debian
  • getting-git-rightMoving a project from one git repository to another while retaining it’s history

Follow me on Twitter

My Tweets
© JGuru 2006 - 2017