sorted by: recent | see : popular

Content Tagged with monitoring + MySQL

I was contacted by the folks at MONyog and asked if I would review MONyog. Since using MONyog is something I have been wanting to do for a while, I jumped at the chance. Of course, “jumped” is relative; Rohit asked me at the MySQL User Conference back in April, and here it is two months later, in June. My apologies to folks for being slow.

This review is an overall review of MONyog as well as specifically reviewing the newest features released in the recent beta (Version 2.5 Beta 2). Feature requests are easily delineated with (feature request). This review is quite long, feel free to bookmark it and read it at your leisure. If you have comments please add them, even if it takes a while for you to read this entire article.

While the webyog website gives some information about what MONyog can do, it is a bit vague about what MONyog is, although there is a link to a PDF whitepaper on What is MONyog? which does answer much of these questions.

The screenshots available from the website are accurate, so I will not reproduce them here. I will note that I have not shared this feedback with the webyog team yet, so I may be upset that a feature is lacking, and the feature may be implemented but I missed it. I will post a follow-up in that case, even though they will likely comment here too.

My reference points — I have used other monitoring and graphing tools such as Nagios, Cacti, and Intermapper as well as MySQL’s Enterprise Monitor.

Overall

As an overall review — MONyog is the best out-of-the-box GUI monitoring tool for MySQL that I have seen. It “just works.” As promised, getting up and running quickly is easy, and having a centralized location for monitoring is very useful. The graphs are beautiful and the statistics that are graphed are useful time-savers.
(more…)

I just uploaded the 1.0.0 release of my MySQL templates for Cacti. Now there’s an actual download under the Downloads tab. I solved a number of issues in this release. The changelog:

2008-06-01: version 1.0.0

	* Fixed when SHOW MASTER LOGS has no File_size column.
	* Fixed Cacti-version-specific problems with include files.
	* Fixed when binary log is not enabled.
	* Fixed some caching issues.
	* Fixed make-template.pl issues when downloaded from SVN.
	* Replication graph shows only slave_lag instead of Seconds_behind_master
	* Generate a version for Cacti 0.8.6i.
	* Support generating custom versions with make-template.pl.

Download MySQL Cacti templates

As promised, I’ve created some improved software for monitoring MySQL via Cacti. I began using the de facto MySQL Cacti templates a while ago, but found some things I needed to improve about them. As time passed, I rewrote everything from scratch. The resulting templates are much improved.

You can grab the templates by browsing the source repository on the project’s homepage.

In no particular order, here are some things I improved:

• Standard polling interval and graph size by default.
• Full captions on every graph; you don’t have to guess at how big the values are. Each graph has current, max, and average values printed at the bottom for every value on it.
• Much more data is captured. I’ve graphed almost everything I could think of.
• The graphs are grouped better. Most graphs have only related values. There are some exceptions, but not many.
• The templates don’t hijack your existing installation. They don’t depend on or alter anything in your default Cacti installation.
• The script that gathers the data is totally rewritten from scratch, and much improved. For example, the math works on 32-bit systems. It has caching built-in so each poll cycle results in just one request to the server, instead of one request per graph. (This is a weakness of Cacti I’m trying to work around). It also has debugging aids and other good coding stuff.
• By default, it assumes you have the same username and password across every server you’re monitoring, so you don’t have to fill in a username and password for every single graph you create.
• One data template == one graph template. This helps work around another Cacti limitation.
• Lots more. Honestly I can’t really remember everything I’ve done. I’m sure you’ll help me remember by asking me how to get X feature working the way you want, and I’ll go “oh, yeah, that’s another thing I improved…”

Cacti templates are very laborious to create if they’re complex at all; it takes a long time and is very error-prone. Instead of doing it through Cacti’s web interface and exporting a huge XML file, I eliminated the redundancies and created a small, easy-to-maintain file from which I generate the XML template with a Perl script. This gives the added benefit of letting me (or you) generate templates with different parameters such as polling interval or graph size. The README file has the full details. However, I’ve pre-generated a set of templates that matches Cacti’s defaults, so you can probably just use that.

This has taken a lot of time. In particular, I spent a lot of time working on it at my former employer, The Rimm-Kaufman Group (kudos to them for letting me open-source the work) and I just spent most of my weekend writing the scripts to convert from the compact format to XML templates, so it’s possible to maintain these beasts. Plus I had to develop the compact format, too. This took a lot of time because I had to understand the Cacti data model, which is pretty complex.

Please enter issue reports for bugs, feature requests, etc at the Google project homepage, not in the comments of this blog post. I do not look through comments on my blog when I’m trying to remember what I should be working on for a software project.

If these templates help you and you feel like visiting my Amazon.com wishlist and sending something my way, I’d appreciate it!

PS: You may also be interested in Alexey Kovyrin’s list of templates for monitoring servers.

I just need to get some basics off of my chest here, it’s by no means a full list but it’s the most basic list I can think of to start with, and it’s basic because I am surprised by some of the slop I’ve seen in production environments.

1. Highly available server clusters - this is different than load balancing cluster, if confused see here.

2. Disaster recovery

-> this means daily,weekly,monthly backups as well as off site backups, and tertiary backups as well as a plan to get those backups imported and running in production as fast as possible. Backups should have consistency checking when they are created.

3. Security

-> perimeter on the network, VLAN’d databases from the web/app servers, firewall, ACLs, etc

-> system level: strong passwords on OS and database accounts (no blank passwords - that *should* be obvious but you’d be surprised what I’ve run into), file permissions, encryption of sensitive database information.

4. Monitoring: monitor everything possible. Log files, disk partitions, service ports, service details (traffic for a service, memory used, tuning parameters: query cache usage, etc), CPU/RAM usage, logged in users, and most importantly being alerted about monitored services. If you’re not getting called when something has passed a threshold, you need to pay more attention to the infrastructure.

Kevin Burton wrote recently about why SHOW SLAVE STATUS is really not a good way to monitor how far behind your slave servers are, and how slave network timeouts can mess up the slave lag. I'd like to chime in and say this is exactly why I thought Jeremy Cole's MySQL Heartbeat script was such a natural fit for the MySQL Toolkit. It measures slave lag in a "show me the money" way: it looks for the effects of up-to-date replication, rather than asking the slave how far behind it thinks it is.

The slave doesn't even need to be running. In fact, the tool doesn't use SHOW SLAVE STATUS at all. This has lots of advantages: for example, it tells you how far the slave lags behind the ultimate master, no matter how deep in the replication daisy-chain it is. In other words, unlike SHOW SLAVE STATUS, it won't tell you a slave is up-to-date just because it's caught up to its master. If a slave's master is an hour behind, it will report that the slave is an hour behind, too -- because it is.

It's a really smart approach. And you can daemonize it, and it'll keep a file up-to-date with running averages (by default it averages the last one, five and fifteen minutes, but of course you can choose that). Now your monitoring scripts can be as simple as "cat /var/log/slave-delay" or some such.

It's not a hard tool to write, and I suspect lots of people have done it, but I bet that between Jeremy, whoever worked on it at Six Apart, and me, we've produced a pretty good version of the tool. It's part of the MySQL Toolkit, and the full manual is online.

This article shows you how to use a little-known InnoDB feature to find out what is holding the lock for which an InnoDB transaction is waiting. I then show you how to use an undocumented feature to make this even easier with innotop.

This release is part of the unstable 1.5 branch. Its features will ultimately go into the stable 1.6 branch. You can download it from the innotop-devel package.

The major change is a new Command Summary' mode (switch to this mode with the 'C' key) that's similar to mytop's 'c' mode. It shows you the relative size of variables from SHOW STATUS and SHOW VARIABLES.