Archive for April 2009

Percona Performance Conference EMT Presentation Slides

I sat down about 20 minutes ago to write a blog post that included a link to the slides of my EMT presentation. It turned into a long post about the presentation, how I feel EMT was received and my feelings on presentations in general. Here is the short post and the link to the slides.

The MySQL conference always inspires me to write so expect a longer post in a few days.

MySQL Brings the Heat

This week throngs of MySQL developers, users, and enthusiasts descended on silicon valley. Apparently the valley’s cooling system can’t keep up because as they arrived the outside temperature went up into the 90s (32s for those of you who choose to use a sane temperature measurement system). I’m not attending the conference this year but I almost wish I was to get some of the air conditioning. It’s supposed to cool off and rain on Saturday. As the brain power leaves I suspect the valley is going to cool and moisture in the air will condense into rain, or tears.

I bet you thought this post was going to be about Oracle and Sun, sorry. I think the weather is more interesting.

Longest beta ever, myisamchk –parallel-recover

I was reading through the manual and noticed that myisamchk parallel recover option is still listed as beta code. The feature was added in 4.0.2 which was released in july 2002. This means it’s been in beta longer than gmail :)

Where did 5.0.79 enterprise come from?

While updating the mirror last week I was surprised to see that the newest MRU MySQL release is numbered 5.0.79. Previously enterprise releases had even numbers and community releases had odd numbers. I posted the question in #mysql-dev and HarrisonF was kind enough to explain it all.

MySQL 5.0 is running out of version numbers. There are limitations in mysql_get_version(), the executable comment syntax, and other places that mean MySQL can only have two digit release version numbers. MySQL Enterprise has started using odd and even version numbers to extend the life of 5.0.

This raises a few questions. What will happen to 5.0 when it runs out of release numbers? Is community going to be sacrificed to give enterprise more versions to use? Are the version restrictions going to be fixed in the future? For example if a feature is implemented in a community release the executable comment version syntax isn’t suitable for preventing it from being executed in a newer enterprise release because the version scheme doesn’t differentiate between enterprise and community.

I think this is now rock solid proof that there were too many features packed into 5.0 and it was released too early. I hope there will be more major releases in the future with fewer features so these problems are prevented. By the way the advanced vs pro enterprise binaries add a whole new layer to the MySQL version issues.

WTF is EMT?

EMT provides an easy way to gather common system performance metrics, as well as providing a simple plugin-based interface to collect custom application-specific metrics. This data can be viewed on the servers that are collecting it or, through the output handler interface, be sent to centralized servers.

I started building EMT because it was very difficult to do ad hoc comparisons of performance metrics when trying to diagnose why systems are or were overloaded. Often times the existing monitoring was only setup to gather data from the operating system such as cpu or network usage. The solution was always to build something quickly by hand, run it for a while, and hope it gathers something meaningful. That’s OK for solving one problem but what about the next time?

EMT runs a set of very light weight scripts out of cron that in turn execute commands and parse data from them. The data can be stored locally (highly recommended) or shipped off to be aggregated on central servers, or both. The power of EMT is really in the local storage. The emt_view command can be used to compare any or all metrics from the system to look for patterns. Example usage can be found in the manual. I highly recommend checking it out.

The installation instructions say to grab the source from svn and use the included script to build an rpm. That’s still the recommended way but I’ve created a snapshot of the source and starter rpm for this blog entry. In the future I’ll setup a more defined release process.

Now for the bad news. EMT is still under heavy development. The view code and how instances work are is going to change a lot before I release it. Things like viewing data from different instances of running programs doesn’t work. That being said it shouldn’t destroy your servers or steal your children if you decide to run it in production. Proven Scaling and a few of our clients have been using it for quite some time with very few issues. If there is enough feedback I’ll bump the version to 0.3 and start stabilizing it for a real release.

If you want to become involved with the project there is a google code page and a google group for discussion. I’ll post development updates to the group page. If you find any issues please report them on the google code issue page.