ScaleCamp - State of the Nation for Monitoring

Where are we now and what's broken with it?

• Cacti


• Collectd


• Munin


• Ganglia


• Hyperic


• Zabbix


• Zenoss

People are almost getting to the point of needing more powerful machines to do the monitoring than the app servers! Maybe something's broken somewhere...

RRDTool - overall, the consensus seemed to be that this was a little dated.

1. Does lots of writes due to the way it stores data


2. throws away data by the way it aggregates - to see fine-grained data of last years sales, you need to keep a backup of the files / graphs, rather than being able to query it.


3. can't be cleansed of bad data, or it's a bitch of a job to do so.

Alternative options

• hbase


• reconnoiter


• Tokyo Tyrant / Cabinet


• timesplicedb looks to be an interesting attempt to provide a replacement. More language bindings needed, don't be shy!

A good start to the conference for me and it gave me a flavour of the depth and breadth of discussions available.

ScaleCamp UK 2009

On Friday I was fortunate enough to attend the inaugural ScaleCamp UK event, organised by Michael Brunton-Spall at the Guardian. This was a great conference. It was a BarCamp-style approach (not that I've been to BarCamp yet!) with the schedule evolving over conversations and planned on a board in the morning. Some of the sessions I took notes at; others were standing room only, so I'll try to remember what was talked about. Obviously, this is a personal perspective focused on my interests; others should be blogging about Javascript and the like.

I met lots of very passionate, smart people doing cool stuff. That bodes well for the economy; if you want to do interesting work, then hooking up with any of the people that attended there wouldn't be a bad place to start.

Maturity

Got home from Font around midnight on Saturday night (Halloween) to see a single carved pumpkin at home with a light inside. In the morning it transpired that someone that evening had taken Connor's pumpkin and smashed it just around the corner. I was taken with his reaction - very laissez-faire and musing on how he had the enjoyment from making the pumpkin. He was just a bit disappointed that I didn't get to see it. He's almost human sometimes!

Font 2009 summary

This was a bonus trip. I had a bad back for a month beforehand thanks to a Connor Kung-Fu Panda drop on me when stretching after a run. So no climbing and whacked out on painkillers - not ideal before a bouldering trip.

Traditional Friday night / Saturday morning driving ferry and drive only to find wet Font greeting us.

Sunday

Apremont

Monday

Cusiniere

Tuesday

Rocher Canon - running

Wednesday

Haute Pleines / Isatis

Thursday

Cuvier - rest day

Friday

95.2

Saturday

Gorge aux Chats

Joined the inveterate tickers club - that and I can't really remember what I've done in Font apart from Carnage, l'Abbatoir and a few others.

Great trip; too hot for getting on some stuff, but rather that that raining all week. Lucky to get stuff done too, with my lack of preparation. Not done 7b in the forest for a few years!

Google Gears for Firefox 3.5 on Ubuntu Karmic 9.10

Built my own - seems to work fine so far.

jabley@miq-jabley:~/work/gears-read-only$ svn info
Path: .
URL: http://gears.googlecode.com/svn/trunk
Repository Root: http://gears.googlecode.com/svn
Repository UUID: fe895e04-df30-0410-9975-d76d301b4276
Revision: 3410
Node Kind: directory
Schedule: normal
Last Changed Author: gears.daemon
Last Changed Rev: 3410
Last Changed Date: 2009-11-10 01:49:08 +0000 (Tue, 10 Nov 2009)

Made some changes:



make mode=OPT

and then install the resulting xpi.

updating git svn local copies

Post-release in maven, I have a lot of svn modules to update:

Settig up Git mirrors of SVN

At work, I've been using git-svn for quite a while. I like the workflow options, and better merging capabilities. As a casualty from the recent laptop hard drive failure, all of my git repositories had gone, checked out from our main SVN server. I'm the only one using git where I work, but I love the workflow that it gives me and going back to SVN is a no-no. That had previously been created by doing a

git svn clone -s svn://svn.example.com/module

That took ages (3 days for all of the stuff I need to work on) and was quite slow when doing commits. My backups are mildly corrupt too, so I've started over, and set it up properly this time. Thanks to the guide here.

On the server, I created a directory to hold the git mirrors, and a text file containing the SVN modules that I wanted initially. Then a simple bash script to loop through the file and create a mirror of each SVN module:

for f in `cat svn-modules.txt` ; do svn2git.sh $f ; done



Then just make the repositories available:

git-daemon --export-all --base-path=/opt/git --verbose

and create a cron job to refresh the git mirrors periodically.

git --bare svn fetch --all

There are other ways, but that's the quick-n-dirty approach. Then a similar script on the client, which used the same list of modules that I wanted to check out.



Benefits of this approach:

• Much faster to set up - it took just over a couple of hours this time.


• Available to other people to try out - not just me.


• Provides a migration path off SVN as we eventually migrate off SVN (my long-term aim, muhahaha)

Separation of concerns

The shower control described here resonated with me. Having a separate control for temperature and volume appeals to me as a software developer. But most sinks don't work this way. Instead, you have a tap for hot water and one for cold. I've seen a sink which did have controls as per the shower. I liked it; most other people complained and eventually it was replaced with a more typical Western arrangement. I wonder why the idea doesn't transport well from shower to sink, or if there are places where that is the norm?

JRuby on Rails Tomcat logging

We have a mixed dev team - Windows and Linux currently, although I'm considering a Mac.

In your environment:

os = java.lang.System.get_property 'os.name'

config.logger = Logger.new('/var/log/my-company/my-app/rails.log', 5, 104857) if os.downcase =~ /linux/

Maven Woes 2

More time spent fighting with maven. Projects which used hibernate starting failing yesterday. We use XDoclet to generate the hibernate mapping files.

Unable to find resource 'xdoclet-plugins:xdoclet-plugin-qtags:jar:1.0.4-SNAPSHOT' in repository
central (http://repo1.maven.org/maven2)
[INFO] ------------------------------------------------------------------------
[ERROR] BUILD ERROR
[INFO] ------------------------------------------------------------------------
[INFO] Failed to resolve artifact.

Missing:
----------
1) xdoclet-plugins:xdoclet-plugin-qtags:jar:1.0.4-SNAPSHOT
Try downloading the file manually from the project website.

  Then, install it using the command: 
      mvn install:install-file -DgroupId=xdoclet-plugins -DartifactId=xdoclet-plugin-qtags -Dversion=1.0.4-SNAPSHOT -Dpackaging=jar -Dfile=/path/to/file

  Alternatively, if you host your own repository you can deploy the file there: 
      mvn deploy:deploy-file -DgroupId=xdoclet-plugins -DartifactId=xdoclet-plugin-qtags -Dversion=1.0.4-SNAPSHOT -Dpackaging=jar -Dfile=/path/to/file -Durl=[url] -DrepositoryId=[id]

For whatever reason, maven wouldn't use the SNAPSHOT version that I had locally. I couldn't find a 1.0.4 released version on t'Internet, so I just deployed the snapshot that I had in my local repository as the 1.0.4 released version. Naughty me. Had to do the same for the xdoclet-plugin-hibernate plugin as well.

UPDATE: And today it's working with the SNAPSHOT versions again. Still just doing mvn clean install, on Ubuntu, Windows Vista and within Hudson on Red Hat. Silently breaks, and then fixes itself. WTF is that all about?

Maven Woes 1

My mental model of Maven is that there is a small kernel and lots of plugins which provide functionality. On a daily basis, maven will try to update plugins that it uses; e.g. for dependency resolution. You can configure maven to not upgrade certain core plugins, but people don't tend to do this. Perhaps they should...

At the beginning of August this year, I started getting this:

 Unable to find resource 'bouncycastle:bctsp-jdk14:jar:138' in repository
central (http://repo1.maven.org/maven2)
[INFO] ------------------------------------------------------------------------
[ERROR] BUILD ERROR
[INFO] ------------------------------------------------------------------------
[INFO] Failed to resolve artifact.

Missing:
----------
1) bouncycastle:bctsp-jdk14:jar:138

  Try downloading the file manually from the project website.

  Then, install it using the command:
      mvn install:install-file -DgroupId=bouncycastle -DartifactId=bctsp-jdk14 -
Dversion=138 -Dpackaging=jar -Dfile=/path/to/file

  Alternatively, if you host your own repository you can deploy the file there:

      mvn deploy:deploy-file -DgroupId=bouncycastle -DartifactId=bctsp-jdk14 -Dv
ersion=138 -Dpackaging=jar -Dfile=/path/to/file -Durl=[url] -DrepositoryId=[id]

This broke not only all of our trunk builds, but also previously released items. Bouncy Castle jars presumably aren't in the main repositories due to crypto-export issues for some countries. We don't need to ship them. The cause of the problem? We had a dependency on jasperreports. This has an open-ended dependency:

<dependency>
  <groupId>com.lowagie</groupId>
  <artifactId>itext</artifactId>
  <version>[1.02b,)</version>
  <scope>compile</scope>
</dependency>

Version 2.1.7 of com.lowagie itext (released a couple of months ago) introduced a dependency on bouncycastle. Before that time, maven had been resolving the com.lowagie itext version to use version 1.3.1. Presumably a plugin was updated to fix a known bug in open-ended dependencies like the one in jasperreports and it exposed us to this problem. We were OK until the bug was fixed! Our current solution is to explicitly define the com.lowagie itext version as 2.1.5, which doesn't have the bouncycastle dependency. The closer dependency wins over the transitive dependency, yada...

Technology predictions revisited

An email that I sent internally about some of our RSS and Atom feed processing, on 2008-03-04:

"The push aspect is a little annoying. We can do polling quite well now, but push via SFTP is messy. One day people will wake up and do this over Jabber, using something based on Atom, but until then, I guess we need to have a scheduled task that polls the SFTP directory and copies content about. Plus ça change, plus c’est la meme chose..."

That would be PubSubHubbub then.