More learning

I like Mercurial. I'm trying to use it more, to see how it can improve my workflow (and trying to resist the complexity of git for the moment). I'm also trying to do a lot with Scheme, thus scratching my itch of learning about language design generally and a Lisp in a more in-depth fashion.

UPDATE: I've actually restricted access to the bitbucket repository at one of the author's request. Leave a comment here if you want access and I'll see about allowing access on a per-person basis. Allowing access will at the very least depend on good verification of the requestor and posting to the group to allow any instructors to veto access.

Socially responsible internet citizenship

I may not yet be knowledgeable enough to contribute editorially, but I can contribute to Wikipedia financially. Just donated.



Update: Not sure how cool a URI this is - not sure they've read Bill de hÓra's post on paging.

mysociety.org - How great!

I've previously used WriteToThem.com to communicate my concerns on various government / EU policies. I didn't realise that the people behind that site also had a host of other initiatives. mysociety.org is a fabulous concept that shows what can be done by moderately organised people with a little bit of technology.

Enforced SVN upgrade

I migrated from Eclipse Europa (3.3) to Eclipse Ganymede (3.4), re-installed the necessary plugins and noticed an issue with using svn on the CLI.


$ svn up
svn: This client is too old to work with working copy '.'; please get a newer Subversion client

$ svn --version
svn, version 1.4.6 (r28521)
compiled Mar 11 2008, 08:26:35

Copyright (C) 2000-2007 CollabNet.
Subversion is open source software, see http://subversion.tigris.org/
This product includes software developed by CollabNet (http://www.Collab.Net/).

The following repository access (RA) modules are available:

* ra_dav : Module for accessing a repository via WebDAV (DeltaV) protocol.
- handles 'http' scheme
- handles 'https' scheme
* ra_svn : Module for accessing a repository using the svn network protocol.
- handles 'svn' scheme
* ra_local : Module for accessing a repository on local disk.
- handles 'file' scheme


Apparently, Subclipse had upgraded my working copies and my CLI client was too old to cope with this. The options appeared to be either upgrade to Ibex, build svn locally or use hardy-backports. Backports seemed like the best option and has a nice option to restrict what I want to install.

$ cat /etc/apt/preference
Package: *
Pin: release a=hardy-backports
Pin-Priority: 400

$ sudo aptitude install subversion=1.5.1dfsg1-1ubuntu2~hardy2
Reading package lists... Done
Building dependency tree
Reading state information... Done
Reading extended state information
Initialising package states... Done
Building tag database... Done
The following packages are BROKEN:
subversion
1 packages upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Need to get 1274kB of archives. After unpacking 713kB will be used.
The following packages have unmet dependencies:
subversion: Depends: libsvn1 (= 1.5.1dfsg1-1ubuntu2~hardy2) but 1.4.6dfsg1-2ubuntu1 is installed.
Resolving dependencies...
The following actions will resolve these dependencies:

Upgrade the following packages:
libsvn1 [1.4.6dfsg1-2ubuntu1 (hardy, now) -> 1.5.1dfsg1-1ubuntu2~hardy2 (hardy-backports)]

Score is 20

Accept this solution? [Y/n/q/?] Y
The following packages will be upgraded:
libsvn1 subversion
2 packages upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Need to get 1995kB of archives. After unpacking 971kB will be used.
Do you want to continue? [Y/n/?] Y
Writing extended state information... Done
Get:1 http://gb.archive.ubuntu.com hardy-backports/main subversion 1.5.1dfsg1-1ubuntu2~hardy2 [1274kB]
Get:2 http://gb.archive.ubuntu.com hardy-backports/main libsvn1 1.5.1dfsg1-1ubuntu2~hardy2 [721kB]
Fetched 1995kB in 27s (72.2kB/s)
(Reading database ... 169200 files and directories currently installed.)
Preparing to replace subversion 1.4.6dfsg1-2ubuntu1 (using .../subversion_1.5.1dfsg1-1ubuntu2~hardy2_i386.deb) ...
Unpacking replacement subversion ...
Preparing to replace libsvn1 1.4.6dfsg1-2ubuntu1 (using .../libsvn1_1.5.1dfsg1-1ubuntu2~hardy2_i386.deb) ...
Unpacking replacement libsvn1 ...
Setting up libsvn1 (1.5.1dfsg1-1ubuntu2~hardy2) ...

Setting up subversion (1.5.1dfsg1-1ubuntu2~hardy2) ...
Installing new version of config file /etc/bash_completion.d/subversion ...
Installing new version of config file /etc/subversion/config ...
Installing new version of config file /etc/subversion/servers ...
Processing triggers for libc6 ...
ldconfig deferred processing now taking place
Reading package lists... Done
Building dependency tree
Reading state information... Done
Reading extended state information
Initialising package states... Done
Building tag database... Done


Subversion upgrade - DONE!

Ubuntu Hardy issues

Just a place holder for stuff as I hit it.

1. Hibernate / Resume has stopped working. Damn, now it takes me 10 minutes to start up and shutdown at the end of the day. That's time my employer is losing. With Gutsy, Sleep was great. Just shut the lid and leg it out at the end of the day to beat the traffic and pick up the kids. Now I have to stop writing code early. Tracking this, but I'm don't have an idea of when I might get the old functionality back.
2. Thunderbird with Lightning 0.8 stopped understanding iCal, VCALENDAR and all remote and local Calendar data disappeared. I eventually narrowed it down to this. With libstdc++5 installed locally, a reinstall of the lightning 0.8 plugin brought all my calendars back. Might be moving more of them into the cloud from now on though.
3. IO seems a slower, or the disk is getting spun quite a lot more. That's just a subjective impression though, and I don't have any Bonnie(++) numbers for my laptop before the upgrade.
4. Network Manager seems to have a hard time with roaming now. At work, I'm on a LAN all the time. At home, I'm on my LAN or WiFi. Switching at home seems a lot more troublesome that it used to be, and /etc/init.d/dbus restart as my usual brute force approach seems to stop the NM applet from getting put back into the System Tray. No bug report for that yet, since that sounds a little woolly!

Java Triple DES example

I had to do this recently, and the examples that I found through Google seemed a little lacking. So a pointer to others:







UPDATE

Changed to use a gist on github. That will probably dilute the search ranking of this post (which is reasonably popular) but gives me nicer formatting!

Environmental influences

Thinking back to leaving Yorkshire also popped into my head how we learn things. I left Yorkshire just before I was 10. When I left, I said "Ow" when I fell over. Within a couple of months of living in Warwickshire, I said "Aie-Yah". WTF? Call it mirroring, blending, whatever. I had changed my behaviour from what was previously defined to suit my new environment.

We've recently had cause to consider how the kids are being raised, after being exposed to some new ideas and rejecting them. That's not what I want to directly talk about now though. It's more how things are learned. We think Connor has a very high tolerance to pain. Obviously, this isn't a boundary that most people want to find the limits of, so I say that based on watching him when he was smaller, and how he'd happily fall over when learning to walk, and had no fear. When I say fall, face-plant would be more accurate. He never seemed to learn to put his hands out, because nothing hurt him, therefore he never used to cry. But he's learned. Callum is much more vocal when he's upset, even if it's just a flesh wound. I think Connor has learned to cry from Callum even though he's not hurt, or in pain to the level where it causes him discomfort. But if he falls over, he's learned that he should cry. Interesting.

Crystal tones

Al's been pointing out a few occasions when I lapse into Northern; e.g. "Look at them cars, boys!". Not the greatest grammar in the world, and the kids have started to pick up on it. It reminded me of my own childhood, when my folks used to find mine and my brother's habit of dropping our H's quite painful. I often wonder if a contributing factor to leaving Yorkshire was to correct our speaking before it became an insurmountable problem. "'Ow do Fred, what's tha 'aving fer t' tea?". Whereas personally I find it problematic that our kids are currently growing up saying 'glarse' rather than glas. Oh well, plus ça change, plus c'est la même chose.

Broken ETags in the Wild

I've had a good teacher. Just noting some of the things I've come across.

Default Apache httpd configuration in a cluster with multiple filesystems

By default, Apache httpd generates an ETag of the format inode-filesize-timestamp. A symptom of this sort of physical architecture is seeing ETags for the same resource and representation that have the same last part; e.g. "518854-3504d-ce290380" and "c8578-3504d-ce290380", and probably the same Last-Modified value too. This can be fixed by changing the Apache configuration or ignoring the ETag and just using Last-Modified. I would normally recommend retaining the ETag and just using the MTime and Size parts to calculate the ETag value.

EScenic CMS

I'm not sure whether this is an application developer issue, or a problem with the EScenic server itself, but the ETag values that I've seen from this server aren't quoted.

Etag: 20080523124147BST-39-6

The ETag value should be a quoted string

Django on Google App Engine 1.0

The version of Django 0.96.1 that shipped with Google App Engine had a similar problem to the EScenic Server, so I'd recommend bundling Django with your app until Google update the bundled version. I went to fix this in Django trunk, only to find someone beat me to it!

Cameron

Something happened earlier this year and I'm only now comfortable enough to able to talk about it. It happened on 22nd June. At the time, it was terrifying, but it has a good ending.

Al had gone away for the weekend and I was at home playing single parent. Sunday morning, not quite woken from my reverie by Cameron crying softly, I got up, changed his nappy then put him on the floor in his room to play. Connor started calling to get me to come help him on the toilet. I left Cameron's room, pulled the stairgate shut and went to the bathroom to help Connor, rubbing the sleep from my eyes. About 10 seconds later I heard the worst sound that I've ever heard. A soft thudding as the boy went down the stairs, followed by immediate screaming. Fuck. I raced down to find a scene from a Tarantino film. I don't know whether the detachment comes from my (long expired) medical training, or that's just what people naturally do in these situations. I noted the pools of blood on the Mediterranean-tiled floor. Then the bump on his forehead where the blood was coming from, as I cradled him. "That's a strange looking bump, more of an indentation. Oh, that's his skull." Reaching for a muslin to try to apply pressure to the wound. It was too wide really anyway to allow effective compression, so I tried to pack the wound while talking to the emergency services operator. It was very hard to hear them with my own state and Cameron either in pain or understandably upset. Got the other two to dress themselves while the ambulance came and I got some clothes on and mopped up the blood. A big part of me even at that point was worrying Al'll kill me if she sees this, and I didn't want his brothers to see all the blood.

Nee-nah

The ambulance came and we all piled in. Callum and Connor were thrilled to be in the van with the sirens going off. I was probably white as and trying to speak calmly to Cameron as they strapped me on the bed and we moved swiftly to Frimley Park. The ambulance crew were great and we got there, Cameron having thrown up on the journey and coming close to closing his eyes a few times.

A&E was very quiet. Lots of fabulous nursing staff were there, and able to take some of the load in terms of getting out toys for the other two boys and Cameron showed an interest as well. Then I could phone Al at her hotel on the hen-weekend and relay the bad news. She took it as well as could be expected and we had lots of crying phone calls as she was driven from Bath to get to the hospital. Due to the lack of urgency in how we were being treated, I was getting more confident at this point that it wasn't life-threatening. By this time, it was the boy's nap time and the nursing staff were happy enough that he could go to sleep. Al arrived maybe 15 minutes after he went to sleep and came into the ward to see him sleeping with his eyes open, head lolling back in my arms and a cut wider maybe a centimetre wide and 6 centimetres long on his forehead, with the bone showing through. It looked ghastly and we were both crying again. We had some fraught communication with the Reg / SHO about where the best place for treatment was. Doubtless that wasn't helped by our frame of mind, and the language issue with the guy. But we were immensely happy with the level of treatment and service that the emergency services provide. You only really miss something when you don't have it anymore; I don't ever want to experience that loss with our primary medical care.

Support

Due to the nature of the head injury, Cameron was kept in overnight for observation. Our friends Alan and Alison provided invaluable help in taking the two older boys with them to a rowing regatta where they got spoiled on BBQ and ice-creams until I went to pick them up later. As an indicator of where my head was, I picked them up and promptly drove the wrong way around a roundabout. Fortunately the guy coming the other way was going fairly slowly, then my brain kicked in and I recognised where I was. Al phoned later with the results of the CT scan (she's very persuasive!).

Cameron and Al were transferred to St George's Tooting to go to the Maxillo Facial surgical unit, where Dr Singh did an amazing job of closing him up. We got to take him home the same day.

Retro

We examined the stairgate. It is a retractable one, which clips onto the opposing wall at the top and the bottom. The top clip was on a block screwed into the wall. I had placed the block 4 years earlier, which had then been removed and replaced by the painter, and endured some abuse from the other two swinging on it. We saw that it was fairly easy to just clip the gate on the top alone, without clipping the attachment on the skirting board as well. When set like this, effectively it acted like a catflap and Cameron would have had hardly any resistance to crawling against it. Not ensuring that this was properly attached was completely my fault. We were so lucky with how this turned out; from the blood pattern it looked like he'd surfed on his stomach down the first 5 steps and hit the skirting board with his head. From there, he'd rolled down the rest of the stairs to the tiled floor below, but somehow not done further damage to himself when he got to the tiles. Then the first order was shooting some video so that both sets of grandparents could see the extent of the damage and be assuaged that they didn't need to travel down to see him straight away.

So lucky, and so blessed.

Facebook Mega-Data

Just noticed that Facebook open-sourced a version of their BigTable data store and it's hosted on Google code, written in Java. I'll omit the obvious snickers about Facebook cloning BigTable and then hosting on Google Code.

FogCreek Copilot - impressed

I was hoping that now I don't have a Windows machine any more that I could wriggle out of IT support duties for the family. Well, Copilot works under Wine on Ubuntu 7.10. Arse. I was wondering about talking my folks through installing a VNC server on their machine, but just tried the Copilot quick free demo, and it all worked. And it's free on the weekend. So I guess that's my IT support position.

How to dismantle an Atomic Bomb (otherwise known as de-stressing by migrating from Vista to Ubuntu)

Finally it's happened! I've been able to get off Windows Vista and install Gutsy Gibbon on my work laptop. Herein follows a few notes on things that I noticed, which may help others take the leap. This will probably get updated as I notice things that I've missed, or that went well.

1. Cygwin. I can't say enough good things about that project. It's been great to have on Windows and ultimately played a very important part in getting stuff off my laptop before wiping the hard-drive. I've found the people involved very helpful and responsive. My initial attempts to rsync data to a safe location failed. If you're using Windows, go fuck yourself, jwz, 2007. Contacting the mailing list led me to try the snapshot versions, report issues and see them fixed very quickly. Thanks Corinna and others.


2. Firefox. My profile was split over ${HOME}/AppData/Local/Mozilla/Firefox and ${HOME}/AppData/Roaming/Mozilla/Firefox. I consolidated the two by copying to ~/.mozilla/firefox. The profiles.ini file needed editing, since it has an incorrect relative path to the profile folder. I also needed to rename extensions.rdf, so that a new one is built on startup. This preserved all of my extensions (apart from Google Toolbar; there is a Linux-specific version?) and associated data such as ModifyHeaders values, UserAgentSwitcher values, etc. I have needed to manually edit some preferences, such as Download location, and then restart Firefox for it to take effect, but otherwise it all seems to be working fine. Similarly for Thunderbird, although I should have exported my Lightning Calendars first - I seem to have lost them, but that's no biggie. I still have all of the emails (MBOX!) (although my tags seem to have disappeared. That's a bit of a pisser). Maybe I should have migrated that to GMail, rather than copying mboxes around, but it's worked.


3. SSH. I made sure that I copied my keys over and that they all worked. I seem to have missed the full known_hosts file; I have a copy, but some entries are missing, which is slightly annoying. Also, I forgot my System32/drives/etc/hosts changes, so I'll need to recreate the local aliases that I have for some servers. Sure I can remember that, so not too painful an omission.


4. sudo aptitude install tofrodos

Ubuntu just flies, versus the same hardware running Vista. That's a shocker, obviously ;-).

Update:

Eclipse was using a lot of file handles (I just install the world in terms of the number of plugins I have), which required an addition to /etc/security/limits.conf.

# raise limits due to Eclipse complaining about too many open files
* soft nofile 5120
* hard nofile 5120


Shell Meme for migrating off Vista

Backing up required files to my home machine first, prior to wiping Vista off the laptop this evening...


jabley@python:~$ history | awk '{a[$2]++ } END{for(i in a){print a[i] " " i}}' | sort -rn | head
89 ll
70 sudo
61 cd
41 exit
38 aptitude
18 svn
15 rsync
10 mkdir
10 cp
6 which


Mouse in your hair is nothing

Just finished Russell Brand's Booky Wook, which is what the title references. Interesting in terms of the honest coverage, but it did feel a little rushed (zeitgeist is everything, darling). At one point he talks about when a mouse lived in his hair (I think it was referenced in the Jonathan Ross interview as well). We can top that. Callum has a nit comb from school; apparently part of the national curriculum to inform hygiene issues. Connor was using it on Al the other night and kept lifting her hair.

Al

Connor, what are you doing?

Connor

Looking for bees, Mummy.

Al

Not nits, Connor?

Connor

No Mummy, BEES!

Minor casualties from the disk crash

There was some loss - mainly source code for personal hacking. My Jython UnicodeData implementation was lost, although I did have a copy of that in my GMail Sent folder which has been contributed to the committers in the hope that someone with more time and enthusiasm than me finds it useful. Apparently they have. The other thing lost was my WideFinder sketching. I surprisingly did have an early warning that the disk was close to failure, when my implementation (which was clocking ten seconds or something) suddenly started taking twenty minutes to complete. Obviously I didn't pick up on this warning; instead blaming it on some horrible mistake in my code. Some learnings from my WideFinder experience.

• NIO can be tricky, and should probably have an easier API in comparison to classical IO. It was nice to look at this anyway, and I'll probably have a read of the Jython and JRuby IO stuff to see how others do it, and also the Apache NIO stuff.
• Java string matching is slow using regexp. I'd not got as far as implementing Boyer-Moore or similar, but I expected to get a big hike out of my 10 second time when switching to that.
• CAS is good, versus synchronized blocks. Brian Goetz has talked about the advantages of non-blocking forms elsewhere.
• Java is quite restrictive. I must have been doing too much Common Lisp, Erlang and Scheme recently!

MTBF

(File under this will happen to you)

Former colleagues may recall how smug I once had occasion to be, when I migrated my old home machine from dual boot with Linux and W2K to solely Ubuntu. My preparation for that was very informed by this, but it was still slightly nerve-wracking to delete everything and start from scratch. After that, my backups got a little slack. A single 40GB IBM Deskstar with content being intermittently copied over to my external USB drive, with very infrequent backups to DVD. Then the IBM drive crashed. Arse. Oh well, still got the USB drive. Then the USB drive crashed two days later. Oh shit, that's five years of my life in pictures and film gone. The first photos of all the boys...
I was feeling physically sick. A few days later and I have a better appreciation for what I've lost. Personal stuff - nothing. The USB drive started working again after leaving it on for half an hour. Everything was taken off that and put on my work laptop, and I still had photos and video on the cameraflash disk and digital tape respectively. Thank fuck for that.

Rebuilt machine - two Maxtor 360GB PATA drives (hey, my motherboard is seven years old!). My motherboard aslo has RAID 1 capabilities, but didn't appear to work too well when I installed Gutsy. It might be possible to do it, but I didn't have the patience and was worried about what happens the next time I upgrade the kernel. So I went for the software RAID option instead. I'm now doing the backups more frequently along with using S3 for anything that would give me serious pain to lose.

So just adding to the general noise around this issue. Look after your data, people!

Clever!!!

Cooking pancakes with the boys today, and Connor was getting a bit impatient waiting for a pancake to cool.

Connor

Can I have it?

Me

No Connor, it's too hot.

Cue the boy going into the cooking draw to get the oven gloves to pick up his pancake. Clever!