What a long gap…


I’m going to write a little more on this blog – admittedly after a 5 year gap its a little difficult to write less!

Anyhow I have a little ranting and a bit of general meandering thought to write down, so here goes.

Pantomime – This year with comedy stage manager on crutches


The start of December for me is Pantomime season – I’m the only one of the stage managers at our theatre who is stupid enough or can actually manage to make the time to handle a 2 week run (1 week build/rehearsal, 1 week performance run) – we had previously tried to do this with multiple stage managers each doing a shorter period with less success.

This year things are slightly different – I’m not back to full fitness, so am running the show in a slightly more executive mode to normal – ie I tell other people to adjust lights rather than me climbing ladders myself.  I’ve only been back at work a couple of weeks, and am having some issues with managing the swelling of my left foot and lower leg, so its probably not the most sensible thing I could be doing – but its for a relatively short period, and is getting back into something approaching normal life.

The other thing that is different this year is that the Rowntree Players have written their own pantomime script.  I very much admire them for that – it has obviously been one hell of a lot of work.  In their favour it saves them the licensing fee that would normally be paid for a script – probably in the order of £500 to £600 – and allows the script to be tailored for what they can do well.

And this script is good.

Its also, from a technical point of view, set so that the changes between scenes can be done very quickly, meaning the audience are not left sitting in the dark whilst shadows move stuff into place on stage – so the whole pace is kept up.

There has also been a lot of work put into the various dance routines – the introductory song has a huge number of things happening in it (very similar to the first part of Disney’s Beauty and the Beast – all the characters pass through during it) and the choreography to Heatwave is also stunning.

So at this point – 4 performances down, 6 to go – I am having a whale of a time, with a great supporting backstage crew and a good performing company.  I may be exhausted (although I’m taking care to pace myself so its not too bad), but this is the sort of thing I do theatre work for!

If you are reading this before the end of the run (12th December), you can get more information on the theatre website, read a preview in the local press (unfortunately at this point they have the wrong photo up on their website) or buy tickets.

You got what???


I’ve written out variations of this story in various media (try using SMS or twitter) several times over the last 2 weeks, so thought it was worth just putting a batch of stuff down and pointing people here to save my fingers…. (and memory).

Please be aware that I am being slightly circumspect about some aspects as potentially there is a legal element involved.

I’ve also just written this straight out.  I may come back and try and fix it up somewhat to improve it – but that lack of concentration I mention further down is starting to bite again!

Short version

I was hit by a car when crossing a road.  My left leg is broken and 4 hours of memory is missing.  I’m getting better but while its going well, its going to be a while.

Full version

I spent the evening of Tuesday 29 September in the theatre I volunteer at stage managing (or in this case simply overseeing my very competent assistant) a production which was having its final pre-dress run.  We finished early (before 21:00) and at this point I lose my sense of time – I should have worked around the building locking up, checked out and left, but I did exactly the same the night before so I have no real idea as to whether my memories are of the previous night or of this night.  The two of us (Stage Manager & Assistant Stage Manager) would have locked the main gates, and set off in opposite directions – me towards the car park after crossing a slightly awkward junction by a roundabout.

The next thing I remember is coming to in A&E – the final stages of a large temporary pot were being applied to my left leg (coming up to my underpants), and the doctor (or more likely Nurse Practitioner) started getting me in a good position to stitch my scalp.  The time was somewhere around 01:00.

Oddly I felt pretty good – not much pain (probably high on all sorts – I guess they would have wanted me reasonably sedated to deal with setting the leg).  I seemed reasonably lucid, although I may not be the best judge of that.

The staff filled me in with what had happened from their point of view – ie they had contacted my wife (I found out later I gave them the wrong number – so I was obviously not unconcious, but maybe not too with it).  I was taken along to X-Ray to check the leg was OK in the new cast – from my point of view I had never seen the technician before, but she had done a full set of X-Rays and a scan on me earlier on in the evening – which all seemed wierd!

So then I was moved to an Orthopeadic ward – being reunited with my posessions (yay – phone still in perfect condition, trousers and boots much less so) on the way.  Thankfully York Hospital has a relatively low key mobile phone policy so I fired off a text to my wife and one to the theatre management team (odd sense of priorities).

The 3 days in hospital are all a bit of a blur – I had surgery on the Thursday (deferred from Wednesday as the surgery list was full) – a tibial nail put in around my knee running the length of my tibia (I have 3 small incisions from the surgery).  There are strange images from those days – the first morning trying to wash off the dried blood and mess, finding that I was supposed to be collecting my daughter from a different theatre (thankfully that had been sorted out).  The ward staff were outstanding people, but it was a confusing, disruptive and noisy (especially at night) place, and I was glad to be able to arrange my discharge for Friday afternoon – as long as I could get through the physio training to use crutches.

So I got home – and it did feel something like a break-out.  The house was rearranged for me – a chair bed moved to the ground floor, although I have actually been OK on stairs with crutches so have slept in a real bed (with no pings, overnight admissions and other interruptions – bliss).  The family have all been really good – I can move round the house OK (slowly, and given notice) but I can’t carry anything so need things to be set for me (ie wash things, drinks, meals etc).

I didn’t see my leg until after surgery (prior to that it was wrapped in the temporary pot).  It didn’t really feel as though it was mine – swollen with the skin tighter than a drum.  The foot is oddly deformed – just due to swelling.  My upper thigh had a glorious red/purple set of mottling and bruising – since faded to a yellow.  There were 3 small surgical incisions under dressings – 2 around knee level and one down by my ankle.  Additionally I had some gravel rash type wounds on the outer back side of the leg.  These have all improved in the subsequent week although there is still significant swelling.

So as for the future, I’m likely to be off work for a couple more weeks and impaired for a couple more months.  I’m carefully doing my physio exercises – which really help, but if you sit down for an hour the leg seizes up and its painful to start walking again.  The type of surgery means I have no cast on and can put full weight on the leg, but I have to take it carefully and slowly.

Mentally there have definitely been some effects.  There is definitely a 4 hour hole in my memory.  My concentration and reading speed is lower than normal, although I think they are improving.  I had one horrid night with strange nightmares and illusions, but other nights have been pretty good.

And as to what happened.  It appears that a car did a U-turn, which I presumably wasn’t expecting – and he wasn’t expecting me to be there.  It was not a sensible thing to do, but you would normally get away with it.  This time he – and I – didn’t.  If I had been at a different point in my stride the car would have swept me onto the windscreen – a shock but most likely much less damage to me.  In this case the bumper hit my outstretched leading leg, knocking me backwards and then running over the leg.

However I’m feeling lucky to be here, and don’t have energy to work the blame game.  I want to concentrate on getting back to something close to normal, retaining my sense of humour, and trying to make sure I treat all of those helping me with respect and courtesy even if I am feeling bad.

Politics – Just the same old same old


In the UK we are getting to the end of the political conference season, with lots of people earnestly telling us their ideas for government which will be tossed aside as soon as the inconvenience of a poll can disposed of.

For me the problem is that I don’t want solutions as much as I want revenge.

Seriously, the people responsible for all bad decisions made, basically take no rap for the problems they cause. At worst they lose a place in parliament – but there is a gold plated pay off plan to allow them to rehabilitate (like prisoners) to the “real world”.  They’ll be able to hit the conference speech circuit, sit on a batch of boards where they can still use their connections into government to keep fiddling.

I frankly want the following:-

  • A row of gallows erected outside the houses of parliament
  • A periodic review of govermental minister (who this is done by is the real sticky point).
  • When a minister is found to be corrupt, incompetent, lying, stupid or guilty of smearing those they interact with, then that minister is executed

Of course this isn’t going to happen…

But for a government that about turned on much of their manifesto (think about the RIP act for example) in the first weeks of government, and hid the evidence (try and find the 1997 manifesto), frankly death is far too good for them.

I voted for the 1997 Labour government – I thought nothing could be worse than the then incumbent administration.  On that day back then I was happy they were in, however the gloss wore off so quickly.

Now with the evidence of snouts in the trough and utter contempt for the people they supposedly serve I am just sick.  I believe the pork produced by those snouts needs to be converted into sausages.   Its time for the servants to realise that they are servants and they are subject to the overall masters.

The relationship needs to change.

The honourable gentlemen aren’t honourable and often not gentlemen.

CPAN RPM Packaging


Seeing as I have been asked about building RPM packages of CPAN modules today I thought it was worth putting some information down in a blog post – I would love comments on this, so that I can improve the information and hopefully make my processes better.

Firstly, I target Centos 4 and 5 i386 only, although I am going to have to start building for x86_64 too) and build stuff for our own internal requirements, the latter meaning that the stuff I package depends very much one what I need for work at the time. Its also a little painful that I cannot easily make this work generally available, but getting these packages into the wild would be complex. Packaging for other RPM flavoured distros should be easy enough, but slightly different.

There are a set of rules we work to for our internal deployments:-

  1. Everything is packaged as RPMs.
  2. You do not cheat on dependancies – if I do yum install foobar then foobar had better bring in all the appropriate required packages/modules
  3. We have 3 repositories – the first 2 are the standard Centos repos (or rather local mirrors) of the base os and updates respositories. The third is our own local repository of packages we build. We require that all packages are signed so have our own internal GPG key for our repos.
  4. All packages in our repository are built by us in a clean build environment and have no dependancies from outside the 3 repos we use.
  5. Packages should pass their own test suite (which we run as part of the build) unless there are exceptional circumstances

Although, as I stated above, we are building things for our own internal use, we do always contribute bug reports and fixes back to the upstream software provider (unless the patch is only needed due to our own bad practice) – maintenance is much easier if you keep close to upstream.

There are really 2 parts to building CPAN RPMs:-

  1. Making a source RPM (or even just the spec file)
  2. Building the binary RPMs

I’m going to consider these separately.

Creating The Source RPM

Mediocre Writers Borrow; Great Writers StealT.S. Eliot

Ideally use a source RPM package from a packager that you trust. In my experience the Fedora guys have now got the best RPM packaging of perl modules – they have a set of package rules they work to, and a good set of tools. If the Fedora folks have packaged a CPAN module then I will use their source RPM in preference to building one myself. If there are problems with that RPM then I will contribute fixes back to their packagers (or the upstream CPAN author). The best place to check for packages is the development sources directories – for example http://mirrors.kernel.org/fedora/development/source/SRPMS/

Fedora also produce the EPEL (Extra Packages for Enterprise Linux) additional packages for RHEL/Centos. Generally the only difference between these and the Fedora packages are possibly slight changes of dependancies due to the different OS environment (and Fedora uses perl 5.10 rather than Centos/RHEL perl 5.8).

If you can’t find a prebuilt source RPM file you will need to create your own. Although this is a relatively simple task (but unfortunately one that always appears to be hard to find documentation about) there are some additional tools to do the heavy lifting for you – these at least produce the skeleton of the package, but in many cases they produce a completely usable package with no intervention required.

I used to use cpan2rpm to produce packages – the version I had was hacked around (not code I was proud of) to produce better build dependancy listings.

However for the last couple of years I have been using cpanspec – another product of those incredible Fedora packagers. This produces a completely usable RPM in the vast majority of cases. A typical invocation (for me) looks like:-

$ cpanspec --follow --srpm SQL::Abstract

The follow flag does its best, but you do normally need to package pre-requisites yourself before packaging the required package.

cpanspec is packaged within the EPEL repositories.

Building RPMs

Basic information on building RPM packages can be found in the RPM Building Crash Course, as well as on the rpm.org wiki.

However it is best practice when building RPMs to build in a clean build environment to ensure that all the dependancy generation is correct, that no unpackaged files affect the build and to make sure the whole process is reproducible. This is done by having a separate chrooted build system that is regenerated for each new build.

Until a couple of years back I used the mach build system to produce clean builds (there is a HowTo document on this) – and I actually still use mach for building Centos 4 packages (because the system hasn’t broke so I’ve had no need to fix it). However for Centos 5 (and for any new environments) I now use mock (again from Fedora).

mock is packaged within the EPEL repositories.

In both cases you will need to configure your build system for the right base OS and repositories.

If you are building significant numbers of packages you will need additional scripting to manage things such as adding built packages to the repository (createrepo), expire off superseded packages (repomanage) and check that your repositories have no unfulfilled dependancies (repoclosure) – these programmes are generally all in the yum or yum-utils packages.

XML Processing


Recently I have had to revisit one of our systems that deal with XML call records (from a VOIP switch).

This system splits out Call Detail Records (CDRs) by customer. The version of this that was running was based on XML::Twig which used to run acceptably fast (this code was written a number of years ago), and has the advantage of being relatively light on memory as the document was processed a chunk at a time rather than being completely read into memory. However the system was getting apparently slower – mainly down the volume of calls being detailed increasing by a substantial factor.

So last week I spent a while trying out different approaches to this problem (as well as investigating approaches for a more database driven storage system for the future).

For the specific problem of splitting the data based on the customer responsible for the CDRs, the fastest approach I managed to put together was based on XML::LibXML. This has the disadvantage that it has to read in the complete XML file (and these are getting to be multi-gigabyte per hour), however the module is relatively light on memory compared to the other methods and a simplified rewrite of my previous programme resulted in a better than factor 20 speed up – rather worth having.

However this was basically a simple filter – splitting data coming in into several output streams based on a very simple criteria. Getting data fields out of the XML records with XML::LibXML appears to be relatively slow (and clumsy) – so for example if I want to extract all the fields into a database then the aggregate cost of accessing all the fields starts to be costly.

XML::Bare converts XML data files into perl hashes – either its own format which includes metadata to aid in reconstruction to XML, or a basic hash format very very similar to that used by the more ancient XML::Simple. Its fairly fast, although appears to be rather more profligate with memory (for some reason it holds the complete XML file as a string as well as hashified version – it also reads the whole file at once). XML::Bare is pretty fast, and if you are doing a lot of manipulation of the data within the XML file it might well be faster than using XML::LibXML

The big advantage of using XML::Twig originally was that its a quite perlish method of manipulating XML, and additionally you can use the simplify operation to convert the XML data into a hash – useful for dealing with individual records within the XML set.

However this cannot be done with XML::LibXML, and XML::Bare is too inflexible. So what would be useful was a fast mechanism for converting an XML::LibXML node into a hash making access within that node much simpler (and quite likely quicker – although there is an initial cost of conversion to a hash, as well as the memory cost).

I’m hoping to be able to set aside a little time to look at this. However I guess people may tell me other approaches – unfortunately the documentation within the various XML modules is somewhat opaque so its quite likely I have missed a great big feature somewhere!

Extending SQL::Abstract


Most of the perl I write currently has ties into DBIx::Class and hence uses SQL::Abstract.

I also have far too much of a preference for boolean items, which I normally encode in the database as a column of type boolean (or the SQLite vague equivalent).

Its been fairly easy to encode a test for boolean value being true with SQL::Abstract – although the syntax

column => \''

which maps to

WHERE column

is a little esoteric. However its downright close to impossible to produce the opposite test without using literal SQL (actually the first version uses literal SQL – except the SQL statement is NULL and comes after the column reference).

So today I have spent a little time extending SQL::Abstract to support the -bool and -not_bool unary operators which allow both positive and negative boolean tests to be encoded in a way that does not resort to literal SQL.

As part of this I refactored a part of SQL::Abstract so that further extensions of this type can be added more easily – adding a unary_ops extension to the constractor in the same way as the previously existing special_ops.

So now:-

-bool      => this_column,
-not_bool  => that_column

will map to

WHERE this_column AND NOT that_column

The code for this is currently sitting in a svn branch – please have a look and comment on this. My own criticism of it is that the syntax is clunky, but we are restricted to the perl datastructure mapping approach of the existing code (which is the real raison d’etre of SQL::Abstract).

Theatre Management with Perl – 1


First some history – in fact this post is almost all history and setting the scene for what we did later.

When I joined the theatre there was a website – however it had been done by someone’s friend, and was a flash only movie containing information on the shows that were coming in the near future.  This suffered massively from being done in spare time and could take months to get updated, so something had to be done.

The first aim was to get a website that we could get event information onto with as little ongoing maintenance as I could get away with.   Unfortunatly the event information we did have was in Excel spreadsheets – and this was arranged in colour coded tabular form (one column per month, one cell per day) which was likely to be hard to parse automagically.

I decided the easiest way to handle the event information was to use an Excel spreadsheet – this would allow other members of the theatre technical team to update the events too.  Each event (which can cover several performance dates) had a single line, with several fields describing it.

A piece of perl (yup, we have finally got there), parsed out the events (Spreadsheet::ParseExcel) and built a set of per-month event pages as well as a front page with current events on it – using HTML::Template.

The page generation was done off-line and the generated html pushed to the webserver – at that time we had to take this approach as the web server was separate to the machines with the appropriate tools to generate stuff.

As a fairly quick and dirty hack this worked extremely well – so much so that it stayed in place for more than 18 months – you can see examples of this on the way back machine – ie this one from March 2004.  Meanwhile we were starting to wake up to the idea of having an events database – even one as primative as the one we had – and realising we could start to use this for our internal backstage management.

We had started to use a wiki for much of the backstage management – we have half a dozen stage managers plus other crew, all volunteers and organising all of us is not an easy task.  The wiki chosen was MoinMoin – at that time I could not see any perl wikis that I was happy with.  The wiki contained another list of events – this time in a way more focused on periods the theatre was booked (which is different to performance dates – it includes rehearsals and some events that have no performances such as training events).

So we were duplicating information – and I was updating a lot of that information I felt this was very much a bad thing.

So I started looking at producing a proper database and bringing a lot of this information together….

Gadget Drooling


Spent yesterday at PLASA Focus at Leeds Royal Armouries.

We went with a fair shopping list – both Lighting Desk and Sound Desk as well as considering what to do about Radio Microphones (the Government playing silly buggers with frequency allocations makes this one a tricky prospect).

We also ended up considering smoke and haze machines – our own is somewhat long in the tooth, and we are considering keeping this sort of equipment in house but charging hirers a fee for use.  The hirer should get kit cheaper than it would cost to commercially hire, plus we would know the equipment, have a revenue source to pay for it, and the kit is likely to be in much better kit than most of that you get from commercial hire.

Recommendations for a lighting desk would be helpful – we currently have a Strand GSX (ie ancient), and that works quite well for us as its fairly simple for our staff to learn to use, and is good for knocking up lighting states on the fly (we have our basic groups of lights on subs which allows a very fast setup).

So we need something with the following characteristics:-

  • Able to handle 80 dimmers plus misc other DMX kit
  • Able to control a few moving heads (we do not have any, but occaisionally hirers bring them in, and if we have the capability then we are more likely to buy some in future).
  • A set of 20 or so sub faders to allow fast immediate use (or show us another fast way of working)
  • Ideally somewhat similar feel to the Strand to make transition easier
  • Ideally with some degree of remote capability so we do not need to move the whole desk when working in the auditorium
  • We have a fairly restricted space in the stage corner where the desk currently resides.  We could reposition, but ideally it would fit into a similar space to a GSX (not too much bigger).

The ones we looked at (only briefly) yesterday were (in no particular order) the Avolite Pearl, the ETC Ion and Element, the Zero88 Juggler TLXtra (actually I wonder if one of their other desks would have been a better fit).

If anyone has any suggestions then drop me a line or leave a comment.