While photographers aren’t producing as much digital assets as video people, a couple of thousands of raw files and a good load of PSDs can build up quite a storage need over the years.

While I was looking for a storage solution, I had four priorities:

• Expandability: I had to copy all my stuff once to get them onto a bigger drive – I never want to repeat that experience.
• Encryption: A good deal of my photos is not meant for everyone’s eyes – I think it’s my responsibility to protect the privacy of my models from anyone who might eventually get their hands on one of the drives.
• Cost: Storage systems can cost nearly unlimited amounts of money – money that I’d rather invest in a new lens.
• Fault tolerance: Obviously.

After some testing I came up this totally insane solution:

Take my existing two 500GB WD USB drives and put two shiny (actually, they’re black.) new 1TB WD USB drives next to them. Next, using Linux software RAID 1, I mirror the identical disks, so I get a redundant 500GB and 1TB volume. These volumes are now being encrypted with three different cipher algorithms (yay for paranoia) using TrueCrypt. The final layer is an LVM volume group using the TrueCrypt volumes as PVs.

Here’s some ASCII art:

  +-----------------------+
  |          LVM          |
  +-----------------------+
       |             |
   TrueCrypt     TrueCrypt
       |             |
     RAID1         RAID1
    |     |       |     |
  +---+ +---+   +---+ +---+
  | 5 | | 5 |   | 1 | | 1 |
  | 0 | | 0 |   | T | | T |
  | 0 | | 0 |   | B | | B |
  | G | | G |   +---+ +---+
  | B | | B |
  +---+ +---+

This setup covers all of my four priorities (the only limitation is that you have to expand it in steps of two identical disks). Naturally, this monster isn’t that fast. According to bonnie++, I get about 10MB/s for writing and about 18MB/s for reading. Enough for me.

If you know a better way to get 1.5 TB expandable, encrypted and fault tolerant storage for less than 250 EUR, feel free to email me.

I’ve spent a lot of time at work tinkering around with the BCFG2 configuration management system. Here’s a neat stunt you can pull off in your bcfg2 repo to achieve great flexibility when it comes to templating.

First off, all my bundles are defined in SGenshi. Now you can obviously use <ConfigFile> and point to a TGenshi template. But you might run into a problem here: A TGenshi template always has a static owner/group/perms tuple set in the info.xml file (which is bad if you need multiple instances of the template that belong to different users). I tried working around this via an extra <Permissions>, but the default info from <ConfigFile> seems to have a higher priority (see BCFG2 bug #627).

My next shot was to use <BoundConfigFile>, which lets you inline the template in the bundle file. However, this method has a number of problems as it tends to clutter your bundle files and forces you to use XML templating for text files.

So here’s my ingenious solution: Genshi allows the use of XInclude in XML templates, but with a special twist: You can tell Genshi to parse the referenced file as a text template! So here we go:

<BoundConfigFile 
    name="/foo/bar" owner="${owner}" group="${group}" perms="0644">
  <xi:include parse="text" href="templates/foo/bar.newtxt" />
</BoundConfigFile>