Hosts Begone!

5 March 2007 | 17:58 | Administration, Linux | No Comments

With my move to virtualisation and seemingly non-stop addition of net-enabled appliances, the hosts file I have on all machines has grown to over 20 entries. I thought it was about time I centralised this configuration and ran a local DNS server.

You always hear how the most common DNS server - bind - is a monolithic beast, but if you have reasonably moderate requirements, the implementation is quite simple.

The setup I wanted to achieve was probably very common for a home network. All machines/devices have static IP addresses and are all part of the same class C subnet. All DNS requests (for both local and remote addresses) should be cached by the local DNS server to improve lookup performance. All external DNS requests should be forwarded to my ISP’s DNS server(s). Adding a machine or changing an IP address should require no more the an update on the actual device holding the IP address in question, and an update to the local DNS server. All other machines should then be able to resolve the new hostname.

If this sounds like something you would be interested in (thanks Entourage - or more specifically) see the installation and configuration on the wiki and say hosts file - begone!.

The next obvious step would be to run a DHCP server too, but that is a story for another day.



The Virtual Bandwagon

19 December 2006 | 10:50 | Administration, Linux | No Comments

It seems that everyone is jumping on the virtualisation bandwagon. Reasons such as server consolidation and improved management seem to be pretty compelling to system administrators, and features such as live migration could be the icing on the cake. Can virtualisation save me too?

The three biggest headaches I have are resource management (e.g. continuously running out of disk space on /), service immobility, and keeping the system current. Regarding the last point, there’s a primal urge to respond when your system status script notifies you that your tar-1.16.r1 has be superseded with 1.16.r2. Who can resist this call to action? Not I! But every time I update installed components I find myself backing up all the configuration files (despite the fact that they haven’t changed since the last update and are included and backed up in my configuration file subversion repository), silently crossing my fingers, and hoping for the best. In addition, my experience/paranoia with hard disks compel me to run the system in a RAID-1 configuration (backup? bah!) which results in the system incurring a non-trivial overhead - particularly with regard to write performance.

Dynamic disk allocation could be achieved with LVM, or - you guessed it - virtualisation.
System portability is by definition a feature of virtualisation, so service separation into different VMs seems to be a reasonable solution to start uncoupling services from physical machines.

With the ability to checkpoint and clone VMs, virtualisation can be used to perform “test updates” - item 3: check!

Now that my spare room has appeared to morph into a data centre, what virtualisation technology is right for me?

There’s Virtual Machines (VMware, Microsoft Virtual Server, Parallels) which emulate all the hardware. It is the most flexible option as you can “pretend” to be running any hardware as far as the guest VMs are concerned, but it’s also most resource intensive. In addition to being isolated from each other, the VMs are isolated from the host machine.

Then there’s Paravirtualisation (Xen) which uses a hypervisor to manage and isolate the VMs either by hardware (using Intel’s VT-x or AMD’s SVM CPU level virtualisation found in recent chipsets) or by loading customised versions of the host OS which has been modified to be aware and behave for the hypervisor. The guest VMs are executing their code natively (albeit under the control of the hypervisor), which means you can only run software targeted for the hardware you are running it on, and if you don’t have a new whiz bang CPU, this limits your guest OS choices to pretty much Linux only. This performs considerably better than the previous option.

Finally there is OS level virtualisation (Solaris Zones, Linux-VServer, Virtuozzo/OpenVZ) which adds a virtualisation layer to the host OS. This is the most limited option, as all your guest VMs need to be the same as your host (although in the case of Linux, not the same distro), but also has the smallest overhead (only 1-2% of resources are spent on virtualisation). Each VM is essentially a chrooted/jailed full file system that is mounted by the host and share the (patched) host kernel (gross simplification - but it is essentially the how it appears to the administrator). What this gives you is full access to the file system of each VM from the host, regardless if it’s running or not.

So where does this leave me? On account of me only running Linux (on the server at least), OS flexibility (or inflexibility) was not a factor. More important was performance - particularly on older hardware, and price. The choice looked between Xen and OpenVZ. When ever I’m faced with a choice, I invariably choose the simplest option (at least initially) and in this case it clearly seemed to be OpenVZ.

Two weeks on and I’m loving it! I can run a debian-based webserver, MySQL under centos, and all my custom services from a gentoo server. I can bind a mount point to each VM (or VE - Virtual Environment - in OpenVZ-speak) to allow the virtual machines share disk which is great both for general storage and things like a portage tree. From the host (or HN - Hardware Node) I can monitor and tune a multitude of resource allocations for each VE which are reflected immediately.

The only area lacking in this solution would appear to be a good monitoring/management interface - but I’m working on it. Stay tuned for further virtual adventures!



WoW Linux!

5 December 2006 | 12:35 | Entertainment, Linux | No Comments

With the latest beta release (6.0.0 beta 3) of CodeWeaver’s CrossOver Linux (formally CrossOver Office) I am finally able to install and run World of Warcraft flawlessly on Linux. In fact, it seems to run better than under Windows XP on my laptop! I tried Cedega and Wine, but neither was as simple and functional as cxlinux.
Although I haven’t used either in quite a while, I’m still concerned about being able to run Adobe Photoshop and Premier, so this may not be the silver bullet to allow me to fully get rid of Windows - but I am definitely getting close. And one 10GB partition out of 4 PC’s and 1.5 TB of disk is a manageable ratio.
People seem to have positive experiences running Photoshop under cxlinux and I’ll be sure to try it out when the need arises.
It it worth the asking US$39.95? HELL YES! Even though I believe pretty much all the code updates are passed upstream to the Wine group, and generally I’m more a DIY person (computer-wise that is), the simplicity of the pre-configured settings for specific applications can not be undervalued.

I can’t wait for the official release!



The cost of small pleasures

7 August 2006 | 18:16 | Administration, General, Linux | No Comments

After reading this article linked to from slashdot (and popular enough to be backslashed), I realised that I too would be happy spending a number of hours reconfiguring my home network just to screw with anyone who had the poor judgement to try and use my wifi.

This as an alternative to leaving my network managed by my routers and its wifi secured with WPA seems like a good use of my time. Sure I need to reconfigure my whole network into multiple segments, install and configure a DHCP server, and setup rules for iptables - but dude - upside down images!

What is it that causes the (admittedly small) pleasures achieved by messing with people - regardless of whether or not you are able to see the effects directly - have such a high personal value that pretty much any effort seems well worth the time and cost it involves?

Maybe it’s just me.



Back it up!

17 July 2006 | 22:38 | Administration, Linux | No Comments

Why are people (including myself) so lazy about backing up their data?

Do we assume the “managed” in the $4.95/month managed server takes care of this for us? Do we think the risk of failure is so small that it will only happen to other people? Do we think it will require a large and ongoing effort? Do we just not know where to start?

With the cost of hard disks being so low these days, these seems to be no reason not to have a copy of at least your most important databases and files on a separate disk to ease the pain when that dark day that you will forever call “the hard disk crash of {insert year here}” comes.

A combination of this thought and the inherent laziness that most programmers have prompted me to look into what this actually would entail. I believe that the end result was simplified by the fact that I run mostly Linux, but I’m sure with cygwin or something similar, the same can be pretty much achieved on windows.

The end result is a set of scripts for backing up MySQL and transferring data (both the MySQL backups, and other data such as my websites) to a remote server. If I create a new database that needs to be backed up, I just have to grant access to it for the backup user and restore user and it will automatically be included in the backup routine. If I want to backup a new folder, I need to add it to my backup script and it will be synced according to the next schedule. Alternatively, I could create a symlink for all top level folders needing backup inside a common “backup” folder and only need to backup that folder, but this reduces the flexibility of being able to backup my websites to machine1 and audio (for example) to machine2. Once a month each backup will be restored to a scrap table for verification. Finally, due to the magical default behaviour of cron, anytime an error is encountered, I will be notified via email.

Information including all scripts for the MySQL backup can be found on the wiki: http://wiki.accordingtokris.com/index.php?title=MySQL_Backup

Information including scripts for content backup can also be found on the wiki: http://wiki.accordingtokris.com/index.php?title=System_Rsync_Backup

Enjoy - and no more excuses if when you have a disk failure!