At time of writing, on a standard Kimsufi, OVH provide you with one static IP, and three “failover IP’s” these “failover” IP’s are just normal IP’s, although they don’t have their own gateway (since only the host machine is allowed to use the gateway). The only thing that’s different about these “failover” IP’s is that if you have more than one Kimsufi or dedicated server with OVH then you can simply change which server the failover IP is routed to, which is quite handy for DR (disaster recovery) or even load balancing.

Let’s get started, first, you need to set up your server with the default CentOS image that is provided by OVH (this may already be the case if you requested this image when you ordered), if not, this is done by selecting your server from the dropdown box at the top of the manager screen, then clicking on “Services” beneath the “Dedicated servers” menu on the left-hand side. While in this menu,  ensure that “Netboot” is set to “hd” so that the boot process is kicked to the hard disk and not OVH’s custom kernels. Next, choose “Reinstall / Change OS” and follow the steps here to install CentOS, the partitioning doesn’t really matter, where possible just choose the default options and continue through the steps until the installation process starts.

The installation process will continue and wipe your server clean and replace it with OVH’s CentOS image (which I might add, isn’t a TRUE CentOS kernel, as provided by the CentOS project). Once you’ve received your email from OVH confirming that the reinstall is complete and have got your new root password, log into the server via SSH. When you have a prompt, type the following commands:

mkdir /cleaninstall
cd /cleaninstall

For 32-bit CentOS:

wget http://ftp.hosteurope.de/mirror/centos.org/5/os/i386/images/pxeboot/initrd.img
wget http://ftp.hosteurope.de/mirror/centos.org/5/os/i386/images/pxeboot/vmlinuz

For 64-bit CentOS:

wget http://ftp.hosteurope.de/mirror/centos.org/5/os/x86_64/images/pxeboot/initrd.img
wget http://ftp.hosteurope.de/mirror/centos.org/5/os/x86_64/images/pxeboot/vmlinuz

Then copy the pxe boot files to the /boot partition

cp vmlinuz /boot/vmlinuz.cent.pxe
cp initrd.img /boot/initrd.img.cent.pxe

Next we need to set up the GRUB bootloader:

yum install grub

Empty the bootloader sequence and insert our new config:

cat /dev/null > /boot/grub/menu.lst
vi /boot/grub/menu.lst

Paste the below text in this file amending the following variables to suit your info:
YOURPASSWORD = A password of your choosing, this is a a temporary password and is only used during the installation via VNC
IPADDR = The IP address of your OVH server (host)
GATEWAY = The gateway IP address of your OVH server (find this by doing: # netstat -r | grep default)

For 32-bit CentOS:

default 0
timeout 30
title Centos Install (PXE)
        root (hd0,0)
        kernel /boot/vmlinuz.cent.pxe vnc vncpassword=YOURPASSWORD headless ip=IPADDR netmask=255.255.255.0 gateway=GATEWAY dns=213.186.33.99 ksdevice=eth0 method=http://ftp.hosteurope.de/mirror/centos.org/5/os/i386/ lang=en_US keymap=us
        initrd /boot/initrd.img.cent.pxe

For 64-bit CentOS:

default 0
timeout 30
title Centos Install (PXE)
        root (hd0,0)
        kernel /boot/vmlinuz.cent.pxe vnc vncpassword=YOURPASSWORD headless ip=IPADDR netmask=255.255.255.0 gateway=GATEWAY dns=213.186.33.99 ksdevice=eth0 method=http://ftp.hosteurope.de/mirror/centos.org/5/os/x86_64/ lang=en_US keymap=us
        initrd /boot/initrd.img.cent.pxe

Save the file (in this sequence hit Escape, colon (:), x, Enter)

Next we need to install GRUB into the MBR of the server disk so…

grub-install /dev/sda
grub-install --recheck /dev/sda

Once that’s done, we need to bounce the box:

shutdown -r now

Wait a couple of minutes for the server to come back online (you can monitor it using ping requests). Once the server is contactable again fire up a VNC client and in the connection box type in the IP address of your server followed by “:1″, for example, if your IP was 192.168.0.5, the VNC connection box will read 192.168.0.5:1

Enter the password you specified as and you will be presented a VNC window at the first step of the CentOS installation screen. Follow the installer (make sure you leave the network options alone – for now at least). After the installation completes and reboots, you now have a PROPER CentOS installation free from OVH’s meddling hands.

Install VMWare Server 2 as you would normally on CentOS, there are tons of HOWTO’s out there on that subject I’m sure. One thing I will say about installing VMWare Server 2 is that when asked about configuring a bridged network interface, to be honest, I wouldn’t configure one, you’ll only need NAT and Host-Only networking. To configure a Bridged interface just invites a headache of having your switch-port blocked if you ever switch on a guest VM which tries to make use of it – which can be very frustrating and WILL cause an interruption to service while OVH sort out unblocking your MAC address from their network.

So, once we’ve got VMware Server 2 installed and running, we need to set up the networking so that the “failover IPs” can be used for the virtual guests. We’re going to be doing this using routed IP’s as it’s the simplest way, without having to involve NAT’ing (ugly).

We’ll start off by configuring the host machine to allow proxied arp requests on the VMware host-only interface, which in this case is the “vmnet1″ device – this is the default interface name that VMware Server creates, however if yours is different you’ll need to amend the below steps accordingly (this is only if you’ve had a non-standard VMware install).

echo "net.ipv4.conf.vmnet1.proxy_arp=1" >> /etc/sysctl.conf && sysctl -p

Next on the host, we have to create a static route to allow traffic to flow to the “failover IP’s”

IPFAILOVER is to be subsituted with one of your failover IP’s. e.g 10.0.0.11

ip route add IPFAILOVER dev vmnet1

A line like the one above is required for each failover IP which you’ll be routing.

This would not survive a reboot (bad) so we have to do something a little more inventive…
We need to create a network config file in the /etc/sysconfig/network-scripts folder to set up the interface dynamically, since VMware doesn’t actually configure the interface using the standard method. In short, we need a config file to set up the interface so that CentOS is “aware” of it, and then we also need to create a route file for the interface too to set up the routing for the interface at boot.

So…
First off we need to find out what interface subnet the vmware installer picked to use for the vmnet1 interface on the host server, so do a

ifconfig vmnet1 | grep "inet addr"

From the output of the above command, use the “inet addr:” address for the IPADDR= and use the “Mask:” for the NETMASK=

Create and populate the /etc/sysconfig/network-scripts/ifcfg-vmnet1 file with the below info:

vi /etc/sysconfig/network-scripts/ifcfg-vmnet1

DEVICE=vmnet1
IPADDR=*.*.*.*
NETMASK=*.*.*.*
ONBOOT=yes

Now that CentOS is aware of that network interface, the routing config to the VM’s fail-over IP needs to go into /etc/sysconfig/network-scripts/route-vmnet1 which is done by:

vi /etc/sysconfig/network-scripts/route-vmnet1

IPFAILOVER dev vmnet1

A new line like the one above is required for each failover IP which you’ll be routing.

Now, unfortunately, that doesn’t solve all our problems because VMware server starts up after the networking service and thus after the virtual device is created. So we need a way to restart the networking service after VMware has started. The quickest and dirtiest way of doing that is by putting something in /etc/rc.local which will do the job. I hated having to do this, purely because it’s not “kosher” with the distro, and in general bad practise, but needs must…

echo "/sbin/service network restart &> /tmp/network-restart" >> /etc/rc.local

Ok, so now we need to allow traffic through to the virtual machines. This is done using the iptables kernel module on the host machine (or can be done on the guest so long as there’s a rule on the host allowing all traffic through to that failover IP). Generally speaking, I like to keep my firewall rules in one place for ease of management, and that’s on the host. So, we’ll open up the iptables config file (or use my script to make it easier):

vim /etc/sysconfig/iptables

Find the line:

-A RH-Firewall-1-INPUT -m state --state NEW -m udp -p udp --dport 1194 -j ACCEPT

and insert this line below it subsituting where necessary (below example is allowing port 22 (ssh) access through to this failover IP):

-A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp --dport 22 -d FAILOVERIP -j ACCEPT

As above, a new line (rule) like the one above is required for each failover IP which you’ll be routing.

As far as the server, or rather, host side is concerned, we’re finished. Now we just need to configure the guest OS with the failover IP address (or addresses if you so choose). The virtual machine should be configured to use the following:

Linux hosts:
IP= FAILOVERIP
NETMASK=255.255.255.255
GATEWAY= FAILOVERIP (yes it’s the same as the interface IP)

Windows hosts:
IP= FAILOVERIP
NETMASK=255.255.255.248
GATEWAY= FAILOVERIP (yes it’s the same as the interface IP)

The DNS nameservers can be set anything you wish to use, so long as they’re addressable and accessible by the VM’s. I tend to use OpenDNS’s servers unless I’m using my own.

#!/bin/bash
TESTING_MINS=2
vim /etc/sysconfig/iptables
clear
QUESTION1="Do you want to restart the firewall now? (hit 't' to test for $TESTING_MINS min(s)) [y/n/t] "
echo -n $QUESTION1

a=""
while test -z "$a"
do
        read -n1 a
        echo ""

 case "$a" in
  Y|y)
        echo -e "Restarting...\n\n"
		/sbin/service iptables restart
  ;;
  N|n)
        exit 0
  ;;
  T|t)
        echo -e "Time is now `date +%H:%M` -firewall service will be stopped at `date +%H:%M -d "+$TESTING_MINS min"`\nIf your test was successful, you will need to manually start the service again by running:\nservice iptables start"
        echo "/sbin/service iptables stop &> /dev/null" | at now + $TESTING_MINS min &> /dev/null
		echo ""
        /sbin/service iptables restart
  ;;
  *)
        a=""
        echo -n $QUESTION1
  ;;
  esac
done

P.S. Any scripts I write and publish here are © Rob Freeman and released under the GPL unless otherwise stated.

yum clean all
mkdir /usr/src/centos
cd /usr/src/centos
wget http://mirror.centos.org/centos/RPM-GPG-KEY-CentOS-5
wget http://mirror.centos.org/centos/5/os/i386/CentOS/centos-release-5-3.el5.centos.1.i386.rpm
wget http://mirror.centos.org/centos/5/os/i386/CentOS/centos-release-notes-5.3-3.i386.rpm
wget http://mirror.centos.org/centos/5/os/i386/CentOS/yum-3.2.19-18.el5.centos.noarch.rpm
wget http://mirror.centos.org/centos/5/os/i386/CentOS/yum-updatesd-0.9-2.el5.noarch.rpm
wget http://mirror.centos.org/centos/5/os/i386/CentOS/yum-fastestmirror-1.1.16-13.el5.centos.noarch.rpm
rpm –-import RPM-GPG-KEY-CentOS-5
/bin/rpm -e --nodeps redhat-release
/bin/rpm -e --nodeps rhn-client-tools
/bin/rpm -e --nodeps yum-rhn-plugin
rpm -Uvh --force *.rpm
yum upgrade

How painless was that?!

Here’s some test footage I made for any people looking at getting a Panasonic TM10 or an SD10. Overall I think it’s a great little camera, can’t wait to take some serious footage in Oz with it. Enjoy! (and yes, I know I need a new windscreen wiperblade)

Click the “Play” image to watch the video, or you can check out the full site, or even the reddit submission.

(Please open the article to see the flash file or player.)

With that said however, it’s somewhat a rite-of-passage that blog writers do (top ten lists – and especially with this as the subject matter), and since I just trawled through my favourites (again) for my “essential addons” for my new installation of Firefox 3.5 RC3, I thought I’d accumulate them all in one post so next time Mozilla make a ridiculously late release of their milestoned browser, I can just rattle them off from one page here.

So, here we go (in order of installation preference):

1. XMarks (previously Foxmarks)
2. TabMixPlus (dev-build)
3. Adblock Plus
4. Flashblock
5. Twitterfox
6. AutoAuth
7. Web Developer
8. Firebug
9. TinyMenu (great for netbooks with limited screen res [Win only])
10. Personas (I use Tranquility)

I just couldn’t have this happening on Red Hat, since the AIX firewall is locked down as tight as can be, with even anomalous outbound tcp/ack’s being disallowed. I know that the portmap service gets its free port numbers from (among other sources) /etc/services so I decided to grab the current port number that mountd was running on…

rpcinfo -p | grep mountd

and make an entry into /etc/services in the hope that rpc.mountd would see the mountd entry and automatically use that port number, and only that port number, such an example entry:

mountd          672/tcp                         # Rob's Edit - binds mountd to a static port
mountd          672/udp                         # Rob's Edit - binds mountd to a static port

I restarted portmap and nfs, and ran rpcinfo again…

service portmap restart
service nfs restart
rpcinfo -p | grep mountd

… and lo-and-behold rpc.mountd had binded to the static port specified.

As soon as I’d reloaded the firewall I could see that NIS had failed (inexplicably) and backed out the change, I had to get NIS back online, and reloading the YP services wasn’t working. With the change backed out, I reloaded the firewall again, this time mkfilt just wasn’t having it. The syntax was fine, but the firewall was now blocking access to all services. Remember, I’m working from home, via an SSH session to a host at work with rlogin access to the wide. As soon as the firewall started blocking traffic my remote session died and I was unable to access it. FUCK!

I get on the phone straight away to the DC and asked a colleague of mine, Brian, to re-run the firewall script from the control workstation, which has a direct, non-IP connection to the wide. About 10mins later I get a call saying he’s been able to restart the firewall ok, and I can access the server from my connection again. Phew. NIS is still down though and still refuses to start-up cleanly. A reboot is in order. By this time, I’m pretty much ready to head into the DC so I can be hands-on with the kit when needed. Brian gets in touch with the client and co-ordinates a graceful shutdown of the databases before we initiate a standard reboot.

By the time I arrive at the DC (15mins away by car fortunately), we’ve managed to arrange “unscheduled maintenance” time, and we bounce both the nodes. Everything comes back up perfectly, and users can log back in just fine. We notify the client, and they can see everything’s ok, the databases have come up and everything’s back the way it was.

I get into finding out what caused NIS and the second firewall reload to spanner completely when we get another call from the company saying that LPR print queue jobs are not being passed from the thin node to a 3rd server which is running Caldera Open Linux linux (yeah, I know!). This Caldera box is running Tarantella which provides client-based printing. Essentially, users printed from a terminal on the thin-node, which is mapped to a remote print queue on the Caldera server, and the Tarantella server then maps the user’s printer to their print queue on the Caldera server. Essentially allowing (in a very round-about way) client-based printing from a terminal. For turn of the century stuff this was quite advanced, since there was no way to do this dynamically, from a web-based (HTTPS) client, and without setting up static routed print-queues on the node.

So that’s the background. Now, when we heard about this printing issue, which had been an intermittent problem since the platform had been introduced, but this had normally been resolved by a simple reboot of the Caldera server. We decided that since the nodes had been down, this had likely caused a bottleneck between the servers and that Caldera needed a reboot in order to enable the bottleneck to clear and allow the print queues to start moving again. We bounce the box and the queues still are being held on the thin node. FRAK! I know beyond a doubt that the issue isn’t software firewall related, since my minor change (a) wouldn’t have affected port 515 communications and (b) the firewall is running ok. My boss, John, had become involved around the time we rebooted both the nodes, as he was interested to know what was going on. After being brought up to speed he was convinced that this was a firewall related issue, since the initial cause was firewall related, and that I’d asked our network manager to add new rules to allow NFS between the new and old platforms. I knew it was highly unlikely that the problem was a firewall one since the changes had been backed out, and the system was in it’s normal, default configuration but  John felt that the timing was just too close for it to be a coincidence with anything other than a firewall issue. It took us a while, looking at the firewall rules in place, to see if any hits were being matched on the Cisco’s (which they weren’t), telnetting to ports etc all of which were fruitless. It was obvious in my mind that there was something on the Caldera box which was not allowing the LPD daemon to respond properly. After looking through the tarantella logs I checked the /var/log/messages log and saw that the LPD daemon faulted at start-up with the error “not enough disk space”. That old chestnut.

After a little more digging, it turned out that the / partition, having only 2GB of space had slowly been filled up by apache access and error log files since the early 2000’s and had caused the disk to become full. Monitoring hadn’t been set up to check disk space usage, which beggared my belief, but there it is. The apache logs had filled the last of the available disk space at pretty much the exact same time as the AIX system had gone down. All of the time spent wasted checking firewall rules and all the printing problem was related to was a frakking simple thing – disk space.

So the moral of this story is, blind co-incidence DOES happen in this profession, and it’s something that I’ll definitely remember for the rest of my career!

<3 “Red Hat is the trusted brand in Linux, and for good reason. Red Hat’s support policies demonstrate an understanding of what Linux customers require: mission-critical support for mission-critical deployments.”

Article: Oracle’s Unbreakable Linux not denting Red Hat | The Open Road – CNET News.

After having set up my .kde settings from 4.1 and then progressing through the betas my .kde folder was unsurprisingly twisted, so this afternoon I cleared it down and re-did my desktops as they were previously.. this is the result…