Green backups with BackupPC and WOL

In an effort to reduce power consumption, I’ve moved most systems I use regularly onto a pair of virtual servers, enabling me to keep many machines off most of the time, except for occasional use.  There are still a few physical machines, and I do prefer to have current backups of everything, so I decided to use the wake-on-LAN protocol, which seems to work for most modern computers hard-wired to ethernet (on the same segment.)

I started with this post by Salik Rafiq, but my first problem was that I didn’t want to turn every machine on, back it up, and then just leave it on.  Nor did I particularly want to turn every machine off after being backed up, since it’s not unusual for a machine to be backed up while I’m actually using it for something.  The obvious solution is to store whether or not a machine is actually on, then return it to that state afterward.

I made a few other changes, which I’ll briefly outline:

  • My BackupPC sends 5 arguments to ping, so I lazily hacked in arguments 3 and 4.  (There are much more elegant ways to do this.) so this script assumes that the last argument is the machine name.
  • I replaced “ethwake” with “wakeonlan,” for the sake of simplicity (not that etherwake doesn’t work, but since WOL packets require a hardware address anyway, why not just use the hardware address?)
  • Related to the above, I retrieve and store the hardware addresses for each machine using the simple command “arp -a machine > machine.wol” which gives me a file with the hardware address for each machine.
  • Note that I use the hardcoded directory “/usr/tools/wol” for both hardware and state information.  The $WOLDIR directory is used for hardware and state information.

As with Salik’s original script, this replaces the “ping” command for BackupPC, and is reasonable to use with every machine, whether or not it supports wake-on-lan.

#!/bin/bash

#this script is totally designed for the backuppc ping command
#which is the first thing it does before it starts a backup
#this is a substitute which pings the machine, if it is not
#awake then it wakes it using a magic packet - using the wol.bsh script
#then pings again to make sure

PING=/bin/ping
argnum=$(($#-1))
ARGS=${@:1:$argnum}
WAKEHOST="${!#}"
ETHWAKE=/usr/bin/wakeonlan
WOLDIR=/usr/tools/wol
SLEEPTIME=3m

logger "Backuppc pinging $ARGS $WAKEHOST"

function getwol {
	if [ -f $WOLDIR/$1.wol ]; then
	   hwaddr=$(cat $WOLDIR/$1.wol | cut -d" " -f4)
	else
	   logger "No MAC address for $1"
	   exit -1
	fi
}

function fwol {
        TO_WAKEUP=$1
	getwol $1
        sudo $ETHWAKE $hwaddr
}

$PING $ARGS $WAKEHOST >>/dev/null 2>&1

if [ $? -ne 0 ]; then
        fwol $WAKEHOST
        if [ "$WOL_RES" = "FAIL" ]; then
                exit 1
        fi
	echo "OFF" > $WOLDIR/$WAKEHOST.state
        sleep $SLEEPTIME
        $PING $ARGS $WAKEHOST
        if [ $? -eq 0 ]
        then
           logger "success waking $WAKEHOST."
        else
           logger "unable to wake $WAKEHOST."
           exit 1
        fi
else
   $PING $ARGS $WAKEHOST
   echo "ON" > $WOLDIR/$WAKEHOST.state
fi

exit 0

To turn the machines back on, I updated my postcmd.sh script (see this post for details) to look up the machine state, and if it was off before, turn it off now:

#!/bin/bash
WINEXE=/usr/bin/winexe
UNAME="Username"
PWD="Password"
WRKGRP="WORKGROUP"
BOX=$1
PID=$($WINEXE -U $UNAME -W $WRKGRP --password=$PWD //$BOX 'cmd /c echo '1'
> c:\backuppc\wake.up')
echo "Rsync and shadow copy unloaded"
if [ -f /usr/tools/wol/$BOX.state ]; then
   read wasoff < /usr/tools/wol/$BOX.state
   if [ "$wasoff" = "OFF" ]; then
      $WINEXE -U $UNAME --password=$PWD //$BOX 'shutdown -f -s -c "Backup
Complete"'
   fi
fi

Although it only recognizes “on” and “off” as valid machine states (it doesn’t detect nor return a system to hibernated, sleeping, etc. states) it at least takes care of the simple function of turning machines on, backing them up, then turning them back off.

For systems that don’t support wake-on-lan, like those connected through wireless, I’m considering using X10.

Share

BackupPC and Bare Metal Restore of Windows XP

While it’s not well documented, it’s possible to do a bare-metal restore of a machine running Windows XP that’s been backed up via BackupPC (assuming, of course, that some method, such as that documented here, has been used to back up open files, such as the registry.)

Step 1:  Build a local tar file using BackupPC_tarCreate

The alternative, BackupPC_zipCreate, proved problematic; the gigantic archives that it created could not be completely recovered, yet tar archives proved intact.  At any rate, this is best accomplished from the command line, as the backuppc user:

BackupPC_tarCreate -t -n -1 -h borkstation -s C / > borkstation.tar

“borkstation” is the name of the host to recover, “-n -1” means the latest backup, and you’ll obviously need to have enough space where the tar file is going to store the entire backup, which will not be compressed.  Note the space between the “C”, which represents the share to restore, and the “/”, which represents the directory to restore.

Step 2:  Prepare bare XP

Basically, you want to install a working copy of the operating system.  Aside from network drivers, it really doesn’t matter if anything’s installed or working, you’re going to wipe it all out anyway.  The filesystem and the partitions will stay, so configure those the way that you want your system to finally end up.

Step 3:  Prepare Recovery Console

Using the same install CD, you can install the recovery console to speed things up a little versus booting from the CD, although either option will work.  To install the recovery console, use Start->Run, then

d:\i386\winnt32.exe /cmdcons

Where D: is the letter if the CD-ROM.  Whether you choose to install it or not isn’t really important, but what is important is that you open up the permissions for Recovery Console, or some of the remaining steps won’t work.

  1. Start->Run, then type secpol.msc
  2. Go to Security settings->Local policies->Security options
  3. Scroll down to “Recovery console: Allow floppy copy and access to all drives and all folders” and make sure it is enabled.

Step 4:  Install Cygwin

You’ll need a “tar” utility you can rely on for the next step, and experimenting with a plethora of them has shown that unless you build your own, you’ll really need Cygwin.  While installing, be sure to select the gnu tar package.  (Unless your backup contains Cygwin, you can just delete the c:\cygwin directory later to get rid of it all.)

Step 5: Unpack the tar file to the local drive

A prerequisite to this step is that you get access to the tar file created in step 1 — which you’ll have to wait for if it isn’t finished being built yet.  The simplest way to get access to the file is to map a drive to the *nix box that has the tar file. I mapped mine to the Z: drive, which was a pretty arbitrary choice.

Next, create a directory to put the recovered files into, because you do not want them going into the root directory yet.  I created a directory called “recovery.”  Launch Cygwin, then:

cd /cygdrive/c/recovery
tar -xvf /cygdrive/z/borkstation.tar

This part takes a while, but when it’s complete, the entire system as backed up will be in the c:\recovery directory.  N.B.:  this goes a little faster if you leave off the “v” parameter, but then you don’t see anything in the way of progress.  It also helps to occlude or minimize the cygwin window.

Step 6:  Rename and move files

There are three hardcoded paths in XP, that are probably the only folders in root on your minimal installation:  Program Files, Documents and Settings, and WINDOWS.  Within the c:\recovery directory, rename these to something else.  I chose “Program Files.recovery” and so on.

Once these three directories are renamed, cut and paste everything from c:\recovery to c:\.  There’s no need to replace any files in c:\, and there shouldn’t be any overlaps, so if Windows asks if you want to overwrite any files, say “no.”

Step 7:  Boot to Recovery Console and rename key folders

Here’s where Step 3 becomes pretty important.  Once you boot into Recovery Console, you’ll need to allow yourself to work in the root of the drive using the “set” command:

set AllowAllPaths = TRUE

Once that’s done, you can rename the three hardcoded directory paths, moving the originals out of the way for your recovered files.  For the sake of having a fallback position, I rename rather than delete them at this point:

ren "Program Files" "Program Files.delete"
ren "Program Files.recovery" "Program Files"
ren "Documents and Settings" "Documents and Settings.delete"
ren "Documents and Settings.recovery" "Documents and Settings"
ren WINDOWS WINDOWS.delete
ren WINDOWS.recovery WINDOWS

The suffix “.delete” has no inherent meaning, it just gets the folders out of the way and signals that I don’t need them any more.  Once this is done, and you’re sure there were no unfortunate typos, you can type “exit” to reboot to a recovered system.

Step 8:  Clean up

After rebooting, pretty much everything should be as it was, although you’ll have three extra directories with a “.deleted” suffix plus the cygwin directory to get rid of.  It may be necessary to take ownership and grant yourself permission to do so, but there’s really no reason not to wipe them all out, since your original folders and files should be intact.

While the “read only” flag has been preserved for recovered files, the “hidden” and “system” attributes have not.  For most files, this doesn’t seem to matter much, but the “desktop.ini” files that dot the drives can have weird side effects, like launching an editor upon boot and showing up.  It’s easy enough to fix from the command line:

cd \
attrib +h +s /s desktop.ini

This will grind away for a while, since it will reset all the desktop.ini files on the drive.  Once complete, you’re back to where you were upon your last backup.

Share

Restoring Selected Keys from the Registry

As I hope everybody does, I have a backup system in place that ensures that I have copies of all critical files, including the system registry.  A combination of Volume Shadow Copy and BackupPC ensure that I’ll have ample copies of the registry available, but the registry itself makes recovery a less-than-straightforward process.

For a bare metal restore, the process is actually not too bad.  To be specific, you can pretty much just copy over the registry files with backups (which is pretty much every file in %systemroot%\system32\config) then rebooting before doing anything else.

More specifically, you want these files:

security, software, system, default, sam

It gets a little weirder from there if you need registry elements from the user hives and security hives, which is conveniently spread out into places like “Documents and Settings\NetworkService\NTUSER.DAT”, “Documents and Settings\LocalService\NTUSER.DAT” and helpfully named things like “UsrClass.dat” spread all over creation.

It wasn’t really my purpose to delineate all these things, so I’ll move on:  instead, my purpose was to point out that what, after losing a drive and reinstalling the operating system, you decide that you don’t want to inherit all the cruft that was lurking in your “old” registry, but instead would like to pull over just a few select keys?

On the plus side, there’s a way to do it.  On the minus side, doing so is about as convoluted as the registry itself.

First, the keys you’re looking for are most likely to be housed in the file “software” or “system,” depending on whether you want some keys describing installed software or hardware, respectively — so restore these files somewhere.  The location doesn’t matter, as long as you don’t put them on top of their current locations.

Second, fire up the registry editor (“regedit” or “regedt32”) which will give you a view of your current registry.  Click on HKEY_LOCAL_MACHINE, which will then make the menu option File->Load Hive available.

Go ahead and pick the “software” file you restored, and you’re immediately prompted for “Key Name.”  This is the key name to mount the hive under, so pick something that’s not a hive already in use.  (In other words, do NOT pick “software” or “hardware” and so on.)  It’s handy, if possible, to pick something absolutely guaranteed not to be in use as a key or value anywhere in the registry, but really anything will do.

Now you can browse the hierarchy of the registry from the file you loaded.  Note that none of these keys are really “in” the registry, but now you can get to them, to select what you need out of the original registry — in my case, it’s almost universally nit picking serial numbers from software I installed long ago and I have an easier time finding the CD than I do where the original serial numbers went.  Once you’ve selected the key, File->Export will allow you to extract it in text form.

Unfortunately, registry exports store the absolute path of whatever you’re exporting, so you’ll have to edit the file to get it back in.  N.B.:  Regedit is notoriously picky about the format of this file, and it will refuse to import a file that isn’t perfect, right down to white space, so you don’t want to edit it with anything that’s going to touch anything but the text (yes, I’m looking at you, TextPad.)  I recommend Notepad with word wrap OFF.

The exported registry file will be littered with references like:

[HKEY_LOCAL_MACHINE\MY_HIVE_KEY\GoodJobSucking]

Your job, of course, is to change every single “MY_HIVE_KEY” to “SOFTWARE” (or whatever hive you’re trying to get the key back into, while leaving everything else alone.  If you were clever about the hive name you selected, this can be a global search-and-replace.

Save the file, then use File->Import to bring the key(s) back in to the actual registry.  When you’re done with this procedure, click on your hive with the crazy name, and select File->Unload Hive to be rid of it.  There’s no “save” in the registry editor, so you’re done at this point.

Share

Backing Up Open Files on Windows with Rsync (and BackupPC)

Update:

Versions of the files below may be downloaded here.  This post is probably still useful as documentation.

 


 

This isn’t specific to BackupPC by any means, but I’ll preface this with a brief explanation:  BackupPC is a “set it and forget it” backup system driven from the server, that allows you to back up the entire network of *nix and Windows PCs.  It doesn’t require any software on the systems it backs up at all, since it relies upon rsync and smbclient, and optionally ssh.

For *nix, this works beautifully.  For Windows, this also works beautifully, except that “open files” can’t be backed up at all.  This problem isn’t unique to BackupPC, any attempt to back up or copy these files will fail, so most commercial backup systems have special “open file” clients to cope with it.

The official Windows solution for XP and later is something called a “volume shadow copy.”  It’s probably far more complex than it possibly needs to be, but essentially, it creates a pseudo-volume for any actual volume, with the difference being that you can actually back up files on it.  So, this can be handily used for rsync in order to make full backups, including every single file…  in theory, anyway.

My goals in getting this working:

  1. The solution should work with off-the-shelf components (i.e., no binaries or code)
  2. Installation and footprint should be minimal
  3. It should “just work” — if it’s too delicate, it’s not all that useful as a backup solution

It took quite a bit of trial-and-error, so I’ll skip what didn’t work, and get straight to what actually does work.  There are a few required components:

  1. winexe, a *nix program for remotely executing commands on Windows systems
  2. vshadow, a Windows program that creates and manages shadow copies
  3. dosdev, a Windows program that maps drive letters to volumes
  4. cwrsync, a Windows version of rsync (the “server” isn’t necessary)

Once all the pieces are assembled, I created a C:\BackupPC directory on the Windows box with all the necessary files.  Note that rsync does not need to be installed as a service, it actually gets loaded on-the-fly.  (Note that this directory is hard-coded in a lot of the files.) Here’s a listing of that directory:

Directory of C:\BackupPC
08/08/2008  07:11 PM                65 backuppc.cmd
08/10/2008  12:56 PM             1,928 cwrsync.cmd
07/22/2008  04:30 PM         1,082,368 cygcrypto-0.9.8.dll
04/11/2008  07:03 AM           999,424 cygiconv-2.dll
04/11/2008  07:03 AM            31,744 cygintl-3.dll
04/11/2008  07:03 AM            20,480 cygminires.dll
07/22/2008  04:30 PM         1,872,884 cygwin1.dll
04/11/2008  07:03 AM            66,048 cygz.dll
09/28/2004  02:07 PM             6,656 dosdev.exe
08/11/2008  11:08 PM             1,000 pre-cmd.vbs
08/11/2008  11:05 PM                44 pre-exec.cmd
07/22/2008  02:26 PM           348,160 rsync.exe
08/11/2008  10:12 PM               161 rsyncd.conf
08/11/2008  10:12 PM                22 rsyncd.secrets
08/11/2008  11:26 PM             1,177 sleep.vbs
06/08/2005  03:17 PM           294,912 vshadow.exe
08/11/2008  10:09 PM               581 vsrsync.cmd
08/11/2008  11:33 PM               308 vss-setvar.cmd

So, here’s how it works.  Before each backup, BackupPC has an option to call a local script first, waiting for that script to finish.  Here’s the execution chain:

  1. preusercmd.sh launches “pre-exec.cmd” on the Windows box
  2. preexec.cmd launches “pre-cmd.vbs”
  3. pre-cmd.vbs cleans up some files, launches “sleep.vbs” in the background (more on this later) and then launches “backuppc.cmd” in the background, and waits for the pid file to appear that signals that rsyncd has been launched
  4. backuppc.cmd launches vshadow, and tells it to execute vsrsync.cmd
  5. vsrsync.cmd maps the shadow volume to B:, and launches rsyncd — it sits and waits here, leaving vshadow and rsync open while the backup or rsync process runs — on the shadow copy on B:

Once the backup is completed, another local script is run — here’s its execution chain:

  1. postusercmd.sh puts a file called “wake.up” in the C:\BackupPC directory
  2. sleep.vbs wakes up, sees this file, reads rsyncd.pid, and kills the rsyncd process
  3. vsrsync.cmd now continues, since the rsync process is dead.  It unmaps the B: drive.  Once this script completes, vshadow automatically deletes the shadow volume.

Sure, it seems simple, but a lot of work went into that, since there are a lot of nuances to sort out.  Here are the file listings:

preusercmd.sh

#!/bin/bash
WINEXE=/usr/bin/winexe
UNAME="Administrator"
PWD="admin.password"
WRKGRP="WORKGROUP"
BOX=$1
$WINEXE --interactive=0 -U $UNAME -W $WRKGRP --password=$PWD //$BOX 'cmd /c c:\backuppc\pre-exec.cmd'
sleep 5
echo "Rsync and shadow copy loaded"
kill $$
# The script needs to be killed, otherwise, winexe waits for input

pre-exec.cmd

cd \backuppc
@echo off
cscript pre-cmd.vbs

pre-cmd.vbs

Const Flag = "C:\BackupPC\rsyncd.pid"
'
' Pid file shouldn't be there already
'
If DoesFileExist(Flag)=0 Then
   Set fso = CreateObject("Scripting.FileSystemObject")
   Set aFile = fso.GetFile(Flag)
   aFile.Delete
End If
'
' Nor should "wake.up"
'
If DoesFileExist("C:\BackupPC\wake.up")=0 Then
   Set fso = CreateObject("Scripting.FileSystemObject")
   Set aFile = fso.GetFile("C:\BackupPC\wake.up")
   aFile.Delete
End If
'
Set objShell = CreateObject("WScript.Shell")
objShell.Exec "cscript C:\BackupPC\sleep.vbs"
'
Set objShell = CreateObject("WScript.Shell")
objShell.Exec "C:\BackupPC\backuppc.cmd > C:\BackupPC\file.out"
'
' Just sleep until the file "rsyncd.pid" appears
'
While DoesFileExist(Flag)
   wscript.sleep 10000
Wend
'
' functions
'
function DoesFileExist(FilePath)
Dim fso
	Set fso = CreateObject("Scripting.FileSystemObject")
	if not fso.FileExists(FilePath) then
		DoesFileExist = -1
	else
		DoesFileExist = 0
	end if
	Set fso = Nothing
end function

sleep.vbs

Const Rsync = "C:\BackupPC\rsyncd.pid"
Const Flag = "C:\BackupPC\wake.up"
'
' Just sleep until the file "rsyncd.pid" appears
'
While DoesFileExist(Rsync)
   wscript.sleep 10000
Wend
'
' Now sleep until the file "wake.up" appears
'
While DoesFileExist(Flag)
   wscript.sleep 10000
Wend
'
Set fso = CreateObject("Scripting.FileSystemObject")
Set aFile = fso.GetFile(Flag)
aFile.Delete
'
' It's time to kill Rsync
'
Set fso = CreateObject("Scripting.FileSystemObject")
Set aReadFile = fso.OpenTextFile(Rsync, 1)
strContents = aReadFile.ReadLine
aReadFile.Close
'
Set objShell = CreateObject("WScript.Shell")
objShell.Run "taskkill /f /pid " & strContents, 0, true
'
' Wait for Rsync to let go
'
wscript.sleep 5000
'
' Delete PID file
'
If DoesFileExist(Rsync)=0 Then
   Set objShell = CreateObject("WScript.Shell")
   objShell.Run "cmd /c del C:\BackupPC\rsyncd.pid", 0, true
End If
'
' functions
'
function DoesFileExist(FilePath)
Dim fso
	Set fso = CreateObject("Scripting.FileSystemObject")
	if not fso.FileExists(FilePath) then
		DoesFileExist = -1
	else
		DoesFileExist = 0
	end if
	Set fso = Nothing
end function

backuppc.cmd

cd \backuppc
vshadow -script=vss-setvar.cmd -exec=vsrsync.cmd c:

vsrsync.cmd

REM @ECHO OFF
call vss-setvar.cmd
cd \BackupPC
SET CWRSYNCHOME=\BACKUPPC
SET CYGWIN=nontsec
SET CWOLDPATH=%PATH%
SET PATH=\BACKUPPC;%PATH%
dosdev B: %SHADOW_DEVICE_1%
REM Go into daemon mode, we'll kill it once we're done
rsync -v -v --daemon --config=rsyncd.conf --no-detach --log-file=diagnostic.txt
dosdev -r -d B:

rsyncd.conf

use chroot = false
strict modes = false
pid file = rsyncd.pid
[C]
path = /cygdrive/B/
auth users = Administrator
secrets file = rsyncd.secrets

postusercmd.sh

#!/bin/bash
WINEXE=/usr/bin/winexe
UNAME="Administrator"
PWD="admin.password"
WRKGRP="WORKGROUP"
BOX=$1
PID=$($WINEXE -U $UNAME -W $WRKGRP --password=$PWD //$BOX 'cmd /c echo '1' > c:\backuppc\wake.up')
echo "Rsync and shadow copy unloaded"
Share