≡ Menu

Data recovery

Linux Create Incremental Backups Using FlyBack GUI Time Machine Backup Software

This is an excellent backup utility for new Linux user.

Apple's Time Machine is a great feature in their OS, and Linux has almost all of the required technology already built in to recreate it. This is a simple GUI to make it easy to use.

Time Machine, like many backup utilities, creates incremental backups of files which can be restored at a later date. It also supports limited restoration of files within applications that are specifically programmed to use Time Machine's functionality.
FlyBack: a Time Machine backup utility for Linux
(Fig. 01: Linux FlyBack Software in Action)

Download and Installation instructions

=> FlyBack - Apple's Time Machine for Linux (via Bernaz's Weblog)

Clonezilla Ghost Like Linux Partition or Disk Clone Software

You can easily clone single hard disk or partition using netcat and your own network. However, software such as Clonezilla offers a partition or disk clone software similar to Ghost. It saves and restores only used blocks in hard drive.

Norton Ghost is popular proprietary commercial software. It is slow and takes lots of time to clone system. There is also Symantec Ghost Corporate Edition with multicasting and fast cloning system.

However, Clonezilla is a free, open source, multitasking and multi operating system software. With DRBL (Diskless Remote Boot in Linux [DRBL] provides a diskless or systemless environment for client machines) and network boot enabled client computers, the only thing you have to prepare is a Clonezilla server. The best thing is ~ you do not even have to prepare a bootable CD or floppy with Partition Image for every client computer.

Clonezilla Ghose like Linux partition or disk clone software in action
(Fig 01: Starting Clonezilla Clone Software)

Clonezilla Ghost like Linux partition or disk clone software
(Fig 02: Clonezilla and DRBL running under Ububtu Linux ~ click to enlarge image)

Download Clonezilla cloning software

=> Visit official project home page to download Clonezilla software.

How to: Setting up a Clonezilla/DRBL server on Ubuntu 7.04

=> A simple howto on setting up Clonezilla/DRBL on Ubuntu Linux

How to: Setting up a Clonezilla/DRBL server on CentOS / Fedora / RHEL

=> A simple documentation to quickly setup up Clonezilla on a PXE server.

Related tutorials / software:

How to Read Ext2 / Ext3 File System From Windows Computer

This software is must if you dual boot between Linux and Windows laptop / desktop computer.

I've already written about Explore2fs and other programs to grant read and write access to Linux ext3 partitions / files from a Windows box. There is a new program called Linux Reader which allows safe and quick access to alternative file systems. This program plays the role of a bridge between your Windows and Ext2/Ext3 Linux file systems. This easy-to-use tool runs under Windows and allows you to browse Ext2/Ext3 Linux file systems and extract files from there. From the project home page:

First of all, DiskInternals Linux Reader is absolutely free. Secondly, the program provides for read-only access and does not allow you to make records in Ext2/Ext3 file system partitions. This guarantees that the interference in an alterative file system will not affect the work of Linux later. Apart from this, it is necessary to note, that it gives you an opportunity to use common Windows Explorer for extracting data. A preview option for pictures is one more pleasant point, which is worth mentioning.

How to Read Ext2 / Ext3 File System From Windows Computer
(Fig 01: Linux Reader in Action under Windows XP [ image credit diskinternals.com ])

Download Linux Reader

=> Download Linux Reader [diskinternals.com]

Humor: Unusual data disaster horror stories for 2007

Ontrack data recovery service has posted unusual data disaster horror stories for 2007. From the article:

An ant-infested hard drive and a failing parachute top a list of data disaster horror stories for 2007.

The list, provided by Kroll Inc.'s Ontrack Data Recovery unit, illustrates some of the strangest and wackiest things that people put electronic storage devices through on a regular basis.

Putting drives in the washing machine. Using oil to stop them from squeaking. These are just two examples of the user bloopers the company's engineers nominated for inclusion on the list. Remarkably, Kroll data recovery specialists were able to recover the data in both instances.

=> Most unusual data disaster horror stories for 2007 [ via slashdot ]

Ubuntu Linux: Turn on 3D Compiz Eye Candy Effects for the X Window System

Compiz brings to life a variety of visual effects that make the Linux desktop easier to use, more powerful and intuitive, and more accessible for users with special needs. It is an OpenGL-based compositing and window-manager. Compiz is the original compositing window manager from Novell's XGL project. It is developed by David Reveman and community.

Compiz is one of the first compositing window managers for the X Window System that uses 3D graphics hardware to create fast compositing desktop effects for window management.

By default Compiz configuration settings manager is not installed under Ubuntu Linux 7.10. So first install compiz manager:
$ sudo apt-get install compizconfig-settings-manager

Please note that compiz only worked with 3D hardware which was supported by Xgl. Most NVIDIA and ATI graphics cards are known to work with Compiz on Xgl. On board Intel card (Intel GMA) also reported working with compiz.

Turn on 3D Compiz Effects

Right Click on Desktop > Change Desktop Background > Select Visual Effect Tab > Select Extra or Custom
Ubuntu Linux: Turn on 3D Compiz Effects for the X Window System
(click to enlarge image)

Save the changes.

How do I use or see the 3D effects?

Now everything is turned on, but how do you use it? Just hit the following key combinations to see effects:

  • ALT + TAB: Switch windows
  • Windows key + Tab: Switch windows
  • Ctrl + Alt + Left/Right Arrow: Switch desktops on cube
  • Ctrl + Alt + Left-click anywhere on wallpaper and drag
  • Try minimizing and maximizing windows
  • Try Dragging windows
  • Double click titlebar
  • Windows key+right-click Zoom-in once
  • Windows key + wheel mouse up : Zoom-in manually

To get idea about 3D effects, please see following youtube video (video may not work inside RSS reader, so click here to view the same):

Further readings:

Compiz Documentation

SSH: Rotate backup shell script to remove directories (old backup files)

Most time you have a limited space on the remote SFTP/ SSH backup server. Here is the script that periodically cleanup old backup files from the server i.e it will remove old directories.


Script will automatically calculate date from today's date. By default it will keep only last 7 days backup on server. You can easily increase / decrease this limit. In order to run script you must meet the following criteria:

  • Remote SSH server with rm command execution permission
  • SSH Keys for password less login (see how to setup RSA / DSA keys for password less login)
  • Accurate date and time on local system (see how to synchronize clock using ntpdate ntp client)
  • Remote backup directory must be in dd-mm-yyyy or mm-dd-yyyy format. For example daily mysql backup should be stored in /mysql/mm-dd-yyyy format.

Sample Script Usage

Run the script as follows:
./rot.backup.sh 7 /mysql "rm -rf"

  • 7 : Remove last 7 days files
  • /mysql : Base directory to clean up. If todays date is 9/Oct/2007, it will remove last 7 days directory /mysql/02-10-2007, /mysql/01-10-2007, .... /mysql/26-09-2007, /mysql/25-09-2007. It means script will only keep last 7 days backup on remote sftp / ssh server.
  • rm -rf : Command to run on directory structure

Sample Shell Script

Install following script:

if [ "$#" == "0" ];then
  echo "$0 upper-limit path {command}"
  exit 1
### SSH Server setup ###
DIR_FORMAT="%d-%m-%Y" # DD-MM-YYYY format
#DIR_FORMAT="%m-%d-%Y" #MM-DD-YYYY format
## do not edit below ##
LIMIT=$( expr $START + $1 )
## default CMD ##
[ "$3" != "" ] && CMD="$3" || :
[ "$2" != "" ] && SSH_PATH="$2" || :
DAYS=$(for d in $(seq $START $LIMIT);do date --date="$d days ago" +"${DIR_FORMAT}"; done)
for d in $DAYS
  ssh ${SSH_USER}@${SSH_SERVER} ${CMD} ${SSH_PATH}/$d

Run above script via cron tab (cronjob):
@daily /path/to/rot.ssh.script 7 "/html" "rm -rf"
@daily /path/to/rot.ssh.script 7 "/mysql" "rm -rf"

Copy hard disk or partition image to another system using a network and netcat (nc)

netcat utility (nc command) considered as TCP/IP swiss army knife. It reads and writes data across network connections, using TCP or UDP protocol. It is designed to be a reliable "back-end" tool that can be used directly or easily driven by other programs and scripts. At the same time, it is a feature-rich network debugging and exploration tool, since it can create almost any kind of connection you would need and has several interesting built-in capabilities.

I also install the netcat package for administering a network and you'd like to use its debugging and network exploration capabilities.

One my favorite usage is to migrating data between two server hard drives using netcat over a network. It is very easy to copy complete drive image from one server to another.

You can also use ssh for the same purpose, but encryption adds its own overheads. This is tried and trusted method (hat tip to karl) .

Make sure you have backup of all important data.

Install netcat

It is possible that nc may not be installed by default under Redhat / CentOS / Debian Linux.

Insall nc under Redhat / CentOS / Fedora Linux

Use yum command as follows:
# yum install nc

Loading "installonlyn" plugin
Loading "rhnplugin" plugin
Setting up Install Process
Setting up repositories
rhel-x86_64-server-vt-5   100% |=========================| 1.2 kB    00:00
rhel-x86_64-server-5      100% |=========================| 1.2 kB    00:00
Reading repository metadata in from local files
Parsing package install arguments
Resolving Dependencies
--> Populating transaction set with selected packages. Please wait.
---> Downloading header for nc to pack into transaction set.
nc-1.84-10.fc6.x86_64.rpm 100% |=========================| 6.9 kB    00:00
---> Package nc.x86_64 0:1.84-10.fc6 set to be updated
--> Running transaction check
Dependencies Resolved
 Package                 Arch       Version          Repository        Size
 nc                      x86_64     1.84-10.fc6      rhel-x86_64-server-5   56 k
Transaction Summary
Install      1 Package(s)
Update       0 Package(s)
Remove       0 Package(s)
Total download size: 56 k
Is this ok [y/N]: y
Downloading Packages:
(1/1): nc-1.84-10.fc6.x86 100% |=========================|  56 kB    00:00
Running Transaction Test
Finished Transaction Test
Transaction Test Succeeded
Running Transaction
  Installing: nc                           ######################### [1/1]
Installed: nc.x86_64 0:1.84-10.fc6

Debian / Ubuntu Linux netcat installation

Simply use apt-get command:
$ sudo apt-get install netcat

WARNING! These examples may result into data loss, ensure there are good backups before doing this, as using command wrong way can be dangerous.

How do I use netcat to copy hard disk image?

Our sample setup

HostA //
HostB //

Your task is copy HostA /dev/sda to HostB's /dev/sdb using netcat command. First login as root user

Command to type on hostB (receiving end ~ write image mode)

You need to open port on hostB using netcat, enter :
# netcat -p 2222 -l |bzip2 -d | dd of=/dev/sdb

  • -p 2222 : Specifies the source port nc should use, subject to privilege restrictions and availability. Make sure port 2222 is not used by another process.
  • -l : Used to specify that nc should listen for an incoming connection rather than initiate a connection to a remote host.
  • bzip2 -d : Compresses image using the Burrows-Wheeler block sorting text compression algorithm, and Huffman coding. This will speed up network transfer ( -d : force decompression mode)
  • dd of=/dev/sda : /dev/sda is your hard disk. You can also specify partition such as /dev/sda1

Command to type on hostA (send data over a network ~ read image mode)

Now all you have to do is start copying image. Again login as root and enter:
# bzip2 -c /dev/sda | netcat hostA 2222
OR use IP address:
# bzip2 -c /dev/sda | netcat 2222

This process takes its own time.

A note about latest netcat version 1.84-10 and above

If you are using latest nc / netcat version above syntax will generate an error. It is an error to use -l option in conjunction with the -p, -s, or -z options. Additionally, any timeouts specified with the -w option are ignored. So use nc command as follows.

On hostA, enter:
# nc -l 2222 > /dev/sdb
On hostB, enter:
# nc hostA 2222< /dev/sda
# nc 2222< /dev/sda

Using a second machine (hostB), connect to the listening nc process at 2222 (hostA), feeding it the file (/dev/sda)which is to be transferred. You can use bzip2 as follows.
On hostA, enter:
# nc -l 2222 | bzip2 -d > /dev/sdb
On hostB, enter:
# bzip2 -c /dev/sda | nc 2222

Further readings

How do I improve performance?

As suggested by anonymous user:

You should definitely use bs=16M or something like that. Otherwise, the copy will take forever. Copying a 300 GB hard drive over a 1 Gbps cross-over cable took about 1 1/2 hours or so using bs=16M Without this option, the same thing would have taken about 7 hours.

In short use command as follows:
# netcat -p 2222 -l |bzip2 -d | dd of=/dev/sdb bs=16M

Updated for accuracy.