Humor: Unusual data disaster horror stories for 2007

Posted on in Categories Data recovery, Hardware, Humor last updated December 7, 2007

Ontrack data recovery service has posted unusual data disaster horror stories for 2007. From the article:

An ant-infested hard drive and a failing parachute top a list of data disaster horror stories for 2007.

The list, provided by Kroll Inc.’s Ontrack Data Recovery unit, illustrates some of the strangest and wackiest things that people put electronic storage devices through on a regular basis.

Putting drives in the washing machine. Using oil to stop them from squeaking. These are just two examples of the user bloopers the company’s engineers nominated for inclusion on the list. Remarkably, Kroll data recovery specialists were able to recover the data in both instances.

=> Most unusual data disaster horror stories for 2007 [ via slashdot ]

Ubuntu Linux: Turn on 3D Compiz Eye Candy Effects for the X Window System

Posted on in Categories Data recovery, Gnome, Hardware, Howto, Linux, Linux desktop, Ubuntu Linux last updated October 18, 2007

Compiz brings to life a variety of visual effects that make the Linux desktop easier to use, more powerful and intuitive, and more accessible for users with special needs. It is an OpenGL-based compositing and window-manager. Compiz is the original compositing window manager from Novell’s XGL project. It is developed by David Reveman and community.

Compiz is one of the first compositing window managers for the X Window System that uses 3D graphics hardware to create fast compositing desktop effects for window management.

By default Compiz configuration settings manager is not installed under Ubuntu Linux 7.10. So first install compiz manager:
$ sudo apt-get install compizconfig-settings-manager

Please note that compiz only worked with 3D hardware which was supported by Xgl. Most NVIDIA and ATI graphics cards are known to work with Compiz on Xgl. On board Intel card (Intel GMA) also reported working with compiz.

Turn on 3D Compiz Effects

Right Click on Desktop > Change Desktop Background > Select Visual Effect Tab > Select Extra or Custom
Ubuntu Linux: Turn on 3D Compiz Effects for the X Window System
(click to enlarge image)

Save the changes.

How do I use or see the 3D effects?

Now everything is turned on, but how do you use it? Just hit the following key combinations to see effects:

  • ALT + TAB: Switch windows
  • Windows key + Tab: Switch windows
  • Ctrl + Alt + Left/Right Arrow: Switch desktops on cube
  • Ctrl + Alt + Left-click anywhere on wallpaper and drag
  • Try minimizing and maximizing windows
  • Try Dragging windows
  • Double click titlebar
  • Windows key+right-click Zoom-in once
  • Windows key + wheel mouse up : Zoom-in manually

To get idea about 3D effects, please see following youtube video (video may not work inside RSS reader, so click here to view the same):

Further readings:

Compiz Documentation

SSH: Rotate backup shell script to remove directories (old backup files)

Posted on in Categories Backup, Data recovery, Howto, RedHat/Fedora Linux, Security, Shell scripting, Sys admin, Tips, Ubuntu Linux, UNIX last updated October 9, 2007

Most time you have a limited space on the remote SFTP/ SSH backup server. Here is the script that periodically cleanup old backup files from the server i.e it will remove old directories.

Requirements

Script will automatically calculate date from today’s date. By default it will keep only last 7 days backup on server. You can easily increase / decrease this limit. In order to run script you must meet the following criteria:

  • Remote SSH server with rm command execution permission
  • SSH Keys for password less login (see how to setup RSA / DSA keys for password less login)
  • Accurate date and time on local system (see how to synchronize clock using ntpdate ntp client)
  • Remote backup directory must be in dd-mm-yyyy or mm-dd-yyyy format. For example daily mysql backup should be stored in /mysql/mm-dd-yyyy format.

Sample Script Usage

Run the script as follows:
./rot.backup.sh 7 /mysql "rm -rf"
Where,

  • 7 : Remove last 7 days files
  • /mysql : Base directory to clean up. If todays date is 9/Oct/2007, it will remove last 7 days directory /mysql/02-10-2007, /mysql/01-10-2007, …. /mysql/26-09-2007, /mysql/25-09-2007. It means script will only keep last 7 days backup on remote sftp / ssh server.
  • rm -rf : Command to run on directory structure

Sample Shell Script

Install following script:

#!/bin/bash
if [ "$#" == "0" ];then
  echo "$0 upper-limit path {command}"
  exit 1
fi
### SSH Server setup ###
SSH_USER="vivek"
SSH_SERVER="nas.nixcraft.in"
START=7
DIR_FORMAT="%d-%m-%Y" # DD-MM-YYYY format
#DIR_FORMAT="%m-%d-%Y" #MM-DD-YYYY format
## do not edit below ##
LIMIT=$( expr $START + $1 )

## default CMD ##
CMD="ls"
SSH_PATH="."

[ "$3" != "" ] && CMD="$3" || :
[ "$2" != "" ] && SSH_PATH="$2" || :

DAYS=$(for d in $(seq $START $LIMIT);do date --date="$d days ago" +"${DIR_FORMAT}"; done)
for d in $DAYS
do
  ssh ${SSH_USER}@${SSH_SERVER} ${CMD} ${SSH_PATH}/$d
done

Run above script via cron tab (cronjob):
@daily /path/to/rot.ssh.script 7 "/html" "rm -rf"
@daily /path/to/rot.ssh.script 7 "/mysql" "rm -rf"

Copy hard disk or partition image to another system using a network and netcat (nc)

Posted on in Categories Backup, CentOS, Data recovery, Debian Linux, File system, FreeBSD, Gentoo Linux, Howto, RedHat/Fedora Linux, Suse Linux, Sys admin, Tips, Ubuntu Linux last updated August 12, 2007

netcat utility (nc command) considered as TCP/IP swiss army knife. It reads and writes data across network connections, using TCP or UDP protocol. It is designed to be a reliable “back-end” tool that can be used directly or easily driven by other programs and scripts. At the same time, it is a feature-rich network debugging and exploration tool, since it can create almost any kind of connection you would need and has several interesting built-in capabilities.

I also install the netcat package for administering a network and you’d like to use its debugging and network exploration capabilities.

One my favorite usage is to migrating data between two server hard drives using netcat over a network. It is very easy to copy complete drive image from one server to another.

You can also use ssh for the same purpose, but encryption adds its own overheads. This is tried and trusted method (hat tip to karl) .

Make sure you have backup of all important data.

Install netcat

It is possible that nc may not be installed by default under Redhat / CentOS / Debian Linux.

Insall nc under Redhat / CentOS / Fedora Linux

Use yum command as follows:
# yum install nc
Output:

Loading "installonlyn" plugin
Loading "rhnplugin" plugin
Setting up Install Process
Setting up repositories
rhel-x86_64-server-vt-5   100% |=========================| 1.2 kB    00:00
rhel-x86_64-server-5      100% |=========================| 1.2 kB    00:00
Reading repository metadata in from local files
Parsing package install arguments
Resolving Dependencies
--> Populating transaction set with selected packages. Please wait.
---> Downloading header for nc to pack into transaction set.
nc-1.84-10.fc6.x86_64.rpm 100% |=========================| 6.9 kB    00:00
---> Package nc.x86_64 0:1.84-10.fc6 set to be updated
--> Running transaction check

Dependencies Resolved

=============================================================================
 Package                 Arch       Version          Repository        Size
=============================================================================
Installing:
 nc                      x86_64     1.84-10.fc6      rhel-x86_64-server-5   56 k

Transaction Summary
=============================================================================
Install      1 Package(s)
Update       0 Package(s)
Remove       0 Package(s)

Total download size: 56 k
Is this ok [y/N]: y
Downloading Packages:
(1/1): nc-1.84-10.fc6.x86 100% |=========================|  56 kB    00:00
Running Transaction Test
Finished Transaction Test
Transaction Test Succeeded
Running Transaction
  Installing: nc                           ######################### [1/1]

Installed: nc.x86_64 0:1.84-10.fc6
Complete!

Debian / Ubuntu Linux netcat installation

Simply use apt-get command:
$ sudo apt-get install netcat

WARNING! These examples may result into data loss, ensure there are good backups before doing this, as using command wrong way can be dangerous.

How do I use netcat to copy hard disk image?

Our sample setup

-----------------------
HostA // 192.168.1.1
------------------------
           sda
        NETWORK
           sdb
------------------------
HostB // 192.168.1.2
-------------------------

Your task is copy HostA /dev/sda to HostB’s /dev/sdb using netcat command. First login as root user

Command to type on hostB (receiving end ~ write image mode)

You need to open port on hostB using netcat, enter :
# netcat -p 2222 -l |bzip2 -d | dd of=/dev/sdb
Where,

  • -p 2222 : Specifies the source port nc should use, subject to privilege restrictions and availability. Make sure port 2222 is not used by another process.
  • -l : Used to specify that nc should listen for an incoming connection rather than initiate a connection to a remote host.
  • bzip2 -d : Compresses image using the Burrows-Wheeler block sorting text compression algorithm, and Huffman coding. This will speed up network transfer ( -d : force decompression mode)
  • dd of=/dev/sda : /dev/sda is your hard disk. You can also specify partition such as /dev/sda1

Command to type on hostA (send data over a network ~ read image mode)

Now all you have to do is start copying image. Again login as root and enter:
# bzip2 -c /dev/sda | netcat hostA 2222
OR use IP address:
# bzip2 -c /dev/sda | netcat 192.168.1.1 2222

This process takes its own time.

A note about latest netcat version 1.84-10 and above

If you are using latest nc / netcat version above syntax will generate an error. It is an error to use -l option in conjunction with the -p, -s, or -z options. Additionally, any timeouts specified with the -w option are ignored. So use nc command as follows.

On hostA, enter:
# nc -l 2222 > /dev/sdb
On hostB, enter:
# nc hostA 2222< /dev/sda
OR
# nc 192.168.1.1 2222< /dev/sda

Using a second machine (hostB), connect to the listening nc process at 2222 (hostA), feeding it the file (/dev/sda)which is to be transferred. You can use bzip2 as follows.
On hostA, enter:
# nc -l 2222 | bzip2 -d > /dev/sdb
On hostB, enter:
# bzip2 -c /dev/sda | nc 192.168.1.1 2222

Further readings

How do I improve performance?

As suggested by anonymous user:

You should definitely use bs=16M or something like that. Otherwise, the copy will take forever. Copying a 300 GB hard drive over a 1 Gbps cross-over cable took about 1 1/2 hours or so using bs=16M Without this option, the same thing would have taken about 7 hours.

In short use command as follows:
# netcat -p 2222 -l |bzip2 -d | dd of=/dev/sdb bs=16M

Updated for accuracy.

Download of the day: AcetoneISO – extract, browse ISO and other CD/DVD formats under Linux

Posted on in Categories Data recovery, Debian Linux, Download of the day, File system, Gentoo Linux, GNU/Open source, Howto, Linux, Linux desktop, RedHat/Fedora Linux, Suse Linux last updated April 18, 2007

AcetoneISO is the disk image emulator that mounts images of DVD and CD media. Both Mac OS X and Linux / other UNIX like oses can mount and use ISO images using loopback device. It is a DAEMON Tools (Microsoft Windows disk image) clone / emulator program with a lot more features.

Using this cool open source software means a user does not have to swap discs to run different programs on local or network computer. You can access software distributed (over Internet) as a disk image such as ISO, DAA, BIN or many other formats (no need to burn a CD/DVD to use disk image). Other usage:

  • Prevent scratching, which can cause permanent damage to a disc
  • Speeds up access times as hard drives are faster than optical drives
  • Provides a backup copy of a disc, in case the original becomes damaged, lost, or stolen

Features:

  • Mount and Unmount ISO, MDF, NRG (if iso-9660 standard)
  • Convert BIN/CUE, MDF, NRG, CCD/IMG, CDI, XBOX, B5I/BWI, PDI, DAA to ISO
  • Burn Your ISO, CUE, TOC images directly in K3b
  • Blank Your CD/DVD ReWritable
  • Verify md5sum of image files and Generate a Md5sum file from ISO
  • Ability to create ISO from Folder and CD/DVD
  • Service-Menu support
  • Play a DVD-Movie ISO with Kaffeine, Mplayer, VLC, Kmplayer
  • Split ISOs in smaller files and Merge them
  • Quick Turbo Mount an ISO file from your Desktop
  • Compress ISO with p7zip and extract
  • Encrypt and Decrypt an ISO
  • Generate a CUE file from a IMG/BIN image
  • Rip a PSX cd to a bin/toc image

AcetoneISO has only one dependencies problem – Kommander. Make sure you have Kommander installed.

Step # 1: Install kommander

Use apt-get command to install kommander ( it consists of an editor and a program executor that produce dialogs that you can execute), which is required by AcetoneISO. You also need p7zip (a file archiver with highest compression ratio) to compress and extract ISO images. Use apt-get command under Debian or Ubuntu Linux as follows:
# apt-get install kommander p7zip
OR
$ sudo apt-get install kommander p7zip

Step # 2: Install AcetoneISO

Download source code or Debian .deb or Suse/Redhat RPM file from official website. Use apt-get / rpm command. Use apt-get command to install .deb file:
# apt-get install AcetoneISO-6.7.deb
OR use rpm package for RPM based distro:
# rpm -ivh AcetoneISO-6.7.noarch.rpm

Step # 2: Start AcetoneISO program

Simply type the command or click on Application > Accessories > AcetoneISO:
$ acetoneiso &
AcetoneISO - extract, browse ISO and other CD/DVD formats under Linux

References:

Linux recover deleted files with lsof command – howto

Posted on in Categories Backup, Data recovery, Linux, RedHat/Fedora Linux, Suse Linux, Ubuntu Linux last updated November 17, 2006

Almost 2 years back I wrote about recovering deleted text file with grep command under UNIX or Linux.

Michael Stutz shows us how to recover deleted files using lsof command.

From the article:
There you are, happily playing around with an audio file you’ve spent all afternoon tweaking, and you’re thinking, “Wow, doesn’t it sound great? Lemme just move it over here.” At that point your subconscious chimes in, “Um, you meant mv, not rm, right?” Oops. I feel your pain — this happens to everyone. But there’s a straightforward method to recover your lost file, and since it works on every standard Linux system, everyone ought to know how to do it.

Briefly, a file as it appears somewhere on a Linux filesystem is actually just a link to an inode, which contains all of the file’s properties, such as permissions and ownership, as well as the addresses of the data blocks where the file’s content is stored on disk. When you rm a file, you’re removing the link that points to its inode, but not the inode itself; other processes (such as your audio player) might still have it open. It’s only after they’re through and all links are removed that an inode and the data blocks it pointed to are made available for writing.

This delay is your key to a quick and happy recovery: if a process still has the file open, the data’s there somewhere, even though according to the directory listing the file already appears to be gone.

Read more at Linux.com

However recovering files under Linux is still hard work for new admins. I highly recommend backing up files regularly and storing backup offsite.

True Tales of Data Disaster and Remarkable Feats of Data Recovery

Posted on in Categories Backup, Data recovery, News last updated November 16, 2006

BBC news has published top 10 data disasters. Hard drives kept in dirty socks and the dangers of oiling your PC feature in a top 10 list of data disasters.

Ontrack Data Recovery has unveiled its annual Top Ten list of remarkable data loss disasters in 2006. Taken from a global poll of Ontrack data recovery experts, this year’s list of data disasters is even more incredible when you consider that in every case cited, Ontrack successfully recovered the data.

It is recommended that you always follow these backup tips.

Use amanda to backup your Linux server – Howto

Posted on in Categories Backup, Data recovery, Howto, Tips last updated November 4, 2006

Amanda is one the best open source Linux backup software.

The Advanced Maryland Automatic Network Disk Archiver (AMANDA), is a backup system that allows the administrator to set up a single master backup server to back up multiple hosts over network to tape drives/changers or disks or optical media.

Novell Cool Solutions has published a small how to about Amanda. From the article:
Amanda is simple to use but tricky to configure. It is worth sticking with Amanda until you get a fully working system. Don’t be put off by the lack of a slick GUI interface or the need to configure the software via a console shell, once it is configured Amanda does its job very efficiently.

The configuration files for Amanda are stored in the /etc/amanda directory. Each configuration that you create goes into a separate directory within /etc/amanda. There is a sample configuration in the “example” directory that you can use as a jumping off point for configuring Amanda

Contents

* Background
* Installing Amanda
* Configuring Amanda
* Creating file and directories
* Preparing tapes for use with Amanda
* Checking the configuration
* Performing the backup
* Checking the backup
* Automating the backup
* Conclusions

Read more, Using Amanda to Backup Your Linux Server at Novell.com

How Linux Live CD could save your life

Posted on in Categories Backup, Data recovery, Linux last updated October 26, 2006

Yet another reason to carry a Live Linux cd 🙂

From the article:
Everyone’s worst nightmare; the normal comforting hum of your computer is disturbed by clicking, pranging, banging…

It happens to everyone because it’s inevitable (hard drives are mechanical, as sure as a car will break down your hard drive will fail eventually). However, no matter how often you see it you never quite get used to it happening, the heartache of all the files you lose forever because you were “just about to back it up, honestly”. This is not a matter of explaining to you how you can best avoid data loss or how to protect against your hard drive dying, this is an article outlining how Ubuntu (or any other LiveCD available distro) could save your life