Copy hard disk or partition image to another system using a network and netcat (nc)

by on August 12, 2007 · 34 comments· LAST UPDATED December 19, 2007

in , ,

netcat utility (nc command) considered as TCP/IP swiss army knife. It reads and writes data across network connections, using TCP or UDP protocol. It is designed to be a reliable "back-end" tool that can be used directly or easily driven by other programs and scripts. At the same time, it is a feature-rich network debugging and exploration tool, since it can create almost any kind of connection you would need and has several interesting built-in capabilities.

I also install the netcat package for administering a network and you'd like to use its debugging and network exploration capabilities.

One my favorite usage is to migrating data between two server hard drives using netcat over a network. It is very easy to copy complete drive image from one server to another.

You can also use ssh for the same purpose, but encryption adds its own overheads. This is tried and trusted method (hat tip to karl) .

Make sure you have backup of all important data.

Install netcat

It is possible that nc may not be installed by default under Redhat / CentOS / Debian Linux.

Insall nc under Redhat / CentOS / Fedora Linux

Use yum command as follows:
# yum install nc
Output:

Loading "installonlyn" plugin
Loading "rhnplugin" plugin
Setting up Install Process
Setting up repositories
rhel-x86_64-server-vt-5   100% |=========================| 1.2 kB    00:00
rhel-x86_64-server-5      100% |=========================| 1.2 kB    00:00
Reading repository metadata in from local files
Parsing package install arguments
Resolving Dependencies
--> Populating transaction set with selected packages. Please wait.
---> Downloading header for nc to pack into transaction set.
nc-1.84-10.fc6.x86_64.rpm 100% |=========================| 6.9 kB    00:00
---> Package nc.x86_64 0:1.84-10.fc6 set to be updated
--> Running transaction check
Dependencies Resolved
=============================================================================
 Package                 Arch       Version          Repository        Size
=============================================================================
Installing:
 nc                      x86_64     1.84-10.fc6      rhel-x86_64-server-5   56 k
Transaction Summary
=============================================================================
Install      1 Package(s)
Update       0 Package(s)
Remove       0 Package(s)
Total download size: 56 k
Is this ok [y/N]: y
Downloading Packages:
(1/1): nc-1.84-10.fc6.x86 100% |=========================|  56 kB    00:00
Running Transaction Test
Finished Transaction Test
Transaction Test Succeeded
Running Transaction
  Installing: nc                           ######################### [1/1]
Installed: nc.x86_64 0:1.84-10.fc6
Complete!

Debian / Ubuntu Linux netcat installation

Simply use apt-get command:
$ sudo apt-get install netcat

WARNING! These examples may result into data loss, ensure there are good backups before doing this, as using command wrong way can be dangerous.

How do I use netcat to copy hard disk image?

Our sample setup

-----------------------
HostA // 192.168.1.1
------------------------
           sda
        NETWORK
           sdb
------------------------
HostB // 192.168.1.2
-------------------------

Your task is copy HostA /dev/sda to HostB's /dev/sdb using netcat command. First login as root user

Command to type on hostB (receiving end ~ write image mode)

You need to open port on hostB using netcat, enter :
# netcat -p 2222 -l |bzip2 -d | dd of=/dev/sdb
Where,

  • -p 2222 : Specifies the source port nc should use, subject to privilege restrictions and availability. Make sure port 2222 is not used by another process.
  • -l : Used to specify that nc should listen for an incoming connection rather than initiate a connection to a remote host.
  • bzip2 -d : Compresses image using the Burrows-Wheeler block sorting text compression algorithm, and Huffman coding. This will speed up network transfer ( -d : force decompression mode)
  • dd of=/dev/sda : /dev/sda is your hard disk. You can also specify partition such as /dev/sda1

Command to type on hostA (send data over a network ~ read image mode)

Now all you have to do is start copying image. Again login as root and enter:
# bzip2 -c /dev/sda | netcat hostA 2222
OR use IP address:
# bzip2 -c /dev/sda | netcat 192.168.1.1 2222

This process takes its own time.

A note about latest netcat version 1.84-10 and above

If you are using latest nc / netcat version above syntax will generate an error. It is an error to use -l option in conjunction with the -p, -s, or -z options. Additionally, any timeouts specified with the -w option are ignored. So use nc command as follows.

On hostA, enter:
# nc -l 2222 > /dev/sdb
On hostB, enter:
# nc hostA 2222< /dev/sda
OR
# nc 192.168.1.1 2222< /dev/sda

Using a second machine (hostB), connect to the listening nc process at 2222 (hostA), feeding it the file (/dev/sda)which is to be transferred. You can use bzip2 as follows.
On hostA, enter:
# nc -l 2222 | bzip2 -d > /dev/sdb
On hostB, enter:
# bzip2 -c /dev/sda | nc 192.168.1.1 2222

Further readings

How do I improve performance?

As suggested by anonymous user:

You should definitely use bs=16M or something like that. Otherwise, the copy will take forever. Copying a 300 GB hard drive over a 1 Gbps cross-over cable took about 1 1/2 hours or so using bs=16M Without this option, the same thing would have taken about 7 hours.

In short use command as follows:
# netcat -p 2222 -l |bzip2 -d | dd of=/dev/sdb bs=16M

Updated for accuracy.

TwitterFacebookGoogle+PDF versionFound an error/typo on this page? Help us!

{ 34 comments… read them below or add one }

1 miljan August 12, 2007 at 11:44 am

Netcat man page says that -l can not be used with -p switch.

-p source_port
Specifies the source port nc should use, subject to privilege restrictions and availability. It is an error to
use this option in conjunction with the -l option.

Reply

2 nixCraft August 12, 2007 at 12:41 pm

miljan,

I don’t have any problem using -p 2222 with -l option. My netcat version is 1.10-32 and I don’t see any such message in man page. Can you specify your netcat version? The post has been updated for latest version. Thanks for quick feedback.

Reply

3 miljan August 12, 2007 at 1:25 pm

This is strange because it seems to be the same version as from your post.

[root@r2d2 ~]# nc -p 2222 -l
usage: nc [-46DdhklnrStUuvzC] [-i interval] [-p source_port]
[-s source_ip_address] [-T ToS] [-w timeout] [-X proxy_version]
[-x proxy_address[:port]] [hostname] [port[s]]

[root@r2d2 ~]# yum list installed nc
Installed Packages
nc.i386 1.84-10.fc6 installed

But anyway, in case someone else has the same problem, -l is enough for netcat to start listening on a desired port.

Reply

4 lefty.crupps August 13, 2007 at 12:51 pm

try putting the port at the end, like this:

nc -l -v -p 2222

Reply

5 Anonymous August 13, 2007 at 1:52 pm

You should definitely use bs=16M or something like that. Otherwise, the copy will take forever. Copying a 300 GB hard drive over a 1 Gbps cross-over cable took about 1 1/2 hours or so using bs=16M Without this option, the same thing would have taken about 7 hours

Reply

6 Karl O. Pinc August 13, 2007 at 2:52 pm

You forgot to mention _why_ someone would want to use netcat to do these things, instead of just plain old ssh which is easier because it can be done in one command from one system. You’d want it when you don’t want the overhead of ssh, and don’t need encryption. (It’s also good to have articles like this because netcat is so useful and if you’ve another problem it’s nice to start with something that you know works.)

Reply

7 foobar August 13, 2007 at 3:41 pm

this is basically copying a compressed partition or disk over the network, unencrypted. you may consider using ssh for safety reasons, e.g:

foo:# cat /dev/sda | bzip | ssh remotehost “cat > /backup/foo.sda.bz2″

Reply

8 Anuarag August 31, 2007 at 4:08 am

Thanks for this guide.

I am confused!! on the host A (i.e. source disk host) command to be typed is:
# bzip2 -c /dev/sda | netcat hostA 2222

Should it be hostA or hostB.

Similarly I am confused about the commands given for the new version of nc: It seems to me commands for host A and B exchanged?

Please correct me.

Thanks
Anurag

Reply

9 g4b January 28, 2008 at 3:01 pm

-l also takes a host as option, so if you really want to be sure, no one else just streams in your data you can add like
nc -l HostSource -p 2222…

also maybe update the whole thing to use “HostSource” and “HostDest” because HostA and HostB is a bit confusing maybe.

thanx for the tutorial, g4b

Reply

10 The Cog February 5, 2008 at 8:46 am

Useful hint, thanks.
There is some confusion betwen host A and B though. All the examples are copying from B to A, but the headings: “Command to type on host…” indicate you are copying from A to B.
Nit-picking, I know.

Reply

11 Ben March 28, 2008 at 2:08 pm

I don’t see any good about using ssh in local network and it would produce extra traffic. Here’s my contrib to backup and restore a linux parition:

# BACKUP – to backupserver (192.168.1.1)
nc -p 2222 -l | bzip2 -c > Image.bz2
# BACKUP – from clienthost (192.168.1.2)
dd if=/dev/sda bs=16M | nc 192.168.1.1 2222

# RESTORE – to clienthost (192.168.1.2)
nc -p 2222 -l > /dev/sda
# RESTORE – from backupserver (192.168.1.1)
dd if=Image.bz2 bs=16M | bunzip2 -c | nc 192.168.1.2 2222

Reply

12 Brian Trauger June 25, 2008 at 4:51 am

This looks very helpful. Can I also specify just 1 partition instead of the whole drive? I have a dual-boot with Xp and want to just backup the Windows partition.. Or should I just use partimage?

Reply

13 chains August 13, 2008 at 4:10 pm

I tried what Ben suggested:

# BACKUP – to backupserver (192.168.1.1)
nc -p 2222 -l | bzip2 -c > Image.bz2
# BACKUP – from clienthost (192.168.1.2)
dd if=/dev/sda bs=16M | nc 192.168.1.1 2222

# RESTORE – to clienthost (192.168.1.2)
nc -p 2222 -l > /dev/sda
# RESTORE – from backupserver (192.168.1.1)
dd if=Image.bz2 bs=16M | bunzip2 -c | nc 192.168.1.2 2222

but when restoring i’m getting the error:
“bunzip2: (stdin) is not a bzip2 file.”

Any suggestions?

Reply

14 S August 23, 2008 at 2:10 am

in response to #11(ben);

why would you first netcat the data uncompressed and only then compress it into an image?

Reply

15 Hannes September 4, 2008 at 8:30 pm

Thanks a lot for this article.
But like Anuarag said, “hostA” should be “hostB” here:

(on hostA, type) # bzip2 -c /dev/sda | netcat hostA 2222

We connect not to ourself, yes? :)

Reply

16 Ionut December 11, 2008 at 1:18 pm

That is correct Hannes!

Someone please make the corrections!

Reply

17 Florian January 6, 2009 at 1:42 am

The command works fine, but if you use netcat to make a diskdump of two directly connected machines, you should avoid the compression, because it will slow down everything.

Works fine for the following scenario:
Host A Host B (with twisted pair)

Reply

18 hockeyrink February 4, 2009 at 5:39 am

I think there’s a minor typo:

In the section labled “Command to type on hostB (receiving end ~ write image mode)”:
You need to open port on hostB using netcat, enter :
# netcat -p 2222 -l |bzip2 -d | dd of=/dev/sdb

dd of=/dev/sda : /dev/sda is your hard disk. You can also specify partition such as /dev/sda1

I think you mean
dd of=/dev/sdb : /dev/sdb is your hard disk. You can also specify partition such as /dev/sdb1

I think? Yes?

Reply

19 rickx March 19, 2009 at 7:37 am

A bit messy tackling errors in the instructions, but doing
sudo su
nc -l -p 2222 | dd of=/dev/sda bs=16M
on the receiver (192.168.0.2) and
sudo su
dd if=/dev/sda bs=16M | nc 192.168.0.2 2222
on the source (192.168.0.1) works.
Use
ifconfig eth0 192.168.0.1[or 2] netmask 255.255.255.0
to configure the nics from the shell on the fly.
Problem is grub then: the entries with GUIDs will not work, so edit the menu.list before rebooting or you’ll get error 2.
Thanks anyway.

Reply

20 casparoo May 1, 2009 at 5:38 am

How can I copy a hard disk image of a Windows machine to a Linux machine through network ( using netcat and dd). Could you please provide the details. Thanks in advance…

Reply

21 Me May 25, 2009 at 5:32 pm

Just skipping the bzip2, is MUCH faster on slower systems..

Reply

22 Paul Johnson July 14, 2009 at 8:33 pm

I have new version of netcat. I wonder if there is a way to use something like the bs option with this new style of command in order to speed it up. Adapting your instruction to my case, here is what I do:

On the TARGET machine, 129.237.61.XX, make sure port 12345 is unfirewalled. Then set netcat to grab the incoming on 12345 and divert it to disk /dev/sdb. Can also replace with particular partition /dev/sdb2:

nc -l 12345 | bzip2 -d > /dev/sdb

After that, go to the SOURCE machine and compress the source disk /dev/sda and pipe it to through nc over to the other system:

bzip2 -c /dev/sda | nc 129.237.61.XX 12345

That does work, but not speedy.

Also, in your instructions, I wonder if you ought to add some information on whether this should be done on a “live” (mounted) file system. From what I can tell, one ought to do this only after booting from a rescue disk or external drive or some sort.

Reply

23 Marco July 16, 2009 at 5:02 pm

This does not work! I tried it using the exact directions given, and it just produces two unbootable drives! This was a useless post, I know this can be done because I know of people who have done it, but this dislexic post is obiously not the correct way to do it.

Reply

24 richard January 24, 2010 at 8:45 am

An excellent HOWTO. Thank you. Take heed of ‘me’ advice. bzip2 will cause a cpu/memory bottleneck on old machines and under utilise the network. I also found using pv to add a progress meter was useful:
Target host (HOST B):
nc -p 2222 -l -vv | pv -r | dd of=/dev/sda bs=16M

Source host (HOST A):
dd if=/dev/sda bs=16M | nc 2222

Reply

25 Hari January 28, 2010 at 2:32 am

Thanks for putting this together. This is a good alternative to have in case ssh option doesn’t workout for some reason. This should also be better than ssh when the hostB is a very lowend device (like iphone).

Reply

26 Hari January 28, 2010 at 2:48 am

I meant to say “hostA” (ie., the source device is a lowend device like iphone)

Reply

27 Ksh February 27, 2010 at 9:41 pm

Why not gzip? In my experience bzip2 take 3-4 times more cpu cycles than “gzip -3″ ;on the other side compression factor could be only 10-20% better.

Reply

28 Salah May 15, 2010 at 1:25 pm

I see that everyone is recommending dd/nc but I wonder if there’s another solution that provide workarounds for network problems (unstable connections => disconnect/resume, bandwidth limitation => schedule,.. etc). While I’m not if it’s possible to use dd/nc that way.

I don’t mind spending days copying a harddisk from a remote location to another if that will go automatically and error free. Anyone?

Reply

29 David Bain May 23, 2010 at 10:52 am

Hmmmm.. when I follow these instructions everything works except it appears to “hang” when no more data is left to pass from the source machine. I added pv to the command “chain”, as noted above, so I pretty much know when no more data is being passed. Isn’t it supposed to close the connection once all the bytes have been moved?

Reply

30 Scott July 2, 2011 at 1:32 pm

Great site and follow through by everybody. I wonder about the output destination partition/file. Is the output partition created before the
dd of=/dev/sdb
if so how is this done?
Can you instead just send this to a file on the receiving end?
dd of=/tftp/HOSTA.img

So far the discussion is about device or hard drive NOT a partition.
If one wants to make a bootable image of the source partition wouldn’t you copy the?
if=/dev/sda1 (if that is it).

Thanks everybody.

Reply

31 Lennart June 4, 2014 at 1:48 am

Scott, forgot to answer the HOW

On the receiving end:
nc -l 2222 > /home/my_account/myfile.img

On the sending end:
nc my_receiving_hostname 2222 < /dev/sda

Reply

32 Roger Casta April 5, 2013 at 9:00 pm

Thanks for this nice guide. I just want to mention a little problem I faced.

Have tried to clone a 60 GB partition over a 100 Mbps network and only acheived a throughput of 5 Mbps on average. I realized the problem was using compression with bzip2. Without compression i’m getting 70 Mbps on average.

Maybe compression is only usefull on very slow networks.

Note: I’m clonning partitions betwen two HP Mini laptops attached directly with an ethernet cable (no switch in between them).

Reply

33 angel March 27, 2014 at 10:28 pm

stupid question, after clone my driver I can start my operative system in the image normally or I need do an extra step??…thank you!

Reply

34 Lennart June 4, 2014 at 1:39 am

You can, however depending on your O/S you might want to change some things to avoid conflicts. On Ubuntu/Debian Linux for example, you might want to mount your root and delete /etc/udev/rules.d/70-persistent-net.rules

This file contains the autodetected MAC addresses for the network adapters, and after cloning they will be wrong. When you delete it and restart it will be regenerated automatically by udev with the correct MAC addresses.

You might also want to set a different hostname. To generate a random one you could do this:

cat /dev/urandom | tr -dc ‘a-zA-Z0-9′ | fold -w 8 | head -n 1 > /etc/hostname

Other than this everything should be dandy.

Reply

Leave a Comment

Tagged as: , , , , , , ,

Previous post:

Next post: