Is there any way to speed up ddrescue?

Question

I had a 500GB drive HDD crash about 5 days ago. I used ddrescue on the important partition a few days ago, and it's been on "Trimming failed blocks" for almost 2 days now.

Original command:

ddrescue -n /dev/rdisk1s2 /Volumes/OSXBackup/rdisk1s2.img /Volumes/OSXBackup/rdisk1s2.log

Current output:

Initial status (read from logfile)
rescued:   248992 MB,  errsize:   1007 MB,  errors:   15867
Current status
rescued:   249021 MB,  errsize:    978 MB,  current rate:    17408 B/s
   ipos:    44405 MB,   errors:   15866,    average rate:     2784 B/s
   opos:    44405 MB,     time from last successful read:       0 s
Trimming failed blocks...

The original command used the ddrescue -n parameter, and I have restarted the process a few times as needed (and it seemed to pick up right where it left off each time).

Is there any way to speed up this process?

Edit: Six hours later, this is the current status:

rescued:   249079 MB,  errsize:    920 MB,  current rate:      409 B/s
   ipos:    39908 MB,   errors:   15851,    average rate:     2698 B/s
   opos:    39908 MB,     time from last successful read:       0 s
Trimming failed blocks...

It appears that while "errors" is counting down excruciatingly slowly, ipos/opos is counting down how much data it has to churn through, and it seems to be working at a rate of 750MB/hour. At this rate, it will complete in ~53 hours. Yikes.

Edit #2: Two days later, still running. However, there is hope. It has moved passed the "Trimming failed blocks" portion, and on to the next phase "Splitting failed blocks". If anything, what should be taken away from viewing this question is that this definitely takes a long time when a good amount of data/errors are involved. My only hope is that I can successfully recover some important data when all is said and done.

rescued:   249311 MB,  errsize:    688 MB,  current rate:        0 B/s
ipos:    26727 MB,   errors:   15905,    average rate:     1331 B/s
opos:    26727 MB,     time from last successful read:      20 s
Splitting failed blocks...

Answer 1

I observed that using the -n (no-split) option together with -r 1 (retry once) and setting -c (cluster size) to a smaller value can help.

My impression is that the splitting step is very slow as ddrescue splits and splits again the damaged areas. This takes a lot of time because ddrescue tries to restore very small portions of data. So, I prefer to use -n (no-split) together with -c 64, -c 32, -c 16, a.s.o.

Probably the -n (no-split) should always be used for one first pass in forward and reverse directions. It seems that the more the data were split, the slower the cloning, although I'm not sure about this. I assume the larger the non-treated areas, the best when running ddrescue again, because more contiguous sectors are to clone.

As I'm using a logfile, I don't hesitate to cancel the command with Ctrl+C when the data read speed becomes two low.

I also use the -R (Reverse) mode and after a first pass it often gives me higher speeds reading backwards than forward.

It's not clear to me how already retried sectors (-r N) are handled when running the ddrescue command again, especially when alternating forward (default) and reverse (-R) cloning commands. I'm not sure if the number of times they were tried is stored in the logfile and probably the work is done again useless.

Probably the -i (input position) flag can help speed up things too.

Answer 2

It can be very hard to see the progress of ddrescue, but there is another command included called ddrescuelog.

A simple command ddrescuelog -t YourLog.txt will output these nice infos:

current pos:     2016 GB,  current status: trimming
domain size:     3000 GB,  in    1 area(s)
rescued:     2998 GB,  in 12802 area(s)  ( 99.91%)
non-tried:         0 B,  in    0 area(s)  (  0%)

errsize:     2452 MB,  errors:   12801  (  0.08%)
non-trimmed:   178896 kB,  in 3395 area(s)  (  0.00%)
non-split:     2262 MB,  in 9803 area(s)  (  0.07%)
bad-sector:    10451 kB,  in 19613 area(s)  (  0.00%)

You can even use it while ddrescue is running...

Answer 3

If your aim is to obtain the bulk of the data intact, then you could speed up its extraction. But if you really want to rescue as much data as possible, then letting ddrecue nibble at each and every is the route to take.

Answer 4

I have found that playing with the -K parameter you can speed things up. From what I've seen if ddrescue finds an error when running with the -n option tries to jump a fixed amount of sectors. If it still can't read it jumps double the size. If you have large damaged areas you can indicate a big K value (for example 100M) and so the jumping on an error will be larger the first time and it will be easier to avoid problematic areas quickly in the first past.

By the way, there is a wonderful graphical application to analyze the log.

http://sourceforge.net/projects/ddrescueview/

Answer 5

One more way to monitor ddrescue's progress (on Linux, at least) is through the use of strace.

First, find the PID for the ddrescue process using "ps aux | grep ddrescue"

root@mojo:~# ps aux | grep ddrescue
root     12083  0.2  0.0  15764  3248 pts/1    D+   17:15   0:04 ddrescue --direct -d -r0 /dev/sdb1 test.img test.logfile
root     12637  0.0  0.0  13588   940 pts/4    S+   17:46   0:00 grep --color=auto ddrescue

Then run "strace" against that process. You'll see something like:

root@mojo:~# strace -p 12083
Process 12083 attached - interrupt to quit
lseek(4, 1702220261888, SEEK_SET)       = 1702220261888
write(4, "\3101\316\335\213\217\323\343o\317\22M\346\325\322\331\3101\316\335\213\217\323\343o\317\22M\346\325\322\331"..., 512) = 512
lseek(3, 1702220261376, SEEK_SET)       = 1702220261376
read(3, "\3101\316\335\213\217\323\343o\317\22M\346\325\322\331\3101\316\335\213\217\323\343o\317\22M\346\325\322\331"..., 512) = 512
lseek(4, 1702220261376, SEEK_SET)       = 1702220261376
write(4, "\3101\316\335\213\217\323\343o\317\22M\346\325\322\331\3101\316\335\213\217\323\343o\317\22M\346\325\322\331"..., 512) = 512
^C

...and so on. The output is fast and ugly, so I then pipe it through "grep" to filter out the stuff I care about:

root@mojo:/media/u02/salvage# nice strace -p 12083 2>&1|grep lseek
lseek(4, 1702212679168, SEEK_SET)       = 1702212679168
lseek(3, 1702212678656, SEEK_SET)       = 1702212678656
lseek(4, 1702212678656, SEEK_SET)       = 1702212678656
lseek(3, 1702212678144, SEEK_SET)       = 1702212678144
lseek(4, 1702212678144, SEEK_SET)       = 1702212678144
lseek(3, 1702212677632, SEEK_SET)       = 1702212677632
lseek(4, 1702212677632, SEEK_SET)       = 1702212677632
lseek(3, 1702212677120, SEEK_SET)       = 1702212677120
lseek(4, 1702212677120, SEEK_SET)       = 1702212677120
lseek(3, 1702212676608, SEEK_SET)       = 1702212676608
^C

In that example, the "1702212676608" equates to "the amount of data that still needs to be processed on that 2 Tb disk you're trying to salvage." (Yeah. Ouch.) ddrescue is spitting out a similar number -- albeit as "1720 GB" -- in its screen output.

strace gives you a MUCH higher granularity data stream for you to examine; it's one more way to evaluate the speed of ddrescue and estimate a completion date.

Running it constantly is probably a bad plan since it would compete with ddrescue for CPU time. I've taken to piping it to "head" so I can grab the first 10 values:

root@mojo:~# strace -p 4073 2>&1 | grep lseek | head

Hope this helps someone.

Answer 6

What is the file system of the hard disk where you save the rescue image and the logfile? I just made the experience that rescuing a 500GB internal hard drive (connected via SATA) on a Laptop running Linux Mint from a USB Stick, saving the rescue image and logfile on an exFat formatted USB hard drive, was starting rather slowly (1-2MB/sec) but after around 250GB it was only crawling at <100KB/sec. It seemed to become slower the larger the rescue image file was growing.

Then I moved the rescue image and logfile to another temporary place, re-formatted the USB hard drive with the ext4 file system, moved the files back on it and resumed the ddrescue process - and now it runs with 1-20MB/sec again (fluctuating but around 7MB/sec on average)!

Seems like exFat does not play very well with very large files (several hundred gigabytes).

Answer 7

For a faster and quick option to rescue the disc you can use a sh script file and run the file with "sh filename.sh". It contains this line shown, just repeat "sudo ddrescue" and "sleep 3" few more times, the sleep is used to make the drive rest some seconds, it can be good for some reasons:

#! /bin/sh -e  
sudo ddrescue -d -r0 -e +0 -T 1s -n /dev/drivepartition file.img log.logfile 
sleep 3

The -r0 is with no replies. The -e +0 is for exit on 1 error. The -T 1s exits with 1 second fail read. There are options that can be used as -d for direct and -n for no scrape which can speed up.

You can use -R after finish with option -A once, that will reverse and remove all errorsize and start again backwards. Means it will read errors differently.

Answer 8

ddrescue --verbose -r 0 -c 32 -f -n --skip-size=100M --unidirectional --min-read-rate=60M /dev/BADDRIVE /dev/GOODDRIVE /home/user/save/log/here.log

In my own testing, passing the log file at the end allows for resume functionality. So it's "safe" to ctrl-c the process and try again.

In my use case, I wanted to get "most" of the data before the drive died. The data was not strictly irreplaceable. I wanted to get the low hanging data without waiting an eternity for 8kbps transfers to complete.

1. verbose: allegedly save more info (doesn't seem very verbose to me)
2. -r 0 / --retry-passes=0: (Default) Exit after the given number of retry passes.
3. -c / --cluster-size=sectors: "Number of sectors to copy at a time. Defaults to 64 KiB / sector_size. Try smaller values for slow drives.". Smaller values can paradoxically improve speed. Try a few values and stop the test with ctrl-c.
4. -f / --force: "Force overwrite of outfile. Needed when outfile is not a regular file, but a device or partition. This option is just a safeguard to prevent the inadvertent destruction of partitions, and is ignored for regular files."
5. -n / --no-scrape: Skip the scraping phase. Avoids spending a lot of time trying to rescue the most difficult parts of the file.
6. --skip-size=[initial][,max]: initial is the size to skip on the first read error or slow read, in bytes. max is the maximum size to skip.
7. unidirectional: only read in one direction
8. --min-read-rate=60M: This is where the magic happens. This is what rocket boosted my drive. Anything below a 60 megaBYTE(?) per second read rate was abandoned after X seconds. I'm giving up on data but gaining a ton of speed. Now my HDD is showing ~160 MBps reads instead of 0.008 MBps reads. I tried with 20 and 30M but they still seemed too slow. For my drive at least, there was either "slow" or "fast" sectors, and I wanted to get the fast sectors while they were still readable.

Combining a few answers as well as my own research and this page. I am using MX Linux 23.5 Xfce live.

Other ideas

Reverse might be good too, if the end of the drive/LBAs is good but the beginning is toast.

I tried shrinking the file system (8TB drive, 5TB used) but it was too broken to successfully shrink.

Answer 9

dd_rhelp is a shell script that uses dd_rescue "[...] on your entire disc, BUT it will try to gather the maximum valid data before trying for ages on bunches of badsectors"

it's quite old (2012) but still works. haven't tried ddrescue yet.