[TriLUG] Copying Files from a Dying HDD...with a twist

Thu Jul 5 13:09:31 EDT 2018

Many years ago, Antonio Diaz heavily updated his ddrescue program to
incorporate a list of my suggestions, including the ability to abort in the
midst of a recovery process and then later resume where it left off, by
creating and frequently updating a "logfile" which kept track of its
progress. At my suggestion, he made the logfile an ASCII text file, so that
other tools could easily work with it.

I then wrote a set of Perl scripts which use his logfiles to do various
useful things. They're on one of my web sites, here:

http://burtonsys.com/download/ddr2sr.zip

Some of those scripts are no longer useful, because Antonio subsequently
implemented their functionality into ddrescue, itself. But some are still
useful, including:

*"ddr2nri.pl <http://ddr2nri.pl>"* -- Read a ddrescue logfile, and create a
.bat script of NFI commands, to identify the damaged files on an NTFS file
system.

*"nficruncher.pl <http://nficruncher.pl>"* -- NTFS File sector Information
tool output cruncher.

Those two Perl scripts use a Microsoft program called NFI ("NTFS File
sector Information utility").  It is a tool which can be used to tell you
which file on an NTFS volume contains a particular sector. *(Caveat: it can
be misleading, if chkdsk has moved the files around!) *

NFI seems to be gone now, because Microsoft loves to delete anything useful
that their people create, leaving no trace of it on their web site. But
there's an archived copy of their information about NFI (among other
things), here:
https://archive.is/4RYVu

The NFI tool, itself, is here:
https://web.archive.org/web/20141117095714/http://download.microsoft.com/download/win2000srv/utility/3.0/nt45/en-us/oem3sr2.zip

The idea is that when ddrescue is done, you use its logfile (which is a
record of which sectors were and were not successfully recovered), to
deduce which files are damaged. The first Perl tool, ddr2nfi.pl, reads the
ddrescue logfile and the output of "fdisk -lu" to create a Windows .bat
script with a long list of NFI commands.

Of course, you can't mount a disk image on Windows with the loop device, so
before you can run that .bat file you'll need to image the recovered image
file back onto a real scratch drive.

Also, since Windows assigns drive letters unpredictably, you might need to
edit the generated .bat script, and change all the drive letters.

Then run the .bat script, which runs the nfi comments, redirecting the
output into a text file.

If there were a lot of bad sectors, that text file might be very large! So
the second Perl tool, nficruncher.pl, reads it, and produces a concise
report, telling you about the damaged files.

ddr2nfi.pl   ("DDRescue to NFI")   v.10, 12-Sep-15

Usage:
   perl -w ddr2nfi.pl {options} nficmds.bat SPIN_LOG.x fdisk_out
LogBeforeSR LogAfterSR

You can omit the 5th parameter (LogAfterSR) if you know that the final
ddrescue
command got no errors (which is the normal case if SpinRite processed the
whole
drive).   If you didn't run SpinRite, then specify "-" in place of the 2nd
parameter (SPIN_LOG.x).  The 3rd parameter (fdisk_out) should be the output
of
the "fdisk -lu /dev/..." command for this drive, or you can specify "-" in
place of the 3rd parameter to omit it.

Note: it often happens that SpinRite successfully recovers a sector but
doesn't
mention that sector in its log file.  Presumably, those sectors were read
successfully after retries, without resorting to DynaStat recovery.  Such
sectors contain good data.  However, if you don't run SpinRite on the entire
drive, that will also result in sectors being omitted from the SpinRite log
file, and those sectors do NOT contain good data.  The only way to tell the
difference is from the LogAfterSR file: the sectors containing good data
will
have been successfully copied, and the sectors containing bad data will not.
So do not omit the 5th parameter if you didn't run SpinRite on the whole
drive.

{options} may be omitted (the usual case), or "-p" to use the physical (lba)
device name and sector number syntax to nfi, or "-v" for verbose/debug, or
"-c" for concise output (no REM lines), or "-nn" where nn is a decimal
integer,
to specify the NTFS cluster size in sectors, used to make nficmds.bat more
concise (the default cluster size is 8, for 4096-byte clusters).  To get nfi
commands for every bad sector, use "-1".

nficruncher.pl     v.8, 30-Mar-08
("NTFS File sector Information tool output cruncher")

The assumption is that you used ddr2nfi.pl to generate a Windows
NT/2K/XP/2003
batch file (perhaps called nficmds.bat), containing NFI commands for the
unrecovered sectors.  This Perl program processes the concatinated output of
those NFI commands.

Usage:
   nficmds.bat >nfioutput.txt
   perl -w nficruncher.pl {options} nfioutput.txt

where {options} can be:
   -v     (verbose debug prints)
   -d     (damaged file report)
   -r     (generate .bat file to rename damaged files)
   -i     (important sectors report, in ddrescue logfile format)
   -u     (unimportant sectors report, in ddrescue logfile format)
   -cNN   (assume cluster size is NN sectors/cluster; default is 8)
   -pXXX  (NTFS partition starts at sector XXX, as reported by 'fdisk -lu')
   -sYYY  (NTFS partition size, in sectors)
   -f     (read 'fdisk-lu_out.txt', the output of Linux 'fdisk -lu ...')
   -ffilename  (read filename', the output of Linux 'fdisk -lu ...')

Note #1:  You can use clustersize.pl to discover the cluster size and
starting
sector number of the NTFS partition.

Note #2:  If you specify -i or -u, nficruncher.pl needs to know the
starting sector of
the NTFS partition (and, for -i, the disk size).  If there is only one NTFS
partition, you can use the '-f' option, otherwise use '-p' and '-s'.

On Thu, Jul 5, 2018 at 9:48 AM, Brian via TriLUG <trilug at trilug.org> wrote:

> ddrescue is underway as we type.  I had not already tried it, it being
> something that fell into one of the many holes in my Linux knowledge! So
> far, I'm quite impressed by its reported progress.  It's already getting
> way better mileage than I was.
>
> On 07/03/2018 05:13 PM, David Burton via TriLUG wrote:
>
>> Better yet, make a couple of copies of the recovered .iso (being sure to
>> also save the ddrescue logfile), and then use its "fill" option to fill
>> the
>> bad blocks two different ways, e.g., with 0x00 bytes in one copy, and with
>> 0xFF bytes in another copy.
>> [then compare files across the two images]
>>
>
> Great idea!  I'll keep that in mind.
>
>> If it were a Windows file system, I'd have some additional suggestions,
>> working from the ddrescue logfile to identify the damaged files.
>>
>
> It is, actually.  It's the godawful-slow HDD that came with my laptop and
> has Windows 10 on it, which I still use for some software synthesizers.  I
> was in the middle of dd-ing it across to a new SSD (with an external
> USB-SATA adapter) when I accidentally knocked it off the desk :-(
>
> -B
>
> --
> This message was sent to: Dave Burton <ncdave4life at gmail.com>
> To unsubscribe, send a blank message to trilug-leave at trilug.org from that
> address.
> TriLUG mailing list : https://www.trilug.org/mailman/listinfo/trilug
> Unsubscribe or edit options on the web  : https://www.trilug.org/mailman
> /options/trilug/ncdave4life%40gmail.com
> Welcome to TriLUG: https://trilug.org/welcome
>