[TriLUG] Bit rot detection without ZFS/btrfs?

[Steve Litt]

On Sun, 30 Jun 2013 20:42:22 -0400
"Randy Barlow" <randy at electronsweatshop.com> wrote:

> Hello fellow Linux people,
> 
> I've been wearing my tin foil hat a little bit too much lately, and
> I've started to become worried about all the files sitting on my
> various hard drives, bit rotting away.
> 
> I am aware that ZFS and btrfs are designed to help with this problem,
> but ZFS has its licensing issues, and btrfs isn't yet "blessed" by
> the Linux elders.
> 
> Do any of you use anything to detect file corruption on your disks?
> I'm mostly interested in detection at this point, as I think I can
> pretty well use my backups or off site backups to recover, but I need
> something to tell me when I need to do that.
> 
> I've considered writing something homegrown to do this, as it's not  
> terribly complicated. I could store a checksum on the FS extended  
> attributes, or maybe just in a database of some kind.
> 
> If anyone knows of a distro package that can do this, I'd love to
> hear about it. There are some interesting challenges to get past
> (lots of opportunities for false positives, for example when
> checksumming a file that is currently being written to by another
> process.)

I wonder if there's a way to do this with rsync. You can make rsync
write to a log. On the destination machine, if a file changed but its
date didn't, that's pretty good evidence of bit rot.

This has the added benefit of acting as a backup server, and if you use
cp -al each time, you keep older versions so you can restore
non-bit-rotted files after changing out your hard disk.

Thanks,

SteveT

Steve Litt                *  http://www.troubleshooters.com/
Troubleshooting Training  *  Human Performance