[TriLUG] mdadm: no devices found for /dev/md0

[T. Bryan]

My debian testing system is kind of useless until I figure this one out.  :(

After recent upgrades, my RAID1 device does not seem to be coming up during 
the boot process any more.

/dev/hda is an IDE disk.  Nothing fancy.  Both /boot and / are mounted from 
partitions on /dev/hda.  For /home and /opt, I was using LVM logical devices 
on a /dev/md0, which is a RAID1 across /dev/hde1 and /dev/hdg1.  

Everything was working beautifully until last week.  I did a reboot for the 
first time in a few weeks, so there are a few weeks of upgrades that could 
have impacted this one.  /dev/md0 is normally not started until later in the 
boot sequence since the root partition doesn't live there.  I'm not sure 
whether that matters for my current problem.

During boot, I hit this error

[/sbin/fsck.ext3 (1) -- /boot] fsck.ext3 -a -C0 /dev/hda1
/boot: clean, 36/52208 files, 41621/104391 blocks
[/sbin/fsck.ext3 (1) -- /home] fsck.ext3 -a -C0 /dev/localvg/lv_home1
fsck.ext3: No such file or directory while trying to open 
/dev/localvg/lv_home1
/dev/localvg/lv_home1:
The superblock could not be read or does not describe a correct ext2
filesystem.  If the device is valid and it really contains an ext2
filesystem (and not swap or ufs or something else), then the superblock
is corrupt, and you might try running e2fsck with an alternate superblock:
    e2fsck -b 8193 <device>
fsck died with exit status 8

Now, I don't think that I'm actually seeing a disk error.  It's more like 
/dev/md0, LVM, and udev are not cooperating during boot up.  After I skip 
past this error and boot my system without access to the LVM partitions, I 
try to stop and restart the RAID device.  I don't think that it's starting 
properly.  Note that it never checks /dev/hdg1 or /dev/hde1.  Those devices 
are not in /dev/, but they are in /dev/.static/dev/.  Even if I use MAKEDEV 
to create devices like /dev/hde1, I still get the same output from mdadm.

# mdadm -S /dev/md0
mdadm: stopped /dev/md0

#mdadm -v -As /dev/md0
mdadm: looking for devices for /dev/md0
mdadm: cannot open device /dev/hdc2: Device or resource busy
mdadm: /dev/hdc2 has wrong uuid.
mdadm: no recogniseable superblock on /dev/hdc1
mdadm: /dev/hdc1 has wrong uuid.
mdadm: cannot open device /dev/hdc: Device or resource busy
mdadm: /dev/hdc has wrong uuid.
mdadm: cannot open device /dev/hda6: Device or resource busy
mdadm: /dev/hda6 has wrong uuid.
mdadm: cannot open device /dev/hda5: Device or resource busy
mdadm: /dev/hda5 has wrong uuid.
mdadm: no recogniseable superblock on /dev/hda4
mdadm: /dev/hda4 has wrong uuid.
mdadm: cannot open device /dev/hda2: Device or resource busy
mdadm: /dev/hda2 has wrong uuid.
mdadm: cannot open device /dev/hda1: Device or resource busy
mdadm: /dev/hda1 has wrong uuid.
mdadm: cannot open device /dev/hda: Device or resource busy
mdadm: /dev/hda has wrong uuid.
mdadm: /dev/hdg has wrong uuid.
mdadm: /dev/hde has wrong uuid.
mdadm: no devices found for /dev/md0

My /etc/mdadm/mdadm.conf still reads

DEVICE partitions
DEVICE /dev/hde1 /dev/hdg1
ARRAY /dev/md0 level=raid1 num-devices=2 
UUID=0474da3c:9651a8fe:0994d30a:26dd9a17
MAILADDR root

I'm seeing one other change during boot.  I used to see

Nov 17 23:27:20 localhost kernel: md: md0 stopped.
Nov 17 23:27:20 localhost kernel: md: bind<hde1>
Nov 17 23:27:20 localhost kernel: md: bind<hdg1>
Nov 17 23:27:20 localhost kernel: raid1: raid set md0 active with 2 out of 2 
mirrors

But now I get

Nov 26 18:28:29 localhost kernel: md: md0 stopped.
Nov 26 18:28:29 localhost kernel: md: bind<hdg>
Nov 26 18:28:29 localhost kernel: md: bind<hde>
Nov 26 18:28:29 localhost kernel: raid1: raid set md0 active with 2 out of 2 
mirrors

I'm not sure whether it's a problem that bind<hde1> became bind<hde>.  

Any suggestions?  I'm really stumped when it comes to troubleshooting this 
one.  I'm just trying to get enough of a handle on this problem so that I can 
search more effectively on Google or figure out which Debian forum to use 
when asking about it.

---Tom