[TriLUG] Debian, LVM, and RAID

[Pat Regan]

Kevin Otte wrote:
> - Mirror partition sizes
> - Change LVM (0x8e) to RAID (0xfd) on the new HD

How did you mirror the partition sizes?  I ask, because you said you
changed the partition type on the new HD.  If you copied the partition
table with dd you might have copied over a duplicated of the pv
superblock on the old hard drive.

> - Created the RAID mirror consisting of elements "missing" and the new hard
>   drive
> - Did a pvcreate on the new MD device
> - Extended the VG into the new PV
> - Performed a pvmove off the PV on the original HD to the new PV in the MD
>   (about to die from an acronym overdose yet?)
> - Did a vgreduce to eliminate the original HD PV from the VG

If I were doing this booted from a rescue cd like you are I would have
created a new volume group and copied the data.  It would likely be faster.

pvmove and vgreduce are going to be slow because they are meant to be
used on a running filesystem with no loss of service.

> - Did a pvremove of the original HD PV
> - Changed LVM (0x8e) to RAID (0xfd) on the old HD
> - Added the old HD to the RAID array
> - Watched it rebuild
> 

I would have hoped that the pvremove and initialization of the array
would have removed the duplicate lvm superblock.  The only thing I can
think of is that it might have caused trouble if you used dd to copy the
partition table.  I don't know the exact details of where md and lvm
store their metadata, but it sounds like it may be possible for them to
coexist well enough on the same block device to confuse themselves.

> By default LVM tries to scan all partitions, regardless of whether they are
> type 0x8e.  I think this is stupid, but I'm no developer, so what do I know. 
> So naturally, when I tried to manually do the pvscan, it found duplicate PVs
> on /dev/hda2 and /dev/hdc2.  Without any way to drop LVM nicely, I boldly
> used mdadm to start the RAID array.  It started as I would expected to.  I
> did a pvscan, and got "No matching physical volumes found."  I tried other
> various incantations of pv and vg commands to try and see my beloved volume
> group, but alas it appeared lost in the void.
> 

I don't think that it is stupid that it scans all partitions.  I am just
terribly surprised that it can find an lvm signature on a block device
that is part of an md device.  I would have thought that the md
signature would get in the way.  Like I said though, I have no idea how
those signatures are stored (I only know they are there :p).

> My question, fair readers, is this: What the heck did I do wrong?!  I have
> installed numerous systems at work using an RAID1 as a PV, but I did the
> configuration of those directly from the installer.  Apparently it does some
> voodoo, as you can see my luck trying to migrate an existing installation to
> this configuration.

The installer shouldn't do any voodoo.

> Side question:
> (*) I tried this in testing using a couple of LVs and it seemed to work, but
> I wondered if anyone else has tried it:
> 
> - lvcreate -n raidt0 -L 512M vg0
> - lvcreate -n raidt1 -L 512M vg0
> - mke2fs -j /dev/vg0/raidt0
> - Mount the FS and drop a test file on it
> - Unmount the FS
> - mdadm -C /dev/md0 -l 1 -n 2 raidt0 missing
>   (This gives a warning about there being an ext filesystem there already)
> - Mount /dev/md0
> - Note the test file is still there and happy.
> 
> I realize this is probably one of those "not recommended" things, but if it
> seems safe enough to some other hackers out here, I'll take it.  I really
> don't want to sit through that insanely long pvmove again.
> 

The fact that this works proves that the lvm and md signatures live in a
space where, at the very least, they do not step on the filesystem
(assuming it still passes an fsck?).  The biggest reason I would say
what you are doing is "not recommended" is because it will only work
with RAID level 1.

Did I guess correctly that you used dd to copy the partition table?  I
can't think of any other way that it would find duplicate PVs...

Pat
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 189 bytes
Desc: OpenPGP digital signature
URL: <http://www.trilug.org/pipermail/trilug/attachments/20060129/4402bb0c/attachment.pgp>