Create RAID 10
How to setup a RAID Nice to read: http://www.tldp.org
Mithilfe von dmesg kontrollieren ob alle Platten gefunden wurden. In der folgenden Übersicht werden die Platten /dev/sda und /dev/sdb am SCSI Bus 0 nicht dargestellt.
Vendor: SEAGATE Model: ST136403LC Rev: 0002 Type: Direct-Access ANSI SCSI revision: 02 (scsi1:A:0): 80.000MB/s transfers (40.000MHz, offset 15, 16bit) Vendor: SEAGATE Model: ST136403LC Rev: 0002 Type: Direct-Access ANSI SCSI revision: 02 (scsi1:A:1): 80.000MB/s transfers (40.000MHz, offset 15, 16bit) Vendor: SEAGATE Model: ST173404LC Rev: 0002 Type: Direct-Access ANSI SCSI revision: 03 (scsi1:A:2): 160.000MB/s transfers (80.000MHz DT, offset 63, 16bit) Vendor: SEAGATE Model: ST173404LC Rev: 0002 Type: Direct-Access ANSI SCSI revision: 03 (scsi1:A:3): 160.000MB/s transfers (80.000MHz DT, offset 63, 16bit) Vendor: SEAGATE Model: ST136403LC Rev: 0002 Type: Direct-Access ANSI SCSI revision: 02 (scsi1:A:4): 80.000MB/s transfers (40.000MHz, offset 15, 16bit) Vendor: SEAGATE Model: ST136403LC Rev: 0002 Type: Direct-Access ANSI SCSI revision: 02 (scsi1:A:5): 80.000MB/s transfers (40.000MHz, offset 15, 16bit) Vendor: SEAGATE Model: ST173404LC Rev: 0002 Type: Direct-Access ANSI SCSI revision: 03 (scsi1:A:6): 160.000MB/s transfers (80.000MHz DT, offset 63, 16bit) Vendor: SEAGATE Model: ST173404LC Rev: 0002 Type: Direct-Access ANSI SCSI revision: 03 (scsi1:A:8): 160.000MB/s transfers (80.000MHz DT, offset 63, 16bit) scsi1:A:0:0: Tagged Queuing enabled. Depth 8 scsi1:A:1:0: Tagged Queuing enabled. Depth 8 scsi1:A:2:0: Tagged Queuing enabled. Depth 8 scsi1:A:3:0: Tagged Queuing enabled. Depth 8 scsi1:A:4:0: Tagged Queuing enabled. Depth 8 scsi1:A:5:0: Tagged Queuing enabled. Depth 8 scsi1:A:6:0: Tagged Queuing enabled. Depth 8 scsi1:A:8:0: Tagged Queuing enabled. Depth 8 Journalled Block Device driver loaded Attached scsi disk sdc at scsi1, channel 0, id 0, lun 0 Attached scsi disk sdd at scsi1, channel 0, id 1, lun 0 Attached scsi disk sde at scsi1, channel 0, id 2, lun 0 Attached scsi disk sdf at scsi1, channel 0, id 3, lun 0 Attached scsi disk sdg at scsi1, channel 0, id 4, lun 0 Attached scsi disk sdh at scsi1, channel 0, id 5, lun 0 Attached scsi disk sdi at scsi1, channel 0, id 6, lun 0 Attached scsi disk sdj at scsi1, channel 0, id 8, lun 0zuerst kontollieren ob genug Divices da sind
turnip:~# ls -l /dev/sd[a-m] brw-rw---- 1 root disk 8, 0 Jan 24 2002 /dev/sda brw-rw---- 1 root disk 8, 16 Jan 24 2002 /dev/sdb brw-rw---- 1 root disk 8, 32 Jan 24 2002 /dev/sdc brw-rw---- 1 root disk 8, 48 Jan 24 2002 /dev/sdd brw-rw---- 1 root disk 8, 64 Jan 24 2002 /dev/sde brw-rw---- 1 root disk 8, 80 Jan 24 2002 /dev/sdf brw-rw---- 1 root disk 8, 96 Jan 24 2002 /dev/sdg brw-rw---- 1 root disk 8, 112 Jan 24 2002 /dev/sdhUnter Umstnden neue Deivices anlegen:
turnip:/dev# MAKEDEV -v sdi create sdi b 8 128 root:disk 0660 create sdi1 b 8 129 root:disk 0660 create sdi2 b 8 130 root:disk 0660 create sdi3 b 8 131 root:disk 0660 create sdi4 b 8 132 root:disk 0660 create sdi5 b 8 133 root:disk 0660 create sdi6 b 8 134 root:disk 0660 create sdi7 b 8 135 root:disk 0660 create sdi8 b 8 136 root:disk 0660 create sdi9 b 8 137 root:disk 0660 create sdi10 b 8 138 root:disk 0660 create sdi11 b 8 139 root:disk 0660 create sdi12 b 8 140 root:disk 0660 create sdi13 b 8 141 root:disk 0660 create sdi14 b 8 142 root:disk 0660 create sdi15 b 8 143 root:disk 0660 turnip:/dev# MAKEDEV -v sdj create sdj b 8 144 root:disk 0660 create sdj1 b 8 145 root:disk 0660 create sdj2 b 8 146 root:disk 0660 create sdj3 b 8 147 root:disk 0660 create sdj4 b 8 148 root:disk 0660 create sdj5 b 8 149 root:disk 0660 create sdj6 b 8 150 root:disk 0660 create sdj7 b 8 151 root:disk 0660 create sdj8 b 8 152 root:disk 0660 create sdj9 b 8 153 root:disk 0660 create sdj10 b 8 154 root:disk 0660 create sdj11 b 8 155 root:disk 0660 create sdj12 b 8 156 root:disk 0660 create sdj13 b 8 157 root:disk 0660 create sdj14 b 8 158 root:disk 0660 create sdj15 b 8 159 root:disk 0660Falls auf den Platten alte Partitionsdaten sind (z.B. von FreeBSD) diese mit dd bügeln, mit fdisk kann man sie nicht löschen
turnip:~# fdisk -l /dev/sdc Disk /dev/sdc: 255 heads, 63 sectors, 4427 cylinders Units = cylinders of 16065 * 512 bytes 8 partitions: # start end size fstype [fsize bsize cpg] c: 1 8914* 8913* unused 0 0 e: 1 8914* 8913* ADOS turnip:~# dd if=/dev/zero of=/dev/sdc bs=1024 count=9999 9999+0 records in 9999+0 records out 10238976 bytes transferred in 0.489498 seconds (20917297 bytes/sec) turnip:~# fdisk -l /dev/sdc Disk /dev/sdc: 255 heads, 63 sectors, 4427 cylinders Units = cylinders of 16065 * 512 bytes Disk /dev/sdc doesn't contain a valid partition table
turnip:~# fdisk /dev/sdc Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel Building a new DOS disklabel. Changes will remain in memory only, until you decide to write them. After that, of course, the previous content won't be recoverable. The number of cylinders for this disk is set to 4427. There is nothing wrong with that, but this is larger than 1024, and could in certain setups cause problems with: 1) software that runs at boot time (e.g., old versions of LILO) 2) booting and partitioning software from other OSs (e.g., DOS FDISK, OS/2 FDISK) Command (m for help): n Command action e extended p primary partition (1-4) p Partition number (1-4): 1 First cylinder (1-4427, default 1):ENTER Using default value 1 Last cylinder or +size or +sizeM or +sizeK (1-4427, default 4427):ENTER Using default value 4427 Command (m for help): w The partition table has been altered! Calling ioctl() to re-read partition table. Syncing disks. turnip:~# fdisk -l /dev/sdc Disk /dev/sdc: 255 heads, 63 sectors, 4427 cylinders Units = cylinders of 16065 * 512 bytes Device Boot Start End Blocks Id System /dev/sdc1 1 4427 35559846 83 LinuxDiese Schritte für alle Platten die in's RAID sollen wiederholen, bei den 8 Platten an turnip von /dev/sdc bis /dev/sdj. Sind die Plattengrössen unterschiedlich legt man auf den grösseren Platten mehrere Partitionen an, diese in der selben grösse wie auf einer kleineren Platte. Ein Beispiel für eine kleine Platte (hier 36 GB). Begin der der Partition ist die Vorgabe (der erste freie Zylinder), als Grösse wird in diesem Beispiel 32000MByte angegeben.
turnip:~# fdisk /dev/sdd Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel Building a new DOS disklabel. Changes will remain in memory only, until you decide to write them. After that, of course, the previous content won't be recoverable. The number of cylinders for this disk is set to 4427. There is nothing wrong with that, but this is larger than 1024, and could in certain setups cause problems with: 1) software that runs at boot time (e.g., old versions of LILO) 2) booting and partitioning software from other OSs (e.g., DOS FDISK, OS/2 FDISK) Command (m for help): p Disk /dev/sdd: 255 heads, 63 sectors, 4427 cylinders Units = cylinders of 16065 * 512 bytes Device Boot Start End Blocks Id System Command (m for help): n Command action e extended p primary partition (1-4) p Partition number (1-4): 1 First cylinder (1-4427, default 1): ENTER Using default value 1 Last cylinder or +size or +sizeM or +sizeK (1-4427, default 4427): +32000M Command (m for help): p Disk /dev/sdd: 255 heads, 63 sectors, 4427 cylinders Units = cylinders of 16065 * 512 bytes Device Boot Start End Blocks Id System /dev/sdd1 1 4080 32772568+ 83 Linux Command (m for help): w The partition table has been altered! Calling ioctl() to re-read partition table. Syncing disks.Ist eine Platte grösser bleibt die Partitionsgrösse 32000MByte, nur legt man mehrere auf der Platte an
turnip:~# fdisk /dev/sde Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel Building a new DOS disklabel. Changes will remain in memory only, until you decide to write them. After that, of course, the previous content won't be recoverable. The number of cylinders for this disk is set to 8924. There is nothing wrong with that, but this is larger than 1024, and could in certain setups cause problems with: 1) software that runs at boot time (e.g., old versions of LILO) 2) booting and partitioning software from other OSs (e.g., DOS FDISK, OS/2 FDISK) Command (m for help): n Command action e extended p primary partition (1-4) p Partition number (1-4): 1 First cylinder (1-8924, default 1): ENTER Using default value 1 Last cylinder or +size or +sizeM or +sizeK (1-8924, default 8924): +32000M Command (m for help): n Command action e extended p primary partition (1-4) p Partition number (1-4): 2 First cylinder (4081-8924, default 4081):ENTER Using default value 4081 Last cylinder or +size or +sizeM or +sizeK (4081-8924, default 8924): +32000M Command (m for help): p Disk /dev/sde: 255 heads, 63 sectors, 8924 cylinders Units = cylinders of 16065 * 512 bytes Device Boot Start End Blocks Id System /dev/sde1 1 4080 32772568+ 83 Linux /dev/sde2 4081 8160 32772600 83 Linux Command (m for help): w The partition table has been altered! Calling ioctl() to re-read partition table. Syncing disks.Wer mag kann genauer rechnen um weniger Platz zu verschwenden, je nachdem sieht das Ergebnis der Mühe etwa so aus:
turnip:~# fdisk -l /dev/sd[c-j] Disk /dev/sdc: 255 heads, 63 sectors, 4427 cylinders Units = cylinders of 16065 * 512 bytes Device Boot Start End Blocks Id System /dev/sdc1 1 4080 32772568+ 83 Linux Disk /dev/sdd: 255 heads, 63 sectors, 4427 cylinders Units = cylinders of 16065 * 512 bytes Device Boot Start End Blocks Id System /dev/sdd1 1 4080 32772568+ 83 Linux Disk /dev/sde: 255 heads, 63 sectors, 8924 cylinders Units = cylinders of 16065 * 512 bytes Device Boot Start End Blocks Id System /dev/sde1 1 4080 32772568+ 83 Linux /dev/sde2 4081 8160 32772600 83 Linux Disk /dev/sdf: 255 heads, 63 sectors, 8924 cylinders Units = cylinders of 16065 * 512 bytes Device Boot Start End Blocks Id System /dev/sdf1 1 4080 32772568+ 83 Linux /dev/sdf2 4081 8160 32772600 83 Linux Disk /dev/sdg: 255 heads, 63 sectors, 4427 cylinders Units = cylinders of 16065 * 512 bytes Device Boot Start End Blocks Id System /dev/sdg1 1 4080 32772568+ 83 Linux Disk /dev/sdh: 255 heads, 63 sectors, 4427 cylinders Units = cylinders of 16065 * 512 bytes Device Boot Start End Blocks Id System /dev/sdh1 1 4080 32772568+ 83 Linux Disk /dev/sdi: 255 heads, 63 sectors, 8924 cylinders Units = cylinders of 16065 * 512 bytes Device Boot Start End Blocks Id System /dev/sdi1 1 4080 32772568+ 83 Linux /dev/sdi2 4081 8160 32772600 83 Linux Disk /dev/sdj: 255 heads, 63 sectors, 8924 cylinders Units = cylinders of 16065 * 512 bytes Device Boot Start End Blocks Id System /dev/sdj1 1 4080 32772568+ 83 Linux /dev/sdj2 4081 8160 32772600 83 LinuxNun wird die /etc/raidtab angelegt. Dort werden zwei Strippe-RAIDs angelegt und aus denen ein Miroor zusammengelegt.
raiddev /dev/md0
raid-level 0
nr-raid-disks 6
persistent-superblock 1
chunk-size 4
device /dev/sdc1
raid-disk 0
device /dev/sdd1
raid-disk 1
device /dev/sde1
raid-disk 2
device /dev/sde2
raid-disk 3
device /dev/sdf1
raid-disk 4
device /dev/sdf2
raid-disk 5
raiddev /dev/md1
raid-level 0
nr-raid-disks 6
persistent-superblock 1
chunk-size 4
device /dev/sdg1
raid-disk 0
device /dev/sdh1
raid-disk 1
device /dev/sdi1
raid-disk 2
device /dev/sdi2
raid-disk 3
device /dev/sdj1
raid-disk 4
device /dev/sdj2
raid-disk 5
raiddev /dev/md2
raid-level 1
nr-raid-disks 2
nr-spare-disks 0
chunk-size 4
#persistent-superblock 1
device /dev/md0
raid-disk 0
device /dev/md1
raid-disk 1
Das Raid wird initialisiert
turnip:/dev# mkraid /dev/md0 handling MD device /dev/md0 analyzing super-block disk 0: /dev/sdc1, 32772568kB, raid superblock at 32772480kB disk 1: /dev/sdd1, 32772568kB, raid superblock at 32772480kB disk 2: /dev/sde1, 32772568kB, raid superblock at 32772480kB disk 3: /dev/sde2, 32772600kB, raid superblock at 32772480kB disk 4: /dev/sdf1, 32772568kB, raid superblock at 32772480kB disk 5: /dev/sdf2, 32772600kB, raid superblock at 32772480kB turnip:/dev# mkraid /dev/md1 handling MD device /dev/md1 analyzing super-block disk 0: /dev/sdg1, 32772568kB, raid superblock at 32772480kB disk 1: /dev/sdh1, 32772568kB, raid superblock at 32772480kB disk 2: /dev/sdi1, 32772568kB, raid superblock at 32772480kB disk 3: /dev/sdi2, 32772600kB, raid superblock at 32772480kB disk 4: /dev/sdj1, 32772568kB, raid superblock at 32772480kB disk 5: /dev/sdj2, 32772600kB, raid superblock at 32772480kBUnd schliesslich der Spiegel
turnip:/dev# mkraid /dev/md2 handling MD device /dev/md2 analyzing super-block disk 0: /dev/md0, 196634880kB, raid superblock at 196634816kB disk 1: /dev/md1, 196634880kB, raid superblock at 196634816kBWir kontrollieren den Status des RAID und legen eine Partition an die anschliessend formatiert und gemountet wird.
turnip:~# cat /proc/mdstat
Personalities : [raid0] [raid1]
read_ahead 1024 sectors
md2 : active raid1 md/1[1] md/0[0]
196634816 blocks [2/2] [UU]
[>....................] resync = 0.0% (16988/196634816) finish=12946.0min speed=251K/sec
md1 : active raid0 scsi/host1/bus0/target8/lun0/part2[5] scsi/host1/bus0/target8/lun0/part1[4] scsi/host1/bus0/target6/lun0/part2[3] scsi/host1/bus0/target6/lun0/part1[2] scsi/host1/bus0/target5/lun0/part1[1] scsi/host1/bus0/target4/lun0/part1[0]
196634880 blocks 4k chunks
md0 : active raid0 scsi/host1/bus0/target3/lun0/part2[5] scsi/host1/bus0/target3/lun0/part1[4] scsi/host1/bus0/target2/lun0/part2[3] scsi/host1/bus0/target2/lun0/part1[2] scsi/host1/bus0/target1/lun0/part1[1] scsi/host1/bus0/target0/lun0/part1[0]
196634880 blocks 4k chunks
unused devices:
turnip:~# fdisk -l /dev/md2
Disk /dev/md2: 2 heads, 4 sectors, 49158704 cylinders
Units = cylinders of 8 * 512 bytes
Disk /dev/md2 doesn't contain a valid partition table
turnip:~# fdisk /dev/md2
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel. Changes will remain in memory only,
until you decide to write them. After that, of course, the previous
content won't be recoverable.
The number of cylinders for this disk is set to 49158704.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
(e.g., DOS FDISK, OS/2 FDISK)
Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-49158704, default 1):
Using default value 1
Last cylinder or +size or +sizeM or +sizeK (1-49158704, default 49158704): ENTER
Using default value 49158704
Command (m for help): p
Disk /dev/md2: 2 heads, 4 sectors, 49158704 cylinders
Units = cylinders of 8 * 512 bytes
Device Boot Start End Blocks Id System
/dev/md2p1 1 49158704 196634814 83 Linux
Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
WARNING: Re-reading the partition table failed with error 22: Invalid argument.
The kernel still uses the old table.
The new table will be used at the next reboot.
Syncing disks.
Nun vergeht viel Zeit bis der Spiegel hergestellt ist! Immerhin müssen alle Blocks abgeglichen werden. Deswegen hängt die benötgte Zeit von der Anzahl der Blöcke, nicht vom Inhalt auf der Platte (noch ist ja nichts drauf) ab. Immerhin ist die angezeigte Zeit (rot) zuverlässig.
Nun wird die Partition gemoutet, ein entsprechender Eintrag in der /etc/fstab Hin und wieder /proc/mdstat kontrollieren oder dieses von einem Script erledigen lassen (nach U_ suchen).
/dev/md2 /export/video auto defaults,errors=remount-ro 0 1Sollte später eine Platte ausfallen wir das RAID als degraded angezeigt.
turnip:~# cat /proc/mdstat
Personalities : [raid0] [raid1]
read_ahead 1024 sectors
md2 : active raid1 md/0[0]
196634816 blocks [2/1] [U_]
md0 : active raid0 scsi/host1/bus0/target3/lun0/part2[5] scsi/host1/bus0/target3/lun0/part1[4] scsi/host1/bus0/target2/lun0/part2[3] scsi/host1/bus0/target2/lun0/part1[2] scsi/host1/bus0/target1/lun0/part1[1] scsi/host1/bus0/target0/lun0/part1[0]
196634880 blocks 4k chunks
unused devices:
Also das System runterfahren, die defekte Platte ersetzen und wieder booten. Falls kein Dateisystem auf der Platte ist wird das beim booten bemerkt (u.U. mit dmes prüfen). Im Bedarfsfall ein Dateisystem anlegen
. . md: scsi/host1/bus0/target0/lun0/part1 [events: 0000000e]<6>(write) scsi/host1/bus0/target0/lun0/part1's sb offset: 32772480 md: ... autorun DONE. [events: 0000000a] md: could not lock sdh1, zero-size? Marking faulty. md: could not import sdh1, trying to run array nevertheless.. . . turnip:~# fdisk /dev/sdh Command (m for help): p Disk /dev/sdh: 73.4 GB, 73407865856 bytes 255 heads, 63 sectors/track, 8924 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Device Boot Start End Blocks Id System Command (m for help): n Command action e extended p primary partition (1-4) p Partition number (1-4): 1 First cylinder (1-8924, default 1): Using default value 1 Last cylinder or +size or +sizeM or +sizeK (1-8924, default 8924): Using default value 8924 Command (m for help): w The partition table has been altered! Calling ioctl() to re-read partition table. Syncing disks.Nun mit mkraid diese Seite des RAID wieder herstellen. Dabei muss der Schalter --force bzw. --really-force angegeben werden. Nach einem weiteren reboot (oder Neustart von md2) wird der Spiegel wieder hergestellt, die Daten können jedoch sofort wieder verwendet werden. To be continued ....