michalczyk.pro/blog
1
0
Fork 0
Code Issues 3 Pull Requests Projects Releases Activity

Reformatting

Browse Source
This commit is contained in:
cryptogopher 2023-08-28 21:51:42 +02:00
parent 57f71f2a8e
commit e1aa7a681a
1 changed files with 15 additions and 22 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

37
_posts/sysadm/2023-06-24-directly-attache-storage-with-redundancy-and-integrity-using-lvm-lvmraid.tl
View File

@ -12,7 +12,8 @@ Data on disk can be lost due to:
* hardware error correction insufficiency or complete disk failure - which is * hardware error correction insufficiency or complete disk failure - which is
obvious and can be remedied using multiple copies of same data on multiple obvious and can be remedied using multiple copies of same data on multiple
disks (RAID). RAID for Logical Volumes is provided by @dm-raid@ kernel module disks (RAID). RAID for Logical Volumes is provided by @dm-raid@ kernel module
and offers functionality similar to @mdadm@ utility. and offers functionality similar to @mdadm@ utility,
* silent data corruption - when erros slip through disk's error detection * silent data corruption - when erros slip through disk's error detection
mechanisms and remain undetected. Risk can be considerably reduced by using mechanisms and remain undetected. Risk can be considerably reduced by using
checksums of data stored on additional integrity sub-LVs using @dm-integrity@ checksums of data stored on additional integrity sub-LVs using @dm-integrity@
@ -21,30 +22,28 @@ Data on disk can be lost due to:
RAID and integrity can be combined: each RAID sub-LV has its per-sector RAID and integrity can be combined: each RAID sub-LV has its per-sector
checksums stored on integrity sub-LVs. Whenever checksum mismatch detects silent checksums stored on integrity sub-LVs. Whenever checksum mismatch detects silent
data corruption, restoration from uncorrupted source is possible thanks to disk data corruption, restoration from uncorrupted source is possible thanks to disk
redundancy. As of LVM tools 2.03.21 (2023-04-21) detecting which disk holds redundancy. As of LVM tools 2.03.21 (2023-04-21), detecting which disk holds
uncorrputed data would be impossible without @dm-integrity@. Even if uncorrputed data would be impossible without @dm-integrity@. Even if
3-disk-RAID1 or RAID6 is used - which theoretically is enough to find source of 3-disk-RAID1 or RAID6 is used - which theoretically is enough to find source of
single sector corruption event - the mechanisms are not implemented. See _man single sector corruption event - the mechanisms are not implemented (see: _man
lvmraid, "Scrubbing Limitations"_. lvmraid, "Scrubbing Limitations"_).
h3. Setup h3. Setup
First create RAID1, then extend it with integrity layer. First create RAID1, then extend it with integrity layer. Integrity block size
cannot be smaller than drive's logical sector size and preferably should match
file system block size. Even before sync is finished, filesystem can be created
on LV:
{% highlight bash %} {% highlight bash %}
lvcreate --type raid1 --mirrors 1 -n backup -L 4t vgbackup /dev/sda /dev/sdb # lvcreate --type raid1 --mirrors 1 -n backup -L 4t vgbackup /dev/sda /dev/sdb
{% endhighlight %} # cat /sys/class/block/sda/queue/physical_block_size
Integrity block size cannot be smaller than drive's logical sector size and
preferably should match file system block size.
{% highlight bash %}
$ cat /sys/class/block/sda/queue/physical_block_size
4096 4096
$ cat /sys/class/block/nvme0n1/queue/logical_block_size # cat /sys/class/block/nvme0n1/queue/logical_block_size
512 512
$ lvconvert --raidintegrity y --raidintegrityblocksize 4096 --raidintegritymode bitmap /dev/vgbackup/backup # lvconvert --raidintegrity y --raidintegrityblocksize 4096 --raidintegritymode bitmap /dev/vgbackup/backup
# mkfs.ext4 -b 4096 -O ^has_journal /dev/vgbackup/backup
{% endhighlight %} {% endhighlight %}
Both operations - creating RAID mirror and adding integrity layer - require Both operations - creating RAID mirror and adding integrity layer - require
@ -54,13 +53,7 @@ initial sync, the progress of which can be monitored:
lvs -a -o name,segtype,devices,sync_percent lvs -a -o name,segtype,devices,sync_percent
{% endhighlight %} {% endhighlight %}
Even before sync is finished, filesystem can be created on LV: Drives can be detached for later synchronization at any time:
{% highlight bash %}
mkfs.ext4 -b 4096 -O ^has_journal /dev/vgbackup/backup
{% endhighlight %}
and the drives can be detached for later synchronization:
{% highlight bash %} {% highlight bash %}
vgchange -an vgbackup vgchange -an vgbackup