setup-storage

NAME

setup-storage - automatically prepare storage devices

SYNOPSIS

setup-storage [-X] [-f filename] [-d] [-h] [-s] [-D disks] [-L directory]

DESCRIPTION

Using FAI disk_config files, setup-storage Computes effective partition and volume sizes and executes the necessary commands to configure storage devices. It manages disk drives, RAID and LVM volumes, along with encryption and tmpfs. Disks and partitions are designated by disk1.2, disk3.1 etc. and may thus be referenced as such. Currently, setup-storage handles following filesystems: ext2/3/4, vFAT(FAT32), msdos(FAT16), reiserFS, XFS and BTRFS but could easily be extended to further types as well. Once the storage devices are prepared, an appropriate fstab(5) (and possibly also a crypttab(5) file is generated.

Without the -X parameter setup-storage runs in test-only mode and does not execute commands other than writing disk labels to a blank disk.

The exit code of setup-storage is 0 if all operations were performed successfully and non-zero if an error occurs.

OPTIONS

-f filename

Normally setup-storage selects an appropriate configuration from $FAI/disk_config/ by picking the class with the highest priority from classes that has an existing file. If however -f is given the configuration in filename is used.

-D disks

Specify the list of disk drives to be configured using setup-storage. Overrides the variable disklist. Do not forget to quote this space separated list.

-L directory

Use the specified directory instead of LOGDIR.

ENVIRONMENT

setup-storage will use the following environment variables:
disklist

If option -D is not used, the disklist variable may contain a space separated list of disk drives available in the system. Their order matters as they may be referred to as disk1, and so on, in disk_config. If unset, fai-disk-info will be called to determine the list.

classes

The list of FAI classes to determine the appropriate configuration to choose.

flag_initial

This variable determines if partitions should be preserved when they are tagged with preserve_reinstall (see below for details). Normally set by the list of FAI flags (FAI_FLAGS).

SS_IGNORE_VG

This variable may contain a list of volume groups, that are ignored by setup-storage. The list can be comma or space separated. It’s used by fai-diskimage.

FILES

If setup-storage executes successfully, an fstab(5) file matching the specified configuration is generated as $LOGDIR/fstab. Furthermore the file $LOGDIR/disk_var.sh is generated. This file defines the following variables, if not yet set: SWAPLIST, ROOT_PARTITION, BOOT_PARTITION (which is only set in case this resides on a disk drive), BOOT_DEVICE and PHYSICAL_BOOT_DEVICES (which contains the list of all physical devices having a bootable partition). Both BOOT_PARTITION and BOOT_DEVICE describe the partition and disk/RAID/LVM device hosting the mount point for /boot. If /boot has no extra mount point, / is used instead. PHYSICAL_BOOT_DEVICES can be used to determine where a bootloader should be installed (this is useful if / is on a LVM or RAID device). You may source $LOGDIR/disk_var.sh to get the variables set. The example config space shipped with FAI sources this file in scripts/GRUB_PC/10-setup. If encryption was configured, a proper crypttab(5) file plus key files will be generated.

The output of setup-storage is also written to the log file format.log.

EXAMPLES

setup-storage configures storage devices according to a FAI disk_config file. The full grammar describing the syntax of these disk_config files is given below. The syntax is similar to the structure of an fstab file. First we show a number of examples to give an intuition what these should look like.
Simple configuration of /dev/sda

disk_config disk1 disklabel:msdos bootable:3

Create a software RAID

disk_config sda

disk_config sdb sameas: sda

disk_config raid

Advanced softRAID features

disk_config raid

Simple LVM example

disk_config sda bootable:1

disk_config lvm

LVM on software RAID

disk_config disk1

disk_config disk2 sameas:disk1

disk_config raid fstabkey:uuid

disk_config lvm fstabkey:uuid

Do not forget to install the packages mdadm and lvm2 into a system using this partition scheme. Also set rd.auto when using dracut as initrd generator.

Crypt example

disk_config /dev/sdb

disk_config cryptsetup

disk_config cryptsetup

Plain disk + LUKS + LVM2 example

disk_config disk1 disklabel:gpt bootable:1 fstabkey:uuid align-at:1M

disk_config cryptsetup

disk_config lvm fstabkey:uuid

tmpfs example

disk_config tmpfs

nfs example

disk_config nfs

Simple BTRFS example

disk_config disk1

disk_config disk2

disk_config disk3 sameas:disk2
disk_config disk4 sameas:disk2

disk_config btrfs fstabkey:uuid

Subvolume names begin with an @ by convention.

External log device example

disk_config /dev/sda fstabkey:uuid bootable:2

disk_config /dev/sdb fstabkey:uuid

Example using a GPT partition table and EFI/ESP partition

disk_config disk1 disklabel:gpt fstabkey:partlabel bootable:1

SYNTAX

This section describes the syntax of disk_config files

file ::= <lines> EOF

lines ::= EOL
/* empty lines or whitespace only */
| <comment> EOL
| <config> EOL

comment ::= #.*

config ::= disk_config lvm( <lvmoption>)*
| disk_config raid( <raidoption>)*
| disk_config cryptsetup( <cryptsetupoption>)*
| disk_config tmpfs
| disk_config end
| disk_config disk[[:digit:]]+( <option>)*
| disk_config [^[:space:]]+( <option>)*
/* fully qualified device-path or short form, like sda, whereby full
* path is assumed to be /dev/sda; may contain shell globbing such
* as /dev/disk/by-id/scsi-* */
| <volume>

lvmoption ::= /* empty */
| preserve_always:([^/,\s\-]+-[^/,\s\-]+(,[^/,\s\-]+-[^/,\s\-]+)*|all)
/* preserve volumes -- always */
| preserve_reinstall:([^/,\s\-]+-[^/,\s\-]+(,[^/,\s\-]+-[^/,\s\-]+)*|all)
/* preserve volumes -- unless the system is installed for the
first time */
| preserve_lazy:([^/,\s\-]+-[^/,\s\-]+(,[^/,\s\-]+-[^/,\s\-]+)*|all)
/* preserve volumes -- unless these don’t exist yet */
| always_format:([^/,\s\-]+-[^/,\s\-]+(,[^/,\s\-]+-[^/,\s\-]+)*|all)
/* run mkfs on the volumes, even if marked as preserve */
| resize:([^/,\s\-]+-[^/,\s\-]+(,[^/,\s\-]+-[^/,\s\-]+)*|all)
/* attempt to resize partitions */
| fstabkey:(device|label|uuid)
/* when creating the fstab, the key used for defining the device
may be the device (/dev/xxx), a label given using -L,
the filesystem uuid, partition label (only when using GPT) or partition uuid
*/

raidoption ::= /* empty */
| preserve_always:([[:digit:]]+(,[[:digit:]]+)*|all)
/* preserve volumes -- always */
| preserve_reinstall:([[:digit:]]+(,[[:digit:]]+)*|all)
/* preserve volumes -- unless the system is installed for the
first time */
| preserve_lazy:([[:digit:]]+(,[[:digit:]]+)*|all)
/* preserve volumes -- unless these don’t exist yet */
| always_format:([[:digit:]]+(,[[:digit:]]+)*|all)
/* run mkfs on the volumes, even if marked as preserve */
| fstabkey:(device|label|uuid)
/* when creating the fstab the key used for defining the device
may be the device (/dev/xxx), a label given using -L, or the uuid
*/

cryptsetupoption ::= /* empty */
| randinit
/* initialise all encrypted partitions with random data */

option ::= /* empty */
| preserve_always:([[:digit:]]+(,[[:digit:]]+)*|all)
/* preserve partitions -- always; the numbers refer to partition
numbers, i.e., preserve_always:5 for /dev/sda refers to /dev/sda5,
which may not necessarily be the 5th line of the configuration */
| preserve_reinstall:([[:digit:]]+(,[[:digit:]]+)*|all)
/* preserve partitions -- unless the system is installed for the
first time. See preserve_always above for the semantics of numbers
used for referring to partitions. */
| preserve_lazy:([[:digit:]]+(,[[:digit:]]+)*|all)
/* preserve partitions -- unless these don’t exist yet */
| always_format:([[:digit:]]+(,[[:digit:]]+)*|all)
/* run mkfs on the partitions, even if marked as preserve */
| resize:([[:digit:]]+(,[[:digit:]]+)*|all)
/* attempt to resize partitions */
| disklabel:(msdos|gpt|gpt-bios)
/* write a disklabel - default is msdos */
| bootable:[[:digit:]]+
/* mark a partition bootable, default is / */
| virtual
/* do not assume the disk to be a physical device, use with xen */
| fstabkey:(device|label|uuid|partuuid|partlabel)
/* when creating the fstab the key used for defining the device
may be the device (/dev/xxx), a label given using -L, or the uuid
*/
| sameas:(disk[[:digit:]]+|[^[:space:]]+)
/* Indicate that this disk will use the same scheme
as the given device. The referenced device must be
defined before the device using this option. Use only
with identical hardware.
*/
| align-at:([[:digit:]]+[kKMGTPiB]*)
/* Align partitions at multiples of the given block size (unit
defaults to MiB, if omitted). Such an alignment, e.g., 4K, might be
important for proper performance of RAID arrays which use a logical
block size other than the sector size of the underlying disks. It
must, however, always be a multiple of this sector size.
*/

volume ::= <type> <mountpoint> <size> <filesystem> <mount_options> <luks_options> <fs_options>
| vg <name> <size> <fs_options>
/* lvm vg */
| tmpfs <mountpoint> <tmpfs_size> <mount_options>
/* tmpfs volume */

type ::= primary
/* for physical disks only */
| logical
/* for physical disks only */
| raw-disk
/* for physical disks only: do not partition this disk, use it as-is */
| m{^p=([^/,;]*).br
/* p=<NAME> set NAME as gpt partition label, NAME may be empty */
| raid[0156]
/* raid level */
| luks
/* encrypted partition using LUKS and auto-generate a key file */
| luks:"[^"]+"
/* encrypted partition using LUKS and use quoted string as passphrase */
| tmp
/* encrypted partition for /tmp usage, will be
recreated with a random key at each boot and
reformatted as ext2 */
| swap
/* encrypted partition for swap space usage, will
be recreated with a random key at each boot and
reformatted as swap space */
| [^/[:space:]]+-[^/[:space:]]+
/* lvm logical volume: vg name and lv name*/

mountpoint ::= (-|swap|/[^[:space:]]*)
/* do not mount, mount as swap, or mount at fully qualified path */

name ::= [^/[:space:]]+
/* lvm volume group name */

sizespec ::= RAM:[[:digit:]]+%|[[:digit:]]+[kKMGTP%iB]*
/* size in kilo (KiB), mega (default, MiB), giga (GiB), tera (TiB),
* petabytes (PiB) or percentage of disk size or RAM size; integers
* only, no decimal numbers.
* Use KB, MB, GB, ... for a factor of 1000 instead of 1024 as
* multiplier */

size ::= <sizespec>(-(<sizespec>)?)?(:resize|:preserve_(always|reinstall|lazy))?
/* size, possibly given as a range; physical partitions or lvm logical
* volumes only */
| -<sizespec>(:resize|:preserve_(always|reinstall|lazy))?
/* size given as upper limit; physical partitions or lvm logical
* volumes only */
| [^,:[:space:]]+(:(spare|missing))*(,[^,:[:space:]]+(:(spare|missing))*)*
/* devices and options for a raid or lvm vg */

tmpfs_size ::= <sizespec>
/* tmpfs size */

mount_options ::= [^[:space:]]+

filesystem ::= -
| swap
| [^[:space:]]
/* mkfs.xxx must exist */

luks_options ::= (lukscreateopts=".*")
/* options to supply to cryptsetup when creating a LUKS
* encrypted filesystem. If no ciper (-c) is specified, then
* aes-xts-plain64 is used. If no key size (-s) is
* specified then 256 is used. */

fs_options ::= (createopts=".*"|tuneopts=".*"|(pv|vg|lv|md)createopts=".*")*
/* options to append to mkfs.xxx and to the filesystem-specific
* tuning tool, pvcreate, vgcreate, lvcreate or mdadm */

CAVEATS

SEE ALSO

This program is part of FAI (Fully Automatic Installation). The FAI homepage is https://fai-project.org.

Further documentation, including coding related information, is available in a wiki page at https://wiki.fai-project.org/index.php/Setup-storage.

AUTHOR

The setup-storage program was written by Michael Tautschnig <mt@debian.org>, with contributions from Christian Kern, Julien Blache <jblache@debian.org>, Kerim Güney, Thomas Lange and others.

The original and primary author of FAI is Thomas Lange <lange@cs.uni-koeln.de>.