PartMan

From Wikitech

This page is about partitioning with preseed. See Ubuntu installer for other parts of the installation process and preseed options.

Background Info

Debian preseed's partman options are an incomprehensible automatic partitioning language.{{fact}}

Standard recipes / disk layout

As of 2020 most hosts are installed via standard partman recipes found in modules/install_server/files/autoinstall/partman/ . There are two main categories of standard recipes: software and hardware RAID, plus VMs for which including partman/flat.cfg virtual.cfg in netboot.cfg is sufficient.

Note: as of 2023 Nov 15th, the netboot.cfg file is now generated by puppet, from configuration located in modules/profile/data/profile/installserver/preseed.yaml. This was done to avoid typos creep through the file, which has caused many engineering hours to debug over time.

Software raid

The host has a number N of (equal size) devices and will be installed with the chosen raid level L (0, 1, 10) plus LVM on top using volume group vg0. The intended usage for the recipe is:

echo partman/standard.cfg partman/raidL-Ndev.cfg

A special case is software raid0 for which a common recipe snippet needs to be included:

echo partman/standard.cfg partman/raid0.cfg partman/raid0-Ndev.cfg

Hardware raid

In this case the host uses on board hardware raid and typically a single block device is presented to the operating system. The host will have LVM on top of the hardware raid, similar to software raid. Intended usage of the recipe is:

echo partman/standard.cfg partman/hwraid-1dev.cfg

LVM layout

The standard recipes provide a / partition of 80GB, swap 1GB and a /srv taking up to 80% of the remaining space on the VG. The space is not allocated all at installation time to provide some leeway in emergencies and a little bit of headroom in capacity planning. The filesystems can be expanded online at any time with e.g. lvextend --resizefs

Non-standard recipes

While a majority of cases are covered by standard recipes, some special cases remain. These are typically storage hosts of some kind and their recipes live in partman/custom/. New hosts/use cases should strive to use the standard recipes if at all possible, if not consider whether a custom recipe can be adapted to your use case.

Help Files

These files are in the Debian and Ubuntu packages, and give some explanation

Important note about RAID

From https://www.debian.org/releases/squeeze/example-preseed.txt

When you use multiraid, you specify the partitions layout that each disk will get. Later on you'll be able to define them via partman-auto-raid/recipe.

## Partitioning using RAID
# The method should be set to "raid".
#d-i partman-auto/method string raid
# Specify the disks to be partitioned. They will all get the same layout,
# so this will only work if the disks are the same size.
#d-i partman-auto/disk string /dev/sda /dev/sdb

# Next you need to specify the physical partitions that will be used. 
#d-i partman-auto/expert_recipe string \
#      multiraid ::                                         \
#              1000 5000 4000 raid                          \
#                      $primary{ } method{ raid }           \
#              .                                            \
#              64 512 300% raid                             \
#                      method{ raid }                       \
#              .                                            \
#              500 10000 1000000000 raid                    \
#                      method{ raid }                       \
#  

In the above example, partman is instructed to create three partitions on each disk, so you'll probably get: sda1/sda2/sda3 and sdb1/sdb2/sdb3 to play with in partman-auto-raid/recipe (to create raid arrays).

Important note about how partman calculates the final size of the partitions

From https://www.bishnet.net/tim/blog/2015/01/29/understanding-partman-autoexpert_recipe/

I recently created the mw-raid1-lvm partman recipe for new app-servers, ending up with the following config:

# Define physical partitions
d-i	partman-auto/expert_recipe	string	multiraid ::	\
	1000	1000	-1	raid				\
		method{ raid }					\
		$lvmignore{ }					\
		.						\
	10000 71000 600000000 ext4				\
		method{ format }				\
		format{ }					\
		use_filesystem{ }				\
		filesystem{ ext4 }				\
		lv_name{ root }					\
		$defaultignore{ }				\
		$lvmok{ }					\
		mountpoint{ / }					\
		.						\
	10000 30000 200000000 ext4				\
		method{ format }				\
		format{ }					\
		use_filesystem{ }				\
		filesystem{ ext4 }				\
		lv_name{ srv }					\
		$defaultignore{ }				\
		$lvmok{ }					\
		mountpoint{ /srv }				\
		.						\
	10000 27500 200000000 ext4				\
		lv_name{ placeholder }				\
		$defaultignore{ }				\
		$lvmok{ }					\
		.

# Parameters are:
# <raidtype> <devcount> <sparecount> <fstype> <mountpoint> \
#	<devices> <sparedevices>
d-i	partman-auto-raid/recipe	string		\
		1	2	0	lvm	-	\
			/dev/sda1#/dev/sdb1		\
		.

In this case each host offers two 1TB disks, and I wanted to have a single software RAID1 device and LVM Volumes on top of it. The hard part was how to find the right values for min/priority/max in the expert-recipe. After a lot of unsuccessful tests with numbers that I thought to have some sense, I found the link at the beginning of this section that enlightened me.

Let's start with the end result of the above example:

elukey@mw1321:~$ df -h
Filesystem                  Size  Used Avail Use% Mounted on
udev                         10M     0   10M   0% /dev
tmpfs                        13G  9.1M   13G   1% /run
/dev/dm-0                   563G  7.8G  527G   2% /
tmpfs                        32G     0   32G   0% /dev/shm
tmpfs                       5.0M     0  5.0M   0% /run/lock
tmpfs                        32G     0   32G   0% /sys/fs/cgroup
tmpfs                       1.0G     0  1.0G   0% /var/lib/nginx
/dev/mapper/mw1321--vg-srv  191G  8.0G  173G   5% /srv

elukey@mw1321:~$ sudo lvs
  LV          VG        Attr       LSize   Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert
  placeholder mw1321-vg -wi-a----- 166.04g
  root        mw1321-vg -wi-ao---- 571.66g
  srv         mw1321-vg -wi-ao---- 193.69g

elukey@mw1321:~$ sudo pvs
  PV         VG        Fmt  Attr PSize   PFree
  /dev/md0   mw1321-vg lvm2 a--  931.38g    0

What I originally wanted was a LVM volume for root with 600GB, a /srv one with 200GB and 200GB free (the placeholder volume, that is required since partman will allocate all the remaining space in the PV to the last volume defined). The end result is pretty close, but I had to do the following calculations:

	10000 71000 600000000 ext4
        # root
	10000 30000 200000000 ext4
        # srv			
	10000 27500 200000000 ext4
	    # placeholder

Each LVM volume starts with a minimum size of 10G (the first value), so ~970GB are left to use (remember that the underlying PV is a sw RAID1 of 1TB usable space). In order to achieve my goal, the following GBs are needed:

  • 590 for root (~61% of the 970GB)
  • 190 for srv (~20% of the 970GB)
  • 190 for placeholder (~17.5% of the 970GB - this value was a mistake since the most correct one should have been 20%, but the last volume defined should get the remaining space anyway. In this case the srv volume gets a bit more space).

These are exactly the values that you can obtain subtracting the priority value (the middle one) with the minimum one (the first one on the left).

Please note that this is my current understanding of how Partman works, do not trust it blindly.

Config language

<owner> <question name> <question type> <value>

<owner>: "d-i" which stands for Debian Installer.

<question name>: partman-auto, partman-auto-raid and partman-auto-lvm are the packages that handle automatic partitioning of various types, and some of the questions (prompts) issued by them are described here: [1]

<question type> says what sort of value to expect (e.g. string, boolean, select (for a menu)...)

<question value> this is where you put the answer that you would otherwise be entering interactively

NOTE that after the question type you can put only one space or tab; any other whitespace will get stuffed in at the front of the value, which you probably don't want.

Dissecting a Semi-working configuration

(by semi-working, I mean that you must hit <enter> twice, but not answer any meaningful questions.)

# Automatic software RAID 1 with LVM partitioning

d-i     partman-auto/method     string  raid

This sentence makes partman know that it will be making a raid

# Use the first two disks
d-i     partman-auto/disk       string  /dev/sda /dev/sdb

after d-i partman-auto/disk you must put the list of hard drives that are going to be used

# Define physical partitions
d-i     partman-auto/expert_recipe      string  \
                multiraid ::                                                    \
                        400000    1000    9500000       raid                    \
                                $primary{ }  method{ raid }                     \
                        .                                                       \

                        4000    1200    4100    linux-swap                      \
                        $primary{ } method{ swap } format{ }                    \
                        .

An interesting thing about partman is that you can put everything on one line, or you can break lines using the character "\". Use a "." in between hard drives.

  • $primary{ } is needed to make the partition a primary partition.
  • method { } is used to tell it what type to format. You can use swap, raid, or format.
  • format { } tells partman to format the partition. Don't put this statement in a section that you will be using for the raid. use_filesystem{ } makes partman use a file system (don't know why this isn't done by the filesystem command)
  • filesystem{ X } use ext3, murderfs, xfs, etc in here to tell it what filesystem to run
  • mountpoint{ X } use things like /, /mnt/sda3, etc

Note that the sizes listed must fit on the physical disks - using 950000 1000 995000 will work fine on a 1T disk but fail on a 250G disk. The characteristics of 'fail' are that it throws you back to the screen that asks "Guided whole disk, guided xxx, or manual?".

# Parameters are:
# <raidtype> <devcount> <sparecount> <fstype> <mountpoint> \
#          <devices> <sparedevices>
d-i     partman-auto-raid/recipe        string  \
                1       2       0       ext3    /                       \
                        /dev/sda1#/dev/sdb1                             \
                .

This snippet is telling us to make a raid 1, with 2 devices, 0 spares, ext3 filesystem, mounted at /, and across /dev/sda1 and /dev/sda2

d-i     partman-md/confirm              boolean true

Theoretically now it confirms stuff automatically. It doesn't. Partman lies

d-i     partman-md/device_remove_md     boolean true

This lets partman remove any existing raids

d-i     partman/confirm_write_new_label boolean true
d-i     partman/choose_partition        select  finish
d-i     partman/confirm                 boolean true

d-i     partman-lvm/device_remove_lvm   boolean true

d-i     mdadm/boot_degraded             boolean true

Most of these possibly do what they appear to do.

Boot degraded is important - if we have a disk failure we'd still like the system to boot, just warn us.

Dissecting another config

# Application server specific configuration

# Implementation specific hack:
d-i	partman-auto/init_automatically_partition	select	20some_device__________/var/lib/partman/devices/=dev=sda
d-i     partman-auto/method     string  regular
d-i	partman-auto/disk	string	/dev/sda

d-i	partman/choose_partition	select Finish partitioning and write changes to disk
d-i	partman/confirm			boolean	      true

# Note, expert_recipe wants to fill up the entire disk
# See http://d-i.alioth.debian.org/svn/debian-installer/installer/doc/devel/partman-auto-recipe.txt
d-i   partman-auto/expert_recipe	string	apache : 3000 5000 8000 ext3 $primary{ } $bootable{ } method{ format } format{ } use_filesystem{ }
 filesystem{ ext3 } mountpoint{ / } . 1000 1000 1000 linux-swap method{ swap } format{ } . 64 1000 10000000 jfs method{ format } format{ } use_fil
esystem{ } filesystem{ jfs } mountpoint{ /a } .
d-i		   partman-auto/choose_recipe	   apache

# Preseeding of other packages
fontconfig   fontconfig/enable_bitmaps	boolean	true

These are preseed file entries that get fed to prompts issued by the installer. Syntax:

<owner> <question name> <question type> <value>

<owner>: "d-i" which stands for Debian Installer.

<question name>: partman-auto, partman-auto-raid and partman-auto-lvm are the packages that handle automatic partitioning of various types, and some of the questions (prompts) issued by them are described here: [2]

<question type> says what sort of value to expect (e.g. string, boolean, select (for a menu)...)

<question value> this is where you put the answer that you would otherwise be entering interactively

NOTE that after the question type you can put only one space or tab; any other whitespace will get stuffed in at the front of the value, which you probably don't want.

The first few lines of the sample config file should be obvious: select the device, choose the partition method and confirm. After that an "expert recipe" for partioning is defined, and given the name "apache". This allows it to be chosen later from the list of recipes.

Let's look at the particular recipe: it says

  • 3000 5000 8000 ext3 $primary{ } $bootable{ } method{ format } format{ } use_filesystem{ } filesystem{ ext3 } mountpoint{ / }
    3000: minumum size of partition in mb
    5000: priority if it and other listed partitions are vying for space on the disk (this is compared with the priorities of the other partitions)
    8000: maximum size of partition in mb (80GB; this is for 80GB disks, which the apaches all have.)
    ext3: filesystem type
    $primary{ }: this is a primary, not logical partition
    $bootable{ }: this is a bootable partition
    method{ format }: set to format to format the partition, to "keep" to not format, and to "swap" for swap partitions
    format{ }: also needed so the partition will be formatted
    use_filesystem{ }: this partition will have a filesystem on it (it won't be swap, lvm, etc)
    filesystem{ ext3 }: what filesystem it gets
    mountpoint{ / }: where it's mounted

The other two lines in the recipe should now be obvious.

  • 1000 1000 1000 linux-swap method{ swap } format{ }
  • 64 1000 10000000 jfs method{ format } format{ } use_filesystem{ } filesystem{ jfs } mountpoint{ /a }


Things that peter has found out

LVM:

I learned these bits when making mailman.cfg, so I'll be dragging things out of there.

(One of) The (many, many) weird thing(s) about partman is that partman-auto/expert_recipe is used for defining partitions on disks and logical volumes. How could this ever not be confusing!?!? I wanted to put some logical volumes on top of raid1, so here is what I did:

I started with a nice

d-i     partman-auto/expert_recipe      string  \
                multiraid ::                                                    \
                        5000    8000    16000   raid                            \
                                $primary{ } $lvmignore{ } method{ raid }        \
                        .                                                       \

and

d-i     partman-auto-raid/recipe        string  \
                1       2       0       ext3    /                       \
                        /dev/sda1#/dev/sdb1                             \
                .                                                       \

Which yields a slice off of both sda and sdb 16GB large, raided together, ext3 formatted, mounted at /. That's some sweet root partition.

Next thing is in the partman-auto/expert-recipe I added

                        64      1000    10000000        raid                    \
                                $primary{ } $lvmignore { } method{ raid }       \
                        .                                                       \

And in partman-auto-raid/recipe I added

                1       2       0       lvm     -                       \
                        /dev/sda2#/dev/sdb2                             \
                .                                                       

Where the first part is basically like "for sda/sdb, grab all remaining physical disk and partition them for some raid" and the second part is like "get those two partitions, raid them together, and let's use it for some LVM"

So now is the part where we make some LVM partitions. This all happens in partman-auto/expert_recipe. A lot of it seems wildly redundant, but so it goes. First, let's create some swap on the lvm, so we can grow it later if need be. For this, we use

                        4000    4000    4000    linux-swap                      \
                                $defaultignore{ } $lvmok{ }                     \
                                lv_name{ swap } method{ swap } format{ }        \
                        .                                                       \

Which is (relatively) high priority. It probably would have been smarter to have used 100% as the upper bound, so that it was equal to the ram on the box. But this is fine. Still allows for performance tuning.

And now to create an xfs formatted logical volume with a mountpoint we put in

 
                        100     1000    300000          xfs                     \
                                $defaultignore{ } $lvmok{ }                     \
                                lv_name{ mailman } method{ format } format{ }   \
                                use_filesystem{ } filesystem{ xfs }             \
                                mountpoint{ /var/lib/mailman }                  \
                        .                                                       \

Relatively self explanitory options. A name is a name. method{ format } format{ } use_filesystem{ } filesystem{ xfs } seems like a rather obtuse way of saying "no really make me some xfs..." and the mountpoint is the mountpoint. We could have more of these as well. For the sake of completeness, I'll include the other logical volume in mailserver.cfg.

                        100     1000    60000           xfs                     \
                                $defaultignore{ } $lvmok{ }                     \
                                lv_name{ exim } method{ format } format{ }      \
                                use_filesystem{ } filesystem{ xfs }             \
                                mountpoint{ /var/spool/exim4 }                  \
                        .

You'll notice that these are the same priority, so if there isn't enough space, the ratio of the sizes will stay the same (after the swap is made).

Another couple of options use/notes:

 d-i     partman-auto-lvm/guided_size    string  80%

That option is something we are fond of at WMF. It allows us to grow some partitions over time. Basically, it means that when making the logical volumes, to only use 80% of the available disk space. The means that if we need more swap later, or if we're running out of room on one partition, we can use some of the space.

Another take away is that basically in partman-auto/expert_recipe

 $primary{ } $lvmignore{ }

means "make a physical partition" whereas

 $defaultignore{ } $lvmok{ }

means "these are logical volumes."

This is what Peter knows.

Other Helpful Documents