PXE boot notes (part 5, kickstart config)

By this point in the process we’ve had a lot of magic:

  • automagically gaining an IP address via BOOTP (and subsequently DHCP)
  • fetching a bootloader via TFTP
  • having the bootloader fetch a kernel and initial ramdisk via TFTP
  • preparing the HTTP distro environment and populating it

But that magic is nothing on the next bit.

If you’re not familiar with Kickstart – why not? Most likely you’re not using a Red Hat derived distribution such as CentOS, Scientific Linux or Fedora. Or you are, but you’ve never had a need to simultaneously provision lots of bare metal installs – I mean, who uses bare metal these days? It’s so 2010!

Oh, people like me.

Anyway – in Part 3 I detailed the config file for the PXE bootloader, which referred to a boot option:

inst.ks=http://10.30.110.40/ks.cfg

That should be pretty clear: go fetch a file called ks.cfg from that webserver.

I already detailed a Kickstart configuration file in Part 1, and this one is more or less the same with a couple of notable differences. Firstly, the method is different – rather than

install
cdrom

we have

install
url --url http://10.30.100.40/centos/
text

The ‘url’ option tells the installer (which was provided over TFTP) to fetch its useful bits from a local webserver.

The next section is as follows:

reboot

lang en_GB.UTF-8
keyboard --vckeymap=gb --xlayouts='gb'
timezone --utc Europe/London

ignoredisk --only-use=sda

network --onboot yes --device enp6s0 --bootproto dhcp

authconfig --enableshadow --passalgo=sha512
rootpw --iscrypted [SHA512_HASH]

selinux --disabled

bootloader --location=mbr --driveorder=sda --append="crashkernel=auto rhgb quiet"

clearpart --all --drives=sda

part /boot --fstype=xfs --size=500
part pv.08002 --grow --size=1

volgroup vg_centos --pesize=4096 pv.08002
logvol / --fstype=ext4 --name=lv_root --vgname=vg_centos --grow --size=1024 --maxsize=204800
logvol swap --name=lv_swap --vgname=vg_centos --grow --size=4096 --maxsize=4096

%packages --nobase
openssh-server
%end

So nothing that special so far – the ‘reboot’ option means the install auto-completes without waiting for human intervention, which makes this a completely unattended installation.

And here is where, as they say, the magic happens: the post-installation section. This starts with ‘%post’ and is usually running in the chroot of the newly installed operating system (it is possible to override that if required but can make things considerably more complicated in terms of referencing the right paths). Firstly:

%post --log=/root/postinstall.log

echo "proxy=http://[PROXY_SERVER]:80" >> /etc/yum.conf

yum -y install strace sysstat at ntp kexec-tools numactl rng-tools wget lsof psmisc postfix iptables dd nfs-utils libaio libaio-devel gcc-c++ net-tools zip unzip tcsh
yum -y update

mkdir /root/postinst
cd /root/postinst
for x in iometer-dynamo.tgz startup_dynamo.sh iperf3-3.0.11-1.fc22.x86_64.rpm jre-8u66-linux-x64.rpm vdbench50403.zip kmod-enic-2.1.1.99-rhel.el7.centos.x86_64.rpm tput.sh sysctl.conf.10GbE sunrpc.conf ntp.conf; do wget http://10.30.110.40/local/$x; done

That section sets up a logfile, sets the proxy server for yum to use, installs a bunch of useful bits and pieces from the distro for this test platform, then fetches another bunch of local requirements which are then processed thus:

# local tools
for x in tput.sh startup_dynamo.sh
do
 cp $x /root && chmod 0700 /root/$x
done

# dynamo
(cd /tmp && tar xvpzf /root/postinst/iometer-dynamo.tgz)
echo "* * * * * root /root/startup_dynamo.sh" >> /etc/cron.d/startup_dynamo

# vdbench
unzip -d /usr/local/vdbench /root/postinst/vdbench50403.zip

mv ntp.conf /etc/ntp.conf
systemctl enable ntpd

yum -y localinstall iperf3-3.0.11-1.fc22.x86_64.rpm jre-8u66-linux-x64.rpm

Now – this was being put together to do some performance testing of a big storage array, as I mentioned right back in Part 1. There were a few things to do which were specific to the testing methodology in that we had 22 separate volumes created for each test machine spread across a number of different devices, using two different NICs (each on a different VLAN and therefore subnet) for the NFS traffic and one for ‘management’. That meant we had to get a bit creative on the addressing and mount front, which required a bit of shell scripting – and yes, there may be far more efficient ways to have done this, but hey ho – it worked. Firstly, setting the NICs up – this depends on the address the management port received over DHCP, and the hostname that was set via the same route (which ended in either an ‘a’ or ‘b’):

# configure secondary NIC for NFS gubbins
ADDR=$( ip ad ls enp6s0 2>&1 | egrep "inet " )
IP=$( echo $ADDR | awk "{ print \$2 }" | sed "s/\/.\+//" )
IP1=$( echo $IP | sed 's/\(^.\+\)\.\([0-9]\+\)/\2/' )
IP2=$( echo $IP | sed 's/\(^.\+\)\.\([0-9]\+\)/\1/' )
echo $HOSTNAME | egrep -q a$
if [ $? -eq 0 ]
then
 MULT=12
 CLASS="prod1"
else
 MULT=106
 CLASS="prod2"
fi
IP3=$(( $IP1+$MULT ))
SECADDR1=$( echo $IP2 | sed 's/\.110/.114/' )
SECADDR2=$( echo $IP2 | sed 's/\.110/.111/' )

MOUNTIP1=$SECADDR1.$IP3
SECINTIP1=$SECADDR1.$IP1
MOUNTIP2=$SECADDR2.$IP3
SECINTIP2=$SECADDR2.$IP1

cat <<EOF > /etc/sysconfig/network-scripts/ifcfg-enp12s0
TYPE=Ethernet
ONBOOT=yes
BOOTPROTO=STATIC
DEFROUTE=no
IPV4_FAILURE_FATAL=no
IPV6INIT=no
IPV6_AUTOCONF=no
IPV6_DEFROUTE=no
IPV6_FAILURE_FATAL=no
NAME=enp12s0
DEVICE=enp12s0
IPADDR=$SECINTIP1
PREFIX=24
MTU=9000

EOF

cat <<EOF > /etc/sysconfig/network-scripts/ifcfg-enp7s0
TYPE=Ethernet
ONBOOT=yes
BOOTPROTO=STATIC
DEFROUTE=no
IPV4_FAILURE_FATAL=no
IPV6INIT=no
IPV6_AUTOCONF=no
IPV6_DEFROUTE=no
IPV6_FAILURE_FATAL=no
NAME=enp7s0
DEVICE=enp7s0
IPADDR=$SECINTIP2
PREFIX=24
MTU=9000

EOF

After that wrangling, some more shell scripting to create the mount points and setup /etc/fstab to mount the appropriate NFS exports from the right device in the right place. Again, this is partially defined by the hostname using $CLASS which we set above:

# sort out NFS mounts
NFSOPTS="rw,bg,vers=3,hard,proto=tcp,timeo=600,rsize=65536,wsize=65536,retry=2 0 0"

for x in {0..9}
do
mkdir /tmp/${CLASS}_${HOSTNAME}_${x}
 cat <<EOF >> /etc/fstab
${MOUNTIP1}:/${CLASS}_${HOSTNAME}_${x} /tmp/${CLASS}_${HOSTNAME}_${x} nfs $NFSOPTS
EOF
done

for x in {10..19}
do
mkdir /tmp/${CLASS}_${HOSTNAME}_${x}
 cat <<EOF >> /etc/fstab
${MOUNTIP2}:/${CLASS}_${HOSTNAME}_${x} /tmp/${CLASS}_${HOSTNAME}_${x} nfs $NFSOPTS
EOF
done

cat <<EOF >> /etc/fstab
${MOUNTIP1}:/${CLASS}_${HOSTNAME} /tmp/${CLASS}_${HOSTNAME} nfs $NFSOPTS
${MOUNTIP2}:/${CLASS}_${HOSTNAME}_clone /tmp/${CLASS}_${HOSTNAME}_clone nfs $NFSOPTS
EOF

mkdir -p /tmp/${CLASS}_${HOSTNAME}
mkdir -p /tmp/${CLASS}_${HOSTNAME}_clone

The next section creates some SSH keys and adds known keys to root’s ‘authorized_keys’ file, which is left as an exercise for the reader. Finally, we add some options to /etc/rc.local which are very specific to this environment (CPU affinity for certain tasks, for example) which are left out as they’re a bit too specific for a general document.

Finally, we create a consistent /etc/hosts and apply it so all the test machines know each other’s names and don’t need to use DNS:

cat <<EOF > /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
10.30.110.21 centos1a
10.30.110.22 centos2a
10.30.110.23 centos3a
10.30.110.24 centos4a
10.30.110.25 centos5a
10.30.110.26 centos6a
10.30.110.27 centos1b
10.30.110.28 centos2b
10.30.110.29 centos3b
10.30.110.30 centos4b
10.30.110.31 centos5b
10.30.110.32 centos6b

10.30.110.40 perf-mgmt
EOF

%end

And with that ‘%end’ option, that’s that.

Setting the machines to boot from the network (using the blade profile management tools) and rebooting them takes us from bare metal to a ready-made test platform in a little under 15 minutes.

Magic!

PS: One last step is to change the pxelinux.cfg file to boot from local disk after they’ve started installing otherwise they’ll just go round and round and round in circles, reinstalling themselves each time. That’s not a desired outcome! I’ve got a shell script to do that, which simply copies in the right file to the right place once the installs have reached the postinstall phase, but again I’ll leave that to the reader to work out. It isn’t that hard!

Advertisements

One comment

  1. Pingback: PXE boot notes (part 4, serving files) « Random Ramblings


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s