OpenHPC:Flexboot Configuration - Revision history

Patrick at 09:50, 19 March 2018

2018-03-19T09:50:09Z

Patrick at 09:48, 19 March 2018

2018-03-19T09:48:54Z

Patrick at 09:36, 19 March 2018

2018-03-19T09:36:49Z

Patrick: /* HWPREFIX and HWADDR */

2018-03-16T16:52:30Z

HWPREFIX and HWADDR

Patrick: /* HWPREFIX and HWADDR */

2018-03-16T16:48:28Z

HWPREFIX and HWADDR

Patrick: /* KARGS */

2018-03-16T14:50:42Z

KARGS

Patrick at 14:48, 16 March 2018

2018-03-16T14:48:41Z

Patrick: Patrick moved page OpenHPC:flexboot to OpenHPC:Flexboot Configuration

2018-03-16T12:30:05Z

Patrick moved page OpenHPC:flexboot to OpenHPC:Flexboot Configuration

Patrick at 12:28, 16 March 2018

2018-03-16T12:28:52Z

Patrick at 12:26, 16 March 2018

2018-03-16T12:26:20Z

@@ Line 139: / Line 139: @@
 wwsh pxe update
 wwsh dhcp update
 </syntaxhighlight>

@@ Line 7: / Line 7: @@
 ===Provisioning Interface===
-Set the 'network device' entry in /etc/warewulf/provision.conf to the correct interface. Fer example if using infiniband and the interface on the headnode is ib0 set it as follows
+Set the 'network device' entry in '''/etc/warewulf/provision.conf''' to the correct interface. For example if using infiniband and the interface on the headnode is ib0, set it as follows
 <syntaxhighlight>
@@ Line 69: / Line 69: @@
 </syntaxhighlight>
-If booting over ethernet then the ib_* modules should not be needed. From what I have been able to gather, just mlx4_core or mlx5_core is needed for ethernet.
+If booting over ethernet then the ib_* modules should not be needed, but there is no harm in including them anyway. From what I have been able to gather, just mlx4_core or mlx5_core is needed for ethernet.
 ===DHCP===
-The default DHCP configuration provided by Warewulf won't work with FlexBoot. Despite FlexBoot being modified iPXE, the dhcp server won't recognise it as such and so will provide incorrect configuration. To get around this edit /etc/warewulf/dhcpd-template.conf to look like below
+The default DHCP configuration provided by Warewulf won't work with FlexBoot. Despite FlexBoot being modified iPXE, the dhcp server won't recognise it as such and so will provide incorrect configuration. To get around this edit '''/etc/warewulf/dhcpd-template.conf''' to look like below
 <syntaxhighlight>
@@ Line 90: / Line 90: @@
 option architecture-type   code 93  = unsigned integer 16;
 filename "http://%{IPADDR}/WW/ipxe/cfg/${mac}";
 subnet %{NETWORK} netmask %{NETMASK} {
@@ Line 120: / Line 104: @@
 Note that the only change made is commenting out the architecture detection and forcing dhcp to provide each node with 'filename "http://%{IPADDR}/WW/ipxe/cfg/${mac}";'
-These files are in /srv/warewulf/ipxe/cfg on the headnode and contain the network information for iPXE. Without this change FlexBoot is detecting incorrectly and tries to use undionly.kpxe which does not work.
+These files are in ''/srv/warewulf/ipxe/cfg''' on the headnode and contain the PXE configuration for each node. Without this change FlexBoot is detecting incorrectly and tries to use undionly.kpxe which does not work, since FlexBoot does not support chainloading PXE (http://www.mellanox.com/related-docs/prod_software/FlexBoot-3_5_305_for_ConnectX4_ConnectX-5_release_notes.pdf see table 7)
 =Infiniband=
@@ Line 157: / Line 141: @@
 </syntaxhighlight>
-To confirm that the changes are applied correctly look at /etc/dhcp/dhcpd.conf. Each host entry show now contain an option called dhcp-client-identifier that is set to the full 20 Byte GUID.
+To confirm that the changes are applied correctly look at '''/etc/dhcp/dhcpd.conf'''. Each host entry show now contain an option called dhcp-client-identifier that is set to the full 20 Byte GUID.
 <syntaxhighlight>
@@ Line 169: / Line 153: @@
 </syntaxhighlight>
-Create links with 6 byte addresses
+===Spoof 6 byte address configs===
-FlexBoot uses 8 byte addresses but PXE only recognises 6 byte addresses. To ensure that pxe can grab the correct configuration, create symbolic links with the 6 byte address that points to the config file for the 8 byte address. To convert from 8 byte addresses to 6 byte addresses, just remove the middle 2 bytes (most likely 03:00).
+FlexBoot uses 8 byte addresses but PXE only recognises 6 byte addresses. To ensure that pxe can grab the correct configuration, create symbolic links in '''/srv/warewulf/ipxe/cfg''' with the 6 byte address that points to the config file for the 8 byte address. To convert from 8 byte addresses to 6 byte addresses, keep the first and last 3 bytes but remove the middle 2 bytes (most likely 03:00).
 <syntaxhighlight>

@@ Line 127: / Line 127: @@
 ===HWPREFIX and HWADDR===
 Infiniband devices have a 20 byte GUID which is comprised of a 12 byte prefix and 8 byte hardware address.
-In warewulf set the hwprefix on each node to 'ff:00:00:00:00:00:02:00:00:02:c9:00'. Set the hwaddr to the 8 byte address.
+In warewulf set the hwprefix on each node to 'ff:00:00:00:00:00:02:00:00:02:c9:00'. This prefix is hard coded in Flexboot and should be constant across all installations. Set the hwaddr to the last 8 bytes of the GUID.
 <syntaxhighlight>

@@ Line 170: / Line 170: @@
 Create links with 6 byte addresses
-FlexBoot uses 8 byte addresses but PXE only recognises 6 byte addresses. To ensure that pxe can grab the correct configuration, create sym links with the 6 byte address that points to the config file for the 8 byte address. To convert from 8 byte adressess to 6 byte addresses, just remove the middle 2 bytes (most likely 03:00).
+FlexBoot uses 8 byte addresses but PXE only recognises 6 byte addresses. To ensure that pxe can grab the correct configuration, create symbolic links with the 6 byte address that points to the config file for the 8 byte address. To convert from 8 byte addresses to 6 byte addresses, just remove the middle 2 bytes (most likely 03:00).
 <syntaxhighlight>
@@ Line 176: / Line 176: @@
 </syntaxhighlight>
-A simple bash script to create these symlinks
+A simple bash script to create these symbolic links
 <syntaxhighlight>

@@ Line 181: / Line 181: @@
 #!/bin/bash
-WW_DIR=/srv/warewulf/ipxe/cfg
+PXE_CONF_DIR=/srv/warewulf/ipxe/cfg
-macs=$(ls ${WW_DIR})
+macs=$(ls ${PXE_CONF_DIR})
 for mac in ${macs[@]}; do
-         ln -s ${WW_DIR}/$mac ${WW_DIR}/${mac:0:9}${mac:15}
+         ln -s ${PXE_CONF_DIR}/$mac ${PXE_CONF_DIR}/${mac:0:9}${mac:15}
 done
 </syntaxhighlight>

@@ Line 63: / Line 63: @@
 ===KARGS===
-During the PXE process, the bootstrap kernel will attempt to run, but it might not load the Mellanox kernel modules. To force the kernel to load the modules set the KARGS variable. At this point in time I an not sure which modules exactly are needed so I just give them all.
+During the PXE process, the bootstrap kernel will attempt to run, but it might not load the Mellanox kernel modules so provisioning will fail. To force the kernel to load the modules set the KARGS variable. At this point in time I am not sure which modules exactly are needed so I just give them all.
 <syntaxhighlight>

← Older revision	Revision as of 12:30, 16 March 2018
(No difference)

← Older revision		Revision as of 12:28, 16 March 2018
Line 2:		Line 2:

	=General setup=		=General setup=
		+
		+	The configuration in this sections applies to both ethernet and ib interfaces.

	===Provisioning Interface===		===Provisioning Interface===