Martin: Created VScaler:OpenStack Performance Tweaks

2018-07-06T15:31:21Z

Created VScaler:OpenStack Performance Tweaks

New page

===Disable HT in the BIOS on nova compute nodes.===

===Enable NUMA topology in the BIOS on nova compute nodes.===

===Enable nested virtualization===

Verify nested virtualization is not already enabled:
On Intel systems use:
<syntaxhighlight>
cat /sys/module/kvm_intel/parameters/nested
N
</syntaxhighlight>

On AMD systems use:
<syntaxhighlight>
cat /sys/module/kvm_amd/parameters/nested
N
</syntaxhighlight>

Shut down all VMs and unload the KVM module:
On Intel systems use:
<syntaxhighlight>
modprobe -r kvm_intel
</syntaxhighlight>

On AMD systems use:
<syntaxhighlight>
modprobe -r kvm_amd
</syntaxhighlight>

Activate nested virtualization until the next reboot:
On Intel systems use:
<syntaxhighlight>
modprobe kvm_intel nested=1
</syntaxhighlight>

On AMD systems use:
<syntaxhighlight>
modprobe kvm_amd nested=1
</syntaxhighlight>

Enable persistent nested virtualization:
Add the following line to
<syntaxhighlight>
/etc/modprobe.d/kvm.conf
</syntaxhighlight>

For Intel systems:
<syntaxhighlight>
options kvm_intel nested=1
</syntaxhighlight>

For AMD systems:
<syntaxhighlight>
options kvm_amd nested=1
</syntaxhighlight>

Verify nested virtualization is enabled:
On Intel systems use:
<syntaxhighlight>
cat /sys/module/kvm_intel/parameters/nested
Y
</syntaxhighlight>

On AMD systems use:
<syntaxhighlight>
cat /sys/module/kvm_amd/parameters/nested
Y
</syntaxhighlight>

===Enable cpu_passthrough on nova compute nodes===
Add the following line to
<syntaxhighlight>
/etc/kolla/config/nova/nova-compute.conf
</syntaxhighlight>

<syntaxhighlight>
[libvirt]
cpu_mode=host-passthrough
</syntaxhighlight>

===Set CPU and RAM allocation ratios===
Add or update the following lines to
<syntaxhighlight>
/etc/kolla/config/nova/nova-compute.conf
</syntaxhighlight>

<syntaxhighlight>
[DEFAULT]
ram_allocation_ratio = 1.0
cpu_allocation_ratio = 1.0
</syntaxhighlight>

===Isolate CPU cores on nova compute nodes===
<syntaxhighlight>
grubby --update-kernel=ALL --args="isolcpus=0,1,2,3"
grub2-install /dev/sda
</syntaxhighlight>
where "isolcpus=0,1,2,3" are the cores reserved for VM use.

===Enable vCPU pinning on nova compute nodes===
Add the following line to /etc/kolla/config/nova-compute.conf or /etc/kolla/config/nova/$nodename/nova-compute.conf if different configurations are required on different nodes.
<syntaxhighlight>
[DEFAULT]
vcpu_pin_set=0-35
</syntaxhighlight>
Where 0-35 is the range of dedicated cores to be allocated.

===Nova scheduler configuration===
Add the following filters to the nova-scheduler configuration located in /etc/kolla/config/nova/nova-scheduler.conf
<syntaxhighlight>
scheduler_default_filters= NUMATopologyFilter,AggregateInstanceExtraSpecsFilter
</syntaxhighlight>

===Enable huge pages on nova compute nodes===
Check huge pages status
<syntaxhighlight>
grep Huge /proc/meminfo
</syntaxhighlight>

Add the following to
<syntaxhighlight>
/etc/default/grub
</syntaxhighlight>
<syntaxhighlight>
GRUB_CMDLINE_LINUX="$GRUB_CMDLINE_LINUX hugepagesz=1G hugepages=160 transparent_hugepage=never"
</syntaxhighlight>
Change the number of huge pages according to the host memory configuration.

Grubby can also be used to add the kernel arguments:
<syntaxhighlight>
grubby --update-kernel=ALL --args="hugepagesz=1G hugepages=160 transparent_hugepage=never"
grub2-install /dev/sda
</syntaxhighlight>

===Add huge pages flavor metadata===
In order to enable allocation of hugepages memory, flavors must have the "hw:mem_page_size" value set to either large or the page size.
<syntaxhighlight>
openstack flavor set m1.large --property hw:mem_page_size=large
</syntaxhighlight>

===Add NUMA zone flavor metadata===
Add hw:numa_mempolicy=preferred and hw:numa_nodes=$number_of_numa_nodes to the flavor metadata
<syntaxhighlight>
openstack flavor set m1.large --property hw:numa_mempolicy=preferred
openstack flavor set m1.large --property hw:numa_nodes=2
</syntaxhighlight>

===Add CPU pinning flavor metadata===
Add hw:cpu_policy=dedicated and hw:cpu_thread_policy=preferred to the flavor metadata
<syntaxhighlight>
openstack flavor set m1.large --property hw:cpu_thread_policy=preferred
openstack flavor set m1.large --property hw:cpu_policy=dedicated
</syntaxhighlight>

===Create a pinned CPU host aggregate===
<syntaxhighlight>
nova aggregate-create performance
nova aggregate-set-metadata performance pinned=true
</syntaxhighlight>

===Add hosts to aggregate===
<syntaxhighlight>
nova aggregate-add-host performance node01
</syntaxhighlight>

===Add aggregate metadata to flavor===
<syntaxhighlight>
openstack flavor set m1.large --property aggregate_instance_extra_specs:pinned=true
</syntaxhighlight>

VScaler:OpenStack Performance Tweaks - Revision history

Martin: Created VScaler:OpenStack Performance Tweaks