Difference between revisions of "KVM: Tips and Tricks"

Revision as of 11:19, 14 May 2015

VNC / Args

CLI Args:
-alt-grab - makes ctrl+alt+shift the key escape rather than ctrl+alt (which makes it impossible for windows logins)
on vncviewer, you can also hit f8 to bring up a menu which will allow
-smp n (number of cores to allocate)
-m 8192 (amount of memory to give the machine)
-net nic -net nic (give the system two nics)

Mounting NFS in VM

ERROR: mountd: refused mount nfs illegal port

In case you’re getting error while mounting a nfs share, and logs on server showing:

Mar 24 10:44:57 localhost mountd[8344]: refused mount request from 192.168.1.2 for /dump (/dump): illegal port 52213

Add insecure option in your exports file:

    /dump  *(ro,sync,no_root_squash,insecure)

    # And do 
    exportfs -r  # or reload/restart nfs service.

Grow a KVM Disk Image (RAW)

E.G to grow a KVM image by 10G (Perhaps this could be a hell of a lot easier with LVM)

Create additional space as a raw qemu-img
cat the two files together
Boot gparted-live to resize partitions/filesystem

[root@kvm vm]$ qemu-img create -f raw guest1-rhel6-64-largerdisk 10G
Formatting 'guest1-rhel6-64-largerdisk', fmt=raw size=10737418240 
[root@kvm vm]$ ll
total 17611676
-rwxrwxrwx. 1 root root 10737418240 Sep 27 14:34 guest1-rhel6-64
-rwxr-xr-x  1 root root 10737418240 Sep 27 13:14 guest1-rhel6-64-clone
'-rw-r--r--  1 root root 10737418240 Sep 27 14:37 guest1-rhel6-64-largerdisk'
[root@kvm vm]$ cat guest1-rhel6-64 guest1-rhel6-64-largerdisk > sl6-30g-puppetcobbler.kvm
[root@kvm vm]$ ll
total 38583200
-rwxrwxrwx. 1 root root 10737418240 Sep 27 14:34 guest1-rhel6-64
-rwxr-xr-x  1 root root 10737418240 Sep 27 13:14 guest1-rhel6-64-clone
-rw-r--r--  1 root root 10737418240 Sep 27 14:37 guest1-rhel6-64-largerdisk
'-rw-r--r--  1 root root 21474836480 Sep 27 14:42 sl6-30g-puppetcobbler.kvm'

Just as easy to convert a raw image to qcow2 and then resize that(qemu-img resize [vmname.qcow2] +100G). Use LVM as below to grow the FS.

Now boot up a ubuntu (or other) live cd (in virt-manager: add hardware -> storage -> select managed.. -> device type -> IDE CDrom)

 Install lvm tools: apt-get install lvm2 
 Fdisk create new partition: 
 fdisk /dev/sda
 delete lvm partition (d -> 2 in this instance)
 create lvm partition (same cylinder start, new end pick the default which will be the last cylinder) (n -> p -> start / finish)
 change type of new partition to lvm (t -> 8e)

Use LVM to grow the volume

root@ubuntu:~# pvdisplay /dev/sda2
  --- Physical volume ---
  PV Name               /dev/sda2
  VG Name               vg_puppetmaster
  PV Size               9.51 GiB / not usable 3.00 MiB
  Allocatable           yes (but full)
  PE Size               4.00 MiB
  Total PE              2434
  Free PE               0
  Allocated PE          2434
  PV UUID               LBxPCN-RL3d-rd7d-qEcr-M5Fq-GfMF-ttiPPd
   
root@ubuntu:~# partprobe
Warning: Unable to open /dev/sr0 read-write (Read-only file system).  /dev/sr0 has been opened read-only.
root@ubuntu:~# pvresize /dev/sda2 
  Physical volume "/dev/sda2" changed
  1 physical volume(s) resized / 0 physical volume(s) not resized
root@ubuntu:~# pvdisplay /dev/sda2
  --- Physical volume ---
  PV Name               /dev/sda2
  VG Name               vg_puppetmaster
  PV Size               19.50 GiB / not usable 3.46 MiB
  Allocatable           yes 
  PE Size               4.00 MiB
  Total PE              4992
  Free PE               2558
  Allocated PE          2434
  PV UUID               LBxPCN-RL3d-rd7d-qEcr-M5Fq-GfMF-ttiPPd
   

t@ubuntu:~# vgchange -a y vg_puppetmaster
  2 logical volume(s) in volume group "vg_puppetmaster" now active
root@ubuntu:~# vgdisplay vg_puppetmaster
  --- Volume group ---
  VG Name               vg_puppetmaster
  System ID             
  Format                lvm2
  Metadata Areas        1
  Metadata Sequence No  4
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                2
  Open LV               0
  Max PV                0
  Cur PV                1
  Act PV                1
  VG Size               19.50 GiB
  PE Size               4.00 MiB
  Total PE              4992
  Alloc PE / Size       2434 / 9.51 GiB
  Free  PE / Size       2558 / 9.99 GiB
  VG UUID               Bu7xaM-OQrj-aZYV-gwY8-MvXg-Az1m-8wZh6p
   
root@ubuntu:~# lvextend -L +10G /dev/vg_puppetmaster/lv_root 
  Extending logical volume lv_root to 15.57 GiB
  Insufficient free space: 2560 extents needed, but only 2558 available

root@ubuntu:~# lvextend -L +9.9G /dev/vg_puppetmaster/lv_root 
  Rounding up size to full physical extent 9.90 GiB
  Extending logical volume lv_root to 15.47 GiB
  Logical volume lv_root successfully resized

root@ubuntu:~# resize2fs /dev/vg_puppetmaster/lv_root 
resize2fs 1.41.14 (22-Dec-2010)
Please run 'e2fsck -f /dev/vg_puppetmaster/lv_root' first.

root@ubuntu:~# e2fsck -f /dev/vg_puppetmaster/lv_root 
e2fsck 1.41.14 (22-Dec-2010)
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
/dev/vg_puppetmaster/lv_root: 50681/365760 files (0.1% non-contiguous), 448889/1460224 blocks
root@ubuntu:~# resize2fs /dev/vg_puppetmaster/lv_root 
resize2fs 1.41.14 (22-Dec-2010)
Resizing the filesystem on /dev/vg_puppetmaster/lv_root to 4056064 (4k) blocks.
The filesystem on /dev/vg_puppetmaster/lv_root is now 4056064 blocks long.

root@ubuntu:~# lvscan 
  ACTIVE            '/dev/vg_puppetmaster/lv_root' [15.47 GiB] inherit
  ACTIVE            '/dev/vg_puppetmaster/lv_swap' [3.94 GiB] inherit

Use old VM img with virt-manager

You'll need to use virt-manager to import an old file. Point this towards an existing raw VM and import. Import didnt add a CD drive, so used the command:

    
[root@kvm vm]$ /usr/libexec/qemu-kvm -hda ./sl6-30g-puppetcobbler.kvm \
      -cdrom /mnt/vstor/gparted-live/gparted-live-0.7.1-1.iso -m 512 -boot d
Using CPU model "cpu64-rhel6"
VNC server running on `::1:5900'

# Then vmcviewer on another terminal
vncviewer localhost:5900

Copying VMs udev renames network

udev seems to be adding lines in /etc/udev/70-persistent-net.rules
Remove entries and set MAC correctly
Also had to update the files in /etc/sysconfig/network-scripts/ifcfg-eth0 to remove/change the MAC address
All this could probably have been avoided if i set the MAC correctly when setting up the machine? (not tested)

Disable DHCP on KVM Server

In this example i wanted to run a virtual DHCP server on the KVM system. The KVM server was getting in the way as it run a DHCP service also. To disable, remove the <dhcp> tags from the file /var/lib/libvirt/network/default.xml

Change from:

<network>
  <name>default</name>
  <uuid>319d4ae5-4b6e-433c-832f-ad20269ed495</uuid>
  <forward mode='nat'/>
  <bridge name='virbr0' stp='on' delay='0' />
  <mac address='52:54:00:09:89:76'/>
  <ip address='192.168.122.1' netmask='255.255.255.0'>
    <dhcp>
      <range start='192.168.122.2' end='192.168.122.254' />
    </dhcp>
  </ip>
</network>

Change to:

<network>
  <name>default</name>
  <uuid>319d4ae5-4b6e-433c-832f-ad20269ed495</uuid>
  <forward mode='nat'/>
  <bridge name='virbr0' stp='on' delay='0' />
  <mac address='52:54:00:09:89:76'/>
  <ip address='192.168.122.1' netmask='255.255.255.0'>
  </ip>
</network>

And then restart libvirtd and dnsmasq:

  service libvirtd restart
  service dnsmasq restart

Important Config Files

Important Configuration And Log Files (Directories) Location

The following files are required to manage and debug KVM problems:

  1. /etc/libvirt/ - Main configuration directory.
  2. /etc/libvirt/qemu/ - Virtual machine configuration directory. All xml files regarding VMs are stored here. You can edit them manually or via virt-manager.
  3. /etc/libvirt/qemu/networks/ - Networking for your KVM including default NAT. NAT is only recommended for small setup or desktops. I strongly suggest you use bridged based networking for performance.
  4. /etc/libvirt/qemu/networks/default.xml - The default NAT configuration used by NAT device virbr0.
  5. /var/log/libvirt/ - The default log file directory. All VM specific logs files are stored here.
  6. /etc/libvirt/libvirtd.conf - Master libvirtd configuration file.
  7. /etc/libvirt/qemu.conf - Master configuration file for the QEMU driver.

Editing the VM XML

libvirt stores it's configuration as xml in '/etc/libvirt/qemu'. The xml is easy to understand, and is similar to VMware *.vmx files. While it is possible to edit these files in place and restart libvirt-bin for the changes to take affect, the recommended method for modifying the attributes of a virtual machine is via virsh (or virt-manager, if it supports changing the hardware you want to change). The concept is simple:

  1. export (aka 'dump') the xml of the virtual machine you want to edit
  2. edit the xml
  3. import (aka 'define') the xml

For example, to edit the machine named 'foo' (you can get a list of your machines with 'virsh list --all'), do:

  $ virsh dumpxml foo > /tmp/foo.xml
  (edit /tmp/foo.xml as needed)
  $ virsh define /tmp/foo.xml

Create an internal network on a KVM host

[root@ivy4 ~]# cat internalnetwork.xml 
<network connections='1'>
  <name>internalnetwork</name>
  <uuid>7a28c269-c1b9-e5b9-d16e-627d085e7587</uuid>
  <bridge name='virbr1' stp='on' delay='0' />
  <mac address='52:54:00:92:40:89'/>
  <ip address='10.10.122.1' netmask='255.255.255.0'>
    <dhcp>
      <range start='10.10.122.2' end='10.10.122.254' />
    </dhcp>
  </ip>
</network>
[root@ivy4 ~]# virsh net-define ./internalnetwork.xml 
Network internalnetwork defined from ./internalnetwork.xml

[root@ivy4 ~]# virsh net-list
Name                 State      Autostart     Persistent
--------------------------------------------------
default              active     yes           yes

[root@ivy4 ~]# virsh net-list --all
Name                 State      Autostart     Persistent
--------------------------------------------------
default              active     yes           yes
internalnetwork      inactive   no            yes

[root@ivy4 ~]# virsh net-autostart internalnetwork
Network internalnetwork marked as autostarted

[root@ivy4 ~]# virsh net-list
Name                 State      Autostart     Persistent
--------------------------------------------------
default              active     yes           yes

[root@ivy4 ~]# virsh net-list --all
Name                 State      Autostart     Persistent
--------------------------------------------------
default              active     yes           yes
internalnetwork      inactive   yes           yes

[root@ivy4 ~]# virsh net-start internalnetwork 
Network internalnetwork started

[root@ivy4 ~]# virsh net-list --all
Name                 State      Autostart     Persistent
--------------------------------------------------
default              active     yes           yes
internalnetwork      active     yes           yes

# Then you can add hardware in the VM; Add NIC; Select internal network; 
# Any nodes on the hosts can communicate on this network, no access to outside world.