Difference between revisions of "KVM: Installation and operation"

From Define Wiki
Jump to navigation Jump to search
(Created page with "===== Check CPU supports KVM ===== <syntaxhighlight> egrep '(vmx|svm)' --color=always /proc/cpuinfo </syntaxhighlight> ===== Install KVM ===== <syntaxhighlight> # on RHEL 5 yum install virt-viewer virt-manag...")
 
 
(2 intermediate revisions by the same user not shown)
Line 1: Line 1:
===== Check CPU supports KVM =====
+
== Check CPU supports KVM ==
 
<syntaxhighlight>
 
<syntaxhighlight>
 
egrep '(vmx|svm)' --color=always /proc/cpuinfo
 
egrep '(vmx|svm)' --color=always /proc/cpuinfo
 
</syntaxhighlight>
 
</syntaxhighlight>
  
===== Install KVM =====
+
== Install KVM ==
 
<syntaxhighlight>
 
<syntaxhighlight>
 
# on RHEL 5
 
# on RHEL 5
Line 16: Line 16:
 
</syntaxhighlight>
 
</syntaxhighlight>
  
===== Install the modules =====
+
== Install the modules ==
 
<syntaxhighlight>
 
<syntaxhighlight>
 
modprobe kvm-intel
 
modprobe kvm-intel
 
</syntaxhighlight>
 
</syntaxhighlight>
  
===== Start libvirtd =====
+
== Start libvirtd ==
 
<syntaxhighlight>
 
<syntaxhighlight>
 
/etc/init.d/libvirtd restart
 
/etc/init.d/libvirtd restart
 
</syntaxhighlight>
 
</syntaxhighlight>
  
===== Load up virtual manager =====
+
== Load up virtual manager ==
 
<syntaxhighlight>
 
<syntaxhighlight>
 
virt-manager
 
virt-manager
 
</syntaxhighlight>
 
</syntaxhighlight>
  
===== Setup Bridged Networking =====
+
== Setup Bridged Networking ==
 
By Default KVM uses NAT (hosts can see outside world but world cannot see the hosts). To setup bridged networking:
 
By Default KVM uses NAT (hosts can see outside world but world cannot see the hosts). To setup bridged networking:
  
Line 72: Line 72:
 
                                                         vnet0
 
                                                         vnet0
  
===== Create the image file =====
+
== Create the image file ==
 
In RHEL6, use virt-install: (dont use any files in /root)
 
In RHEL6, use virt-install: (dont use any files in /root)
 
<syntaxhighlight>
 
<syntaxhighlight>
Line 95: Line 95:
 
OR
 
OR
 
qemu-img create -f qcow windows.img 12G
 
qemu-img create -f qcow windows.img 12G
 +
</syntaxhighlight>
 +
 +
Another one for installing off the cobbler system (in an VM on a physical system which is on the cobbler network)
 +
<syntaxhighlight>
 +
[root@x8-0 ~]# qemu-img create -f qcow2 -o preallocation=metadata /vm/images/centos6vm.qcow2 20G
 +
Formatting '/vm/images/centos6vm.qcow2', fmt=qcow2 size=21474836480 encryption=off cluster_size=65536 preallocation='metadata'
 +
[root@x8-0 ~]# virt-install --connect qemu:///system --name centos6vm --ram 1024 --vcpus 1 --disk \
 +
path=/vm/images/centos6vm.qcow2,format=qcow2,bus=virtio,cache=none --network=bridge:br0,model=virtio \
 +
--vnc --os-type=linux --os-variant=rhel6 --accelerate --noautoconsole --keymap=en-gb --pxe
 
</syntaxhighlight>
 
</syntaxhighlight>
  
Line 106: Line 115:
 
* Change CDs using '''virt-manger''' -> Open Guest -> Show Virtual Hardware -> CDROM
 
* Change CDs using '''virt-manger''' -> Open Guest -> Show Virtual Hardware -> CDROM
  
===== Start qemu =====
+
== Start qemu ==
 
<syntaxhighlight>
 
<syntaxhighlight>
 
/usr/libexec/qemu-kvm -hda vig_xp.img  -cdrom vig_xp.iso -m 512 -boot d
 
/usr/libexec/qemu-kvm -hda vig_xp.img  -cdrom vig_xp.iso -m 512 -boot d
 
</syntaxhighlight>
 
</syntaxhighlight>
  
===== Other Handy Bits =====
+
== Clone a VM ==
====== VNC / Args ======
+
This can only be done when the image is shutdown or suspended
<syntaxhighlight>
 
CLI Args:
 
-alt-grab - makes ctrl+alt+shift the key escape rather than ctrl+alt (which makes it impossible for windows logins)
 
on vncviewer, you can also hit f8 to bring up a menu which will allow
 
-smp n (number of cores to allocate)
 
-m 8192 (amount of memory to give the machine)
 
-net nic -net nic (give the system two nics)
 
</syntaxhighlight>
 
 
 
====== Mounting NFS in VM ======
 
ERROR: mountd: refused mount nfs illegal port
 
 
 
In case you’re getting error while mounting a nfs share, and logs on server showing:
 
<syntaxhighlight>
 
Mar 24 10:44:57 localhost mountd[8344]: refused mount request from 192.168.1.2 for /dump (/dump): illegal port 52213
 
</syntaxhighlight>
 
 
 
Add insecure option in your exports file:
 
<syntaxhighlight>
 
    /dump  *(ro,sync,no_root_squash,insecure)
 
 
 
    # And do
 
    exportfs -r  # or reload/restart nfs service.
 
</syntaxhighlight>
 
 
 
====== Grow a KVM Disk Image (RAW) ======
 
E.G to grow a KVM image by 10G (Perhaps this could be a hell of a lot easier with LVM)
 
* Create additional space as a raw qemu-img
 
* cat the two files together
 
* Boot gparted-live to resize partitions/filesystem
 
<syntaxhighlight>
 
[root@kvm vm]$ qemu-img create -f raw guest1-rhel6-64-largerdisk 10G
 
Formatting 'guest1-rhel6-64-largerdisk', fmt=raw size=10737418240
 
[root@kvm vm]$ ll
 
total 17611676
 
-rwxrwxrwx. 1 root root 10737418240 Sep 27 14:34 guest1-rhel6-64
 
-rwxr-xr-x  1 root root 10737418240 Sep 27 13:14 guest1-rhel6-64-clone
 
'-rw-r--r--  1 root root 10737418240 Sep 27 14:37 guest1-rhel6-64-largerdisk'
 
[root@kvm vm]$ cat guest1-rhel6-64 guest1-rhel6-64-largerdisk > sl6-30g-puppetcobbler.kvm
 
[root@kvm vm]$ ll
 
total 38583200
 
-rwxrwxrwx. 1 root root 10737418240 Sep 27 14:34 guest1-rhel6-64
 
-rwxr-xr-x  1 root root 10737418240 Sep 27 13:14 guest1-rhel6-64-clone
 
-rw-r--r--  1 root root 10737418240 Sep 27 14:37 guest1-rhel6-64-largerdisk
 
'-rw-r--r--  1 root root 21474836480 Sep 27 14:42 sl6-30g-puppetcobbler.kvm'
 
</syntaxhighlight>
 
 
 
Just as easy to convert a raw image to qcow2 and then resize that(<tt>qemu-img resize [vmname.qcow2] +100G</tt>). Use LVM as below to grow the FS.
 
 
 
 
 
Now boot up a ubuntu (or other) live cd (in virt-manager: add hardware -> storage -> select managed.. ->  device type -> IDE CDrom) 
 
  Install lvm tools: apt-get install lvm2
 
  Fdisk create new partition:
 
  fdisk /dev/sda
 
  delete lvm partition (d -> 2 in this instance)
 
  create lvm partition (same cylinder start, new end pick the default which will be the last cylinder) (n -> p -> start / finish)
 
  change type of new partition to lvm (t -> 8e)
 
 
 
 
 
Use LVM to grow the volume
 
<syntaxhighlight>
 
root@ubuntu:~# pvdisplay /dev/sda2
 
  --- Physical volume ---
 
  PV Name              /dev/sda2
 
  VG Name              vg_puppetmaster
 
  PV Size              9.51 GiB / not usable 3.00 MiB
 
  Allocatable          yes (but full)
 
  PE Size              4.00 MiB
 
  Total PE              2434
 
  Free PE              0
 
  Allocated PE          2434
 
  PV UUID              LBxPCN-RL3d-rd7d-qEcr-M5Fq-GfMF-ttiPPd
 
 
 
root@ubuntu:~# partprobe
 
Warning: Unable to open /dev/sr0 read-write (Read-only file system).  /dev/sr0 has been opened read-only.
 
root@ubuntu:~# pvresize /dev/sda2
 
  Physical volume "/dev/sda2" changed
 
  1 physical volume(s) resized / 0 physical volume(s) not resized
 
root@ubuntu:~# pvdisplay /dev/sda2
 
  --- Physical volume ---
 
  PV Name              /dev/sda2
 
  VG Name              vg_puppetmaster
 
  PV Size              19.50 GiB / not usable 3.46 MiB
 
  Allocatable          yes
 
  PE Size              4.00 MiB
 
  Total PE              4992
 
  Free PE              2558
 
  Allocated PE          2434
 
  PV UUID              LBxPCN-RL3d-rd7d-qEcr-M5Fq-GfMF-ttiPPd
 
 
 
 
 
t@ubuntu:~# vgchange -a y vg_puppetmaster
 
  2 logical volume(s) in volume group "vg_puppetmaster" now active
 
root@ubuntu:~# vgdisplay vg_puppetmaster
 
  --- Volume group ---
 
  VG Name              vg_puppetmaster
 
  System ID           
 
  Format                lvm2
 
  Metadata Areas        1
 
  Metadata Sequence No  4
 
  VG Access            read/write
 
  VG Status            resizable
 
  MAX LV                0
 
  Cur LV                2
 
  Open LV              0
 
  Max PV                0
 
  Cur PV                1
 
  Act PV                1
 
  VG Size              19.50 GiB
 
  PE Size              4.00 MiB
 
  Total PE              4992
 
  Alloc PE / Size      2434 / 9.51 GiB
 
  Free  PE / Size      2558 / 9.99 GiB
 
  VG UUID              Bu7xaM-OQrj-aZYV-gwY8-MvXg-Az1m-8wZh6p
 
 
 
root@ubuntu:~# lvextend -L +10G /dev/vg_puppetmaster/lv_root
 
  Extending logical volume lv_root to 15.57 GiB
 
  Insufficient free space: 2560 extents needed, but only 2558 available
 
 
 
root@ubuntu:~# lvextend -L +9.9G /dev/vg_puppetmaster/lv_root
 
  Rounding up size to full physical extent 9.90 GiB
 
  Extending logical volume lv_root to 15.47 GiB
 
  Logical volume lv_root successfully resized
 
 
 
root@ubuntu:~# resize2fs /dev/vg_puppetmaster/lv_root
 
resize2fs 1.41.14 (22-Dec-2010)
 
Please run 'e2fsck -f /dev/vg_puppetmaster/lv_root' first.
 
 
 
root@ubuntu:~# e2fsck -f /dev/vg_puppetmaster/lv_root
 
e2fsck 1.41.14 (22-Dec-2010)
 
Pass 1: Checking inodes, blocks, and sizes
 
Pass 2: Checking directory structure
 
Pass 3: Checking directory connectivity
 
Pass 4: Checking reference counts
 
Pass 5: Checking group summary information
 
/dev/vg_puppetmaster/lv_root: 50681/365760 files (0.1% non-contiguous), 448889/1460224 blocks
 
root@ubuntu:~# resize2fs /dev/vg_puppetmaster/lv_root
 
resize2fs 1.41.14 (22-Dec-2010)
 
Resizing the filesystem on /dev/vg_puppetmaster/lv_root to 4056064 (4k) blocks.
 
The filesystem on /dev/vg_puppetmaster/lv_root is now 4056064 blocks long.
 
 
 
root@ubuntu:~# lvscan
 
  ACTIVE            '/dev/vg_puppetmaster/lv_root' [15.47 GiB] inherit
 
  ACTIVE            '/dev/vg_puppetmaster/lv_swap' [3.94 GiB] inherit
 
 
 
</syntaxhighlight>
 
 
 
====== Use old VM img with virt-manager ======
 
You'll need to use virt-manager to import an old file. Point this towards an existing raw VM and import. Import didnt add a CD drive, so used the command:
 
 
 
<syntaxhighlight>   
 
[root@kvm vm]$ /usr/libexec/qemu-kvm -hda ./sl6-30g-puppetcobbler.kvm \
 
      -cdrom /mnt/vstor/gparted-live/gparted-live-0.7.1-1.iso -m 512 -boot d
 
Using CPU model "cpu64-rhel6"
 
VNC server running on `::1:5900'
 
 
 
# Then vmcviewer on another terminal
 
vncviewer localhost:5900
 
</syntaxhighlight>
 
 
 
====== Copying VMs udev renames network ======
 
* udev seems to be adding lines in /etc/udev/70-persistent-net.rules
 
* Remove entries and set MAC correctly
 
* Also had to update the files in /etc/sysconfig/network-scripts/ifcfg-eth0 to remove/change the MAC address
 
* All this could probably have been avoided if i set the MAC correctly when setting up the machine? (not tested)
 
 
 
====== Disable DHCP on KVM Server ======
 
In this example i wanted to run a virtual DHCP server on the KVM system. The KVM server was getting in the way as it run a DHCP service also. To disable, remove the <tt><dhcp></tt> tags from the file <tt>/var/lib/libvirt/network/default.xml</tt>
 
 
 
Change from:
 
<syntaxhighlight>
 
<network>
 
  <name>default</name>
 
  <uuid>319d4ae5-4b6e-433c-832f-ad20269ed495</uuid>
 
  <forward mode='nat'/>
 
  <bridge name='virbr0' stp='on' delay='0' />
 
  <mac address='52:54:00:09:89:76'/>
 
  <ip address='192.168.122.1' netmask='255.255.255.0'>
 
    <dhcp>
 
      <range start='192.168.122.2' end='192.168.122.254' />
 
    </dhcp>
 
  </ip>
 
</network>
 
</syntaxhighlight>
 
 
 
Change to:
 
<syntaxhighlight>
 
<network>
 
  <name>default</name>
 
  <uuid>319d4ae5-4b6e-433c-832f-ad20269ed495</uuid>
 
  <forward mode='nat'/>
 
  <bridge name='virbr0' stp='on' delay='0' />
 
  <mac address='52:54:00:09:89:76'/>
 
  <ip address='192.168.122.1' netmask='255.255.255.0'>
 
  </ip>
 
</network>
 
</syntaxhighlight>
 
 
 
And then restart libvirtd and dnsmasq:
 
<syntaxhighlight>
 
  service libvirtd restart
 
  service dnsmasq restart
 
</syntaxhighlight>
 
 
 
====== Important Config Files ======
 
Important Configuration And Log Files (Directories) Location
 
 
 
The following files are required to manage and debug KVM problems:
 
 
 
  1. /etc/libvirt/ - Main configuration directory.
 
  2. /etc/libvirt/qemu/ - Virtual machine configuration directory. All xml files regarding VMs are stored here. You can edit them manually or via virt-manager.
 
  3. /etc/libvirt/qemu/networks/ - Networking for your KVM including default NAT. NAT is only recommended for small setup or desktops. I strongly suggest you use bridged based networking for performance.
 
  4. /etc/libvirt/qemu/networks/default.xml - The default NAT configuration used by NAT device virbr0.
 
  5. /var/log/libvirt/ - The default log file directory. All VM specific logs files are stored here.
 
  6. /etc/libvirt/libvirtd.conf - Master libvirtd configuration file.
 
  7. /etc/libvirt/qemu.conf - Master configuration file for the QEMU driver.
 
 
 
====== Editing the VM XML ======
 
libvirt stores it's configuration as xml in '/etc/libvirt/qemu'. The xml is easy to understand, and is similar to VMware *.vmx files. While it is possible to edit these files in place and restart libvirt-bin for the changes to take affect, the recommended method for modifying the attributes of a virtual machine is via virsh (or virt-manager, if it supports changing the hardware you want to change). The concept is simple:
 
 
 
  1. export (aka 'dump') the xml of the virtual machine you want to edit
 
  2. edit the xml
 
  3. import (aka 'define') the xml
 
 
 
For example, to edit the machine named 'foo' (you can get a list of your machines with 'virsh list --all'), do:
 
 
 
 
<syntaxhighlight>
 
<syntaxhighlight>
  $ virsh dumpxml foo > /tmp/foo.xml
+
virt-clone --connect=qemu:///system -o centos6vm -n openstack1 --auto-clone
  (edit /tmp/foo.xml as needed)
+
Cloning centos6vm.qcow2                                                              24% [==================                                                        ] 202 MB/s | 4.8 GB    01:17 ETA
  $ virsh define /tmp/foo.xml
 
 
</syntaxhighlight>
 
</syntaxhighlight>

Latest revision as of 15:19, 2 September 2014

Check CPU supports KVM

egrep '(vmx|svm)' --color=always /proc/cpuinfo

Install KVM

# on RHEL 5
yum install virt-viewer virt-manager
yum install kvm kvm-qemu-img
OR
yum groupinstall KVM

# on RHEL 6
yum groupinstall  Virtualisation 'Virtualization Client' 'Virtualization Platform' 'Virtualization Tools'

Install the modules

modprobe kvm-intel

Start libvirtd

/etc/init.d/libvirtd restart

Load up virtual manager

virt-manager

Setup Bridged Networking

By Default KVM uses NAT (hosts can see outside world but world cannot see the hosts). To setup bridged networking:

  • Disable NetworkManager
 # chkconfig NetworkManager off
 # chkconfig network on
 # service NetworkManager stop
 # service network start
  • Edit the file: /etc/sysconfig/network-scripts/ifcfg-eth0
 DEVICE="eth0"
 ONBOOT="yes"
 BRIDGE=br0


  • Create the bridge script/file: /etc/sysconfig/network-scripts/ifcfg-br0
 DEVICE=br0
 TYPE=Bridge
 IPADDR=172.28.10.58
 NETMASK=255.255.0.0
 NETWORK=172.28.0.0
 DNS1=172.28.1.3
 ONBOOT=yes
 DELAY=0
  • Configure iptables to allow traffic be forwarded across the bridge
 # iptables -I FORWARD -m physdev --physdev-is-bridged -j ACCEPT
 # service iptables save
 # service iptables restart
  • Reload libvirtd
 # service libvirtd reload
  • Verify it's working fine:
 # brctl show 
 bridge name	bridge id		STP enabled	interfaces
 br0		8000.0025902efb56	no		eth0
                                                       vnet2
 virbr0	8000.525400098976	yes		virbr0-nic
                                                       vnet0

Create the image file

In RHEL6, use virt-install: (dont use any files in /root)

[root@kvm kvms]$ virt-install \
                 --name=guest1-sl6-64 \
                 --disk path=/vm/guest1-rhel6-64,size=10,sparse=false \
                 --vnc \
                 --vcpus=2 \
                 --ram=2048 \
                 --cdrom=/vm/SL-61-x86_64-2011-07-27-Everything-DVD1.iso \
                 --network bridge=br0 \
                 --os-type=linux \
                 --os-variant=rhel6

Starting install...
Creating storage file guest1-rhel6-64    8% [==-                           ]  17 MB/s | 873 MB     09:13 ETA

Notes from RHEL5 on creating a qcow filesystem (virt-install runs this under the covers)

dd if=/dev/zero of=/home/viglen/vm_winxp.img bs=1024k count=12000
OR
qemu-img create -f qcow windows.img 12G

Another one for installing off the cobbler system (in an VM on a physical system which is on the cobbler network)

[root@x8-0 ~]# qemu-img create -f qcow2 -o preallocation=metadata /vm/images/centos6vm.qcow2 20G 
Formatting '/vm/images/centos6vm.qcow2', fmt=qcow2 size=21474836480 encryption=off cluster_size=65536 preallocation='metadata' 
[root@x8-0 ~]# virt-install --connect qemu:///system --name centos6vm --ram 1024 --vcpus 1 --disk \
path=/vm/images/centos6vm.qcow2,format=qcow2,bus=virtio,cache=none --network=bridge:br0,model=virtio \ 
--vnc --os-type=linux --os-variant=rhel6 --accelerate --noautoconsole --keymap=en-gb --pxe

Creating files in /root resulted in the following error:

Starting install...
ERROR    internal error process exited while connecting to monitor: char device redirected to /dev/pts/1
qemu: could not open disk image /root/scratch/SL-61-x86_64-2011-07-27-Everything-DVD1.iso: Permission denied
  • Change CDs using virt-manger -> Open Guest -> Show Virtual Hardware -> CDROM

Start qemu

/usr/libexec/qemu-kvm -hda vig_xp.img  -cdrom vig_xp.iso -m 512 -boot d

Clone a VM

This can only be done when the image is shutdown or suspended

virt-clone --connect=qemu:///system -o centos6vm -n openstack1 --auto-clone
Cloning centos6vm.qcow2                                                               24% [==================                                                         ] 202 MB/s | 4.8 GB     01:17 ETA