Libvirt cheatsheet

Remote connection over ssh

virt-manager -c qemu+ssh://root@10.0.0.1/system

In the installation/cloning examples below, I precreated LV using lvcreate. The path's given point to the LVs (For example /dev/xen02/member.root)
Installing without virtio might end up with a slow VM.

virt-install --name=member \
--ram 1024 \
--os-type linux --os-variant debianlenny \
--file /dev/xen02/member.root \
--network network=default \
--vnc \
--noautoconsole \
--hvm \
--cdrom /home/lmwangi/debian-505-amd64-netinst.iso
--force yes

Virtio speeds up things a lot.

virt-install --name=nms01 \
--ram 512 \
--os-type linux --os-variant debianlenny \
--disk path=/dev/dm-8,bus=virtio \
--network network=default,model=virtio \
--vnc \
--noautoconsole \
--cdrom /home/lmwangi/debian-505-amd64-netinst.iso
--virt-type kvm \
--force yes

Cloning is easy. Create/Install a template VM and clone it to create your workhorses.

virt-clone \
--prompt \
--connect qemu:///system \
--original template \
--name "champ" \
--file /dev/xen02/champ.root

By default, your default network should be started.

virsh net-start default && virsh net-autostart default

You can also view an existing VM's definition.

# virsh net-dumpxml default

default
900a487e-d5a6-b731-7bb0-3997fc0455fb







.....

You can also hotplug disks. The VM wasn't stable though :(

virsh # detach-disk
error: command 'detach-disk' requires option
error: command 'detach-disk' requires option
virsh # detach-disk nosql vdb
Disk detached successfully

virsh # attach-disk nosql /dev/xen02/nosql.storage vdb
Disk attached successfully

On the VM, you may need to

modprobe acpiphp

Occasionally, I have legacy VMs that were not properly created (Someone bypassed libvirt.) Cloning is a pain. In essence, I create a similar LV (same size), define a VM using the new LV and then dd over the LV

[root@vm01 ~]# lvdisplay cnix/box1_root
--- Logical volume ---
LV Name /dev/cnix/box1_root
VG Name cnix
LV UUID xxxx-xxxx.....
LV Write Access read/write
LV Status available
# open 1
LV Size 19.53 GB
Current LE 5000
Segments 1
Allocation inherit
Read ahead sectors auto
- currently set to 256
Block device 253:12

[root@vm01 ~]# lvcreate -L 19.53G -n signer sdbvg
Rounding up size to full physical extent 19.53 GB
Logical volume "signer" created

[root@vm01 ~]# virt-clone --prompt --original legacy1 --name signer --file /dev/sdbvg/signer.root --file /dev/sdbvg/signer.swap
This will overwrite the existing path '/dev/sdbvg/signer.root'!
Do you really want to use this disk (yes or no) yes
This will overwrite the existing path '/dev/sdbvg/signer.swap'!

Do you really want to use this disk (yes or no) yes
Cloning /dev/cnix/legacy1_root | 20 GB 02:11
Cloning /dev/cnix/legacy1_swap | 2.0 GB 00:08

Clone 'signer' created successfully.

Sometimes this is not always flawless

virt-clone --prompt --original master --name master1 --file /dev/sdbvg/master1.root --file /dev/sdbvg/master1.swap
ERROR Domain 'master' was not found.
What is the name of the original virtual machine? master
ERROR Domain 'master' was not found.
What is the name of the original virtual machine? legacy1
ERROR Domain 'legacy1' was not found.
What is the name of the original virtual machine? ERROR

So we have to dd the image to the new hdd

[root@vm01 ~]# dd if=/dev/cnix/legacy1_root of=/dev/sdbvg/master1.root
dd if=/dev/cnix/legacy2_root of=/dev/sdbvg/member.root
40960000+0 records in
40960000+0 records out
20971520000 bytes (21 GB) copied, 1237.21 seconds, 17.0 MB/s

And boot the new VM

Cloning a VM that uses LVM using libvirt utilities

Get specs for your existing VM storage and create an LV for the new VM

# lvdisplay /dev/sdbvg/eng3.example.net_root
--- Logical volume ---
LV Name /dev/sdbvg/eng3.example.net_root
VG Name sdbvg
LV UUID 9APovE-4ad5-QYe9-UB3U-ndP2-lW3p-IcZWt1
LV Write Access read/write
LV Status available
# open 1
LV Size 40.00 GB
Current LE 10240
Segments 1
Allocation inherit
Read ahead sectors auto
- currently set to 256
Block device 253:46

[root@vm01 ~]# lvcreate -L 40G -n template.example.net_swap /dev/sdbvg
Logical volume "template.example.net_swap" created

Do the same thing for swap or any other partitions you may have.

Then call virt-clone

virt-clone --prompt --original eng3.example.net --name template.example.net --file /dev/sdbvg/template.example.net_root --file/dev/sdbvg/template.example.net_swap

Cloning /dev/sdbvg/eng3..example.net_root | 40 GB 05:55

Cloning /dev/sdbvg/eng3.example.nett_swap | 4.0 GB 00:34

Libvirt utilities over TLS

If you are running libvirt on a different host and you don't trust the devices in your network path, you should use TLS. You can of course use ssh :). To use TLS/SSL with libvirt, you should have:

• Access to CA henceforth known as 'the CA'.
• Server key, server cert signed by the CA.
• Client key, client cert signed by the CA.

If you don't have a CA, it's easy to generate one using openSSL or better use certtool (man 1 certtool) from gnutls binaries.

Once you have a CA, have a look at the libvirtd help on your server.

xen02:~/pki# libvirtd -h

libvirtd: invalid option -- 'h'

Usage:

libvirtd [options]

Options:

-v | --verbose Verbose messages.

-d | --daemon Run as a daemon & write PID file.

-l | --listen Listen for TCP/IP connections.

-t | --timeout Exit after timeout period.

-f | --config Configuration file.

| --version Display version information.

-p | --pid-file Change name of PID file.

.......

TLS:

CA certificate: /etc/pki/CA/cacert.pem

Server certificate: /etc/pki/libvirt/servercert.pem

Server private key: /etc/pki/libvirt/private/serverkey.pem

.......

From the output above, it's evident that you will need to copy the CA's certificate to /etc/pki/CA/cacert.pem, generate a server key, a server CSR then get the CSR to the CA. The CA will then provision you with a server cert which you can place in the appropriate path.

You will need to enable TLS in /etc/libvirt/libvirtd.conf

listen_tls = 1

You will probably want to enable TLS authentication for VNC as well as listening on interfaces other than localhost:

vnc_listen = "0.0.0.0"

vnc_tls = 1

#Read the documentation for this

vnc_tls_x509_cert_dir = "/etc/pki/libvirt-vnc"

vnc_tls_x509_verify = 1

Here's my PKI directory

PKI directory listing

$ find /etc/pki/

/etc/pki/

/etc/pki/libvirt-vnc

/etc/pki/libvirt-vnc/ca-cert.pem

/etc/pki/libvirt-vnc/server-key.pem

/etc/pki/libvirt-vnc/server-cert.pem

/etc/pki/libvirt

/etc/pki/libvirt/servercert.pem

/etc/pki/libvirt/private

/etc/pki/libvirt/private/serverkey.pem

/etc/pki/CA

/etc/pki/CA/cacert.pem

To make life easier, You can make VNC and libvirt to share the same keys and certs.

Restart the libvirt service

/etc/init.d/libvirt-bin restart

The client requires a similar PKI setup. The same CA should sign the server and client certs.

$ find /etc/pki/

/etc/pki/

/etc/pki/CA

/etc/pki/CA/cacert.pem

/etc/pki/libvirt

/etc/pki/libvirt/private

/etc/pki/libvirt/private/clientkey.pem

/etc/pki/libvirt/clientcert.pem

Add a connection from your virt-manager that utilizes TLS or use the virsh tools:

virt-viewer -c qemu+tls://xen02.example.net/system box2

virsh -c qemu+tls://xen02.example.net/system list

Id Name State

----------------------------------

7 box1 running

10 box2 running

11 box3 running

13 box4 running

14 box5 running

15 box6 running

16 box7 running