Troubleshooting KVM Virtualization Problem With Log Files in Linux

Posted on in Categories , , , , , , , last updated May 6, 2017

There are various log files stored on the host system to assist with debugging KVM related problems. In this tutorial, I will cover log file locations and tools used to track down KVM problems.

The files are as follows:

  1. $HOME/.virtinst/virt-install.log – virt-install tool log file.
  2. $HOME/.virt-manager/virt-manager.log – virt-manager tool log file.
  3. /var/log/libvirt/qemu/ – Log files for each running virtual machine. If centos is virtual machine name, than log file is /var/log/libvirt/qemu/centos.log.

You can use the grep and other Linux tools to view this files:
# tail -f /var/log/libvirt/qemu/freebsd.log
# grep something $HOME/.virtinst/virt-install.log
$ sudo tail -f /var/log/libvirt/qemu/openbsd.log

Sample outputs:

2016-11-08 17:14:46.909+0000: starting up libvirt version: 1.3.1, package: 1ubuntu10.5 (Stefan Bader <stefan.bader@canonical.com> Thu, 06 Oct 2016 13:07:20 +0200), qemu version: 2.5.0 (Debian 1:2.5+dfsg-5ubuntu10.5), hostname: server1.cyberciti.biz
....
.....
..
=29,id=hostnet1,vhost=on,vhostfd=30 -device virtio-net-pci,netdev=hostnet1,id=net1,mac=52:54:00:8f:31:5f,bus=pci.0,addr=0x4 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -vnc 127.0.0.1:1 -device cirrus-vga,id=video0,bus=pci.0,addr=0x2 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x7 -msg timestamp=on
char device redirected to /dev/pts/8 (label charserial0)

Connecting To Console

Use the virsh command to connect to guest serial console as follows :
$ virsh list
$ virsh console freebsd

This is useful to troubleshoot problem such as networking and much more from host itself. However, you need to configure guest operating system for a serial console. See how to setup a serial console for FreeBSD and CentOS virtual machines.

KVM Configuration Files

You can edit configuration file to define hardware properties for VMs or other setup. They are located in /etc/libvirt/qemu/ directory. Here is sample FreeBSD guest configuration file.
# less /etc/libvirt/qemu/freebsd.xml
OR
$ virsh dumpxml freebsd
Sample outputs:

Fig.01: virsh dumxml command in action
Fig.01: virsh dumxml command in action

When you create guests with the virt-manager or virt-install, the guests configuration files are created automatically in the /etc/libvirt/qemu/ directory. You can use this file error checking. You can edit this file using a text editor or virsh command itself:
virsh edit freebsd
virsh edit centos

KVM tools

Type the following command to install kvm-tools package which contains some diagnostics and debugging tools for KVM, such as kvmtrace and kvm_stat, enter:
# yum -y install kvm_tools
You need to mount debugfs, enter:
# mount -t debugfs debugfs /sys/kernel/debug
# kvm_stat

Sample outputs:

Fig.02: Getting stats about KVM with kvm_stat
Fig.02: Getting stats about KVM with kvm_stat

You can use normal tools to troubleshoot and get other information using the following tools:

  • ps, pstree, and top
  • vmstat, iostat, and lsof
  • tcpdump, brctl, ip, and ifconfig
This entry is 6 of 14 in the CentOS / Redhat (RHEL) KVM Virtulization series. Keep reading the rest of the series:
  1. CentOS / Redhat: Install KVM Virtualization Software
  2. CentOS / Redhat: KVM Bridged Network Configuration
  3. KVM virt-manager: Install CentOS As Guest Operating System
  4. KVM virt-install: Install FreeBSD / CentOS As Guest Operating System
  5. KVM: Install CentOS / RHEL Using Kickstart File (Automated Installation)
  6. Troubleshooting KVM Virtualization Problem With Log Files
  7. KVM Virsh: Redirect FreeBSD Console To A Serial Port
  8. KVM: Starting / Stopping Guest Operating Systems With virsh Command
  9. Linux KVM: Disable virbr0 NAT Interface
  10. FreeBSD / OpeBSD Running in KVM Does Not Accept FTP Traffic
  11. KVM: Start a Virtual Machine / Guest At Boot Time
  12. KVM virt-install: Install OpenBSD As Guest Operating System
  13. Linux KVM: OpenBSD Guest Hangs At Starting tty Flags
  14. KVM Virtualization: Start VNC Remote Access For Guest Operating Systems

Posted by: Vivek Gite

The author is the creator of nixCraft and a seasoned sysadmin and a trainer for the Linux operating system/Unix shell scripting. He has worked with global clients and in various industries, including IT, education, defense and space research, and the nonprofit sector. Follow him on Twitter, Facebook, Google+.