Qemu-guest-agent support
- Vitaliy Kuznetsov
Requirements and motivation:
Implement the ability to create snapshots consistent with applications.
Prevent data corruption when taking snapshots.
Get rid of the need to shutdown applications to create or rollback a snapshot.
An additional option - is the ability to receive other information from the guest application.
(For example - metrics or logs)
Why Qemu-guest-agent?
Advantages:
Ready-made solution for working in the application;
Multiple ways of interaction between host and application;
Satisfies all conditions for taking snapshots on the host side without data corruption on the application side;
Supported by QEMU;
Availability in repositories;
Easy to install and run.
Flaws:
If qemu-guest-agent is not running or listening on the wrong cid when using VSOCK or chardev when using virtio-serial,
operations will not be available.No XEN support
How will it work?
There are several options for creating a connection between a host and a guest application by
using between host and qemu-guest-agent on the guest side.
Namely, through:
-device virtio-serial
or
-device vhost-vsock-pci
-device virtio-serial
In the first case, with -device virtio-serial, the guest agent is exposed to the host via a separate QEMU chardev
device (generally, a UNIX socket) that communicates with the agent using the QMP wire protocol (minus the negotiation)
over a virtio-serial or isa-serial channel to the guest. Assuming the agent will be listening inside the guest using
the virtio-serial device at /dev/virtio-ports/org.qemu.guest_agent.0 (the default).
Virtio-serial overview
Virtio device for serial ports and console
The host can add/remove ports
Ports can be named (e.g. org.qemu.guest_agent.0)
Guest agent opens /dev/virtio- ports/<name> character device
The client app connects to QEMU chardev (UNIX domain socket, named pipe, etc)
Flaws:
N:1 connections are clunky over the 1:1 serial port are not convenient and require an additional
a layer of abstraction to manage. This complicates the implementation of this functionality.Applications have to multiplex over 1 stream
Libvirt has to arbitrate access between qemu-guest-agent client.
Creating a new chardev on every App creation
-device vhost-vsock-pci
virtio-vsock is a host/guest communications device. It allows applications in the guest and host to communicate.
This can be used to implement hypervisor services and guest agents (like qemu-guest-agent or SPICE vdagent).
Advantages:
Socket listeners (ex. Host) can accept connections from multiple clients (unlike virtio-serial char devices).
This allows you not to create a new chardev for each application for communication.No address setting is required in the guest system. The technology itself supports this functionality,
but apparently, qemu-ga has not yet been taught how to read system information from /dev/vsock at startup.
Although this is stated on the wiki page. Perhaps this will be corrected in the future.Can be used with VMs that have no network interfaces
An accessible library for Golang and very simple implementation on the EVE side.
Flaws:
An application on that hypervisor with kernel 4.8+
it is not possible for qemu-ga not to specify CID:Port when using VSOCK.
The documentation says that "No address setting is required inside the guest", but in practice this
does not work yet (Perhaps they will fix it in new versions). For this reason, it is necessary to send
information about (CID:Port) to the controller so that the user can run/configure qemu-ga on the application side.
Or we can use a mirrored CID, which will be equal to the TCP port. (For example 5891,...,n).
Thus, the CID will always be similar to the port and there will be no need to send additional information to the controller.
vhost-vsock-pci overview
Sockets are created with the AF_VSOCK address family. The SOCK_STREAM socket type is currently implemented for in-order, guaranteed stream semantics.
In EVE, we plan to implement exactly this method of communication with applications.
The proposed implementation of the functionality on the EVE side for working with snapshots.
To support consistent zfs snapshots in EVE without shutdown applications, it is proposed to add 3 functions.
Namely:
The function of checking the state of the file system in the application. (freeze, thawed)
The function of resetting memory and freezing the file system in the application
Function to thawed the file system in the application
These features will also use synchronization between EVE and the VM to guarantee command execution.
In addition, each of these functions will return the result of the command from the application
This functionality does not require sending any information to the controller, so the API in EVE will not change.
Suggested workflow for working with snapshots
Receive a command from the controller to create or roll back a snapshot;
Checking the state of the file system in the application;
Make sure the file system is working normally;
Flush RAM to disk and freeze the file system in the application;
Make sure the file system in the application is frozen;
Create a new snapshot or rollback to an existing snapshot;
After completing the command with snapshot, send the command to unfreeze the file system in the application;
Make sure that the file system is unfrozen and works normally;
Sending up-to-date information to the controller.
Links
Available queries for working with qemu-guest-agent