L4Re - L4 Runtime Environment
Uvmm, the virtual machine monitor

Setting up guest memory

In the most simple setup, memory for the guest can be provided via a simple dataspace. In your ned script, create a new dataspace of the required size and hand it into uvmm as the ram capability:

local ramds = L4.Env.user_factory:create(L4.Proto.Dataspace, 60 * 1024 * 1024)

L4.default_loader::startv({caps = {ram = ramds:m("rw")}}, "rom/uvmm")

The memory will be mapped to the most appropriate place and a memory node added to the device tree, so that the guest can find the memory.

For a more complex setup, the memory can be configured via the device tree. uvmm scans for memory nodes and tries to set up the memory from them. A memory device node should look like this:

memory@0 {
  device_type = "memory";
  reg = <0x00000000 0x00100000
         0x00200000 0xffffffff>;
  l4vmm,dscap = "memcap";
  l4vmm,physmap;
  dma-ranges = <>;
};

The device_type property is mandatory and needs to be set to memory.

l4vmm,dscap contains the name of the capability containing the dataspace to be used for the RAM. reg describe the memory regions to use for the memory. The regions will be filled up to the size of the supplied dataspace. If they are larger, then the remaining area will be cut.

l4vmm,physmap indicates that uvmm should try to map the dataspace to its actual physical address when no IOMMU is available. If the physical address cannot be determined or an IOMMU is available, then the memory will be mapped to the addresses supplied in regs. It is possible to omit the regs property when l4vmm,physmap is set. In this case, uvmm will fail to start if the physical address cannot be determined.

If a dma-ranges property is given, the host-physical address ranges for the memory regions will be added here. Note that the property is not cleared first, so it should be left empty.

Memory layout

uvmm populates the RAM with the following data:

  • kernel binary
  • (optional) ramdisk
  • (optional) device tree

The kernel binary is put at the predefined address. For ELF binaries, this is an absolute physical address. If the binary supports relative addressing, the binary is put to the requested offset relative to beginning of the first 'memory' region defined in the device tree.

The ramdisk and device tree are placed as far as possible to the end of the regions defined in the first 'memory' node.

If there is a part of RAM that must remain empty, then define an extra memory node for it in the device tree. uvmm only writes to memory in the first memory node it finds.

Warning: uvmm does not touch any unpopulated memory. In particular, it does not ensure that the memory is cleared. It is the responsibility of the provider of the RAM dataspace to make sure that no data leakage can happen. Normally this is not an issue because dataspaces are guaranteed to be cleaned when they are newly created but users should be careful when reusing memory or dataspaces, for example, when restarting the uvmm.

Forwarding hardware resources to the guest

Hardware resources must be specified in two places: the device tree contains the description of all hardware devices the guest could see and the Vbus describes which resources are actually available to the uvmm.

The vbus allows the uvmm access to hardware resources in the same way as any other L4 application. uvmm expects a capability named 'vbus' where it can access its hardware resources. It is possible to leave out the capability for purely virtual guests (Note that this is not actually practical on some architectures. On ARM, for example, the guest needs hardware access to the interrupt controller. Without a 'vbus' capability, interrupts will not work.) For information on how to configure a vbus, see the IO documentation.

The device tree needs to contain the hardware description the guest should see. For hardware devices this usually means to use a device tree that would also be used when running the guest directly on hardware.

On startup, uvmm scans the device tree for any devices that require memory or interrupt resources and compares the required resources with the ones available from its vbus. When all resources are available, it sets up the appropriate forwarding, so that the guest now has direct access to the hardware. If the resources are not available, the device will be marked as 'disabled'. This mechanism allows to work with a standard device tree for all guests in the system while handling the actual resource allocation in a flexible manner via the vbus configuration.

The default mechanism assigns all resources 1:1, i.e. with the same memory address and interrupt number as on hardware. It is also possible to map a hardware device to a different location. In this case, the assignment between vbus device and device tree device must be known in advance and marked in the device tree using the l4vmm,vbus-dev property.

The following device will for example be bound with the vbus device with the HID 'l4-test,dev':

test@e0000000 {
    compatible = "memdev,bar";
    reg = <0 0xe0000000 0 0x50000>,
          <0 0xe1000000 0 0x50000>;
    l4vmm,vbus-dev = "l4-test,dev";
    interrupts-extended = <&gic 0 139 4>;
};

Resources are then matched by name. Memory resources in the vbus must be named reg0 to reg9 to match against the address ranges in the device tree reg property. Interrupts must be called irq0 to irq9 and will be matched against interrupts or interrupts-extended entries in the device tree. The vbus must expose resources for all resources defined in the device tree entry or the initialisation will fail.

An appropriate IO entry for the above device would thus be: MEM = Io.Hw.Device(function() Property.hid = "l4-test,dev" Resource.reg0 = Io.Res.mmio(0x41000000, 0x4104ffff) Resource.reg1 = Io.Res.mmio(0x42000000, 0x4204ffff) Resource.irq0 = Io.Res.irq(134); end)

Please note that HIDs on the vbus are not necessarily unique. If multiple devices with the HID given in l4vmm,vbus-dev are available on the vbus, then one device is chosen at random.

If no vbus device with the given HID is available, the device is disabled.

How to enable guest suspend/resume

Note
Currently only supported on ARM. It should work fine with Linux version 4.4 or newer.

Uvmm (partially) implements the power state coordination interface (PSCI), which is the standard ARM power management interface. To make use of this interface, you have to announce its availability to the guest operating system via the device tree like so:

psci {
      compatible = "arm,psci-0.2";
      method = "hvc";
};

The Linux guest must be configured with at least these options:

CONFIG_SUSPEND=y
CONFIG_ARM_PSCI=y

How to communicate power management (PM) events

Uvmm can be instructed to inform a PM manager of PM events through the L4::Platform_control interface. To that end, uvmm may be equipped with a pfc cap. On suspend, uvmm will call l4_platform_ctl_system_suspend().

The pfc cap can also be implemented by IO. In that case the guest can start a machine suspend/shutdown/reboot.

KVM clock for uvmm/amd64 guests

When executing L4Re + uvmm on QEMU, the PIT as clock source is not reliable. The paravirtualized KVM clock provides the guest with a stable clock source.

A KVM clock device is available to the guest, if the device tree contains the corresponding entry:

kvm_clock {
    compatible = "kvm-clock";
    reg = <0x0 0x0 0x0 0x0>;
};

To make use of this clock, the Linux guest must be built with the following configuration options:

CONFIG_HYPERVISOR_GUEST=y CONFIG_KVM_GUEST=y CONFIG_PTP_1588_CLOCK_KVM is not set

Note: KVM calls besides the KVM clock are unhandled and lead to failure in the uvmm, e.g. vmcall 0x9 for the PTP_1588_CLOCK_KVM.

This is considered a development feature. The KVM clock is not required when running on physical hardware as TSC calibration via the PIT works as expected.