halt-polling.rst

  1.. SPDX-License-Identifier: GPL-2.0
  2
  3===========================
  4The KVM halt polling system
  5===========================
  6
  7The KVM halt polling system provides a feature within KVM whereby the latency
  8of a guest can, under some circumstances, be reduced by polling in the host
  9for some time period after the guest has elected to no longer run by cedeing.
 10That is, when a guest vcpu has ceded, or in the case of powerpc when all of the
 11vcpus of a single vcore have ceded, the host kernel polls for wakeup conditions
 12before giving up the cpu to the scheduler in order to let something else run.
 13
 14Polling provides a latency advantage in cases where the guest can be run again
 15very quickly by at least saving us a trip through the scheduler, normally on
 16the order of a few micro-seconds, although performance benefits are workload
 17dependent. In the event that no wakeup source arrives during the polling
 18interval or some other task on the runqueue is runnable the scheduler is
 19invoked. Thus halt polling is especially useful on workloads with very short
 20wakeup periods where the time spent halt polling is minimised and the time
 21savings of not invoking the scheduler are distinguishable.
 22
 23The generic halt polling code is implemented in:
 24
 25	virt/kvm/kvm_main.c: kvm_vcpu_block()
 26
 27The powerpc kvm-hv specific case is implemented in:
 28
 29	arch/powerpc/kvm/book3s_hv.c: kvmppc_vcore_blocked()
 30
 31Halt Polling Interval
 32=====================
 33
 34The maximum time for which to poll before invoking the scheduler, referred to
 35as the halt polling interval, is increased and decreased based on the perceived
 36effectiveness of the polling in an attempt to limit pointless polling.
 37This value is stored in either the vcpu struct:
 38
 39	kvm_vcpu->halt_poll_ns
 40
 41or in the case of powerpc kvm-hv, in the vcore struct:
 42
 43	kvmppc_vcore->halt_poll_ns
 44
 45Thus this is a per vcpu (or vcore) value.
 46
 47During polling if a wakeup source is received within the halt polling interval,
 48the interval is left unchanged. In the event that a wakeup source isn't
 49received during the polling interval (and thus schedule is invoked) there are
 50two options, either the polling interval and total block time[0] were less than
 51the global max polling interval (see module params below), or the total block
 52time was greater than the global max polling interval.
 53
 54In the event that both the polling interval and total block time were less than
 55the global max polling interval then the polling interval can be increased in
 56the hope that next time during the longer polling interval the wake up source
 57will be received while the host is polling and the latency benefits will be
 58received. The polling interval is grown in the function grow_halt_poll_ns() and
 59is multiplied by the module parameters halt_poll_ns_grow and
 60halt_poll_ns_grow_start.
 61
 62In the event that the total block time was greater than the global max polling
 63interval then the host will never poll for long enough (limited by the global
 64max) to wakeup during the polling interval so it may as well be shrunk in order
 65to avoid pointless polling. The polling interval is shrunk in the function
 66shrink_halt_poll_ns() and is divided by the module parameter
 67halt_poll_ns_shrink, or set to 0 iff halt_poll_ns_shrink == 0.
 68
 69It is worth noting that this adjustment process attempts to hone in on some
 70steady state polling interval but will only really do a good job for wakeups
 71which come at an approximately constant rate, otherwise there will be constant
 72adjustment of the polling interval.
 73
 74[0] total block time:
 75		      the time between when the halt polling function is
 76		      invoked and a wakeup source received (irrespective of
 77		      whether the scheduler is invoked within that function).
 78
 79Module Parameters
 80=================
 81
 82The kvm module has 4 tunable module parameters to adjust the global max polling
 83interval, the initial value (to grow from 0), and the rate at which the polling
 84interval is grown and shrunk. These variables are defined in
 85include/linux/kvm_host.h and as module parameters in virt/kvm/kvm_main.c, or
 86arch/powerpc/kvm/book3s_hv.c in the powerpc kvm-hv case.
 87
 88+-----------------------+---------------------------+-------------------------+
 89|Module Parameter	|   Description		    |	     Default Value    |
 90+-----------------------+---------------------------+-------------------------+
 91|halt_poll_ns		| The global max polling    | KVM_HALT_POLL_NS_DEFAULT|
 92|			| interval which defines    |			      |
 93|			| the ceiling value of the  |			      |
 94|			| polling interval for      | (per arch value)	      |
 95|			| each vcpu.		    |			      |
 96+-----------------------+---------------------------+-------------------------+
 97|halt_poll_ns_grow	| The value by which the    | 2			      |
 98|			| halt polling interval is  |			      |
 99|			| multiplied in the	    |			      |
100|			| grow_halt_poll_ns()	    |			      |
101|			| function.		    |			      |
102+-----------------------+---------------------------+-------------------------+
103|halt_poll_ns_grow_start| The initial value to grow | 10000		      |
104|			| to from zero in the	    |			      |
105|			| grow_halt_poll_ns()	    |			      |
106|			| function.		    |			      |
107+-----------------------+---------------------------+-------------------------+
108|halt_poll_ns_shrink	| The value by which the    | 2			      |
109|			| halt polling interval is  |			      |
110|			| divided in the	    |			      |
111|			| shrink_halt_poll_ns()	    |			      |
112|			| function.		    |			      |
113+-----------------------+---------------------------+-------------------------+
114
115These module parameters can be set from the sysfs files in:
116
117	/sys/module/kvm/parameters/
118
119Note: these module parameters are system-wide values and are not able to
120      be tuned on a per vm basis.
121
122Any changes to these parameters will be picked up by new and existing vCPUs the
123next time they halt, with the notable exception of VMs using KVM_CAP_HALT_POLL
124(see next section).
125
126KVM_CAP_HALT_POLL
127=================
128
129KVM_CAP_HALT_POLL is a VM capability that allows userspace to override halt_poll_ns
130on a per-VM basis. VMs using KVM_CAP_HALT_POLL ignore halt_poll_ns completely (but
131still obey halt_poll_ns_grow, halt_poll_ns_grow_start, and halt_poll_ns_shrink).
132
133See Documentation/virt/kvm/api.rst for more information on this capability.
134
135Further Notes
136=============
137
138- Care should be taken when setting the halt_poll_ns module parameter as a large value
139  has the potential to drive the cpu usage to 100% on a machine which would be almost
140  entirely idle otherwise. This is because even if a guest has wakeups during which very
141  little work is done and which are quite far apart, if the period is shorter than the
142  global max polling interval (halt_poll_ns) then the host will always poll for the
143  entire block time and thus cpu utilisation will go to 100%.
144
145- Halt polling essentially presents a trade-off between power usage and latency and
146  the module parameters should be used to tune the affinity for this. Idle cpu time is
147  essentially converted to host kernel time with the aim of decreasing latency when
148  entering the guest.
149
150- Halt polling will only be conducted by the host when no other tasks are runnable on
151  that cpu, otherwise the polling will cease immediately and schedule will be invoked to
152  allow that other task to run. Thus this doesn't allow a guest to cause denial of service
153  of the cpu.