Commit graph

35 commits

Author SHA1 Message Date
Andreas Sandberg
e7d230ede0 kvm: x86: Always assume segments to be usable
When transferring segment registers into kvm, we need to find the
value of the unusable bit. We used to assume that this could be
inferred from the selector since segments are generally unusable if
their selector is 0. This assumption breaks in some weird corner
cases.  Instead, we just assume that segments are always usable. This
is what qemu does so it should work.
2014-03-03 14:34:33 +01:00
Andreas Sandberg
739cc0128b kvm: Initialize signal handlers from startupThread()
Signal handlers in KVM are controlled per thread and should be
initialized from the thread that is going to execute the CPU. This
changeset moves the initialization call from startup() to
startupThread().
2014-03-03 14:31:39 +01:00
Andreas Sandberg
0d6009e8dc kvm: Add support for multi-system simulation
The introduction of parallel event queues added most of the support
needed to run multiple VMs (systems) within the same gem5
instance. This changeset fixes up signal delivery so that KVM's
control signals are delivered to the thread that executes the CPU's
event queue. Specifically:

  * Timers and counters are now initialized from a separate method
    (startupThread) that is scheduled as the first event in the
    thread-specific event queue. This ensures that they are
    initialized from the thread that is going to execute the CPUs
    event queue and enables signal delivery to the right thread when
    exiting from KVM.

  * The POSIX-timer-based KVM timer (used to force exits from KVM) has
    been updated to deliver signals to the thread that's executing KVM
    instead of the process (thread is undefined in that case). This
    assumes that the timer is instantiated from the thread that is
    going to execute the KVM vCPU.

  * Signal masking is now done using pthread_sigmask instead of
    sigprocmask. The behavior of the latter is undefined in threaded
    applications.

  * Since signal masks can be inherited, make sure to actively unmask
    the control signals when setting up the KVM signal mask.

There are currently no facilities to multiplex between multiple KVM
CPUs in the same event queue, we are therefore limited to
configurations where there is only one KVM CPU per event queue. In
practice, this means that multi-system configurations can be
simulated, but not multiple CPUs in a shared-memory configuration.
2014-02-20 15:43:53 +01:00
Andreas Sandberg
4b8be6a90b kvm: Set the perf exclude_host attribute if available
The performance counting framework in Linux 3.2 and onwards supports
an attribute to exclude events generated by the host when running
KVM. Setting this attribute allows us to get more reliable
measurements of the guest machine. For example, on a highly loaded
system, the instruction counts from the guest can be severely
distorted by the host kernel (e.g., by page fault handlers).

This changeset introduces a check for the attribute and enables it in
the KVM CPU if present.
2013-10-15 10:09:23 +02:00
Andreas Sandberg
e5d63d0535 kvm: Remove the unused hostFreq member from BaseKvmCPU 2013-11-26 17:40:58 +01:00
Steve Reinhardt ext:(%2C%20Nilay%20Vaish%20%3Cnilay%40cs.wisc.edu%3E%2C%20Ali%20Saidi%20%3CAli.Saidi%40ARM.com%3E)
de366a16f1 sim: simulate with multiple threads and event queues
This patch adds support for simulating with multiple threads, each of
which operates on an event queue.  Each sim object specifies which eventq
is would like to be on.  A custom barrier implementation is being added
using which eventqs synchronize.

The patch was tested in two different configurations:
1. ruby_network_test.py: in this simulation L1 cache controllers receive
   requests from the cpu. The requests are replied to immediately without
   any communication taking place with any other level.
2. twosys-tsunami-simple-atomic: this configuration simulates a client-server
   system which are connected by an ethernet link.

We still lack the ability to communicate using message buffers or ports. But
other things like simulation start and end, synchronizing after every quantum
are working.

Committed by: Nilay Vaish
2013-11-25 11:21:00 -06:00
Andreas Sandberg
cc42e87b85 kvm: Fix latency calculation of IPR accesses
When handling IPR accesses in doMMIOAccess, the KVM CPU used
clockEdge() to convert between cycles and ticks. This is incorrect
since doMMIOAccess is supposed to return a latency in ticks rather
than when the access is done. This changeset fixes this issue by
returning clockPeriod() * ipr_delay instead.
2013-10-16 18:12:15 +02:00
Andreas Sandberg
0dd6f87e63 kvm: Service events in the instruction event queues
This changset adds calls to the service the instruction event queues
that accidentally went missing from commit [0063c7dd18ec]. The
original commit only included the code needed to schedule instruction
stops from KVM and missed the functionality to actually service the
events.
2013-10-03 11:00:18 +02:00
Andreas Sandberg
469f2e31cf kvm: Add support for thread-specific instruction events
Instruction events are currently ignored when executing in KVM. This
changeset adds support for triggering KVM exits based on instruction
counts using hardware performance counters. Depending on the
underlying performance counter implementation, there might be some
inaccuracies due to instructions being counted in the host kernel when
entering/exiting KVM.

Due to limitations/bugs in Linux's performance counter interface, we
can't reliably change the period of an overflow counter. We work
around this issue by detaching and reattaching the counter if we need
to reconfigure it.
2013-09-30 09:53:52 +02:00
Andreas Sandberg
86bade714e kvm: FPU synchronization support on x86
This changeset adds support for synchronizing the FPU and SIMD state
of a virtual x86 CPU with gem5. It supports both the XSave API and the
KVM_(GET|SET)_FPU kernel API. The XSave interface can be disabled
using the useXSave parameter (in case of kernel
issues). Unfortunately, KVM_(GET|SET)_FPU interface seems to be buggy
in some kernels (specifically, the MXCSR register isn't always
synchronized), which means that it might not be possible to
synchronize MXCSR on old kernels without the XSave interface.

This changeset depends on the __float80 type in gcc and might not
build using llvm.
2013-09-30 09:43:43 +02:00
Andreas Sandberg
30841926a3 kvm: x86: Fix segment registers to make them VMX compatible
There are cases when the segment registers in gem5 are not compatible
with VMX. This changeset works around all known such issues. Specifically:

* The accessed bits in CS, SS, DD, ES, FS, GS are forced to 1.
* The busy bit in TR is forced to 1.
* The protection level of SS is forced to the same protection level as
  CS. The difference /seems/ to be caused by a bug in gem5's x86
  implementation.
2013-09-30 09:36:54 +02:00
Andreas Sandberg
e5c319db43 kvm: Add x86 segment register verification to help debugging 2013-09-25 12:35:21 +02:00
Andreas Sandberg
599b59b387 kvm: Initial x86 support
This changeset adds support for KVM on x86. Full support is split
across a number of commits since some features are relatively
complex. This changeset includes support for:

 * Integer state synchronization (including segment regs)
 * CPUID (gem5's CPUID values are inserted into KVM)
 * x86 legacy IO (remapped and handled by gem5's memory system)
 * Memory mapped IO
 * PCI
 * MSRs
 * State dumping

Most of the functionality is fairly straight forward. There are some
quirks to support PCI enumerations since this is done in the TLB(!) in
the simulated CPUs. We currently replicate some of that code.

Unlike the ARM implementation, the x86 implementation of the virtual
CPU does not use the cycles hardware counter. KVM on x86 simulates the
time stamp counter (TSC) in the kernel. If we just measure host cycles
using perfevent, we might end up measuring a slightly different number
of cycles. If we don't get the cycle accounting right, we might end up
rewinding the TSC, with all kinds of chaos as a result.

An additional feature of the KVM CPU on x86 is extended state
dumping. This enables Python scripts controlling the simulator to
request dumping of a subset of the processor state. The following
methods are currenlty supported:

 * dumpFpuRegs
 * dumpIntRegs
 * dumpSpecRegs
 * dumpDebugRegs
 * dumpXCRs
 * dumpXSave
 * dumpVCpuEvents
 * dumpMSRs

Known limitations:
  * M5 ops are currently not supported.
  * FPU synchronization is not supported (only affects CPU switching).

Both of the limitations will be addressed in separate commits.
2013-09-25 12:24:26 +02:00
Andreas Sandberg
cd9cd85ce9 kvm: Correctly handle the return value from handleIpr(Read|Write)
The KVM base class incorrectly assumed that handleIprRead and
handleIprWrite both return ticks. This is not the case, instead they
return cycles. This changeset converts the returned cycles to ticks
when handling IPR accesses.
2013-09-19 17:55:04 +02:00
Andreas Sandberg
211c10b46d kvm: Fix a case where the run timers weren't armed properly
There is a possibility that the timespec used to arm a timer becomes
zero if the number of ticks used when arming a timer is close to the
resolution of the timer. Due to the semantics of POSIX timers, this
actually disarms the timer. This changeset fixes this issue by
eliminating the rounding error (we always round away from zero
now). It also reuses the minimum number of cycles, which were
previously only used for cycle-based timers, to calculate a more
useful resolution.
2013-09-19 17:55:03 +02:00
Andreas Sandberg
6151c0f7f4 kvm: Use the address finalization code in the TLB
Reuse the address finalization code in the TLB instead of replicating
it when handling MMIO. This patch also adds support for injecting
memory mapped IPR requests into the memory system.
2013-06-18 16:10:22 +02:00
Andreas Sandberg
64270b19c3 kvm: Add more VM stats
This changeset adds the following stats to KVM:
 * numVMHalfEntries: Number of entries into KVM to finalize pending
   IO operations without executing guest instructions. These typically
   happen as a result of a drain where the guest must finalize some
   operations before the guest state is consistent.
 * numExitSignal: Number of VM exits that have been triggered by a
   signal. These usually happen as a result of the timer that limits
   the time spent in KVM.
2013-06-11 09:43:05 +02:00
Andreas Sandberg
c97a99110b kvm: Separate host frequency from simulated CPU frequency
We used to use the KVM CPU's clock to specify the host frequency. This
was not ideal for several reasons. One of them being that the clock
parameter of a CPU determines the frequency of some of the components
connected to the CPU. This changeset adds a separate hostFreq
parameter that should be used to specify the host frequency until we
add code to autodetect it. The hostFactor should still be used to
specify the conversion factor between the host performance and that of
the simulated system.
2013-06-11 09:24:55 +02:00
Andreas Sandberg
4f002930bc kvm: Don't handle IO and execute in the same tick
We currently execute instructions in the guest and then handle any IO
request right after we break out of the virtualized environment. This
has the effect of executing IO requests in the exact same tick as the
first instruction in the sequence that was just run. There seem to be
cases where this simplification upsets some timing-sensitive devices.

This changeset splits execute and IO (and other services) across
multiple ticks. This is implemented by adding a separate
RunningService state to the CPU state machine. When a VM requires
service, it enters into this state and pending IO is then serviced in
the future instead of immediately. The delay between getting the
request and servicing it depends on the number of cycles executed in
the guest, which allows other components to catch up with the CPU.
2013-06-11 09:24:51 +02:00
Andreas Sandberg
df059f45a0 kvm: Maintain a local instruction counter and update totalNumInsts
Update the system's totalNumInst counter when exiting from KVM and
maintain an internal absolute instruction count instead of relying on
the one from perf.
2013-06-11 09:24:40 +02:00
Andreas Sandberg
c2ec232920 kvm: Allow architectures to override the cycle accounting mechanism
Some architectures have special registers in the guest that can be
used to do cycle accounting. This is generally preferrable since the
prevents the guest from seeing a non-monotonic clock. This changeset
adds a virtual method, getHostCycles(), that the architecture-specific
code can override to implement this functionallity. The default
implementation uses the hwCycles counter.
2013-06-03 13:39:11 +02:00
Andreas Sandberg
15f81b6ed9 kvm: Add handling of EAGAIN when creating timers
timer_create can apparently return -1 and set errno to EAGAIN if the
kernel suffered a temporary failure when allocating a timer. This
happens from time to time, so we need to handle it.
2013-06-03 13:38:59 +02:00
Andreas Sandberg
2b65fce5d9 kvm: Add a call to thread->startup() in startup()
It is now required to initialize the thread context by calling
startup() on it. Failing to do so currently causes decoder in
x86-based CPUs to get very confused when restoring from checkpoints.
2013-06-03 12:36:56 +02:00
Andreas Sandberg
4e52789c6d kvm: Add support for disabling coalesced MMIO
Add the option useCoalescedMMIO to the BaseKvmCPU. The default
behavior is to disable coalesced MMIO since this hasn't been heavily
tested.
2013-05-14 16:02:45 +02:00
Andreas Sandberg
3ba93822cc kvm: Dump state before panic in KVM exit handlers 2013-05-14 15:59:43 +02:00
Andreas Sandberg
98483ba858 kvm: Fix the memory interface used by KVM
The CpuPort class was removed before the KVM patches were committed,
which means that the KVM interface currently doesn't compile. This
changeset adds the BaseKvmCPU::KVMCpuPort class which derives from
MasterPort. This class is used on the data and instruction ports
instead of the old CpuPort.
2013-05-14 15:56:04 +02:00
Andreas Sandberg
e316e4e5fe kvm: Add a stat counting number of instructions executed
This changeset adds a 'numInsts' stat to the KVM-based CPU. It also
cleans up the variable names in kvmRun to make the distinction between
host cycles and estimated simulated cycles clearer. As a bonus
feature, it also fixes a warning (unreferenced variable) when
compiling in fast mode.
2013-05-02 12:03:43 +02:00
Andreas Sandberg
fa249461ca kvm: Add checkpoint debug print
Add a debug print (when the Checkpoint debug flag is set) on serialize
and unserialize. Additionally, dump the KVM state before
serializing. The KVM state isn't dumped after unserializing since the
state is loaded lazily on the next KVM entry.
2013-05-02 12:02:19 +02:00
Andreas Sandberg
41156c8196 kvm: Make MMIO requests uncacheable
Device accesses are normally uncacheable. This change probably doesn't
make any difference since we normally disable caching when KVM is
active. However, there might be devices that check this, so we'd
better enable this flag to be safe.
2013-05-02 12:01:50 +02:00
Andreas Sandberg
33ab8f735d kvm: Add support for pseudo-ops on ARM
This changeset adds support for m5 pseudo-ops when running in
kvm-mode. Unfortunately, we can't trap the normal gem5 co-processor
entry in KVM (it doesn't seem to be possible to trap accesses to
non-existing co-processors). We therefore use BZJ instructions to
cause a trap from virtualized mode into gem5. The BZJ instruction is
becomes a normal branch to the gem5 fallback code when running in
simulated mode, which means that this patch does not need to change
the ARM ISA-specific code.

Note: This requires a patched host kernel.
2013-04-22 13:20:32 -04:00
Andreas Sandberg
32ecd72b6e kvm: Add support for state dumping on ARM 2013-04-22 13:20:32 -04:00
Andreas Sandberg
f156020158 kvm: Add basic support for ARM
Architecture specific limitations:
 * LPAE is currently not supported by gem5. We therefore panic if LPAE
   is enabled when returning to gem5.
 * The co-processor based interface to the architected timer is
   unsupported. We can't support this due to limitations in the KVM
   API on ARM.
 * M5 ops are currently not supported. This requires either a kernel
   hack or a memory mapped device that handles the guest<->m5
   interface.
2013-04-22 13:20:32 -04:00
Andreas Sandberg
f8f66fa3df kvm: Add experimental support for a perf-based execution timer
Add support for using the CPU cycle counter instead of a normal POSIX
timer to generate timed exits to gem5. This should, in theory, provide
better resolution when requesting timer signals.

The perf-based timer requires a fairly recent kernel since it requires
a working PERF_EVENT_IOC_PERIOD ioctl. This ioctl has existed in the
kernel for a long time, but it used to be completely broken due to an
inverted match when the kernel copied things from user
space. Additionally, the ioctl does not change the sample period
correctly on all kernel versions which implement it. It is currently
only known to work reliably on kernel version 3.7 and above on ARM.
2013-04-22 13:20:32 -04:00
Andreas Sandberg
2607efded8 kvm: Avoid synchronizing the TC on every KVM exit
Reduce the number of KVM->TC synchronizations by overloading the
getContext() method and only request an update when the TC is
requested as opposed to every time KVM returns to gem5.
2013-04-22 13:20:32 -04:00
Andreas Sandberg
f485ad1908 kvm: Basic support for hardware virtualized CPUs
This changeset introduces the architecture independent parts required
to support KVM-accelerated CPUs. It introduces two new simulation
objects:

KvmVM -- The KVM VM is a component shared between all CPUs in a shared
         memory domain. It is typically instantiated as a child of the
         system object in the simulation hierarchy. It provides access
         to KVM VM specific interfaces.

BaseKvmCPU -- Abstract base class for all KVM-based CPUs. Architecture
	      dependent CPU implementations inherit from this class
	      and implement the following methods:

                * updateKvmState() -- Update the
                  architecture-dependent KVM state from the gem5
                  thread context associated with the CPU.

                * updateThreadContext() -- Update the thread context
                  from the architecture-dependent KVM state.

                * dump() -- Dump the KVM state using (optional).

	      In order to deliver interrupts to the guest, CPU
	      implementations typically override the tick() method and
	      check for, and deliver, interrupts prior to entering
	      KVM.

Hardware-virutalized CPU currently have the following limitations:
 * SE mode is not supported.
 * PC events are not supported.
 * Timing statistics are currently very limited. The current approach
   simply scales the host cycles with a user-configurable factor.
 * The simulated system must not contain any caches.
 * Since cycle counts are approximate, there is no way to request an
   exact number of cycles (or instructions) to be executed by the CPU.
 * Hardware virtualized CPUs and gem5 CPUs must not execute at the
   same time in the same simulator instance.
 * Only single-CPU systems can be simulated.
 * Remote GDB connections to the guest system are not supported.

Additionally, m5ops requires an architecture specific interface and
might not be supported.
2013-04-22 13:20:32 -04:00