Add a set of scripts to automatically test checkpointing in the
regression framework. The checkpointing tests are similar to the
switcheroo tests, but instead of switching between CPUs, they
checkpoint the system and restore from the checkpoint again. This is
done at regular intervals, typically while booting Linux.
The implementation is fairly straight forward, with the exception that
we have to work around gem5's inability to restore from a checkpoint
after a system has been instantiated. We work around this by forking
off child processes that does the actual simulation and never
instantiate a system in the parent process unless a maximum checkpoint
count is reached (in which case we just simulate the system to
completion in the parent).
Checkpoint testing is currently only enabled 32- and 64-bit ARM
systems using atomic CPUs.
Note: An unfortunate side-effect of forking is that every new process
will overwrite the stats and terminal output from the previous
process. This means that the output directory only contains data from
the last checkpoint.
This patch introduces a few subclasses to the CoherentXBar and
NoncoherentXBar to distinguish the different uses in the system. We
use the crossbar in a wide range of places: interfacing cores to the
L2, as a system interconnect, connecting I/O and peripherals,
etc. Needless to say, these crossbars have very different performance,
and the clock frequency alone is not enough to distinguish these
scenarios.
Instead of trying to capture every possible case, this patch
introduces dedicated subclasses for the three primary use-cases:
L2XBar, SystemXBar and IOXbar. More can be added if needed, and the
defaults can be overridden.
The MemTest class really only tests false sharing, and as such there
was a lot of old cruft that could be removed. This patch cleans up the
tester, and also makes it more clear what the assumptions are. As part
of this simplification the reference functional memory is also
removed.
The regression configs using MemTest are updated to reflect the
changes, and the stats will be bumped in a separate patch. The example
config will be updated in a separate patch due to more extensive
re-work.
In a follow-on patch a new tester will be introduced that uses the
MemChecker to implement true sharing.
This patch removes the three MIPS and SPARC regressions that use the
deprecated InOrderCPU.
This is the first step in completely removing the code from the tree,
avoiding confusion, and focusing all development efforts on the
MinorCPU. Brave new world.
Changes due to speculative execution of an unaligned PC, introduction
of TLB stats, changes and re-work of the prefetcher, and the
introduction of rank-wise refresh in the DRAM controller.
Re-use the existing traffic generator regression, and enable the stack
distance calculation in the comm monitor, along with the verification
stack.
The traffic generator config is also tuned to not increase the
run-time too much (and actually have some address re-use).
This is a simple test program for the new mwait implemenation. It is uses
m5threads to create to threads of execution in syscall emulation mode that
interact using the mwait instruction.
Committed by: Nilay Vaish <nilay@cs.wisc.edu>
This patch is the final in the series. The whole series and this patch in
particular were written with the aim of interfacing ruby's directory controller
with the memory controller in the classic memory system. This is being done
since ruby's memory controller has not being kept up to date with the changes
going on in DRAMs. Classic's memory controller is more up to date and
supports multiple different types of DRAM. This also brings classic and
ruby ever more close. The patch also changes ruby's memory controller to
expose the same interface.
Both ruby and the system used to maintain memory copies. With the changes
carried for programmed io accesses, only one single memory is required for
fs simulations. This patch sets the copy of memory that used to reside
with the system to null, so that no space is allocated, but address checks
can still be carried out. All the memory accesses now source and sink values
to the memory maintained by ruby.
This changes the default ARM system to a Versatile Express-like system that supports
2GB of memory and PCI devices and updates the default kernels/file-systems for
AArch64 ARM systems (64-bit) to support up to 32GB of memory and PCI devices. Some
platforms that are no longer supported have been pruned from the configuration files.
In addition a set of 64-bit ARM regressions have been added to the regression system.
As a result of the fixes, the full-system dual-core ARM regressions
are slightly changed. Hopefully this also means there will no longer
be any discrepancies between the results observed on different hosts.
This patch bumps the stats to reflect the addition of the snoop filter
and snoop stats, the change from bus to crossbar, and the updates to
the ARM regressions that are now using a different CPU and cache
configuration. Lastly, some minor changes are expected due to the
activation cleanup of the CPUs.
This patch changes the CPU and cache configurations used in the ARM SE and FS
regressions to make them more representative, and also get better code
coverage by exercising different replacement policies and use an L2
prefetcher.
This patch changes the name of the Bus classes to XBar to better
reflect the actual timing behaviour. The actual instances in the
config scripts are not renamed, and remain as e.g. iobus or membus.
As part of this renaming, the code has also been clean up slightly,
making use of range-based for loops and tidying up some comments. The
only changes outside the bus/crossbar code is due to the delay
variables in the packet.
--HG--
rename : src/mem/Bus.py => src/mem/XBar.py
rename : src/mem/coherent_bus.cc => src/mem/coherent_xbar.cc
rename : src/mem/coherent_bus.hh => src/mem/coherent_xbar.hh
rename : src/mem/noncoherent_bus.cc => src/mem/noncoherent_xbar.cc
rename : src/mem/noncoherent_bus.hh => src/mem/noncoherent_xbar.hh
rename : src/mem/bus.cc => src/mem/xbar.cc
rename : src/mem/bus.hh => src/mem/xbar.hh
This patch changes the perlbmk regression script from the large to the
medium dataset to reduce the regression run time. For all ISAs and CPU
models, the total perlbmk host CPU time with the large dataset is
roughly 12 hours (constituting >30% of the total regression host
time). There is, most likely, almost no added value in terms of code
coverage for this rather excessive run time.
This patch avoids building the 'inorder' CPU model for any permutation
of ALPHA, and also removes the ALPHA regressions using the 'inorder'
CPU. The 'minor' CPU is already providing a broader test coverage.
This patch changes the CPU configuration used for the full-system ARM
regressions to increase the test coverage. Note that it is only the
core configuration, and not the caches etc.
This patch fixes scripts related to ruby by adding the ruby clock domain.
Now the L1 controllers and the Sequencer shares the cpu clock domain,
while the rest of the components use the ruby clock domain.
Before this patch, running simulations with the cpu clock set at 2GHz or
1GHz will output the same time results and could distort power measurements.
Committed by: Nilay Vaish <nilay@cs.wisc.edu>
Only printing one rather than two args for the ignored syscall
warning means the count of register accesses has changed on
a few runs. Oddly only Alpha Tru64 seems to have any ignored
syscalls in the regression tests.
Mostly small differences in total ticks, but O3 stall causes
shifted significantly.
30.eon does speed up by ~6% on Alpha and ARM, and 50.vortex
by 4.5% on ARM. At the other extreme, X86 70.twolf is 0.8%
slower.
This patch reflects the recent name change in the DRAM TrafficGen
tests and also tidies up the test directory.
--HG--
rename : tests/configs/tgen-simple-dram.py => tests/configs/tgen-dram-ctrl.py
rename : tests/quick/se/70.tgen/ref/null/none/tgen-simple-dram/config.ini => tests/quick/se/70.tgen/ref/null/none/tgen-dram-ctrl/config.ini
rename : tests/quick/se/70.tgen/ref/null/none/tgen-simple-dram/simerr => tests/quick/se/70.tgen/ref/null/none/tgen-dram-ctrl/simerr
rename : tests/quick/se/70.tgen/ref/null/none/tgen-simple-dram/simout => tests/quick/se/70.tgen/ref/null/none/tgen-dram-ctrl/simout
rename : tests/quick/se/70.tgen/ref/null/none/tgen-simple-dram/stats.txt => tests/quick/se/70.tgen/ref/null/none/tgen-dram-ctrl/stats.txt
rename : tests/quick/se/70.tgen/tgen-simple-dram.cfg => tests/quick/se/70.tgen/tgen-dram-ctrl.cfg
This patch encompasses several interrelated and interdependent changes
to the ISA generation step. The end goal is to reduce the size of the
generated compilation units for instruction execution and decoding so
that batch compilation can proceed with all CPUs active without
exhausting physical memory.
The ISA parser (src/arch/isa_parser.py) has been improved so that it can
accept 'split [output_type];' directives at the top level of the grammar
and 'split(output_type)' python calls within 'exec {{ ... }}' blocks.
This has the effect of "splitting" the files into smaller compilation
units. I use air-quotes around "splitting" because the files themselves
are not split, but preprocessing directives are inserted to have the same
effect.
Architecturally, the ISA parser has had some changes in how it works.
In general, it emits code sooner. It doesn't generate per-CPU files,
and instead defers to the C preprocessor to create the duplicate copies
for each CPU type. Likewise there are more files emitted and the C
preprocessor does more substitution that used to be done by the ISA parser.
Finally, the build system (SCons) needs to be able to cope with a
dynamic list of source files coming out of the ISA parser. The changes
to the SCons{cript,truct} files support this. In broad strokes, the
targets requested on the command line are hidden from SCons until all
the build dependencies are determined, otherwise it would try, realize
it can't reach the goal, and terminate in failure. Since build steps
(i.e. running the ISA parser) must be taken to determine the file list,
several new build stages have been inserted at the very start of the
build. First, the build dependencies from the ISA parser will be emitted
to arch/$ISA/generated/inc.d, which is then read by a new SCons builder
to finalize the dependencies. (Once inc.d exists, the ISA parser will not
need to be run to complete this step.) Once the dependencies are known,
the 'Environments' are made by the makeEnv() function. This function used
to be called before the build began but now happens during the build.
It is easy to see that this step is quite slow; this is a known issue
and it's important to realize that it was already slow, but there was
no obvious cause to attribute it to since nothing was displayed to the
terminal. Since new steps that used to be performed serially are now in a
potentially-parallel build phase, the pathname handling in the SCons scripts
has been tightened up to deal with chdir() race conditions. In general,
pathnames are computed earlier and more likely to be stored, passed around,
and processed as absolute paths rather than relative paths. In the end,
some of these issues had to be fixed by inserting serializing dependencies
in the build.
Minor note:
For the null ISA, we just provide a dummy inc.d so SCons is never
compelled to try to generate it. While it seems slightly wrong to have
anything in src/arch/*/generated (i.e. a non-generated 'generated' file),
it's by far the simplest solution.
Splits the CommMonitor trace_file parameter into three parameters. Previously,
the trace was only enabled if the trace_file parameter was set, and would be
written to this file. This patch adds in a trace_enable and trace_compress
parameter to the CommMonitor.
No trace is generated if trace_enable is set to False. If it is set to True, the
trace is written to a file based on the name of the SimObject in the simulation
hierarchy. For example, system.cluster.il1_commmonitor.trc. This filename can be
overridden by additionally specifying a file name to the trace_file parameter
(more on this later).
The trace_compress parameter will append .gz to any filename if set to True.
This enables compression of the generated traces. If the file name already ends
in .gz, then no changes are made.
The trace_file parameter will override the name set by the trace_enable
parameter. In the case that the specified name does not end in .gz but
trace_compress is set to true, .gz is appended to the supplied file name.
The patch removes the ruby_fs.py file. The functionality is being moved to
fs.py. This would being ruby fs simulations in line with how ruby se
simulations are started (using --ruby option). The alpha fs config functions
are being combined for classing and ruby memory systems. This required
renaming the piobus in ruby to iobus. So, we will have stats being renamed
in the stats file for ruby fs regression.
Piobus was recently added to se scripts for ruby so that the interrupt
controller can be connected to something (required since the interrupt
controller sends address range messages). This patch removes the piobus
and instead, the pio port of ruby port will now ignore the range change
messages in se mode.
Couple of errors were discovered in 4eec7bdde5b0 which necessitated this patch.
Firstly, we create interrupt controllers in the se mode, but no piobus was
being created. RubyPort, which earlier used to ignore range changes now
forwards those to the piobus. The lack of piobus resulted in segmentation
fault. This patch creates a piobus even in se mode. It is not created only
when some tester is running. Secondly, I had missed out on modifying port
connections for other coherence protocols.
Currently, the interrupt controller in x86 is connected to the io bus
directly. Therefore the packets between the io devices and the interrupt
controller do not go through ruby. This patch changes ruby port so that
these packets arrive at the ruby port first, which then routes them to their
destination. Note that the patch does not make these packets go through the
ruby network. That would happen in a subsequent patch.
For some reason, the default x86 kernel is specified in
tests/configs/x86_generic.py and not in configs/common/FSConfig.py,
where the kernels for all the other ISAs are. This means that
running configs/example/fs.py for x86 fails because no kernel
is specified. Moving the specification over fixes this problem.
There is another problem that this uncovers, which is that going
past the init stage (i.e., past where the regression test stops)
fails because the fsck test on the disk device fails, but that's
a separate issue.
The output from the switcheroo tests is voluminous and
(because it includes timestamps) highly sensitive to
minor changes, leading to extremely large updates to the
reference outputs. This patch addresses this problem
by suppressing output from the tests. An internal
parameter can be set to enable the output. Wiring that
up to a command-line flag (perhaps even the rudimantary
-v/-q options in m5/main.py) is left for future work.
Update stats for recent changes. Mostly minor changes
in register access stats due to addition of new cc
register type and slightly different (and more accurate)
classification of int vs. fp register accesses.
In the unusual case that regressions are run with --update-ref
when there is no existing regression output, scons gets
confused because it depends on stats.txt to trigger the
update, but it has no indication that running the test will
generate the stats.txt file. (In the typical case where
stats.txt already exists, scons doesn't care about where
it came from.)
It's easy to fix this just by adding the stats.txt file
to the target list for the test action.
This patch simply brings the stats for the pc-simple-timing-ruby
regression up to date. The particular regression seems to give
different results on different systems unfortunately, and this update
reflects the current behaviour on zizzer.
The updates to the x87 caused the stats for several regressions to
change. This was mainly caused by the addition of a working 32-bit and
80-bit FP load instruction and xsave support.
Apparently only stats.txt was updated the last time, so
this changeset updates other reference output files
(config.ini, simout, simerr, ruby.stats) so that
test output diffs should not be cluttered with irrelevant
changes. There are a few stats.txt updates too, but
they are in the minority.
This patch simply takes a first step to use the NULL ISA build for
tests that do not make use of a CPU. Most of the Ruby tests could go
the same way, but to avoid duplicating a lot of compilation targets
that will have to wait until Ruby is built as a library and linked in
independently.
--HG--
rename : tests/quick/se/50.memtest/ref/alpha/linux/memtest/config.ini => tests/quick/se/50.memtest/ref/null/none/memtest/config.ini
rename : tests/quick/se/50.memtest/ref/alpha/linux/memtest/simerr => tests/quick/se/50.memtest/ref/null/none/memtest/simerr
rename : tests/quick/se/50.memtest/ref/alpha/linux/memtest/simout => tests/quick/se/50.memtest/ref/null/none/memtest/simout
rename : tests/quick/se/50.memtest/ref/alpha/linux/memtest/stats.txt => tests/quick/se/50.memtest/ref/null/none/memtest/stats.txt
rename : tests/quick/se/70.tgen/ref/arm/linux/tgen-simple-dram/simerr => tests/quick/se/70.tgen/ref/null/none/tgen-simple-dram/simerr
rename : tests/quick/se/70.tgen/ref/arm/linux/tgen-simple-dram/simout => tests/quick/se/70.tgen/ref/null/none/tgen-simple-dram/simout
rename : tests/quick/se/70.tgen/ref/arm/linux/tgen-simple-dram/stats.txt => tests/quick/se/70.tgen/ref/null/none/tgen-simple-dram/stats.txt
rename : tests/quick/se/70.tgen/ref/arm/linux/tgen-simple-mem/simerr => tests/quick/se/70.tgen/ref/null/none/tgen-simple-mem/simerr
rename : tests/quick/se/70.tgen/ref/arm/linux/tgen-simple-mem/simout => tests/quick/se/70.tgen/ref/null/none/tgen-simple-mem/simout
rename : tests/quick/se/70.tgen/ref/arm/linux/tgen-simple-mem/stats.txt => tests/quick/se/70.tgen/ref/null/none/tgen-simple-mem/stats.txt
The number of transitions per cycle that a controller can carry out is
a proxy for the number of ports that a controller has. This value is
currently 32 which is way too high. The patch introduces an option
for the number of ports and uses this option in the protocol files
to set the number of transitions. The default value is being set to
4. None of the se regressions change. Ruby stats for the fs regression
change and are being updated.
This patch updates the stats to reflect the: 1) addition of the
internal queue in SimpleMemory, 2) moving of the memory class outside
FSConfig, 3) fixing up of the 2D vector printing format, 4) specifying
burst size and interface width for the DRAM instead of relying on
cache-line size, 5) performing merging in the DRAM controller write
buffer, and 6) fixing how idle cycles are counted in the atomic and
timing CPU models.
The main reason for bundling them up is to minimise the changeset
size.
This patch changes the default parameter value of conf_table_reported
to match the common case. It also simplifies the regression and config
scripts to reflect this change.
This patch adds the notion of voltage domains, and groups clock
domains that operate under the same voltage (i.e. power supply) into
domains. Each clock domain is required to be associated with a voltage
domain, and the latter requires the voltage to be explicitly set.
A voltage domain is an independently controllable voltage supply being
provided to section of the design. Thus, if you wish to perform
dynamic voltage scaling on a CPU, its clock domain should be
associated with a separate voltage domain.
The current implementation of the voltage domain does not take into
consideration cases where there are derived voltage domains running at
ratio of native voltage domains, as with the case where there can be
on-chip buck/boost (charge pumps) voltage regulation logic.
The regression and configuration scripts are updated with a generic
voltage domain for the system, and one for the CPUs.
This patch moves the instantiation of the memory controller outside
FSConfig and instead relies on the mem_ranges to pass the information
to the caller (e.g. fs.py or one of the regression scripts). The main
motivation for this change is to expose the structural composition of
the memory system and allow more tuning and configuration without
adding a large number of options to the makeSystem functions.
The patch updates the relevant example scripts to maintain the current
functionality. As the order that ports are connected to the memory bus
changes (in certain regresisons), some bus stats are shuffled
around. For example, what used to be layer 0 is now layer 1.
Going forward, options will be added to support the addition of
multi-channel memory controllers.
This patch removes the sparse histogram total from the CommMonitor
stats. It also bumps the stats after the unit fixes in the atomic
cache access. Lastly, it updates the stats to match the new port
ordering. All numbers are the same, and the only thing that changes is
which master corresponds to what port index.
This patch adds the notion of source- and derived-clock domains to the
ClockedObjects. As such, all clock information is moved to the clock
domain, and the ClockedObjects are grouped into domains.
The clock domains are either source domains, with a specific clock
period, or derived domains that have a parent domain and a divider
(potentially chained). For piece of logic that runs at a derived clock
(a ratio of the clock its parent is running at) the necessary derived
clock domain is created from its corresponding parent clock
domain. For now, the derived clock domain only supports a divider,
thus ensuring a lower speed compared to its parent. Multiplier
functionality implies a PLL logic that has not been modelled yet
(create a separate clock instead).
The clock domains should be used as a mechanism to provide a
controllable clock source that affects clock for every clocked object
lying beneath it. The clock of the domain can (in a future patch) be
controlled by a handler responsible for dynamic frequency scaling of
the respective clock domains.
All the config scripts have been retro-fitted with clock domains. For
the System a default SrcClockDomain is created. For CPUs that run at a
different speed than the system, there is a seperate clock domain
created. This domain incorporates the CPU and the associated
caches. As before, Ruby runs under its own clock domain.
The clock period of all domains are pre-computed, such that no virtual
functions or multiplications are needed when calling
clockPeriod. Instead, the clock period is pre-computed when any
changes occur. For this to be possible, each clock domain tracks its
children.
This patch extends the existing system builders to also include a
syscall-emulation builder. This builder is deployed in all
syscall-emulation regressions that do not involve Ruby,
i.e. o3-timing, simple-timing and simple-atomic, as well as the
multi-processor regressions o3-timing-mp, simple-timing-mp and
simple-atomic-mp (the latter are only used by SPARC at this point).
The values chosen for the cache sizes match those that were used in
the existing config scripts (despite being on the large
side). Similarly, a mem_class parameter is added to the builder base
class to enable simple-atomic to use SimpleMemory and o3-timing to use
the default DDR3 configuration.
Due to the different order the ports are connected, the bus stats get
shuffled around for the multi-processor regressions. A separate patch
bumps the port indices. Besides this, all behaviour is exactly the
same.
This patch adds a 'sys_clock' command-line option and use it to assign
clocks to the system during instantiation.
As part of this change, the default clock in the System class is
removed and whenever a system is instantiated a system clock value
must be set. A default value is provided for the command-line option.
The configs and tests are updated accordingly.
This patch removes the explicit setting of the clock period for
certain instances of CoherentBus, NonCoherentBus and IOCache where the
specified clock is same as the default value of the system clock. As
all the values used are the defaults, there are no performance
changes. There are similar cases where the toL2Bus is set to use the
parent CPU clock which is already the default behaviour.
The main motivation for these simplifications is to ease the
introduction of clock domains.
This patch prunes the 00.gzip regressions with the main motivation
being that it adds little (or no) coverage and requires a substantial
amount of run time.
A complete regression run, including compilation from a clean repo, is
almost 20% faster(!).
This patch changes the regression script such that it is possible to
identify the runs that fail with an exit code, and those that finish
with stats differences. The ones that truly fail are reported as
FAILED, and those that finish with changed stats as CHANGED.
The yellow colour has been reclaimed from the skipped regressions and
is now used for the changed ones. With no obvious good option left the
skipped ones are now in cyan.
While I was editing the script I also bumped any occurence of M5 to
gem5.
Ruby's controller statistics have been mostly moved to stats.txt now.
Plus stats.txt for solaris/t1000-simple-atomic and arm/20.parser are
also being updated.
This patch updates the stats to reflect the addition of the bus stats,
and changes to the bus layers. In addition it updates the stats to
match the addition of the static pipeline latency of the memory
conotroller and the addition of a stat tracking the bytes per
activate.
This patch changes the class names of the variuos DRAM configurations
to better reflect what memory they are based on. The speed and
interface width is now part of the name, and also the alias that is
used to select them on the command line.
Some minor changes are done to the actual parameters, to better
reflect the named configurations. As a result of these changes the
regressions change slightly and the stats will be bumped in a separate
patch.
This patch enables selection of the memory controller class through a
mem-type command-line option. Behind the scenes, this option is
treated much like the cpu-type, and a similar framework is used to
resolve the valid options, and translate the short-hand description to
a valid class.
The regression scripts are updated with a hardcoded memory class for
the moment. The best solution going forward is probably to get the
memory out of the makeSystem functions, but Ruby complicates things as
it does not connect the memory controller to the membus.
--HG--
rename : configs/common/CpuConfig.py => configs/common/MemConfig.py
This changeset adds support for initializing a KVM VM in the
BaseSystem test class and adds the following methods in run.py:
require_file -- Test if a file exists and abort/skip if not.
require_kvm -- Test if KVM support has been compiled into gem5 (i.e.,
BaseKvmCPU exists) and the KVM device exists on the
host.
Add the options 'panic_on_panic' and 'panic_on_oops' to the
LinuxArmSystem SimObject. When these option are enabled, the simulator
panics when the guest kernel panics or oopses. Enable panic on panic
and panic on oops in ARM-based test cases.
The new changeset that can reorder Ruby profilers will cause the ruby.stats
files to reordered statistics (the point of the patch). Update the references
to ensure that these changes are reflected in regressions.
CPU switching consists of the following steps:
1. Drain the system
2. Switch out old CPUs (cpu.switchOut())
3. Change the system timing mode to the mode the new CPUs require
4. Flush caches if switching to hardware virtualization
5. Inform new CPUs of the handover (cpu.takeOverFrom())
6. Resume the system
m5.switchCpus() previously only did step 2 & 5. Since information
about the new processors' memory system requirements is now exposed,
do all of the steps above.
This patch adds automatic memory system switching and flush (if
needed) to switchCpus(). Additionally, it adds optional draining to
switchCpus(). This has the following implications:
* changeToTiming and changeToAtomic are no longer needed, so they have
been removed.
* changeMemoryMode is only used internally, so it is has been renamed
to be private.
* switchCpus requires a reference to the system containing the CPUs as
its first parameter.
WARNING: This changeset breaks compatibility with existing
configuration scripts since it changes the signature of
m5.switchCpus().
This patch moves the default DRAM parameters from the SimpleDRAM class
to two different subclasses, one for DDR3 and one for LPDDR2. More can
be added as we go forward.
The regressions that previously used the SimpleDRAM are now using
SimpleDDR3 as this is the most similar configuration.
