With this change, the test script will output a warning when it reads
an incomplete (e.g., when a regression is still running) or corrupt
status file instead of throwing an exception. When the scipt is used
to show the results the corrupt file is skipped; when it is used to
test if all regressions run successfully it will return an error value
(2).
Change-Id: Ie7d9b457b200e3abc7ae6238e3efbf3d18cf4297
Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-on: https://gem5-review.googlesource.com/2320
Reviewed-by: Jason Lowe-Power <jason@lowepower.com>
Maintainer: Jason Lowe-Power <jason@lowepower.com>
It's currently possible to change the log level in gem5 by tweaking a
set of global variables. These variables are currently exposed to
Python using SWIG. This mechanism is far from ideal for two reasons:
First, changing the log level requires that the Python world enables
or disables individual levels. Ideally, this should be a single call
where a log level is selected. Second, exporting global variables is
poorly supported by most Python frameworks. SWIG puts variables in
their own namespace and PyBind doesn't seem to support it at all.
This changeset refactors the logging code to create a more abstract
interface. Each log level is associated with an instance of a Logger
class. This class contains common functionality, an enable flag, and a
verbose flag.
Available LogLevels are described by the LogLevel class. Lower log
levels are used for more critical messages (PANIC being level 0) and
higher levels for less critical messages. The highest log level that
is printed is controlled by calling Logger:setLevel().
Change-Id: I31e44299d242d953197a8e62679250c91d6ef776
Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-by: Gabor Dozsa <gabor.dozsa@arm.com>
Reviewed-by: Curtis Dunham <curtis.dunham@arm.com>
Reviewed-by: Jason Lowe-Power <jason@lowepower.com>
Don't output verbose text descriptions in stat files when running
tests. This saves a lot of space when storing reference data.
Change-Id: I2a7ead4843586e800ecf83846694b73f0c356373
Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
Reviewed-by: Sascha Bischoff <sascha.bischoff@arm.com>
Reviewed-by: Curtis Dunham <curtis.dunham@arm.com>
Reviewed-by: Jason Lowe-Power <jason@lowepower.com>
Reviewed-by: Tony Gutierrez <anthony.gutierrez@amd.com>
The EIOProcess class was removed recently and it was the only other class
which derived from Process. Since every Process invocation is also a
LiveProcess invocation, it makes sense to simplify the organization by
combining the fields from LiveProcess into Process.
Names of DRAM configurations were updated to reflect both
the channel and device data width.
Previous naming format was:
<DEVICE_TYPE>_<DATA_RATE>_<CHANNEL_WIDTH>
The following nomenclature is now used:
<DEVICE_TYPE>_<DATA_RATE>_<n>x<w>
where n = The number of devices per rank on the channel
x = Device width
Total channel width can be calculated by n*w
Example:
A 64-bit DDR4, 2400 channel consisting of 4-bit devices:
n = 16
w = 4
The resulting configuration name is:
DDR4_2400_16x4
Updated scripts to match new naming convention.
Added unique configurations for DDR4 for:
1) 16x4
2) 8x8
3) 4x16
Change-Id: Ibd7f763b7248835c624309143cb9fc29d56a69d1
Reviewed-by: Radhika Jagtap <radhika.jagtap@arm.com>
Reviewed-by: Curtis Dunham <curtis.dunham@arm.com>
Provides a helpful error when tests.py is invoked without the gem5 binary.
Before:
Running 0 tests
After:
gem5 binary 'quick/...' not an executable file
Change-Id: I1566802206c9e21ca89bd03e91db22844168a085
Reviewed-by: Andreas Sandberg <andreas.sandberg@arm.com>
The system calls tested in rv64i.cpp in RISC-V's insttest suite have
different behavior depending on the operating system and file system they
are run on. This patch ignores the output of those tests and only
ensures that the instructions in RV64I complete successfully.
[Change deletion of ECALL test to block comment.]
[Restore ECALL test but remove test output to test only for completion
without error.]
[Update patch description and again try to push EMPTY files for rv64i
tests.]
In some newer Linux distributions, env python default to Python 3.0. This
patch explicitly uses "python2" instead of just "python" for all scripts
that use #!
Reported-by: Sanchayan Maity <maitysanchayan@gmail.com>
Signed-off-by: Jason Lowe-Power <jason@lowepower.com>
Swig wrappers for native objects currently share the _m5.internal name
space with Python code. This is undesirable if we ever want to switch
from Swig to some other framework for native binding (e.g., PyBind11
or Boost::Python). This changeset moves all of such wrappers to the
_m5 namespace, which is now reserved for native code.
Change-Id: I2d2bc12dbc05b57b7c5a75f072e08124413d77f3
Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-by: Curtis Dunham <curtis.dunham@arm.com>
Reviewed-by: Jason Lowe-Power <jason@lowepower.com>
This patch is the eighth patch in a series adding RISC-V to gem5, and
third of the bonus patches to the original series of five. It adds some
regression tests to RISC-V.
Regression tests included:
- se/00.hello
- se/02.insttest (split into several binaries which are not included due
to large size)
The tests added to 00.insttest will need to be build manually; to
facilitate this, a Makefile is included. The required toolchain and
compiler (riscv64-unknown-elf-gcc) can be built from the riscv-tools
GitHub repository at https://github.com/riscv/riscv-tools.
Note that because EBREAK only makes sense when gdb is running or while in
FS mode, it is not included in the linux-rv64i insttest. ERET is not
included because it does not make sense in SE mode and, in fact, causes
a panic by design.
Note also that not every system call is tested in linux-rv64i; of the ones
defined in linux/process.hh, some have been given numbers but not
definitions for the toolchain, or are merely stubs that always return 0. Of
the ones that do work properly, only a subset are tested due to similar
functionality.
Signed-off by: Alec Roelke
Signed-off by: Jason Lowe-Power <jason@lowepower.com>
This patch avoids compiling ALPHA six times as part of running
'util/regress', and instead relis on NULL with different protocols to
run the rubytest. All we need is the memory system, so there is really
no need to compile the ISA over and over again.
The one downside is the removal of running 'hello' for the variuos
ALPHA and protocol combinations, but if this is a concern we should
rather beef up the synthetic tests for the variuos protocols.
--HG--
rename : build_opts/NULL => build_opts/NULL_MESI_Two_Level
rename : build_opts/NULL => build_opts/NULL_MOESI_CMP_directory
rename : build_opts/NULL => build_opts/NULL_MOESI_CMP_token
rename : build_opts/NULL => build_opts/NULL_MOESI_hammer
rename : tests/quick/se/60.rubytest/ref/alpha/linux/rubytest-ruby-MESI_Two_Level/config.ini => tests/quick/se/60.rubytest/ref/null/none/rubytest-ruby-MESI_Two_Level/config.ini
rename : tests/quick/se/60.rubytest/ref/alpha/linux/rubytest-ruby-MESI_Two_Level/simerr => tests/quick/se/60.rubytest/ref/null/none/rubytest-ruby-MESI_Two_Level/simerr
rename : tests/quick/se/60.rubytest/ref/alpha/linux/rubytest-ruby-MESI_Two_Level/simout => tests/quick/se/60.rubytest/ref/null/none/rubytest-ruby-MESI_Two_Level/simout
rename : tests/quick/se/60.rubytest/ref/alpha/linux/rubytest-ruby-MESI_Two_Level/stats.txt => tests/quick/se/60.rubytest/ref/null/none/rubytest-ruby-MESI_Two_Level/stats.txt
rename : tests/quick/se/60.rubytest/ref/alpha/linux/rubytest-ruby-MOESI_CMP_directory/config.ini => tests/quick/se/60.rubytest/ref/null/none/rubytest-ruby-MOESI_CMP_directory/config.ini
rename : tests/quick/se/60.rubytest/ref/alpha/linux/rubytest-ruby-MOESI_CMP_directory/simerr => tests/quick/se/60.rubytest/ref/null/none/rubytest-ruby-MOESI_CMP_directory/simerr
rename : tests/quick/se/60.rubytest/ref/alpha/linux/rubytest-ruby-MOESI_CMP_directory/simout => tests/quick/se/60.rubytest/ref/null/none/rubytest-ruby-MOESI_CMP_directory/simout
rename : tests/quick/se/60.rubytest/ref/alpha/linux/rubytest-ruby-MOESI_CMP_directory/stats.txt => tests/quick/se/60.rubytest/ref/null/none/rubytest-ruby-MOESI_CMP_directory/stats.txt
rename : tests/quick/se/60.rubytest/ref/alpha/linux/rubytest-ruby-MOESI_CMP_token/config.ini => tests/quick/se/60.rubytest/ref/null/none/rubytest-ruby-MOESI_CMP_token/config.ini
rename : tests/quick/se/60.rubytest/ref/alpha/linux/rubytest-ruby-MOESI_CMP_token/simerr => tests/quick/se/60.rubytest/ref/null/none/rubytest-ruby-MOESI_CMP_token/simerr
rename : tests/quick/se/60.rubytest/ref/alpha/linux/rubytest-ruby-MOESI_CMP_token/simout => tests/quick/se/60.rubytest/ref/null/none/rubytest-ruby-MOESI_CMP_token/simout
rename : tests/quick/se/60.rubytest/ref/alpha/linux/rubytest-ruby-MOESI_CMP_token/stats.txt => tests/quick/se/60.rubytest/ref/null/none/rubytest-ruby-MOESI_CMP_token/stats.txt
rename : tests/quick/se/60.rubytest/ref/alpha/linux/rubytest-ruby-MOESI_hammer/config.ini => tests/quick/se/60.rubytest/ref/null/none/rubytest-ruby-MOESI_hammer/config.ini
rename : tests/quick/se/60.rubytest/ref/alpha/linux/rubytest-ruby-MOESI_hammer/simerr => tests/quick/se/60.rubytest/ref/null/none/rubytest-ruby-MOESI_hammer/simerr
rename : tests/quick/se/60.rubytest/ref/alpha/linux/rubytest-ruby-MOESI_hammer/simout => tests/quick/se/60.rubytest/ref/null/none/rubytest-ruby-MOESI_hammer/simout
rename : tests/quick/se/60.rubytest/ref/alpha/linux/rubytest-ruby-MOESI_hammer/stats.txt => tests/quick/se/60.rubytest/ref/null/none/rubytest-ruby-MOESI_hammer/stats.txt
rename : tests/quick/se/60.rubytest/ref/alpha/linux/rubytest-ruby/config.ini => tests/quick/se/60.rubytest/ref/null/none/rubytest-ruby/config.ini
rename : tests/quick/se/60.rubytest/ref/alpha/linux/rubytest-ruby/simerr => tests/quick/se/60.rubytest/ref/null/none/rubytest-ruby/simerr
rename : tests/quick/se/60.rubytest/ref/alpha/linux/rubytest-ruby/simout => tests/quick/se/60.rubytest/ref/null/none/rubytest-ruby/simout
rename : tests/quick/se/60.rubytest/ref/alpha/linux/rubytest-ruby/stats.txt => tests/quick/se/60.rubytest/ref/null/none/rubytest-ruby/stats.txt
Continue along the same line as the recent patch that made the
Ruby-related config scripts Python packages and make also the
configs/common directory a package.
All affected config scripts are updated (hopefully).
Note that this change makes it apparent that the current organisation
and naming of the config directory and its subdirectories is rather
chaotic. We mix scripts that are directly invoked with scripts that
merely contain convenience functions. While it is not addressed in
this patch we should follow up with a re-organisation of the
config structure, and renaming of some of the packages.
This patch moves the addition of network options into the Ruby module
to avoid the regressions all having to add it explicitly. Doing this
exposes an issue in our current config system though, namely the fact
that addtoPath is relative to the Python script being executed. Since
both example and regression scripts use the Ruby module we would end
up with two different (relative) paths being added. Instead we take a
first step at turning the config modules into Python packages, simply
by adding a __init__.py in the configs/ruby, configs/topologies and
configs/network subdirectories.
As a result, we can now add the top-level configs directory to the
Python search path, and then use the package names in the various
modules. The example scripts are also updated, and the messy
path-deducing variations in the scripts are unified.
The memtest and memcheck are not designed to test timing. Make them
functional only to make ref diffs less noisy in the future.
Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>
The switcheroo tests only really serve to check functional
correctness. Checking for stat differences in them just increases the
size of reference diffs.
Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>
Switcheroo and checkpoint tests should generally be considered to be
successful if they run to completion. Remove all reference output
files from the switcheroo and checkopint tests to make them purely
functional.
Change-Id: I70b47853bd662b7a33716d9e0d2154b16077f9dc
Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-by: Curtis Dunham <curtis.dunham@arm.com>
Modify the ClassicTest class to only emit a stat verification test
unit if there is a reference stat file. This makes it possible to
design tests that don't care about stat changes.
To generate purely functional tests, we need to be able to create
empty test reference directories. This does not work well with many
revision control systems. As a workaround, add a file named EMPTY to
the list of ignored files in the test harness. This file can be used
as a placeholder in otherwise empty test directories.
Change-Id: I583c8c4e55479f0d48fa99d0b0d1eac9221e6652
Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-by: Curtis Dunham <curtis.dunham@arm.com>
An email sent to gem5-users and gem5-dev asking if anyone was
still using EIO traces got no responses, so it seems like it's
not worth maintaining this any longer.
This patch changes the default behaviour of the SystemXBar, adding a
snoop filter. With the recent updates to the snoop filter allocation
behaviour this change no longer causes problems for the regressions
without caches.
Change-Id: Ibe0cd437b71b2ede9002384126553679acc69cc1
Reviewed-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
Reviewed-by: Jason Lowe-Power <jason@lowepower.com>
Reviewed-by: Tony Gutierrez <anthony.gutierrez@amd.com>
There are cases where we need to ignore files with specific extensions
(e.g., when Mercurial litters the file system with patch
rejects). Implement this functionality using a helper class
(FileIgnoreList) that supports both regular expressions and basic
string comparisons.
Change-Id: I34549754bd2e10ed230ffb2dc057403349f8fa78
Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>
This patch provides the example test script to configure different HMC
architecture and run traffic through traffic generator.
Committed by Jason Lowe-Power <jason@lowepower.com>
Add a "test" command to tests.py that queries a test pickle file and
returns different exit codes depending on the outcome of the tests in
the file. The following exit codes can currently be returned:
* 0: All tests were successful or skipped.
* 1: General fault in the script such as incorrect parameters or
failing to parse a pickle file.
* 2: At least one test failed to run. This is what the summary
formatter usually shows as a 'FAILED'.
* 3: All tests ran correctly, but at least one failed to verify
its output. When displaying test output using the summary
formatter, such a test would show up as 'CHANGED'.
The command can be invoked like this:
./tests/tests.py test `find build/ARM/tests/opt/ -name status.pickle`
Change-Id: I7e6bc661516f38ff08dfda7c4359a1e10bf97864
Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-by: Curtis Dunham <curtis.dunham@arm.com>
The test base class already assumes that test cases consists of a run
stage and a verification stage. Reflect this in the results class to
make it possible to detect cases where a run was successful, but
didn't verify.
Change-Id: I31ef393e496671221c5408aca41649cd8dda74ca
Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-by: Curtis Dunham <curtis.dunham@arm.com>
The traffic generator currently resolves relative trace paths relative
to gem5's current working directory. This can lead to surprising
results for relative paths where the expectation would normally be
that they are resolved relative to the configuration file. This
changeset implements config-relative trace file lookups. The old
behavior is kept as a fallback for configs that expect that behavior.
Change-Id: I1bda4e16725842666ffc37dcb6838c23a6ff138c
Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-by: Curtis Dunham <curtis.dunham@arm.com>
The traffic generator tests currently assume that they are run from
the root of the source directory. This sometimes breaks tests when
they are run using the new test framework.
Change-Id: I6538a7902694c5d2c980295e076ea1c09acc4291
Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-by: Curtis Dunham <curtis.dunham@arm.com>
Limit the test configs to Ruby-only configs when testing a Ruby target
that isn't MI_example. This avoids re-running configs that has already
been tested by the generic (non-Ruby) ISA target. This behavior was
the expected behavior prior to switching to the new test framework.
Change-Id: I3f138dbf9c7071ce862d1073aaec57c59afbc921
Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-by: Curtis Dunham <curtis.dunham@arm.com>
Rewrite the SCons script responsible for launching tests to use the
new test framework. The biggest visible change after this changeset is
that SCons no longer produces a "status" file in test build
directories. Instead, it creates a status.pickle file. That file can
be inspected using the new tests.py script. Another visible change is
that timed out tests are now reported as failed rather than a using a
separate summary message.
Since the pickle file will remain in the build directory after running
tests, it's now possible to convert the test results into other
formats. For example:
./tests/tests.py show --format junit -o junit.xml \
`find build/ -name status.pickle`
To better facilitate running outside of scons, there is now a set of
targets that generate test listings that can be used by the test
script. There are currently three targets, quick.list, long.list, and
all.list. For example:
scons build/ARM/tests/opt/all.list
for F in `cat build/ARM/tests/opt/all.list`; do
./tests/tests.py run build/ARM/gem5.opt $F
done
Change-Id: I2c0f8ca0080d7af737362e198eda4cb3a72e6c36
Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-by: Curtis Dunham <curtis.dunham@arm.com>
