Commit graph

838 commits

Author SHA1 Message Date
Andreas Hansson
607c277291 stats: Update stats to reflect recent changes to floats
Mostly just splitting out the floats ops and corresponding
reads/writes.
2016-10-19 06:20:04 -04:00
Andreas Hansson
2f5262eb67 config: Make configs/common a Python package
Continue along the same line as the recent patch that made the
Ruby-related config scripts Python packages and make also the
configs/common directory a package.

All affected config scripts are updated (hopefully).

Note that this change makes it apparent that the current organisation
and naming of the config directory and its subdirectories is rather
chaotic. We mix scripts that are directly invoked with scripts that
merely contain convenience functions. While it is not addressed in
this patch we should follow up with a re-organisation of the
config structure, and renaming of some of the packages.
2016-10-14 10:37:38 -04:00
Curtis Dunham
c87b717dbd stats: update references 2016-10-13 23:21:40 +01:00
Andreas Hansson
68fdccb30b ruby: Fix regressions and make Ruby configs Python packages
This patch moves the addition of network options into the Ruby module
to avoid the regressions all having to add it explicitly. Doing this
exposes an issue in our current config system though, namely the fact
that addtoPath is relative to the Python script being executed. Since
both example and regression scripts use the Ruby module we would end
up with two different (relative) paths being added. Instead we take a
first step at turning the config modules into Python packages, simply
by adding a __init__.py in the configs/ruby, configs/topologies and
configs/network subdirectories.

As a result, we can now add the top-level configs directory to the
Python search path, and then use the package names in the various
modules. The example scripts are also updated, and the messy
path-deducing variations in the scripts are unified.
2016-10-13 03:17:19 -04:00
Andreas Sandberg
7eb168746c tests, arm: Reinstate accidentally removed switcheroo tests
Two of the switcheroo tests were accidentally removed due to
unexpected Mercurial behavior.

Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>
2016-09-22 12:10:47 +01:00
Andreas Sandberg
d0ffd2f9b8 test: Make the memtest and memcheck tests functional only
The memtest and memcheck are not designed to test timing. Make them
functional only to make ref diffs less noisy in the future.

Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>
2016-09-22 10:49:09 +01:00
Andreas Sandberg
39663dc77c tests: Make remaining switcheroo tests functional only
The switcheroo tests only really serve to check functional
correctness. Checking for stat differences in them just increases the
size of reference diffs.

Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>
2016-09-22 10:49:08 +01:00
Andreas Sandberg
ada0e2f02f tests, arm: Make switcheroo and checkpoint tests functional
Switcheroo and checkpoint tests should generally be considered to be
successful if they run to completion.  Remove all reference output
files from the switcheroo and checkopint tests to make them purely
functional.

Change-Id: I70b47853bd662b7a33716d9e0d2154b16077f9dc
Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-by: Curtis Dunham <curtis.dunham@arm.com>
2016-09-16 09:14:31 +01:00
Andreas Sandberg
1ecc3628a8 tests: Add support for functional only tests
Modify the ClassicTest class to only emit a stat verification test
unit if there is a reference stat file. This makes it possible to
design tests that don't care about stat changes.

To generate purely functional tests, we need to be able to create
empty test reference directories. This does not work well with many
revision control systems. As a workaround, add a file named EMPTY to
the list of ignored files in the test harness. This file can be used
as a placeholder in otherwise empty test directories.

Change-Id: I583c8c4e55479f0d48fa99d0b0d1eac9221e6652
Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-by: Curtis Dunham <curtis.dunham@arm.com>
2016-09-16 09:04:20 +01:00
David Hashe
9d324e135c tests: Add example of using KVM acceleration with an app
Add #ifdef's to gpu-hello.cpp demonstrating how to annotate an application
for KVM acceleration.
2016-08-22 11:41:37 -04:00
Steve Reinhardt
608a37c844 tests: remove EIO tests
An email sent to gem5-users and gem5-dev asking if anyone was
still using EIO traces got no responses, so it seems like it's
not worth maintaining this any longer.
2016-08-13 23:07:28 -04:00
Andreas Sandberg
55ed9609f1 stats: Update to match classic memory changes 2016-08-12 14:12:59 +01:00
Andreas Hansson
080d4e08d6 mem: Add snoop filter to SystemXBar by default
This patch changes the default behaviour of the SystemXBar, adding a
snoop filter. With the recent updates to the snoop filter allocation
behaviour this change no longer causes problems for the regressions
without caches.

Change-Id: Ibe0cd437b71b2ede9002384126553679acc69cc1
Reviewed-by: Nikos Nikoleris <nikos.nikoleris@arm.com>
Reviewed-by: Jason Lowe-Power <jason@lowepower.com>
Reviewed-by: Tony Gutierrez <anthony.gutierrez@amd.com>
2016-08-12 14:11:45 +01:00
Curtis Dunham
ae445c0348 stats: update references 2016-08-02 11:34:32 +01:00
Andreas Sandberg
c20b6c56f6 tests: Add regex-based ignore rules for ref files
There are cases where we need to ignore files with specific extensions
(e.g., when Mercurial litters the file system with patch
rejects). Implement this functionality using a helper class
(FileIgnoreList) that supports both regular expressions and basic
string comparisons.

Change-Id: I34549754bd2e10ed230ffb2dc057403349f8fa78
Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>
2016-07-22 15:24:20 +01:00
Curtis Dunham
84f138ba96 stats: update references 2016-07-21 17:19:18 +01:00
Abdul Mutaal Ahmad
9c20880fb5 mem: tester for new HMC configuration
This patch provides the example test script to configure different HMC
architecture and run traffic through traffic generator.

Committed by Jason Lowe-Power <jason@lowepower.com>
2016-07-01 09:48:43 -05:00
Andreas Sandberg
9c8710430e stats: Update stats to reflect ARM changes 2016-06-21 16:42:04 +01:00
Andreas Sandberg
dd0f54fed6 tests: Add a test command to get test status as an exit code
Add a "test" command to tests.py that queries a test pickle file and
returns different exit codes depending on the outcome of the tests in
the file. The following exit codes can currently be returned:

  * 0: All tests were successful or skipped.

  * 1: General fault in the script such as incorrect parameters or
    failing to parse a pickle file.

  * 2: At least one test failed to run. This is what the summary
    formatter usually shows as a 'FAILED'.

  * 3: All tests ran correctly, but at least one failed to verify
    its output. When displaying test output using the summary
    formatter, such a test would show up as 'CHANGED'.

The command can be invoked like this:

./tests/tests.py test `find build/ARM/tests/opt/ -name status.pickle`

Change-Id: I7e6bc661516f38ff08dfda7c4359a1e10bf97864
Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-by: Curtis Dunham <curtis.dunham@arm.com>
2016-06-20 14:50:43 +01:00
Andreas Sandberg
96b74fd31b tests: Split test results into running and verification
The test base class already assumes that test cases consists of a run
stage and a verification stage. Reflect this in the results class to
make it possible to detect cases where a run was successful, but
didn't verify.

Change-Id: I31ef393e496671221c5408aca41649cd8dda74ca
Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-by: Curtis Dunham <curtis.dunham@arm.com>
2016-06-20 14:50:34 +01:00
Andreas Sandberg
efb7fb6f85 mem: Resolve TrafficGen trace relative to the config
The traffic generator currently resolves relative trace paths relative
to gem5's current working directory. This can lead to surprising
results for relative paths where the expectation would normally be
that they are resolved relative to the configuration file. This
changeset implements config-relative trace file lookups. The old
behavior is kept as a fallback for configs that expect that behavior.

Change-Id: I1bda4e16725842666ffc37dcb6838c23a6ff138c
Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-by: Curtis Dunham <curtis.dunham@arm.com>
2016-06-20 14:49:37 +01:00
Steve Reinhardt
54aeb1a187 stats: update EIO stats 2016-06-12 20:02:49 -04:00
Andreas Sandberg
85997e66a0 stats: Add power stats to test references
Change-Id: Ic827213134b199446822f128b81d4a480e777fee
2016-06-06 17:16:44 +01:00
Steve Reinhardt
672c06a01d stats: update EIO stats 2016-06-06 00:18:34 -04:00
Andreas Sandberg
8a52029adc stats: Update ref stats to match ARM TLB changes 2016-06-02 20:34:39 +01:00
Andreas Sandberg
1a65e94636 tests: Remove working dir assumption in tgen tests
The traffic generator tests currently assume that they are run from
the root of the source directory. This sometimes breaks tests when
they are run using the new test framework.

Change-Id: I6538a7902694c5d2c980295e076ea1c09acc4291
Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-by: Curtis Dunham <curtis.dunham@arm.com>
2016-06-02 15:20:24 +01:00
Andreas Sandberg
1d933447fc stats: Update to match ARM ISA changes 2016-06-02 14:14:36 +01:00
Curtis Dunham
53ae19bb5d tests: add 'CHANGED' output to pickle viewer
Change-Id: I64c69fde8657c273adea69122877c5348a4f867a
Reviewed-by: Andreas Sandberg <andreas.sandberg@arm.com>
2016-06-02 10:50:52 +01:00
Andreas Sandberg
1088003135 tests: Only run Ruby tests when testing Ruby targets
Limit the test configs to Ruby-only configs when testing a Ruby target
that isn't MI_example. This avoids re-running configs that has already
been tested by the generic (non-Ruby) ISA target. This behavior was
the expected behavior prior to switching to the new test framework.

Change-Id: I3f138dbf9c7071ce862d1073aaec57c59afbc921
Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-by: Curtis Dunham <curtis.dunham@arm.com>
2016-06-02 10:48:45 +01:00
Curtis Dunham
dafec4a515 stats: update and fix e273e86a873d 2016-05-31 16:55:47 +01:00
Andreas Sandberg
44e9b81f74 scons: Use the new test framework from scons
Rewrite the SCons script responsible for launching tests to use the
new test framework. The biggest visible change after this changeset is
that SCons no longer produces a "status" file in test build
directories. Instead, it creates a status.pickle file. That file can
be inspected using the new tests.py script. Another visible change is
that timed out tests are now reported as failed rather than a using a
separate summary message.

Since the pickle file will remain in the build directory after running
tests, it's now possible to convert the test results into other
formats. For example:

./tests/tests.py show --format junit -o junit.xml \
    `find build/ -name status.pickle`

To better facilitate running outside of scons, there is now a set of
targets that generate test listings that can be used by the test
script. There are currently three targets, quick.list, long.list, and
all.list. For example:

scons build/ARM/tests/opt/all.list
for F in `cat build/ARM/tests/opt/all.list`; do
    ./tests/tests.py run build/ARM/gem5.opt $F
done

Change-Id: I2c0f8ca0080d7af737362e198eda4cb3a72e6c36
Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-by: Curtis Dunham <curtis.dunham@arm.com>
2016-05-31 11:27:00 +01:00
Andreas Sandberg
dbf64aa2c2 tests: Fix incorrect stat.txt ignore when updating refs
ClassicTest was incorrectly ignoring stats.txt when updating reference
statistics. This was caused by ignore rules being applied too
aggressively when listing reference files. This changeset splits the
ignore rules into two different lists: 1) diff_ignore_files that lists
the files that shouldn't be diff:ed using the normal diff tool, and 2)
ref_ignore_files which lists files that should be ignored by the test
system.

Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>
2016-05-31 11:26:59 +01:00
Curtis Dunham
62b6ff22ec stats: update for snoop filter tweak
--HG--
extra : source : 2323557eb4f4866fa1ea1575a9f5969e0022adc1
2016-05-31 11:07:18 +01:00
Stephan Diestelhorst
71a02f624e mem, config: Selective use of snoop filter
Disable the default snoop filter in the SystemXBar so that the
typical membus does not have a snoop filter by default. Instead,
add the snoop filter only when there are caches added to the system
(with the caches / l2cache options).

The underlying problem is that the snoop filter grows without
bounds (for now) if there are no caches to tell it that lines have
been evicted. This causes slow regression runs for all the atomic
regressions. This patch fixes this behaviour.

--HG--
extra : source : f97c20511828209757440839ed48d741d02d428f
2016-05-27 17:05:58 +01:00
Andreas Sandberg
f385adc8af tests: Add test infrastructure as a Python module
Implement gem5's test infrastructure as a Python module and a run
script that can be used without scons. The new implementation has
several features that were lacking from the previous test
infrastructure such as support for multiple output formats, automatic
runtime tracking, and better support for being run in a cluster
environment.

Tests consist of one or more steps (TestUnit). Units are run in two
stages, the first a run stage and then a verify stage. Units in the
verify stage are automatically skipped if any unit run stage wasn't
run. The library currently contains TestUnit implementations that run
gem5, diff stat files, and diff output files.

Existing tests are implemented by the ClassicTest class and "just
work". New tests can that don't rely on the old "run gem5 once and
diff output" strategy can be implemented by subclassing the Test base
class or ClassicTest.

Test results can be output in multiple formats. The module currently
supports JUnit, text (short and verbose), and Python's pickle
format. JUnit output allows CI systems to automatically get more
information about test failures. The pickled output contains all state
necessary to reconstruct a tests results object and is mainly intended
for the build system and CI systems.

Since many JUnit parsers parsers assume that test suite names look
like Java package names. We currently output path-like names with
slashes separating components. Test names are translated according to
these rules:

  * '.' -> '-"
  * '/' -> '.'

The test tool, tests.py, supports the following features:

  * Test listing. Example: ./tests.py list arm/quick

  * Running tests. Example:
    ./tests.py run -o output.pickle --format pickle \
        ../build/ARM/gem5.opt \
        quick/se/00.hello/arm/linux/simple-timing

  * Displaying pickled results. Example:
    ./tests.py show --format summary *.pickle

Change-Id: I527164bd791237aacfc65e7d7c0b67b695c5d17c
Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-by: Curtis Dunham <curtis.dunham@arm.com>
Reviewed-by: Joel Hestness <jthestness@gmail.com>
2016-05-26 11:56:24 +01:00
Andreas Sandberg
8b412fcfd6 tests: Enable test running outside of gem5's source tree
The learning gem5 scripts currently assumes that the current working
directory is the root of gem5's source tree. This isn't necessarily
the case when running the tests using gem5's new test runner.

Change-Id: Ief569bbe77b1b3e2b0fb0e6c575fb0705bbba9b3
Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-by: Curtis Dunham <curtis.dunham@arm.com>
2016-05-09 11:32:07 +01:00
Steve Reinhardt
2ae8b365c1 tests: update EIO ref stats for removed cache stats
Complaints about changes in EIO tests were due to reference files
that still have removed cache stats from cset 11454:e55afadc4e19.
2016-05-07 14:43:06 -04:00
Andreas Sandberg
b80f568bcf tests: Remove stale reference output files
Remove test reference files that are not generated any more:

    * chair.cook.ppm: This file should be generated by eon and not
      mcf, so it shouldn't be included as an output from mcf.

    * system.pc.terminal: The terminal device has been renamed so this
      file is no longer generated.

Change-Id: I3962efe1ff25479ca276115f7564eccb5fac8cf9
Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>
2016-04-28 15:16:52 +01:00
Andreas Hansson
8845aae4da tests: Add a basic memcheck regression
This patch adds a simple regression that calls the existing
memcheck.py script.

--HG--
rename : tests/configs/learning-gem5-p1-simple.py => tests/configs/memcheck.py
rename : tests/quick/se/70.tgen/test.py => tests/quick/se/51.memcheck/test.py
2016-04-25 03:46:12 -04:00
Jason Power
bf7e27fe45 tests: Update learning gem5 tests scripts with copyright 2016-04-21 17:25:31 -05:00
Andreas Hansson
b006ad26d4 stats: Update stats to reflect cache changes
Removed unused stats, now counting WriteLineReq, and changed how
uncacheable writes are handled while responses are outstanding.
2016-04-21 04:48:24 -04:00
Andreas Hansson
d9193d1b20 stats: Match current behaviour
Small changes to the branch predictor and BTB caused stats changes
throughout.
2016-04-09 12:13:40 -04:00
Curtis Dunham
1d61224a8b stats: update stats for thermals, indirect BP 2016-04-08 11:01:45 -05:00
Steve Reinhardt
d7c083864c stats: update stats for ld.so support
Additional auxv entries leads to more instructions in start-up
while walking the list, along with different cache conflicts
wrt stack entries.
2016-03-17 10:32:53 -07:00
Steve Reinhardt
4fc69db8f8 stats: update stats for mmap changes 2016-03-17 10:30:58 -07:00
Steve Reinhardt
9d8fec0d90 stats: update stats for mmap() change.
SE O3 runs see an additional reg read per mmap() call.
2016-03-17 10:25:11 -07:00
Andreas Hansson
7a40e7864a stats: Bump stats to match cache changes
Update stats to match current behaviour. As a result of the earlier
conflict check we are seeing a few prefetch requests being ignored
before being sent as upward snoops.
2016-03-17 09:51:21 -04:00
Steve Reinhardt
f5d1dd75e5 stats: overdue updates to long regressions 2016-03-16 13:03:49 -07:00
Steve Reinhardt
807e2705b4 stats: update gpu-ruby-GPU_RfO stats
Output changed way back in this cset:

changeset:   11345:b6a66a90e0a1
user:        John Kalamatianos <john.kalamatianos@amd.com>
summary:     gpu: fix bugs with MemFence, Flat Instrs and Resource utilization
2016-02-18 10:42:03 -05:00
Krishnendra Nathella
cabd4768c7 cpu: Fix LLSC atomic CPU wakeup
Writes to locked memory addresses (LLSC) did not wake up the locking
CPU. This can lead to deadlocks on multi-core runs. In AtomicSimpleCPU,
recvAtomicSnoop was checking if the incoming packet was an invalidation
(isInvalidate) and only then handled a locked snoop. But, writes are
seen instead of invalidates when running without caches (fast-forward
configurations). As as simple fix, now handleLockedSnoop is also called
even if the incoming snoop packet are from writes.
2015-07-19 15:03:30 -05:00