This patch adds a first version of a script that processes the debug
output and generates a command trace for DRAMPower. This is work in
progress and is intended as a snapshot of ongoing work at this point.
The longer term plan is to link in DRAMPower as a library and have one
instance of the model per rank, and instantiate it based on a struct
passed from gem5. Each command will then be a call to the model and no
parsing of traces will be necessary.
The style checker used to traverse symlinks if they pointed to files, which can
result in style checker failure if the pointed-to file doesn't exist. This
style check is actually unnecessary, since symlinks either point to other files
that are already style checked, or files outside gem5, which shouldn't be
checked. Skip symlinks.
Upon aggregating records, serialize system's cache-block size, as the
cache-block size can be different when restoring from a checkpoint. This way,
we can correctly read all records when restoring from a checkpoints, even if
the cache-block size is different.
Note, that it is only possible to restore from a checkpoint if the
desired cache-block size is smaller or equal to the cache-block size
when the checkpoint was taken; we can split one larger request into
multiple small ones, but it is not reliable to do the opposite.
Committed by: Nilay Vaish <nilay@cs.wisc.edu>
1) fixes a typo for clean target libgemOpJni.so -> libgem5OpJni.so
2) addes jni_gem5Op.h to clean since it is added during make
3) links the m5 utility statically since it won't work on some images otherwise
This patch changes the decode script to output the optional fields of
the proto message Packet, namely id and flags. The flags field is set
by the communication monitor.
The id field is useful for CPU trace experiments, e.g. linking the
fetch side to decode side. It had to be renamed because it clashes
with a built in python function id() for getting the "identity" of an
object.
This patch also takes a few common function definitions out from the
multiple scripts and adds them to a protolib python module.
Note: AArch64 and AArch32 interworking is not supported. If you use an AArch64
kernel you are restricted to AArch64 user-mode binaries. This will be addressed
in a later patch.
Note: Virtualization is only supported in AArch32 mode. This will also be fixed
in a later patch.
Contributors:
Giacomo Gabrielli (TrustZone, LPAE, system-level AArch64, AArch64 NEON, validation)
Thomas Grocutt (AArch32 Virtualization, AArch64 FP, validation)
Mbou Eyole (AArch64 NEON, validation)
Ali Saidi (AArch64 Linux support, code integration, validation)
Edmund Grimley-Evans (AArch64 FP)
William Wang (AArch64 Linux support)
Rene De Jong (AArch64 Linux support, performance opt.)
Matt Horsnell (AArch64 MP, validation)
Matt Evans (device models, code integration, validation)
Chris Adeniyi-Jones (AArch64 syscall-emulation)
Prakash Ramrakhyani (validation)
Dam Sunwoo (validation)
Chander Sudanthi (validation)
Stephan Diestelhorst (validation)
Andreas Hansson (code integration, performance opt.)
Eric Van Hensbergen (performance opt.)
Gabe Black
The checkpoint aggregation script had become outdated due to numerous changes
to checkpoints over the past couple of years. This updates the script. It
now supports aggregation for x86 architecture instead of alpha. Also a couple
of new options have been added that specify the size of the memory file to be
created and whether or not the memory file should be compressed.
Thumb2 ARM kernels may access the TEEHBR via thumbee_notifier
in arch/arm/kernel/thumbee.c. The Linux kernel code just seems
to be saving and restoring the register. This patch adds support
for the TEEHBR cp14 register. Note, this may be a special case
when restoring from an image that was run on a system that
supports ThumbEE.
Newer linux kernels and distros exercise more functionality in the IDE device
than previously, exposing 2 races. The first race is the handling of aborted
DMA commands would immediately report the device is ready back to the kernel
and cause already in flight commands to assert the simulator when they returned
and discovered an inconsitent device state. The second race was due to the
Status register not being handled correctly, the interrupt status bit would get
stuck at 1 and the driver eventually views this as a bad state and logs the
condition to the terminal. This patch fixes these two conditions by making the
device handle aborted commands gracefully and properly handles clearing the
interrupt status bit in the Status register.
Changes to make m5ops work under virtualization seemed to break them working
with non-virtualized systems and the recently added m5 fail command makes
the m5op binary not compile. For now remove the code for virtualization.
This Python script generates an ARM DS-5 Streamline .apc project based
on gem5 run. To successfully convert, the gem5 runs needs to be run
with the context-switch-based stats dump option enabled (The guest
kernel also needs to be patched to allow gem5 interrogate its task
information.) See help for more information.
In order to support m5ops in virtualized environments, we need to use
a memory mapped interface. This changeset adds support for that by
reserving 0xFFFF0000-0xFFFFFFFF and mapping those to the generic IPR
interface for m5ops. The mapping is done in the
X86ISA::TLB::finalizePhysical() which means that it just works for all
of the CPU models, including virtualized ones.
This patch makes it possible to once again build gem5 without any
ISA. The main purpose is to enable work around the interconnect and
memory system without having to build any CPU models or device models.
The regress script is updated to include the NULL ISA target. Currently
no regressions make use of it, but all the testers could (and perhaps
should) transition to it.
--HG--
rename : build_opts/NOISA => build_opts/NULL
rename : src/arch/noisa/SConsopts => src/arch/null/SConsopts
rename : src/arch/noisa/cpu_dummy.hh => src/arch/null/cpu_dummy.hh
rename : src/cpu/intr_control.cc => src/cpu/intr_control_noisa.cc
Using arm-linux-gnueabi-gcc 4.7.3-1ubuntu1 on Ubuntu 13.04 to compiled
the m5 binary yields the error:
m5op_arm.S: Assembler messages:
m5op_arm.S:85: Error: selected processor does not support ARM mode `bxj lr'
For each of of the SIMPLE_OPs. Apparently, this compiler doesn't like the
interworking of these code types for the default arch. Adding -march=armv7-a
makes it compile. Another alternative that I found to work is replacing the
bxj lr instruction with mov pc, lr, but I don't know how that affects the
KVM stuff and if bxj is needed.
This patch adds checkpointing support to x86 tlb. It upgrades the
cpt_upgrader.py script so that previously created checkpoints can
be updated. It moves the checkpoint version to 6.
This patch simplifies the usage of the packet trace encoder/decoder by
attempting to automatically generating the packet proto definitions in
case they cannot be found.
Changeset 5ca6098b9560 accidentally broke the m5 utility. This
changeset adds the missing co-processor call used to trigger the
pseudo-op in ARM mode and fixes an alignment issue that caused some
pseudo-ops to leave thumb mode.
This changeset adds support for m5 pseudo-ops when running in
kvm-mode. Unfortunately, we can't trap the normal gem5 co-processor
entry in KVM (it doesn't seem to be possible to trap accesses to
non-existing co-processors). We therefore use BZJ instructions to
cause a trap from virtualized mode into gem5. The BZJ instruction is
becomes a normal branch to the gem5 fallback code when running in
simulated mode, which means that this patch does not need to change
the ARM ISA-specific code.
Note: This requires a patched host kernel.
This patch adds a simple Python script that reads the protobuf-encoded
packet traces (not gzipped), and prints them to an ASCII trace file.
The script can also be used as a template for other packet output
formats.
This patch adds a simple Python script that reads a simple ASCII trace
format and encodes it as protobuf output compatible with the traffic
generator.
The script can also be used as a template for other packet input
formats that should be converted to the gem5 packet protobuf format.
Changeset 02321b16685f added m5_writefile to m5op_x86.S a second time,
which causes a compilation error on when compiling for x86. This
changeset reverts that changeset and fixes the error.
Fixes the tick used from rename:
- previously this gathered the tick on leaving rename which was always 1 less
than the dispatch. This conflated the decode ticks when back pressure built
in the pipeline.
- now picks up tick on entry.
Added --store_completions flag:
- will additionally display the store completion tail in the viewer.
- this highlights periods when large numbers of stores are outstanding (>16 LSQ
blocking)
Allows selection by tick range (previously this caused an infinite loop)
Used as a command in full-system scripts helps the user ensure the benchmarks have finished successfully.
For example, one can use:
/path/to/benchmark args || /sbin/m5 fail 1
and thus ensure gem5 will exit with an error if the benchmark fails.
The number of arguments specified when calling parse_int_args() in
do_exit() is incorrect. This leads to stack corruption since it causes
writes past the end of the ints array.
In order to see all registers independent of the current CPU mode, the
ARM architecture model uses the magic MISCREG_CPSR_MODE register to
change the register mappings without actually updating the CPU
mode. This hack is no longer needed since the thread context now
provides a flat interface to the register file. This patch replaces
the CPSR_MODE hack with the flat register interface.
After making the ISA an independent SimObject, it is serialized
automatically by the Python world. Previously, this just resulted in
an empty ISA section. This patch moves the contents of the ISA to that
section and removes the explicit ISA serialization from the thread
contexts, which makes it behave like a normal SimObject during
serialization.
Note: This patch breaks checkpoint backwards compatibility! Use the
cpt_upgrader.py utility to upgrade old checkpoints to the new format.
Updated the util/m5/Makefile.arm so that m5op_arm.S is used to create
a static library - libm5.a. Allowing users to insert m5
psuedo-instructions into their applications for fine-grained
checkpointing, switching cpus or dumping statistics. e.g.
#include <m5op.h>
void foo(){
...
m5_reset_stats(<delay>,<period>)
m5_work_begin(<workid>,<threadid>);
...
m5_work_end(<workid>,<threadid>);
m5_dump_stats(<delay>,<period>);
}
This patch enables dumping statistics and Linux process information on
context switch boundaries (__switch_to() calls) that are used for
Streamline integration (a graphical statistics viewer from ARM).
This patch moves all the memory backing store operations from the
independent memory controllers to the global physical memory. The main
reason for this patch is to allow address striping in a future set of
patches, but at this point it already provides some useful
functionality in that it is now possible to change the number of
memory controllers and their address mapping in combination with
checkpointing. Thus, the host and guest view of the memory backing
store are now completely separate.
With this patch, the individual memory controllers are far simpler as
all responsibility for serializing/unserializing is moved to the
physical memory. Currently, the functionality is more or less moved
from AbstractMemory to PhysicalMemory without any major
changes. However, in a future patch the physical memory will also
resolve any ranges that are interleaved and properly assign the
backing store to the memory controllers, and keep the host memory as a
single contigous chunk per address range.
Functionality for future extensions which involve CPU virtualization
also enable the host to get pointers to the backing store.
This script (util/diff_config.pl) takes two config.ini files and compares them.
It highlights value changes, as well as displaying which parts are unique to
a specific config.ini file. This is useful when trying to replicate an earlier
experiment and when trying to make small changes to an existing configuration.
When the "all" test is specified, the "tests" list should have two elements in
it, "quick" and "long", not a single element "quick,long". The later would be
appropriate as the default for one of the command line options which are split
at commas, but at that point "tests" should already be a list.
This patch updates the regress script to reflect the merge of the
SE/FS builds and the new structure of the test directories. It adds a
"mode" flag to the script, that defaults to both se and fs.
Usage: m5 writefile <filename>
File will be created in the gem5 output folder with the identical filename.
Implementation is largely based on the existing "readfile" functionality.
Currently does not support exporting of folders.
These ops allow gem5 ops to be called from within java programs like the following:
import jni.gem5Op;
public class HelloWorld {
public static void main(String[] args) {
gem5Op gem5 = new gem5Op();
System.out.println("Rpns0:" + gem5.rpns());
System.out.println("Rpns1:" + gem5.rpns());
}
static {
System.loadLibrary("gem5OpJni");
}
}
When building you need to make sure classpath include gem5OpJni.jar:
javac -classpath $CLASSPATH:/path/to/gem5OpJni.jar HelloWorld.java
and when running you need to make sure both the java and library path are set:
java -classpath $CLASSPATH:/path/to/gem5OpJni.jar -Djava.library.path=/path/to/libgem5OpJni.so HelloWorld
Implemented a pipeline activity viewer as a python script (util/o3-pipeview.py)
and modified O3 code base to support an extra trace flag (O3PipeView) for
generating traces to be used as inputs by the tool.
The end of the COPYING file was generated with:
% python ./util/find_copyrights.py configs src system tests util
Update -C command line option to spit out COPYING file
Changed the --variants option to --test-variants and added a new
--compile-variants option for variants that are only compiled
(not tested). The former still defaults to 'opt' and the latter
defaults to 'debug,fast'.
Also changed the behavior when no tests are specified from just
compiling to running the 'quick' tests.
As a result, a plain 'util/regress' invocation will now compile
(but not test) the debug and fast builds, and compile and run the
quick regressions on the opt build. This should be the default
set of tests that are run before committing. Since the nightly
regressions use this same script, this will also be the new
nightly regression behavior.
Test-only regressions can still be done by setting --compile=''.
Compile-only regressions can be done by setting --test=''.
This change replaces the mkblankimage.sh script, used for creating new disk
images, with a new gem5img.py script. The new version is written in python
instead of bash, takes its parameters from command line arguments instead of
prompting for them, and finds a free loopback device dynamically instead of
hardcoding /dev/loop1. The file system used is now optionally configurable,
and the blank image is filled by a "hole" left by lseek and write instead of
literally filling it with zeroes.
The functionality of the new script is broken into subcommands "init",
"mount", "umount", "new", "partition", and "format". "init" creates a new file
of the appropriate size, partitions it, and then formats the first (and only)
new parition. "mount" attaches a new loopback device to the first parition of
the image file and mounts it to the specified mount point. "umount" unmounts
the specified mount point and identifies and cleans up the underlying loopback
device. "new", "partition", and "format" are the individual stages of "init"
but broken out so they can be run individually. That's so an image can be
reinitialized in place if needed.
Two features of the original script are being dropped. The first is the
ability to specify a source directory to copy into the new file system. The
second is the ability to specify a list of commands to run which are expected
to (but not required to) update the permissions of the files in the new fs.
Both of these seem easy enough to do manually, especially given the "mount"
and "umount" commands, that removing them would meaningfully simplify the
script without making it less useful.
I didn't realize that the perl version existed when I started this,
this version has a lot more features than the previous one since it will
sort and separate python, system, and m5 headers in separate groups, it
will remove duplicates, it will also convert c headers to stl headers
I've renamed the check_whitespace operation to check_style. You're going to
need to change your .hg/hgrc file. While you're at it, add a pre-qrefresh
hook please.
clean up the code a little bit while we're at it.
I recommend that everyone adds the pre-qrefresh hook below since it
will make qref run the style hook and not just commit/qpush
[extensions]
style = <m5 path>/util/style.py
[hooks]
pretxncommit.style = python:style.check_whitespace
pre-qrefresh.style = python:style.check_whitespace
break it up into reading one page at a time. Also, avoid redoing a aggregating a checkpoint that's
already done.
--HG--
rename : util/checkpoint-aggregator.py => util/checkpoint_aggregator.py
add -n/--no-exec which doesn't execute scons, but just prints the command line
add -j0 which tries to calculate how many cpus you have
add -D/--build-dir to specify a build directory other than ./build
This patch includes the necessary regression updates to test the new ruby
configuration system. The patch includes support for multiple ruby protocols
and adds the ruby random tester. The patch removes atomic mode test for
ruby since ruby does not support atomic mode acceses. These tests can be
added back in when ruby supports atomic mode for real.
--HG--
rename : tests/quick/50.memtest/test.py => tests/quick/60.rubytest/test.py
1) Move alpha-specific code out of page_table.cc:serialize().
2) Begin serializing M5_pid and unserializing it, but adding an function to do optional paramIn so that old checkpoints don't need to be fixed up.
3) Fix up alpha startup code so that the unserialized M5_pid value is properly written to DTB_IPR_ASN.
4) Fix the memory unserialize that I forgot somehow in the last changeset.
5) Add in an agg_se.py to handle aggregated checkpoints. --bench foo-bar plus positional arguments foo bar are the only changes in usage from se.py.
Note this aggregation stuff has only been tested for Alpha and nothing else, though it should take a very minimal amount of work to get it to work with another ISA.
This is simply a translation of the C++ slicc into python with very minimal
reorganization of the code. The output can be verified as nearly identical
by doing a "diff -wBur".
Slicc can easily be run manually by using util/slicc
The early call to child->step() was removed earlier because it confused the
new differences-only protocol ARM sendState() was using. It's necessary that
that gets called at least once before attempting to print the initial stack
frame, though, because otherwise statetrace doesn't know what the stack
pointer is. By putting the first call to child->step() in a common spot, both
needs are met.
