Determine if gem5 is running in a batch environment by checking if
STDIN is wired to a TTY or not. If the simulator is running in a batch
environment, disable all listeners by default. This behavior can be
overridden using the --enable-listeners option.
Change-Id: I404c709135339144216bf08a2769c016c543333c
Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-by: Sean McGoogan <sean.mcgoogan@arm.com>
Reviewed-on: https://gem5-review.googlesource.com/2322
Reviewed-by: Jason Lowe-Power <jason@lowepower.com>
Add a mechanism to configure the stat output format using a URL-like
syntax. This makes it possible to specify both an output format
(currently, only text is supported) and override default
parameters.
On the Python-side, this is implemented using a helper function
(m5.stats.addStatVisitor) that adds a visitor to the list of active
stat visitors. The helper function parses a URL-like stat
specification to determine the stat output type. Optional parameters
can be specified to change how stat visitors behave.
For example, to output stats in text format without stat descriptions:
m5.stats.addStatVisitor("text://stats.txt?desc=False")
From the command line:
gem5.opt --stats-file="text://stats.txt?desc=False"
Internally, the stat framework uses the _url_factory decorator
to wrap a Python function with the fn(path, **kwargs) signature in a
function that takes a parsed URL as its only argument. The path and
keyword arguments are automatically derived from the URL in the
wrapper function.
New output formats can be registered in the m5.stats.factories
dictionary. This dictionary contains a mapping between format names
(URL schemes) and factory methods.
To retain backwards compatibility, the code automatically assumes that
the user wants text output if no format has been specified (i.e., when
specifying a plain path).
Change-Id: Ic4dce93ab4ead07ffdf71e55a22ba0ae5a143061
Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-by: Curtis Dunham <curtis.dunham@arm.com>
Reviewed-by: Sascha Bischoff <sascha.bischoff@arm.com>
Reviewed-by: Ilias Vougioukas <ilias.vougioukas@arm.com>
Reviewed-by: Jason Lowe-Power <jason@lowepower.com>
Reviewed-by: Tony Gutierrez <anthony.gutierrez@amd.com>
Swig wrappers for native objects currently share the _m5.internal name
space with Python code. This is undesirable if we ever want to switch
from Swig to some other framework for native binding (e.g., PyBind11
or Boost::Python). This changeset moves all of such wrappers to the
_m5 namespace, which is now reserved for native code.
Change-Id: I2d2bc12dbc05b57b7c5a75f072e08124413d77f3
Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-by: Curtis Dunham <curtis.dunham@arm.com>
Reviewed-by: Jason Lowe-Power <jason@lowepower.com>
Call the stat visitor from the stat itself rather than casting stats
in Python. This reduces the number of ways visitors are called.
Change-Id: Ic4d0b7b32e3ab9897b9a34cd22d353f4da62d738
Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-by: Sascha Bischoff <sascha.bischoff@arm.com>
Reviewed-by: Curtis Dunham <curtis.dunham@arm.com>
Reviewed-by: Jason Lowe-Power <jason@lowepower.com>
Reviewed-by: Joe Gross <joseph.gross@amd.com>
The headers declared in export_method_cxx_predecls are redundant since a
SimObject's main header is automatically included.
Change-Id: Ied9e84630b36960e54efe91d16f8c66fba7e0da0
Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-by: Curtis Dunham <curtis.dunham@arm.com>
Reviewed-by: Joe Gross <joseph.gross@amd.com>
Reviewed-by: Jason Lowe-Power <jason@lowepower.com>
Some configuration scripts need periodic stat dumps. One of the ways
this can be achieved is by using the pariodicStatDump helper
function. This function was previously only exported in the internal
name space. Export it as a normal function in m5.stat instead.
Change-Id: Ic88bf1fd33042a62ab436d5944d8ed778264ac98
Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-by: Sascha Bischoff <sascha.bischoff@arm.com>
Currently only 'start' and 'end' of AddrRange are printed in config.ini.
This causes address ranges to be overlapping when loading a c++-only
config with interleaved addresses using CxxConfigManger. This patch adds
prints for the interleave and XOR bits to config.ini such that address
ranges are properly setup with cxx config.
This changeset updates the dot output to bail out if it is unable to
resolve the voltage or clock domains (which will cause it to raise an
AttributeError). Additionally, the DVFS dot output is disabled by
default for speed purposes.
Minor fixup for 0aeca8f.
This patch adds a secondary dot output file which shows the DVFS domains. This
has been done separately for now to avoid cluttering the already existing
diagram. Due to the way that the clock domains are assigned to components in
gem5, this output must be generated after the C++ objects have been
instantiated. This further motivates the need to generate this file separately
to the current dot output, and not to replace it entirely.
Refactor the style checker into a Python module that can be reused by
command line tools that integrate with git. In particular:
* Create a style package in util
* Move style validators from style.py to the style/validators.py.
* Move style verifiers from style.py to the style/verifiers.py.
* Move utility functions (sort_includes, region handling,
file_types) into the style package
* Move generic code from style.py to style/style.py.
Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-by: Curtis Dunham <curtis.dunham@arm.com>
Reviewed-by: Steve Reinhardt <steve.reinhardt@amd.com>
--HG--
rename : util/style.py => util/hgstyle.py
rename : util/sort_includes.py => util/style/sort_includes.py
extra : rebase_source : ad6cf9b9a18c48350dfc7b7c77bea6c5344fb53c
This changeset adds forking capabilities to the gem5 python scripts. A fork
method is added to simulate.py. This method is responsible for forking the
simulator itself, and will direct all output files to a new output directory
based on the fork sequence number. The default name of the output directory is
the same as the parent with the suffix ".fN" added where N is the fork sequence
number. The fork method provides the option to specify if the system should be
drained prior to forking, or not. By default the system is drained to ensure
that there are no in-flight transactions.
When forking the simulator, the fork method returns the PID of the child
process, or returns 0 if running in the child. This is in line with the standard
Python forking interface.
Signed-off-by: Andreas Sandberg <andreas@sandberg.pp.se>
[sascha.bischoff@arm.com: Rebased patches onto a newer gem5 version]
Signed-off-by: Sascha Bischoff <sascha.bischoff@arm.com>
[andreas.sandberg@arm.com: Updated to comply with modern draining semantics ]
Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>
When forking a gem5 process, some objects need to clean up resources
(mainly file descriptions) shared between the child and the parent of
the fork. This changeset adds the notifyFork() method to Drainable,
which is called in the child process.
Signed-off-by: Andreas Sandberg <andreas@sandberg.pp.se>
[sascha.bischoff@arm.com: Rebased patches onto a newer gem5 version]
Signed-off-by: Sascha Bischoff <sascha.bischoff@arm.com>
Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>
This patch adds a --debug-end flag to main.py so that debug output can be
stoped at a specified tick, while allowing the simulation to continue. It is
useful in situations where you would like to produce a trace for a region of
interest while still collecting stats for the entire run. This is in contrast
to the currently existing --debug-break flag, which terminates the simulation
at the tick.
Just changes the metavar for --debug-start from TIME
to TICK in cset 72046b9b3323 and didn't notice that the
comment "must be in ticks" is now redundant.
By ignoring SIG_TRAP, using --debug-break <N> when not connected to
a debugger becomes a no-op. Apparently this was intended to be a
feature, though the rationale is not clear.
If we don't ignore SIG_TRAP, then using --debug-break <N> when not
connected to a debugger causes the simulation process to terminate
at tick N. This is occasionally useful, e.g., if you just want to
collect a trace for a specific window of execution then you can combine
this with --debug-start to do exactly that.
In addition to not ignoring the signal, this patch also updates
the --debug-break help message and deletes a handful of unprotected
calls to Debug::breakpoint() that relied on the prior behavior.
The gem5 option '--list-sim-objects' is supposed to list all available
SimObjects and their parameters. It currently chokes on SimObjects
with parameters that have an object instance as their default
value. This is caused by __str__ in SimObject trying to resolve its
complete path. When the path resolution method reaches the parent
object (a MetaSimObject since it hasn't been instantiated), it dies
with a Python exception.
This changeset adds a guard to stop path resolution if the parent
object is a MetaSimObject.
Added the missing types EthernetAddr and Current to the JSON/INI file
reader example configs/example/read_config.py.
Also added __str__ to EthernetAddr to make values appear in the same form
in JSON an INI files.
This information is useful if you have a bunch of simulations running
and want to know which one to kill, but you've neglected to take
advantage of the previous patch and embed the pid in your output path.
The DTRACE() macro tests both Trace::enabled and the specific flag. This
change uses the same administrative interface for enabling/disabling
tracing, but masks the SimpleFlags settings directly. This eliminates a
load for every DTRACE() test, e.g. DPRINTF.
Draining is currently done by traversing the SimObject graph and
calling drain()/drainResume() on the SimObjects. This is not ideal
when non-SimObjects (e.g., ports) need draining since this means that
SimObjects owning those objects need to be aware of this.
This changeset moves the responsibility for finding objects that need
draining from SimObjects and the Python-side of the simulator to the
DrainManager. The DrainManager now maintains a set of all objects that
need draining. To reduce the overhead in classes owning non-SimObjects
that need draining, objects inheriting from Drainable now
automatically register with the DrainManager. If such an object is
destroyed, it is automatically unregistered. This means that drain()
and drainResume() should never be called directly on a Drainable
object.
While implementing the new functionality, the DrainManager has now
been made thread safe. In practice, this means that it takes a lock
whenever it manipulates the set of Drainable objects since SimObjects
in different threads may create Drainable objects
dynamically. Similarly, the drain counter is now an atomic_uint, which
ensures that it is manipulated correctly when objects signal that they
are done draining.
A nice side effect of these changes is that it makes the drain state
changes stricter, which the simulation scripts can exploit to avoid
redundant drains.
The memWriteback() and memInvalidate() calls used to live in the
Serializable interface. In this series of patches, the Serializable
interface will be redesigned to make serialization independent of the
object graph and always work on the entire simulator. This means that
the Serialization interface won't be useful to perform maintenance of
the caches in a sub-graph of the entire SimObject graph. This
changeset moves these memory maintenance methods to the SimObject
interface instead.
When the Python helper code switches CPU models, it sometimes also
needs to change the memory mode of the simulator. When this happens,
it accidentally tried to drain the simulator despite having done so
already. This changeset removes the redundant drain.
Objects that are can be serialized are supposed to inherit from the
Serializable class. This class is meant to provide a unified API for
such objects. However, so far it has mainly been used by SimObjects
due to some fundamental design limitations. This changeset redesigns
to the serialization interface to make it more generic and hide the
underlying checkpoint storage. Specifically:
* Add a set of APIs to serialize into a subsection of the current
object. Previously, objects that needed this functionality would
use ad-hoc solutions using nameOut() and section name
generation. In the new world, an object that implements the
interface has the methods serializeSection() and
unserializeSection() that serialize into a named /subsection/ of
the current object. Calling serialize() serializes an object into
the current section.
* Move the name() method from Serializable to SimObject as it is no
longer needed for serialization. The fully qualified section name
is generated by the main serialization code on the fly as objects
serialize sub-objects.
* Add a scoped ScopedCheckpointSection helper class. Some objects
need to serialize data structures, that are not deriving from
Serializable, into subsections. Previously, this was done using
nameOut() and manual section name generation. To simplify this,
this changeset introduces a ScopedCheckpointSection() helper
class. When this class is instantiated, it adds a new /subsection/
and subsequent serialization calls during the lifetime of this
helper class happen inside this section (or a subsection in case
of nested sections).
* The serialize() call is now const which prevents accidental state
manipulation during serialization. Objects that rely on modifying
state can use the serializeOld() call instead. The default
implementation simply calls serialize(). Note: The old-style calls
need to be explicitly called using the
serializeOld()/serializeSectionOld() style APIs. These are used by
default when serializing SimObjects.
* Both the input and output checkpoints now use their own named
types. This hides underlying checkpoint implementation from
objects that need checkpointing and makes it easier to change the
underlying checkpoint storage code.
Currently if there are shell special characters in a
command-line argument, you can't copy and paste the
echoed command line onto a shell prompt because the
characters aren't quoted properly. This patch fixes
that problem.
This patch extends the current address interleaving with basic hashing
support. Instead of directly comparing a number of address bits with a
matching value, it is now possible to use two independent set of
address bits XOR'ed together. This avoids issues where strided address
patterns are heavily biased to a subset of the interleaved ranges.
The Float param was not settable on the command line
due to a typo in the class definition in
python/m5/params.py. This corrects the typo and allows
floats to be set on the command line as intended.
This patch adds sorting based on the SimObject name or parameter name
for all situations where we iterate over dictionaries. This should
ensure a deterministic and consistent order across the host systems
and hopefully avoid regression results differing across python
versions.
This patch adds the ability to load in config.ini files generated from
gem5 into another instance of gem5 built without Python configuration
support. The intended use case is for configuring gem5 when it is a
library embedded in another simulation system.
A parallel config file reader is also provided purely in Python to
demonstrate the approach taken and to provided similar functionality
for as-yet-unknown use models. The Python configuration file reader
can read both .ini and .json files.
C++ configuration file reading:
A command line option has been added for scons to enable C++ configuration
file reading: --with-cxx-config
There is an example in util/cxx_config that shows C++ configuration in action.
util/cxx_config/README explains how to build the example.
Configuration is achieved by the object CxxConfigManager. It handles
reading object descriptions from a CxxConfigFileBase object which
wraps a config file reader. The wrapper class CxxIniFile is provided
which wraps an IniFile for reading .ini files. Reading .json files
from C++ would be possible with a similar wrapper and a JSON parser.
After reading object descriptions, CxxConfigManager creates
SimObjectParam-derived objects from the classes in the (generated with this
patch) directory build/ARCH/cxx_config
CxxConfigManager can then build SimObjects from those SimObjectParams (in an
order dictated by the SimObject-value parameters on other objects) and bind
ports of the produced SimObjects.
A minimal set of instantiate-replacing member functions are provided by
CxxConfigManager and few of the member functions of SimObject (such as drain)
are extended onto CxxConfigManager.
Python configuration file reading (configs/example/read_config.py):
A Python version of the reader is also supplied with a similar interface to
CxxConfigFileBase (In Python: ConfigFile) to config file readers.
The Python config file reading will handle both .ini and .json files.
The object construction strategy is slightly different in Python from the C++
reader as you need to avoid objects prematurely becoming the children of other
objects when setting parameters.
Port binding also needs to be strictly in the same port-index order as the
original instantiation.
Add the ability to build libgem5 without embedded Python or the
ability to configure with Python.
This is a prelude to a patch to allow config.ini files to be loaded
into libgem5 using only C++ which would make embedding gem5 within
other simulation systems easier.
This adds a few registration interfaces to things which cross
between Python and C++. Namely: stats dumping and SimObject resolving
This patch adds the Python parameter type Current, which is used for
the DRAM power modelling (to start with). With this addition we avoid
implicit unit assumptions.
This patch changes the name of the Bus classes to XBar to better
reflect the actual timing behaviour. The actual instances in the
config scripts are not renamed, and remain as e.g. iobus or membus.
As part of this renaming, the code has also been clean up slightly,
making use of range-based for loops and tidying up some comments. The
only changes outside the bus/crossbar code is due to the delay
variables in the packet.
--HG--
rename : src/mem/Bus.py => src/mem/XBar.py
rename : src/mem/coherent_bus.cc => src/mem/coherent_xbar.cc
rename : src/mem/coherent_bus.hh => src/mem/coherent_xbar.hh
rename : src/mem/noncoherent_bus.cc => src/mem/noncoherent_xbar.cc
rename : src/mem/noncoherent_bus.hh => src/mem/noncoherent_xbar.hh
rename : src/mem/bus.cc => src/mem/xbar.cc
rename : src/mem/bus.hh => src/mem/xbar.hh
This patch 'completes' .json config files generation by adding in the
SimObject references and String-valued parameters not currently
printed.
TickParamValues are also changed to print in the same tick-value
format as in .ini files.
This allows .json files to describe a system as fully as the .ini files
currently do.
This patch adds a new function config_value (which mirrors ini_str) to
each ParamValue and to SimObject. This function can then be explicitly
changed to give different .json and .ini printing behaviour rather than
being written in terms of ini_str.
Parsing vectorparams from the command was slightly broken
in that it wouldn't accept the input that the help message
provided to the user and it didn't do the conversion
on the second code path used to convert the string input
to the actual internal representation. This patch fixes these bugs.
The new configuration scripts need the ability to splice
a simobject between a pair of ports that are already connected.
The primary use case is when a CommMonitor needs to be
created after the system is configured and then spliced between
the pair of ports it will monitor.
When passed from a configuration script with a hexadecimal value (like
"0x80000000"), gem5 would error out. This is because it would call
"toMemorySize" which requires the argument to end with a size specifier (like
1MB, etc).
This modification makes it so raw hex values can be passed through Addr
parameters from the configuration scripts.
This patch adds helper functions to SimObject.py, params.py and
simulate.py to enable the new configuration system. Functions like
enumerateParams() in SimObject lets the config system auto-generate
command line options for simobjects to be modified on the command
line.
Params in params.py have __call__() added
to their definition to allow the argparse module to use them
as a type to check command input is in the proper format.
This patch adds a the member function StaticInst::printFlags to allow all
of an instruction's flags to be printed without using the individual
is... member functions or resorting to exposing the 'flags' vector
It also replaces the enum definition StaticInst::Flags with a
Python-generated enumeration and adds to the enum generation mechanism
in src/python/m5/params.py to allow Enums to be placed in namespaces
other than Enums or, alternatively, in wrapper structs allowing them to
be inherited by other classes (so populating that class's name-space
with the enumeration element names).
The unproxy code for Parent.any can generate a circular reference
in certain situations with classes hierarchies like those in ClockDomain.py.
This patch solves this by marking ouself as visited to make sure the
search does not resolve to a self-reference.
The probe patch is motivated by the desire to move analytical and trace code
away from functional code. This is achieved by the probe interface which is
essentially a glorified observer model.
What this means to users:
* add a probe point and a "notify" call at the source of an "event"
* add an isolated module, that is being used to carry out *your* analysis (e.g. generate a trace)
* register that module as a probe listener
Note: an example is given for reference in src/cpu/o3/simple_trace.[hh|cc] and src/cpu/SimpleTrace.py
What is happening under the hood:
* every SimObject maintains has a ProbeManager.
* during initialization (src/python/m5/simulate.py) first regProbePoints and
the regProbeListeners is called on each SimObject. this hooks up the probe
point notify calls with the listeners.
FAQs:
Why did you develop probe points:
* to remove trace, stats gathering, analytical code out of the functional code.
* the belief that probes could be generically useful.
What is a probe point:
* a probe point is used to notify upon a given event (e.g. cpu commits an instruction)
What is a probe listener:
* a class that handles whatever the user wishes to do when they are notified
about an event.
What can be passed on notify:
* probe points are templates, and so the user can generate probes that pass any
type of argument (by const reference) to a listener.
What relationships can be generated (1:1, 1:N, N:M etc):
* there isn't a restriction. You can hook probe points and listeners up in a
1:1, 1:N, N:M relationship. They become useful when a number of modules
listen to the same probe points. The idea being that you can add a small
number of probes into the source code and develop a larger number of useful
analysis modules that use information passed by the probes.
Can you give examples:
* adding a probe point to the cpu's commit method allows you to build a trace
module (outputting assembler), you could re-use this to gather instruction
distribution (arithmetic, load/store, conditional, control flow) stats.
Why is the probe interface currently restricted to passing a const reference:
* the desire, initially at least, is to allow an interface to observe
functionality, but not to change functionality.
* of course this can be subverted by const-casting.
What is the performance impact of adding probes:
* when nothing is actively listening to the probes they should have a
relatively minor impact. Profiling has suggested even with a large number of
probes (60) the impact of them (when not active) is very minimal (<1%).
This patch makes the Clock a TickParamValue just like
Latency/Frequency. There is no longer any need to distinguish it
(originally needed to support multiplication).
If you successfully export a C++ SimObject method, but try to
invoke it from Python before the C++ object is created, you
get a confusing error that says the attribute does not exist,
making you question whether you successfully exported the
method at all. In reality, your only problem is that you're
calling the method too soon. This patch enhances the error
message to give you a better clue.
