This patch adds a mechanism to collect run time samples for specific portions
of a benchmark, using work_begin and work_end pseudo instructions.It also enhances
the histogram stat to report geometric mean.
To make gem5 compile and run with swig 2.0.4 a few minor fixes are
necessary, the fail label issues by swig must not be treated as an
error by gcc (tested with gcc 4.2.1), and the vector wrappers must
have SWIGPY_SLICE_ARG defined which happens in pycontainer.swg,
included through std_container.i. By adding the aforementioned include
to the vector wrappers everything seems to work.
Replace the (broken as of previous changeset) swig_objdecl() method
that allowed/forced you to substitute a whole new C++ struct
definition for SWIG to wrap with a set of export_method* hooks
that let you just declare a set of C++ methods (or other declarations)
that get inserted in the auto-generated struct.
Restore the System get/setMemoryMode methods, and use this mechanism
to specialize SimObject as well, eliminating teh need for sim_object.i.
Needed bits of sim_object.i are moved to the new pyobject.i.
Also sucked a little SimObject specialization into cxx_param_decl()
allowing us to get rid of src/sim/sim_object_params.hh. Now the
generation and wrapping of the base SimObject param struct is more
in line with how derived objects are handled.
--HG--
rename : src/python/swig/sim_object.i => src/python/swig/pyobject.i
- Move the random bits of SWIG code generation out of src/SConscript
file and into methods on the objects being wrapped.
- Cleaned up some variable naming and added some comments to make
the process a little clearer.
- Did a little generated file/module renaming:
- vptype_Foo now Foo_vector
- init_Foo is now Foo_init
This makes it easier to see all the Foo-related files in a
sorted directory listing.
- Made cxx_predecls and swig_predecls normal SimObject classmethods.
- Got rid of swig_objdecls hook, even though this breaks the System
objects get/setMemoryMode method exports. Will be fixing this in
a future changeset.
Print IpAddress params in dot notation for readability.
Properly compare IpAddress objects (by value and not object identity).
Also fix up derived param classes (IpNetmask and IpWithPort)
similarly.
This makes it possible to use the grammar multiple times and use the multiple
instances concurrently. This makes implementing an include statement as part
of a grammar possible.
The end of the COPYING file was generated with:
% python ./util/find_copyrights.py configs src system tests util
Update -C command line option to spit out COPYING file
Last summer's big rewrite of the initialization code (in
particular cset 6efc3672733b) got rid of the implicit parenting
that used to occur when an unparented SimObject was assigned as
a parameter value to another SimObject. The idea was that the
new adoptOrphanParams() step would catch these anyway so it was
unnecessary.
Unfortunately it turns out that adoptOrphanParams() has some
inherent instability in that the parent that does the adoption
depends on the config tree traversal order. Even making this
order deterministic (e.g., by traversing children in
alphabetical order) can introduce unwanted and unexpected
hierarchy changes between similar configs (e.g., when adding a
switch_cpu in place of a cpu), causing problems when trying to
restore checkpoints across similar configs. The hierarchy
created by implicit parenting is more stable and more
controllable, so this patch turns that behavior back on.
This patch also cleans up some long-standing holes regarding
parenting of SimObjects that are created in class definitions
(either in the body of the class, or as default parameters).
To avoid breaking some existing config files, this necessitated
changing the error on reparenting children to a warning. This
change fixes another bug where attempting to print the prior
error message would fail on reparenting SimObjectVectors
because they lack a _parent attribute. Some further issues
with SimObjectVectors were cleaned up by getting rid of the
get_parent() call (which could cause errors with some
SimObjectVectors where there was no single parent to return)
with has_parent() (since all the uses of get_parent() were just
boolean tests anyway).
Finally, since the adoptOrphanParam() step turned out to be so
problematic, we now issue a warning when it actually has to do
an adoption. Future cleanup of config files will get rid of
current warnings.
At the same time, rename the trace flags to debug flags since they
have broader usage than simply tracing. This means that
--trace-flags is now --debug-flags and --trace-help is now --debug-help
This is basically like the range_map stuff in src/base (range already
exists in Python). This code is like a set of ranges. I'm using it
to keep track of changed lines in source code, but it could be use to
keep track of memory ranges and holes in memory regions. It could
also be used in memory allocation type stuff. (Though it's not at all
optimized.)
Thanks to swig this was interfering with the standard Python
random module. The only function in that module was seed(),
which erroneously called srand48(). Moved the function to
m5.internal.core, renamed it seedRandom(), and made it call
random_mt.init() instead.
This causes a lot of rebuilds that could have otherwise possibly been
avoided, and, more annoyingly, a lot of unnecessary rerunning of the
regressions. The benefits of having the revision in the output haven't
materialized, so this change removes it.
I like the brevity of Ali's recent change, but the ambiguity of
sometimes showing the source and sometimes the target is a little
confusing. This patch makes scons typically list all sources and
all targets for each action, with the common path prefix factored
out for brevity. It's a little more verbose now but also more
informative.
Somehow Ali talked me into adding colors too, which is a whole
'nother story.
Ran all the source files through 'perl -pi' with this script:
s|\s*(};?\s*)?/\*\s*(end\s*)?namespace\s*(\S+)\s*\*/(\s*})?|} // namespace $3|;
s|\s*};?\s*//\s*(end\s*)?namespace\s*(\S+)\s*|} // namespace $2\n|;
s|\s*};?\s*//\s*(\S+)\s*namespace\s*|} // namespace $1\n|;
Also did a little manual editing on some of the arch/*/isa_traits.hh files
and src/SConscript.
New parameter forms are:
IP address in the format "a.b.c.d" where a-d are from decimal 0 to 255.
IP address with netmask which is an IP followed by "/n" where n is a netmask
length in bits from decimal 0 to 32 or by "/e.f.g.h" where e-h are from
decimal 0 to 255 and which is all 1 bits followed by all 0 bits when
represented in binary. These can also be specified as an integral IP and
netmask passed in separately.
IP address with port which is an IP followed by ":p" where p is a port index
from decimal 0 to 65535. These can also be specified as an integral IP and
port value passed in separately.
Move generated enums into internal.params, which gets
imported into object.params, restoring backward
compatibility for scripts that expect to find them there.
This is necessary because versions of swig older than 1.3.39 fail to
do the right thing and try to do relative imports for everything (even
with the package= option to %module). Instead of putting params in
the m5.internal.params package, put params in the m5.internal package
and make all param modules start with param_. Same thing for
m5.internal.enums.
Also, stop importing all generated params into m5.objects. They are
not necessary and now with everything using relative imports we wound
up with pollution of the namespace (where builtin-range got overridden).
--HG--
rename : src/python/m5/internal/enums/__init__.py => src/python/m5/internal/enums.py
rename : src/python/m5/internal/params/__init__.py => src/python/m5/internal/params.py
Instead of putting all object files into m5/object/__init__.py, interrogate
the importer to find out what should be imported.
Instead of creating a single file that lists all of the embedded python
modules, use static object construction to put those objects onto a list.
Do something similar for embedded swig (C++) code.
It doesn't appear to be necessary and it is somewhat odd. I'm pretty
sure that the package parameter to %module does whatever this might
have been before. It's necessary in future revisions anyway.
Replace direct call to unserialize() on each SimObject with a pair of
calls for better control over initialization in both ckpt and non-ckpt
cases.
If restoring from a checkpoint, loadState(ckpt) is called on each
SimObject. The default implementation simply calls unserialize() if
there is a corresponding checkpoint section, so we get backward
compatibility for existing objects. However, objects can override
loadState() to get other behaviors, e.g., doing other programmed
initializations after unserialize(), or complaining if no checkpoint
section is found. (Note that the default warning for a missing
checkpoint section is now gone.)
If not restoring from a checkpoint, we call the new initState() method
on each SimObject instead. This provides a hook for state
initializations that are only required when *not* restoring from a
checkpoint.
Given this new framework, do some cleanup of LiveProcess subclasses
and X86System, which were (in some cases) emulating initState()
behavior in startup via a local flag or (in other cases) erroneously
doing initializations in startup() that clobbered state loaded earlier
by unserialize().
The separate restoreCheckpoint() call is gone; just pass
the checkpoint dir as an optional arg to instantiate().
This change is a precursor to some more extensive
reworking of the startup code.
The old code for handling SimObject children was kind of messy,
with children stored both in _values and _children, and
inconsistent and potentially buggy handling of SimObject
vectors. Now children are always stored in _children, and
SimObject vectors are consistently handled using the
SimObjectVector class.
Also, by deferring the parenting of SimObject-valued parameters
until the end (instead of doing it at assignment), we eliminate
the hole where one could assign a vector of SimObjects to a
parameter then append to that vector, with the appended objects
never getting parented properly.
This patch induces small stats changes in tests with data races
due to changes in the object creation & initialization order.
The new code does object vectors in order and so should be more
stable.
Orphan SimObjects (not in the config hierarchy) could get
created implicitly if they have a port connection to a SimObject
that is in the hierarchy. This means that there are objects on
the C++ SimObject list (created via the C++ SimObject
constructor call) that are unknown to Python and will get
skipped if we walk the hierarchy from the Python side (as we are
about to do). This patch detects this situation and prints an
error message.
Also fix the rubytester config script which happened to rely on
this behavior.
Enforce that the Python Root SimObject is instantiated only
once. The C++ Root object already panics if more than one is
created. This change avoids the need to track what the root
object is, since it's available from Root.getInstance() (if it
exists). It's now redundant to have the user pass the root
object to functions like instantiate(), checkpoint(), and
restoreCheckpoint(), so that arg is gone. Users who use
configs/common/Simulate.py should not notice.
It would be nice if python had a tree class that would do this for real,
but since we don't, we'll just keep a sorted list of keys and update
it on demand.
If the user sets the environment variable M5_OVERRIDE_PY_SOURCE to
True, then imports that would normally find python code compiled into
the executable will instead first check in the absolute location where
the code was found during the build of the executable. This only
works for files in the src (or extras) directories, not automatically
generated files.
This is a developer feature!
This tidbit was pulled from a larger patch for Tim's sake, so
the comment reflects functions that haven't been exported yet.
I hope to commit them soon so it didn't seem worth cleaning up.
Expand the help text on the --remote-gdb-port option so
people know you can use it to disable remote gdb without
reading the source code, and thus don't waste any time
trying to add a separate option to do that.
Clean up some gdb-related cruft I found while looking
for where one would add a gdb disable option, before
I found the comment that told me that I didn't need
to do that.
Also, make Formulas work on AverageVector. First, Stat::Average (and thus
Stats::AverageVector) was broken when coming out of a checkpoint and on resets,
this fixes that. Formulas also didn't work with AverageVector, but added
support for that.
Get rid of misc.py and just stick misc things in __init__.py
Move utility functions out of SCons files and into m5.util
Move utility type stuff from m5/__init__.py to m5/util/__init__.py
Remove buildEnv from m5 and allow access only from m5.defines
Rename AddToPath to addToPath while we're moving it to m5.util
Rename read_command to readCommand while we're moving it
Rename compare_versions to compareVersions while we're moving it.
--HG--
rename : src/python/m5/convert.py => src/python/m5/util/convert.py
rename : src/python/m5/smartdict.py => src/python/m5/util/smartdict.py
Enable more or less takes the place of check, but also allows stats to
do some other configuration. Prepare moves all of the code that readies
a stat for dumping into a separate function in preparation for supporting
serialization of certain pieces of statistics data.
While we're at it, clean up the visitor code and some of the python code.
We need to add a reference when an object is put on the C++ queue, and remove
a reference when the object is removed from the queue. This was not happening
before and caused a memory problem.
The major thrust of this change is to limit the amount of code
duplication surrounding the code for these functions. This code also
adds two new message types called info and hack. Info is meant to be
less harsh than warn so people don't get confused and start thinking
that the simulator is broken. Hack is a way for people to add runtime
messages indicating that the simulator just executed a code "hack"
that should probably be fixed. The benefit of knowing about these
code hacks is that it will let people know what sorts of inaccuracies
or potential bugs might be entering their experiments. Finally, I've
added some flags to turn on and off these message types so command
line options can change them.
If the same dictionary option is seen in several options, those
dictionaries are composed. If you define the same dictionary key in
multiple options, the system flags an error.
Also, clean up the jobfile code so that it is more debuggable.
Make them easier to express by only having the cxx_type parameter which
has the full namespace name, and drop the cxx_namespace thing.
Add support for multiple levels of namespace.
Since the early days of M5, an event needed to know which event queue
it was on, and that data was required at the time of construction of
the event object. In the future parallelized M5, this sort of
requirement does not work well since the proper event queue will not
always be known at the time of construction of an event. Now, events
are created, and the EventQueue itself has the schedule function,
e.g. eventq->schedule(event, when). To simplify the syntax, I created
a class called EventManager which holds a pointer to an EventQueue and
provides the schedule interface that is a proxy for the EventQueue.
The intent is that objects that frequently schedule events can be
derived from EventManager and then they have the schedule interface.
SimObject and Port are examples of objects that will become
EventManagers. The end result is that any SimObject can just call
schedule(event, when) and it will just call that SimObject's
eventq->schedule function. Of course, some objects may have more than
one EventQueue, so this interface might not be perfect for those, but
they should be relatively few.
I've done a few things here. First, I invoke the script a little bit
differently so that pdb doesn't get confused. Second, I've stored the
actual filename in the module's __file__ so that pdb can find the
source file on your machine.
When invoking several copies of m5 on the same machine at the same
time, there can be a race for TCP ports for the terminal connections
or remote gdb. Expose a function to disable those ports, and have the
regression scripts disable them. There are some SimObjects that have
no other function than to be used with ports (NativeTrace and
EtherTap), so they will panic if the ports are disabled.
This should allow m5 to be more easily embedded into other simulators.
The m5 binary adds a simple main function which then calls into the m5
libarary to start the simulation. In order to make this work
correctly, it was necessary embed python code directly into the
library instead of the zipfile hack. This is because you can't just
append the zipfile to the end of a library the way you can a binary.
As a result, Python files that are part of the m5 simulator are now
compile, marshalled, compressed, and then inserted into the library's
data section with a certain symbol name. Additionally, a new Importer
was needed to allow python to get at the embedded python code.
Small additional changes include:
- Get rid of the PYTHONHOME stuff since I don't think anyone ever used
it, and it just confuses things. Easy enough to add back if I'm wrong.
- Create a few new functions that are key to initializing and running
the simulator: initSignals, initM5Python, m5Main.
The original code for creating libm5 was inspired by a patch Michael
Adler, though the code here was done by me.
I decided that separating some of the scons code into generate.py was
just a bad idea because it caused the dependency system to get all
messed up. If separation is the right way to go in the future, we
should probably use the sconscript mechanism, not the mechanism that I
just removed.
they're all in the same place. This also involves having just one
jobfile.py and moving it into the utils directory to avoid
duplication. Lots of improvements to the utility as well.
--HG--
rename : src/python/m5/attrdict.py => src/python/m5/util/attrdict.py
rename : util/pbs/jobfile.py => src/python/m5/util/jobfile.py
rename : src/python/m5/util.py => src/python/m5/util/misc.py
rename : src/python/m5/multidict.py => src/python/m5/util/multidict.py
rename : util/stats/orderdict.py => src/python/m5/util/orderdict.py
