This patch bumps the supported version of gcc from 4.4 to 4.6, and
clang from 2.9 to 3.0. This enables, amongst other things, range-based
for loops, lambda expressions, etc. The STL implementation shipping
with 4.6 also has a full functional implementation of unique_ptr and
shared_ptr.
This patch encompasses several interrelated and interdependent changes
to the ISA generation step. The end goal is to reduce the size of the
generated compilation units for instruction execution and decoding so
that batch compilation can proceed with all CPUs active without
exhausting physical memory.
The ISA parser (src/arch/isa_parser.py) has been improved so that it can
accept 'split [output_type];' directives at the top level of the grammar
and 'split(output_type)' python calls within 'exec {{ ... }}' blocks.
This has the effect of "splitting" the files into smaller compilation
units. I use air-quotes around "splitting" because the files themselves
are not split, but preprocessing directives are inserted to have the same
effect.
Architecturally, the ISA parser has had some changes in how it works.
In general, it emits code sooner. It doesn't generate per-CPU files,
and instead defers to the C preprocessor to create the duplicate copies
for each CPU type. Likewise there are more files emitted and the C
preprocessor does more substitution that used to be done by the ISA parser.
Finally, the build system (SCons) needs to be able to cope with a
dynamic list of source files coming out of the ISA parser. The changes
to the SCons{cript,truct} files support this. In broad strokes, the
targets requested on the command line are hidden from SCons until all
the build dependencies are determined, otherwise it would try, realize
it can't reach the goal, and terminate in failure. Since build steps
(i.e. running the ISA parser) must be taken to determine the file list,
several new build stages have been inserted at the very start of the
build. First, the build dependencies from the ISA parser will be emitted
to arch/$ISA/generated/inc.d, which is then read by a new SCons builder
to finalize the dependencies. (Once inc.d exists, the ISA parser will not
need to be run to complete this step.) Once the dependencies are known,
the 'Environments' are made by the makeEnv() function. This function used
to be called before the build began but now happens during the build.
It is easy to see that this step is quite slow; this is a known issue
and it's important to realize that it was already slow, but there was
no obvious cause to attribute it to since nothing was displayed to the
terminal. Since new steps that used to be performed serially are now in a
potentially-parallel build phase, the pathname handling in the SCons scripts
has been tightened up to deal with chdir() race conditions. In general,
pathnames are computed earlier and more likely to be stored, passed around,
and processed as absolute paths rather than relative paths. In the end,
some of these issues had to be fixed by inserting serializing dependencies
in the build.
Minor note:
For the null ISA, we just provide a dummy inc.d so SCons is never
compelled to try to generate it. While it seems slightly wrong to have
anything in src/arch/*/generated (i.e. a non-generated 'generated' file),
it's by far the simplest solution.
the current implementation of the fetch buffer in the o3 cpu
is only allowed to be the size of a cache line. some
architectures, e.g., ARM, have fetch buffers smaller than a cache
line, see slide 22 at:
http://www.arm.com/files/pdf/at-exploring_the_design_of_the_cortex-a15.pdf
this patch allows the fetch buffer to be set to values smaller
than a cache line.
This patch changes the ProtoBuf builder such that the generated source
and header is placed in the build directory of the proto file. This
was previously not the case for the directories included as EXTRAS. To
make this work, we also ensure that the build directory for the EXTRAS
are added to the include path (which does not seem to automatically be
the case).
There's not much to do about it other than disable the offending
warning anyway, so it's not worth terminating the build over.
Also suppress uninitialized variable warnings on gcc (happens
at least with gcc 4.4 and swig 1.3.40).
This patch restructures and unifies the flags used by gcc and clang as
they are largely the same. The common parts are now dealt with in a
shared block of code, and the few bits and pieces that are
specifically affecting either gcc or clang are done separately.
This patch enables a warning for deleting derived classes that do not
have a virtual destructor. The patch merely adds additional checks,
and there are currently no cases that had to be fixed.
This patch simply prunes the SUNCC and ICC compiler options as they
are both sufficiently stale that they would have to be re-written from
scratch anyhow. The patch serves to clean things up before shifting to
a build environment that enforces basic c++11 compliance as done in
the following patch.
This patch enables the use of protobuf input files in the build
process, thus allowing .proto files to be added to input. Each .proto
file is compiled using the protoc tool and the newly created C++
source is added to the list of sources.
The first location where the protobufs will be used is in the
capturing and replay of memory traces, involving the communication
monitor and the trace-generator state of the traffic generator. This
will follow in the next patch.
This patch does add a dependency on the availability of the BSD
licensed protobuf library (and headers), and the protobuf compiler,
protoc. These dependencies are checked in the SConstruct, similar to
e.g. swig. The user can override the use of protoc from the PATH by
specifying the PROTOC environment variable.
Although the dependency on libprotobuf and protoc might seem like a
big step, they add significant value to the project going
forward. Execution traces and other types of traces could easily be
added and parsers for C++ and Python are automatically generated. We
could also envision using protobufs for the checkpoints, description
of the traffic-generator behaviour etc. The sky is the limit. We could
also use the GzipOutputStream from the protobuf library instead of the
current GPL gzstream.
Currently, only the C++ source and header is generated. Going forward
we might want to add the Python output to support simple command-line
tools for displaying and editing the traces.
When casting objects in the generated SWIG interfaces, SWIG uses
classical C-style casts ( (Foo *)bar; ). In some cases, this can
degenerate into the equivalent of a reinterpret_cast (mainly if only a
forward declaration of the type is available). This usually works for
most compilers, but it is known to break if multiple inheritance is
used anywhere in the object hierarchy.
This patch introduces the cxx_header attribute to Python SimObject
definitions, which should be used to specify a header to include in
the SWIG interface. The header should include the declaration of the
wrapped object. We currently don't enforce header the use of the
header attribute, but a warning will be generated for objects that do
not use it.
This patch adds an explicit dependency between param_%s.i and the
Python source file defining the object. Previously, the build system
didn't rebuild SWIG interfaces correctly when an object's Python
sources were updated.
This patch adds Link-Time Optimization when building the fast target
using gcc >= 4.6, and adds a scons flag to disable it (-no-lto). No
check is performed to guarantee that the linker supports LTO and use
of the linker plugin, so the user has to ensure that binutils GNU ld
>= 2.21 or the gold linker is available. Typically, if gcc >= 4.6 is
available, the latter should not be a problem. Currently the LTO
option is only useful for gcc >= 4.6, due to the limited support on
clang and earlier versions of gcc. The intention is to also add
support for clang once the LTO integration matures.
The same number of jobs is used for the parallel phase of LTO as the
jobs specified on the scons command line, using the -flto=n flag that
was introduced with gcc 4.6. The gold linker also supports concurrent
and incremental linking, but this is not used at this point.
The compilation and linking time is increased by almost 50% on
average, although ARM seems to be particularly demanding with an
increase of almost 100%. Also beware when using this as gcc uses a
tremendous amount of memory and temp space in the process. You have
been warned.
After some careful consideration, and plenty discussions, the flag is
only added to the fast target, and the warning that was issued in an
earlier version of this patch is now removed. Similarly, the flag used
to enable LTO, now the default is to use it, and the flag has been
modified to disable LTO. The rationale behind this decision is that
opt is used for development, whereas fast is only used for long runs,
e.g. regressions or more elaborate experiments where the additional
compile and link time is amortized by a much larger run time.
When it comes to the return on investment, the regression seems to be
roughly 15% faster with LTO. For a bit more detail, I ran twolf on
ARM.fast, with three repeated runs, and they all finish within 42
minutes (+- 25 seconds) without LTO and 31 minutes (+- 25 seconds)
with LTO, i.e. LTO gives an impressive >25% speed-up for this case.
Without LTO (ARM.fast twolf)
real 42m37.632s
user 42m34.448s
sys 0m0.390s
real 41m51.793s
user 41m50.384s
sys 0m0.131s
real 41m45.491s
user 41m39.791s
sys 0m0.139s
With LTO (ARM.fast twolf)
real 30m33.588s
user 30m5.701s
sys 0m0.141s
real 31m27.791s
user 31m24.674s
sys 0m0.111s
real 31m25.500s
user 31m16.731s
sys 0m0.106s
This patch adds a new target called 'perf' that facilitates profiling
using google perftools rather than gprof. The perftools CPU profiler
offers plenty useful information in addition to gprof, and the latter
is kept mostly to offer profiling also on non-Linux hosts.
This patch restructures the ccflags such that the common parts are
defined in a single location, also capturing all the target types in a
single place.
The patch also adds a corresponding ldflags in preparation for
google-perf profiling support and the addition of Link-Time
Optimization.
This patch disables a warning for unused values which causes problems
when compiling the swig-generated sources using recent llvm-based
compilers like llvm-gcc and clang.
This patch addresses a number of minor issues that cause problems when
compiling with clang >= 3.0 and gcc >= 4.6. Most importantly, it
avoids using the deprecated ext/hash_map and instead uses
unordered_map (and similarly so for the hash_set). To make use of the
new STL containers, g++ and clang has to be invoked with "-std=c++0x",
and this is now added for all gcc versions >= 4.6, and for clang >=
3.0. For gcc >= 4.3 and <= 4.5 and clang <= 3.0 we use the tr1
unordered_map to avoid the deprecation warning.
The addition of c++0x in turn causes a few problems, as the
compiler is more stringent and adds a number of new warnings. Below,
the most important issues are enumerated:
1) the use of namespaces is more strict, e.g. for isnan, and all
headers opening the entire namespace std are now fixed.
2) another other issue caused by the more stringent compiler is the
narrowing of the embedded python, which used to be a char array,
and is now unsigned char since there were values larger than 128.
3) a particularly odd issue that arose with the new c++0x behaviour is
found in range.hh, where the operator< causes gcc to complain about
the template type parsing (the "<" is interpreted as the beginning
of a template argument), and the problem seems to be related to the
begin/end members introduced for the range-type iteration, which is
a new feature in c++11.
As a minor update, this patch also fixes the build flags for the clang
debug target that used to be shared with gcc and incorrectly use
"-ggdb".
Partial backout of cset 8b223e308b08.
Although it's great that there's currently no need
for Werror=false in the current tree, some of us
have uncommitted code that still needs this option.
This patch removes the overriding of "-Werror" in a handful of
cases. The code compiles with gcc 4.6.3 and clang 3.0 without any
warnings, and thus without any errors. There are no functional changes
introduced by this patch. In the future, rather than ypassing
"-Werror", address the warnings.
1. --implicit-cache behavior is default.
2. makeEnv in src/SConscript is conditionally called.
3. decider set to MD5-timestamp
4. NO_HTML build option changed to SLICC_HTML (defaults to False)
This patch adds the necessary flags to the SConstruct and SConscript
files for compiling using clang 2.9 and later (on Ubuntu et al and OSX
XCode 4.2), and also cleans up a bunch of compiler warnings found by
clang. Most of the warnings are related to hidden virtual functions,
comparisons with unsigneds >= 0, and if-statements with empty
bodies. A number of mismatches between struct and class are also
fixed. clang 2.8 is not working as it has problems with class names
that occur in multiple namespaces (e.g. Statistics in
kernel_stats.hh).
clang has a bug (http://llvm.org/bugs/show_bug.cgi?id=7247) which
causes confusion between the container std::set and the function
Packet::set, and this is currently addressed by not including the
entire namespace std, but rather selecting e.g. "using std::vector" in
the appropriate places.
To make gem5 compile and run with swig 2.0.4 a few minor fixes are
necessary, the fail label issues by swig must not be treated as an
error by gcc (tested with gcc 4.2.1), and the vector wrappers must
have SWIGPY_SLICE_ARG defined which happens in pycontainer.swg,
included through std_container.i. By adding the aforementioned include
to the vector wrappers everything seems to work.
- Move the random bits of SWIG code generation out of src/SConscript
file and into methods on the objects being wrapped.
- Cleaned up some variable naming and added some comments to make
the process a little clearer.
- Did a little generated file/module renaming:
- vptype_Foo now Foo_vector
- init_Foo is now Foo_init
This makes it easier to see all the Foo-related files in a
sorted directory listing.
- Made cxx_predecls and swig_predecls normal SimObject classmethods.
- Got rid of swig_objdecls hook, even though this breaks the System
objects get/setMemoryMode method exports. Will be fixing this in
a future changeset.
The default generated binary is now gem5.<type> instead of m5.<type>.
The latter does still work but gem5.<type> will be generated first and
then m5.<type> will be hard linked to it.
The end of the COPYING file was generated with:
% python ./util/find_copyrights.py configs src system tests util
Update -C command line option to spit out COPYING file
This is similar to guards on mercurial queues and they're used for selecting
which files are compiled into some given object. We already do something
similar, but it's mostly hard coded for the m5 binary and the m5 library
and I'd like to make it more flexible to better support the unittests
At the same time, rename the trace flags to debug flags since they
have broader usage than simply tracing. This means that
--trace-flags is now --debug-flags and --trace-help is now --debug-help
This causes a lot of rebuilds that could have otherwise possibly been
avoided, and, more annoyingly, a lot of unnecessary rerunning of the
regressions. The benefits of having the revision in the output haven't
materialized, so this change removes it.
I like the brevity of Ali's recent change, but the ambiguity of
sometimes showing the source and sometimes the target is a little
confusing. This patch makes scons typically list all sources and
all targets for each action, with the common path prefix factored
out for brevity. It's a little more verbose now but also more
informative.
Somehow Ali talked me into adding colors too, which is a whole
'nother story.
Ran all the source files through 'perl -pi' with this script:
s|\s*(};?\s*)?/\*\s*(end\s*)?namespace\s*(\S+)\s*\*/(\s*})?|} // namespace $3|;
s|\s*};?\s*//\s*(end\s*)?namespace\s*(\S+)\s*|} // namespace $2\n|;
s|\s*};?\s*//\s*(\S+)\s*namespace\s*|} // namespace $1\n|;
Also did a little manual editing on some of the arch/*/isa_traits.hh files
and src/SConscript.
The build_dir parameter name has been deprecated and replaced with
variant_dir. This change switches us over to avoid warning spew in newer
versions of scons.
This is necessary because versions of swig older than 1.3.39 fail to
do the right thing and try to do relative imports for everything (even
with the package= option to %module). Instead of putting params in
the m5.internal.params package, put params in the m5.internal package
and make all param modules start with param_. Same thing for
m5.internal.enums.
Also, stop importing all generated params into m5.objects. They are
not necessary and now with everything using relative imports we wound
up with pollution of the namespace (where builtin-range got overridden).
--HG--
rename : src/python/m5/internal/enums/__init__.py => src/python/m5/internal/enums.py
rename : src/python/m5/internal/params/__init__.py => src/python/m5/internal/params.py
Instead of putting all object files into m5/object/__init__.py, interrogate
the importer to find out what should be imported.
Instead of creating a single file that lists all of the embedded python
modules, use static object construction to put those objects onto a list.
Do something similar for embedded swig (C++) code.
If the user sets the environment variable M5_OVERRIDE_PY_SOURCE to
True, then imports that would normally find python code compiled into
the executable will instead first check in the absolute location where
the code was found during the build of the executable. This only
works for files in the src (or extras) directories, not automatically
generated files.
This is a developer feature!
This causes builds to happen in sorted order rather than in
declaration order. This gets annoying when you make a global change
and then you notice that the files that are being compiled are jumping
around the directory hierarchy.
