Commit graph

167 commits

Author SHA1 Message Date
Andreas Hansson
3ede4dceb8 scons: Use python-config instead of distutils
This patch changes how we determine the Python-related compiler and
linker flags. The previous approach used the internal LINKFORSHARED
which is not intended as part of the external API
(http://bugs.python.org/issue3588) and causes failures on recent OSX
installations.

Instead of using distutils we now rely on python-config and scons
ParseConfig. For backwards compatibility we also parse out the
includes and libs although this could safely be dropped. The drawback
of this patch is that Python 2.5 is now required, but hopefully that
is an acceptable compromise as any system with gcc 4.4 most likely
will have Python >= 2.5.
2013-07-18 08:29:28 -04:00
Ali Saidi
2b582ad9bb scons: ammend swig warning error to version 2.0.10 as well 2013-06-04 15:17:04 -05:00
Andreas Sandberg
f156020158 kvm: Add basic support for ARM
Architecture specific limitations:
 * LPAE is currently not supported by gem5. We therefore panic if LPAE
   is enabled when returning to gem5.
 * The co-processor based interface to the architected timer is
   unsupported. We can't support this due to limitations in the KVM
   API on ARM.
 * M5 ops are currently not supported. This requires either a kernel
   hack or a memory mapped device that handles the guest<->m5
   interface.
2013-04-22 13:20:32 -04:00
Andreas Sandberg
f485ad1908 kvm: Basic support for hardware virtualized CPUs
This changeset introduces the architecture independent parts required
to support KVM-accelerated CPUs. It introduces two new simulation
objects:

KvmVM -- The KVM VM is a component shared between all CPUs in a shared
         memory domain. It is typically instantiated as a child of the
         system object in the simulation hierarchy. It provides access
         to KVM VM specific interfaces.

BaseKvmCPU -- Abstract base class for all KVM-based CPUs. Architecture
	      dependent CPU implementations inherit from this class
	      and implement the following methods:

                * updateKvmState() -- Update the
                  architecture-dependent KVM state from the gem5
                  thread context associated with the CPU.

                * updateThreadContext() -- Update the thread context
                  from the architecture-dependent KVM state.

                * dump() -- Dump the KVM state using (optional).

	      In order to deliver interrupts to the guest, CPU
	      implementations typically override the tick() method and
	      check for, and deliver, interrupts prior to entering
	      KVM.

Hardware-virutalized CPU currently have the following limitations:
 * SE mode is not supported.
 * PC events are not supported.
 * Timing statistics are currently very limited. The current approach
   simply scales the host cycles with a user-configurable factor.
 * The simulated system must not contain any caches.
 * Since cycle counts are approximate, there is no way to request an
   exact number of cycles (or instructions) to be executed by the CPU.
 * Hardware virtualized CPUs and gem5 CPUs must not execute at the
   same time in the same simulator instance.
 * Only single-CPU systems can be simulated.
 * Remote GDB connections to the guest system are not supported.

Additionally, m5ops requires an architecture specific interface and
might not be supported.
2013-04-22 13:20:32 -04:00
Andreas Sandberg
e28e6246fc scons: Try to use 'tcmalloc' before 'tcmalloc_minimal'
tcmalloc_minimal doesn't support the heap checker on Debian, while
tcmalloc does. Instead of always linking with tcmalloc_minimal, if it
exists, we first check for tcmalloc and then use tcmalloc_minimal as a
fallback.
2013-03-18 11:24:56 +01:00
Andreas Sandberg
468ad10f50 scons: Avoid malloc/free compiler optimization when using tcmalloc
According to the tcmalloc readme, the recommended way of compiling
applications that make use of tcmalloc is to disable compiler
optimizations that make assumptions about malloc and friends. This
changeset adds the necessary compiler flags for both gcc and clang.

From the tcmalloc readme:
"NOTE: When compiling with programs with gcc, that you plan to link
with libtcmalloc, it's safest to pass in the flags

 -fno-builtin-malloc -fno-builtin-calloc
 -fno-builtin-realloc -fno-builtin-free

when compiling."
2013-03-18 10:57:26 +01:00
Andreas Sandberg
9e9a47cb9a scons: Don't explicitly add tcmalloc_minimal to LIBS
SCons automatically adds a library to LIBS if conf.CheckLib succeeds,
so there is no need to explicitly add the library.
2013-03-18 10:44:34 +01:00
Andreas Sandberg
3e3475138b scons: Include flags required to link statically with Python
Python requires the flags in LINKFORSHARED to be added the linker
flags when linking with a statically with Python. Failing to do so can
lead to errors from the Python's dynamic module loader at start up.

--HG--
extra : rebase_source : e7a8daf72f4ede7ee5a4a5398a0b12e978a919b9
2013-03-18 10:22:21 +01:00
Andreas Sandberg
9544e8fbe0 scons: Check for known buggy version of SWIG (2.0.9)
SWIG version 2.0.9 uses fully qualified module names despite of the
importing module being in the same package as the imported
module. This has the unfortunate consequence of causing the following
error when importing m5.internal.event:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "src/python/importer.py", line 75, in load_module
    exec code in mod.__dict__
  File "src/python/m5/__init__.py", line 35, in <module>
    import internal
  File "src/python/importer.py", line 75, in load_module
    exec code in mod.__dict__
  File "src/python/m5/internal/__init__.py", line 32, in <module>
    import event
  File "src/python/importer.py", line 75, in load_module
    exec code in mod.__dict__
  File "build/X86/python/swig/event.py", line 107, in <module>
    class Event(m5.internal.serialize.Serializable):
AttributeError: 'module' object has no attribute 'internal'

When 'event' is loaded, it triggers 'serialize' to be loaded. However,
it seems like the dictionary of 'm5' isn't updated until after
__init__.py terminates, which means that 'event' never sees the
'internal' attribute on 'm5'. Older versions of SWIG didn't include
the fully qualified module name if the modules were in the same
package.
2013-03-14 16:08:55 +01:00
Andreas Hansson
08a5fd328b scons: Unify the flags shared by gcc and clang
This patch restructures and unifies the flags used by gcc and clang as
they are largely the same. The common parts are now dealt with in a
shared block of code, and the few bits and pieces that are
specifically affecting either gcc or clang are done separately.
2013-02-19 05:56:07 -05:00
Andreas Hansson
0acd2a96e5 scons: Add warning for overloaded virtual functions
A derived function with a different signature than a base class
function will result in the base class function of the same name being
hidden. The parameter list and return type for the member function in
the derived class must match those of the member function in the base
class, otherwise the function in the derived class will hide the
function in the base class and no polymorphic behaviour will occur.

This patch addresses these warnings by ensuring a unique function name
to avoid (unintentionally) hiding any functions.
2013-02-19 05:56:06 -05:00
Andreas Hansson
d670fa60a1 scons: Add warning for missing field initializers
This patch adds a warning for missing field initializers for both gcc
and clang, and addresses the warnings that were generated.
2013-02-19 05:56:06 -05:00
Anthony Gutierrez
ba983f9387 ext lib: add libfdt to enable flattened device tree support
this patch adds libfdt, a library necessary for supporting
flattened device tree support in current and future versions of
the linux/android kernel for ARM.
2013-02-15 18:48:59 -05:00
Andreas Hansson
62544f938a scons: Disable protobuf if pkg-config and CheckLib fails
This patch changes the use of pkg-config such that protobuf is still
evaluated with CheckLib even if it fails. This is to allow setups
where libprotobuf is available, but not configured through
protobuf. Moreover, if CheckLib fails to use libprotobuf then all the
tracing is disabled, but scons is allowed to continue with a warning.
2013-01-21 09:20:18 -05:00
Andreas Hansson
406891c62a scons: Enforce gcc >= 4.4 or clang >= 2.9 and c++0x support
This patch checks that the compiler in use is either gcc >= 4.4 or
clang >= 2.9. and enables building with --std=c++0x in all cases. As a
consequence, we can tidy up the hashmap and always have static_assert
available. If anyone wants to use alternative compilers, icc for
example supports c++0x to a similar level and could be added if
needed.

This patch opens up for a more elaborate use of c++0x features that
are present in gcc 4.4 and clang 2.9, e.g. auto typed variables,
variadic templates, rvalues and move semantics, and strongly typed
enums. There will be no going back on this one...
2013-01-07 13:05:39 -05:00
Andreas Hansson
221302335b scons: Remove stale compiler options
This patch simply prunes the SUNCC and ICC compiler options as they
are both sufficiently stale that they would have to be re-written from
scratch anyhow. The patch serves to clean things up before shifting to
a build environment that enforces basic c++11 compliance as done in
the following patch.
2013-01-07 13:05:39 -05:00
Andreas Sandberg
4c06be60fc scons: Whitelist useful environment variables
Scons normally removes all environment variables that aren't
whitelisted from the build environment. This messes up things like
ccache, distcc, and the clang static analyzer. This changeset adds the
DISTCC_, CCACHE_, and CCC_ prefixes to the environment variable
whitelist.
2013-01-07 13:05:39 -05:00
Andreas Hansson
41f228c2ea scons: Add support for google protobuf building
This patch enables the use of protobuf input files in the build
process, thus allowing .proto files to be added to input. Each .proto
file is compiled using the protoc tool and the newly created C++
source is added to the list of sources.

The first location where the protobufs will be used is in the
capturing and replay of memory traces, involving the communication
monitor and the trace-generator state of the traffic generator. This
will follow in the next patch.

This patch does add a dependency on the availability of the BSD
licensed protobuf library (and headers), and the protobuf compiler,
protoc. These dependencies are checked in the SConstruct, similar to
e.g. swig. The user can override the use of protoc from the PATH by
specifying the PROTOC environment variable.

Although the dependency on libprotobuf and protoc might seem like a
big step, they add significant value to the project going
forward. Execution traces and other types of traces could easily be
added and parsers for C++ and Python are automatically generated. We
could also envision using protobufs for the checkpoints, description
of the traffic-generator behaviour etc. The sky is the limit. We could
also use the GzipOutputStream from the protobuf library instead of the
current GPL gzstream.

Currently, only the C++ source and header is generated. Going forward
we might want to add the Python output to support simple command-line
tools for displaying and editing the traces.
2013-01-07 13:05:37 -05:00
Andreas Sandberg
4544f3def4 base: Check for static_assert support and provide fallback
C++11 has support for static_asserts to provide compile-time assertion
checking. This is very useful when testing, for example, structure
sizes to make sure that the compiler got the right alignment or vector
sizes.
2012-09-25 11:49:40 -05:00
Andreas Hansson
166afc43e0 Scons: Verbose messages when dependencies are not installed
This patch adds a few more checks to ensure that a compiler is present
on the system, along with swig. It references the relevant packages on
Ubuntu/RedHat, and also adds a similar line for the Python headers.
2012-09-21 10:11:22 -04:00
Andreas Hansson
d1f3a3b91a gcc: Enable Link-Time Optimization for gcc >= 4.6
This patch adds Link-Time Optimization when building the fast target
using gcc >= 4.6, and adds a scons flag to disable it (-no-lto). No
check is performed to guarantee that the linker supports LTO and use
of the linker plugin, so the user has to ensure that binutils GNU ld
>= 2.21 or the gold linker is available. Typically, if gcc >= 4.6 is
available, the latter should not be a problem. Currently the LTO
option is only useful for gcc >= 4.6, due to the limited support on
clang and earlier versions of gcc. The intention is to also add
support for clang once the LTO integration matures.

The same number of jobs is used for the parallel phase of LTO as the
jobs specified on the scons command line, using the -flto=n flag that
was introduced with gcc 4.6. The gold linker also supports concurrent
and incremental linking, but this is not used at this point.

The compilation and linking time is increased by almost 50% on
average, although ARM seems to be particularly demanding with an
increase of almost 100%. Also beware when using this as gcc uses a
tremendous amount of memory and temp space in the process. You have
been warned.

After some careful consideration, and plenty discussions, the flag is
only added to the fast target, and the warning that was issued in an
earlier version of this patch is now removed. Similarly, the flag used
to enable LTO, now the default is to use it, and the flag has been
modified to disable LTO. The rationale behind this decision is that
opt is used for development, whereas fast is only used for long runs,
e.g. regressions or more elaborate experiments where the additional
compile and link time is amortized by a much larger run time.

When it comes to the return on investment, the regression seems to be
roughly 15% faster with LTO. For a bit more detail, I ran twolf on
ARM.fast, with three repeated runs, and they all finish within 42
minutes (+- 25 seconds) without LTO and 31 minutes (+- 25 seconds)
with LTO, i.e. LTO gives an impressive >25% speed-up for this case.

Without LTO (ARM.fast twolf)

real	42m37.632s
user	42m34.448s
sys	0m0.390s

real	41m51.793s
user	41m50.384s
sys	0m0.131s

real	41m45.491s
user	41m39.791s
sys	0m0.139s

With LTO (ARM.fast twolf)

real	30m33.588s
user	30m5.701s
sys	0m0.141s

real	31m27.791s
user	31m24.674s
sys	0m0.111s

real	31m25.500s
user	31m16.731s
sys	0m0.106s
2012-09-14 12:13:22 -04:00
Andreas Hansson
806a1144ce scons: Use c++0x with gcc >= 4.4 instead of 4.6
This patch shifts the version of gcc for which we enable c++0x from
4.6 to 4.4 The more long term plan is to see what the c++0x features
can bring and what level of support would be enabled simply by bumping
the required version of gcc from 4.3 to 4.4.

A few minor things had to be fixed in the code base, most notably the
choice of a hashmap implementation. In the Ruby Sequencer there were
also a few minor issues that gcc 4.4 was not too happy about.
2012-09-14 12:13:18 -04:00
Jason Power
aa8bcd15ec Ruby: Modify Scons so that we can put .sm files in extras
Also allows for header files which are required in slicc generated
code to be in a directory other than src/mem/ruby/slicc_interface.
2012-09-12 14:52:04 -05:00
Andreas Hansson
55bfe13705 scons: Add LIBRARY_PATH from the user environment to Scons
This patch adds the LIBRARY_PATH from the users OS environment to
Scons build environment. This path is used when linking to search for
libraries, and this patch enables tcmalloc to be used during the build
even if it is not placed in the default search paths.
2012-07-12 12:56:11 -04:00
Andreas Hansson
c64731af61 swig: Use SWIG from environment when determining version
This patch fixes a minor issue in the SConstruct where a hardcoded
swig is used instead of the environment SWIG when determining the
version.
2012-06-20 19:32:42 -04:00
Andreas Hansson
eaf19d2815 Build: Point to the appropriate tcmalloc package
This patch updates the message printed if the user does not have
tcmalloc available. It turns out that the correct package (which
creates all required symlinks etc) is libgoogle-perftools-dev. This
has been verified on Ubuntu 12.04.
2012-06-18 15:43:12 -04:00
Ali Saidi
12ba92055d scons: Make compiler version error more verbose and easier to debug. 2012-06-11 10:54:37 -04:00
Ali Saidi
aec7a44116 sim: Add support for tcmalloc if it's installed and available.
This package is available in Ubuntu, Debian, and Redhat as google-perftools.
With multiple tests on a single machine I've seen a little over 10% performance
gain with tcmalloc.
2012-06-05 01:23:09 -04:00
Ali Saidi
1b370431d0 sim: Remove FastAlloc
While FastAlloc provides a small performance increase (~1.5%) over regular malloc it isn't thread safe.
After removing FastAlloc and using tcmalloc I've seen a performance increase of 12% over libc malloc
when running twolf for ARM.
2012-06-05 01:23:08 -04:00
Steve Reinhardt
18e8d7ed2d scons: allow override of SWIG binary on command line 2012-05-08 07:49:57 -07:00
Steve Reinhardt
867ff2b5f1 scons: update minimum SWIG version to 1.3.34
We should try to keep this synced with the wiki
(which I also just updated, but which was
previously inconsistent).
2012-04-23 09:25:16 -07:00
Andreas Hansson
b9bc530ad2 Regression: Add ANSI colours to highlight test status
This patch adds a very basic pretty-printing of the test status
(passed or failed) to highlight failing tests even more: green for
passed, and red for failed. The printing only uses ANSI it the target
output is a tty and supports ANSI colours. Hence, any regression
scripts that are outputting to files or sending e-mails etc should
still be fine.
2012-04-14 05:44:27 -04:00
Andreas Hansson
b6aa6d55eb clang/gcc: Fix compilation issues with clang 3.0 and gcc 4.6
This patch addresses a number of minor issues that cause problems when
compiling with clang >= 3.0 and gcc >= 4.6. Most importantly, it
avoids using the deprecated ext/hash_map and instead uses
unordered_map (and similarly so for the hash_set). To make use of the
new STL containers, g++ and clang has to be invoked with "-std=c++0x",
and this is now added for all gcc versions >= 4.6, and for clang >=
3.0. For gcc >= 4.3 and <= 4.5 and clang <= 3.0 we use the tr1
unordered_map to avoid the deprecation warning.

The addition of c++0x in turn causes a few problems, as the
compiler is more stringent and adds a number of new warnings. Below,
the most important issues are enumerated:

1) the use of namespaces is more strict, e.g. for isnan, and all
   headers opening the entire namespace std are now fixed.

2) another other issue caused by the more stringent compiler is the
   narrowing of the embedded python, which used to be a char array,
   and is now unsigned char since there were values larger than 128.

3) a particularly odd issue that arose with the new c++0x behaviour is
   found in range.hh, where the operator< causes gcc to complain about
   the template type parsing (the "<" is interpreted as the beginning
   of a template argument), and the problem seems to be related to the
   begin/end members introduced for the range-type iteration, which is
   a new feature in c++11.

As a minor update, this patch also fixes the build flags for the clang
debug target that used to be shared with gcc and incorrectly use
"-ggdb".
2012-04-14 05:43:31 -04:00
Marc Orr
24fe7cdf00 build: remove implicit-cache setting of scons from recent build faster patch 2012-03-16 03:29:07 -07:00
Geoffrey Blake
043709fdfa CheckerCPU: Make CheckerCPU runtime selectable instead of compile selectable
Enables the CheckerCPU to be selected at runtime with the --checker option
from the configs/example/fs.py and configs/example/se.py configuration
files.  Also merges with the SE/FS changes.
2012-03-09 09:59:27 -05:00
Marc Orr
eb43883bef build scripts: Made minor modifications to reduce build overhead time.
1. --implicit-cache behavior is default.
2. makeEnv in src/SConscript is conditionally called.
3. decider set to MD5-timestamp
4. NO_HTML build option changed to SLICC_HTML (defaults to False)
2012-03-06 19:07:41 -08:00
Steve Reinhardt
3418ebb5f8 SConstruct: rename and document AddM5Option
Got rid of gratuitous M5 reference in the function name.
In the process, wondered why we have this function at all and
spent time trying to get rid of it and eventually firgured out
why it's needed.  Put the answer in a comment so we don't have
to wonder again.
2012-03-02 14:05:00 -08:00
Steve Reinhardt
f71a5c5e62 SConstruct: update comments & doc strings
Lots of references to 'M5' still, and some to SE/FS
2012-03-02 13:53:52 -08:00
Gabe Black
ea8b347dc5 Merge with head, hopefully the last time for this batch. 2012-01-31 22:40:08 -08:00
Koan-Sin Tan
7d4f187700 clang: Enable compiling gem5 using clang 2.9 and 3.0
This patch adds the necessary flags to the SConstruct and SConscript
files for compiling using clang 2.9 and later (on Ubuntu et al and OSX
XCode 4.2), and also cleans up a bunch of compiler warnings found by
clang. Most of the warnings are related to hidden virtual functions,
comparisons with unsigneds >= 0, and if-statements with empty
bodies. A number of mismatches between struct and class are also
fixed. clang 2.8 is not working as it has problems with class names
that occur in multiple namespaces (e.g. Statistics in
kernel_stats.hh).

clang has a bug (http://llvm.org/bugs/show_bug.cgi?id=7247) which
causes confusion between the container std::set and the function
Packet::set, and this is currently addressed by not including the
entire namespace std, but rather selecting e.g. "using std::vector" in
the appropriate places.
2012-01-31 12:05:52 -05:00
Gabe Black
22a076a6d5 SE/FS: Get rid of the FULL_SYSTEM config option. 2012-01-28 07:24:56 -08:00
Andreas Hansson
c2dbfc1d6c MAC: Make gem5 compile and run on MacOSX 10.7.2
Adaptations to make gem5 compile and run on OSX 10.7.2, with a stock
gcc 4.2.1 and the remaining dependencies from macports, i.e. python
2.7,.2 swig 2.0.4, mercurial 2.0. The changes include an adaptation of
the SConstruct to handle non-library linker flags, and Darwin-specific
code to find the memory usage of gem5. A number of Ruby files relied
on ambigious uint (without the 32 suffix) which caused compilation
errors.
2012-01-09 18:08:20 -06:00
Gabe Black
2ee59cee1b GCC: Guard some gcc flags so they're used when available and needed. 2011-11-09 21:48:28 -08:00
Nilay Vaish
821dfc1289 BuildEnv: Eliminate RUBY as build environment variable
This patch replaces RUBY with PROTOCOL in all the SConscript files as
the environment variable that decides whether or not certain components
of the simulator are compiled.
2011-08-08 10:50:13 -05:00
Gabe Black
1c68c32cc9 Scons: Make some Action objects fit the abreviated output format. 2011-08-02 03:22:11 -07:00
Nilay Vaish
1b49c56679 Scons: Drop RUBY as compile time option.
This patch drops RUBY as a compile time option. Instead the PROTOCOL option
is used to figure out whether or not to build Ruby. If the specified protocol
is 'None', then Ruby is not compiled.
2011-08-02 00:10:08 -05:00
Gabe Black
f8ac16b348 SCons: Only print all the SConsopts being read if verbose is turned on. 2011-07-19 02:56:02 -07:00
Steve Reinhardt
5a78c2d001 SConstruct: automatically update .hg/hgrc with style hooks.
Seems easier than pestering people about it.
Note also that path is now absolute, so you don't get errors
when invoking hg from subdirectories.
Also whacked unused mercurial_bin_not_found message (the
code that used this was deleted a couple months ago in
rev 5138d1e453f1).
2011-06-02 21:23:02 -07:00
Nathan Binkert
f7b3900c13 stats: delete mysql support
we can add it back within python in some future changeset
2011-05-12 11:19:35 -07:00
Steve Reinhardt
8ce85d3db6 scons: interpret paths relative to launch directory
Make sure all command-line targets and EXTRAS directories
are interpreted relative to the launch directory.  This
turns out to be very useful when building code from an
EXTRAS directory using SCons's -C option.

We were trying to do this with targets but it didn't actually
work since we didn't update BUILD_TARGETS (so SCons got
confused internally).  We weren't even trying with EXTRAS.

To simplify the code, the default target is also interpreted
relative to the launch dir even though it was explicitly
handled as relative to the m5 dir before... I doubt anyone
really uses this anyway so it didn't seem worth the complexity.
(Maybe we should get rid of it?)
2011-05-02 12:40:32 -07:00