minix/distrib/sets
David van Moolenbroek e4d99eb9b0 Basic live rerandomization infrastructure
This commits adds a basic infrastructure to support Address Space
Randomization (ASR).  In a nutshell, using the already imported ASR
LLVM pass, multiple versions can be generated for the same system
service, each with a randomized, different address space layout.
Combined with the magic instrumentation for state transfer, a system
service can be live updated into another ASR-randomized version at
runtime, thus providing live rerandomization.

Since MINIX3 is not yet capable of running LLVM linker passes, the
ASR-randomized service binaries have to be pregenerated during
crosscompilation.  These pregenerated binaries can then be cycled
through at runtime.  This patch provides the basic proof-of-concept
infrastructure for both these parts.

In order to support pregeneration, the clientctl host script has
been extended with a "buildasr" command.  It is to be used after
building the entire system with bitcode and magic support, and will
produce a given number of ASR-randomized versions of all system
services.  These services are placed in /usr/service/asr in the
image that is generated as final step by the "buildasr" command.

In order to support runtime updating, a new update_asr(8) command
has been added to MINIX3.  This command attempts to live-update the
running system services into their next ASR-randomized versions.
For now, this command is not run automatically, and thus must be
invoked manually.

Technical notes:

- For various reasons, magic instrumentation is x86-only for now,
  and ASR functionality is therefore to be used on x86 only as well.
- The ASR-randomized binaries are placed in numbered subdirectories
  so as not to have to change their actual program names, which are
  assumed to be static in various places (system.conf, procfs).
- The root partition is typically too small to contain all the
  produced binaries, which is why we introduce /usr/service.  There
  is a symlink from /service/asr to /usr/service/asr for no other
  reason than to let userland continue to assume that all services
  are reachable through /service.
- The ASR count field (r_asr_count/ASRcount) maintained by RS is not
  used within RS in any way; it is only passed through procfs to
  userland in order to allow update_asr(8) to keep track of which
  version is currently loaded without having to maintain own state.
- Ideally, pre-instrumentation linking of a service would remove all
  its randomized versions.  Currently, the user is assumed not to
  perform ASR instrumentation and then recompile system services
  without performing ASR instrumentation again, as the randomized
  binaries included in the image would then be stale.  This aspect
  has to be improved later.
- Various other issues are flagged in the comments of the various
  parts of this patch.

Change-Id: I093ad57f31c18305591f64b2d491272288aa0937
2015-09-17 17:15:03 +00:00
..
lists Basic live rerandomization infrastructure 2015-09-17 17:15:03 +00:00
attrs Upgrading build system to new NetBSD revision 2012-11-15 16:07:29 +01:00
checkflist Basic live rerandomization infrastructure 2015-09-17 17:15:03 +00:00
comments Synchronize on NetBSD-CVS (2013/12/1 12:00:00 UTC) 2014-07-28 17:05:06 +02:00
culldeps Upgrading build system to new NetBSD revision 2012-11-15 16:07:29 +01:00
deps Synchronize on NetBSD-CVS (2013/12/1 12:00:00 UTC) 2014-07-28 17:05:06 +02:00
descrs Synchronize on NetBSD-CVS (2013/12/1 12:00:00 UTC) 2014-07-28 17:05:06 +02:00
getdirs.awk Synchronize on NetBSD-CVS (2013/12/1 12:00:00 UTC) 2014-07-28 17:05:06 +02:00
join.awk Upgrading build system to new NetBSD revision 2012-11-15 16:07:29 +01:00
listpkgs Upgrading build system to new NetBSD revision 2012-11-15 16:07:29 +01:00
Makefile Synchronize on NetBSD-CVS (2013/12/1 12:00:00 UTC) 2014-07-28 17:05:06 +02:00
makeflist Synchronize on NetBSD-CVS (2013/12/1 12:00:00 UTC) 2014-07-28 17:05:06 +02:00
makeobsolete Upgrading build system to new NetBSD revision 2012-11-15 16:07:29 +01:00
makeplist Upgrading build system to new NetBSD revision 2012-11-15 16:07:29 +01:00
makesrctars Upgrading build system to new NetBSD revision 2012-11-15 16:07:29 +01:00
makesums Upgrading build system to new NetBSD revision 2012-11-15 16:07:29 +01:00
maketars Synchronize on NetBSD-CVS (2013/12/1 12:00:00 UTC) 2014-07-28 17:05:06 +02:00
metalog.subr Upgrading build system to new NetBSD revision 2012-11-15 16:07:29 +01:00
mkvars.mk Synchronize on NetBSD-CVS (2013/12/1 12:00:00 UTC) 2014-07-28 17:05:06 +02:00
README Synchronize on NetBSD-CVS (2013/12/1 12:00:00 UTC) 2014-07-28 17:05:06 +02:00
regpkg Synchronize on NetBSD-CVS (2013/12/1 12:00:00 UTC) 2014-07-28 17:05:06 +02:00
regpkgset Synchronize on NetBSD-CVS (2013/12/1 12:00:00 UTC) 2014-07-28 17:05:06 +02:00
sets.subr Synchronize on NetBSD-CVS (2013/12/1 12:00:00 UTC) 2014-07-28 17:05:06 +02:00
sort-list Synchronize on NetBSD-CVS (2013/12/1 12:00:00 UTC) 2014-07-28 17:05:06 +02:00
syspkgdeps Upgrading build system to new NetBSD revision 2012-11-15 16:07:29 +01:00
TODO Upgrading build system to new NetBSD revision 2012-11-15 16:07:29 +01:00
versions Upgrading build system to new NetBSD revision 2012-11-15 16:07:29 +01:00

# $NetBSD: README,v 1.13 2013/08/06 22:33:59 soren Exp $

the scripts should be run from the directory where they reside.

makeflist:	output the list of files that should be in a
		distribution, according to the contents of the
		'lists' directory.

checkflist:	check the file list (as internally generated
		by makeflist) against the tree living in $DESTDIR.
		(that tree should be made with 'make distribution'.)

maketars:	make tarballs of the various sets in the distribution,
		based on the contents of the lists, the tree in
		$DESTDIR, and put the tarballs in $RELEASEDIR.
		Note that this script _doesn't_ create the 'secr'
		distribution, because (for now) it requires
		manual intervention to get the binaries right...
		(i'll add another script to create that dist, later.)

what's in 'lists':

lists describing file sets.  There are two sets of lists per file
set: machine dependent and machine-independent files. (there's
also another file in the 'man' dir, which is used by the 'man'
and 'misc' sets, but that's explained later.)

There is one machine-independent file, named "mi".  There are
N machine-dependent files (one per architecture), named "md.${ARCH}".

the sets are as follows:

	base:	the base binary set.  excludes everything described
		below.

	comp:	compiler tools.  All of the tools relating to C, C++,
		and FORTRAN (yes, there are two!) that are in the
		tree.  This includes includes, the linker, tool chain,
		and the .a versions of the libraries.  (obviously,
		base includes ldd, ld.so, and the shared versions.
		base also includes 'cpp', because that's used by X11.)
		includes the man pages for all the binaries contained
		within.  Also, includes all library and system call
		manual pages.

	debug:	Debugging libraries (_g.a/MKDEBUGLIB) and (.debug/MKDEBUG)
		binaries.

	etc:	/etc, and associated files (/var/cron/tabs, /root,
		etc.).  things that shouldn't be blindly reinstalled
		on an upgrade.

	games:	the games and their man pages.

	man:	all of the man pages for the system, except those
		listed elsewhere (e.g. in comp, games, misc, text).
		Includes machine-dependent man pages for this CPU.

	misc:	share/dict, share/doc, and the machine-dependent
		man pages for other CPUs which happen to always
		be installed.

	modules:	stand/${MACHINE}/${OSRELEASE}/modules kernel modules

	tests:	unit, regression, integration and stress tests for the
		whole system.

	text:	text processing tools.  groff and all of its friends.
		includes man pages for all bins contained within.

Each set must contain "./etc/mtree/set.<set name>" within the mi
list.  Failure to add this will break unprivileged builds.