Commit graph

168 commits

Author SHA1 Message Date
Erik van der Kouwe
ad898517ac sprofdiff for comparing sprofile results
This patch adds the sprofdiff tool, which compares two sets of profiling
output files. It sorts processes and symbols by difference in average
number of samples, placing those that took more time on the left first
and those that took more time on the right last. If multiple runs are
combined, a standard deviation is computed and this is used to compute
the significance level, which gives an indication of which differences
are likely to be due to chance.

This tool is run not on the raw profiling files, but on the output of
sprofalyze -d (a new option). Though having to use two tools and an
intermediate file seems a bit awkward, the advantage is that the
original source tree is not needed to resolve the symbols. For
comparisons, this is very useful. Also, the intermediate file is in a
text format that can easily be processed by scripts, which may be useful
for other purposes as well.
2012-08-11 22:09:42 +00:00
Erik van der Kouwe
fc42f78836 Re-write sprofalyze in C for better performance, more options and to fix problems with the gap the kernel now has 2012-08-07 13:09:02 +00:00
Ben Gras
50e2064049 No more intel/minix segments.
This commit removes all traces of Minix segments (the text/data/stack
memory map abstraction in the kernel) and significance of Intel segments
(hardware segments like CS, DS that add offsets to all addressing before
page table translation). This ultimately simplifies the memory layout
and addressing and makes the same layout possible on non-Intel
architectures.

There are only two types of addresses in the world now: virtual
and physical; even the kernel and processes have the same virtual
address space. Kernel and user processes can be distinguished at a
glance as processes won't use 0xF0000000 and above.

No static pre-allocated memory sizes exist any more.

Changes to booting:
        . The pre_init.c leaves the kernel and modules exactly as
          they were left by the bootloader in physical memory
        . The kernel starts running using physical addressing,
          loaded at a fixed location given in its linker script by the
          bootloader.  All code and data in this phase are linked to
          this fixed low location.
        . It makes a bootstrap pagetable to map itself to a
          fixed high location (also in linker script) and jumps to
          the high address. All code and data then use this high addressing.
        . All code/data symbols linked at the low addresses is prefixed by
          an objcopy step with __k_unpaged_*, so that that code cannot
          reference highly-linked symbols (which aren't valid yet) or vice
          versa (symbols that aren't valid any more).
        . The two addressing modes are separated in the linker script by
          collecting the unpaged_*.o objects and linking them with low
          addresses, and linking the rest high. Some objects are linked
          twice, once low and once high.
        . The bootstrap phase passes a lot of information (e.g. free memory
          list, physical location of the modules, etc.) using the kinfo
          struct.
        . After this bootstrap the low-linked part is freed.
        . The kernel maps in VM into the bootstrap page table so that VM can
          begin executing. Its first job is to make page tables for all other
          boot processes. So VM runs before RS, and RS gets a fully dynamic,
          VM-managed address space. VM gets its privilege info from RS as usual
          but that happens after RS starts running.
        . Both the kernel loading VM and VM organizing boot processes happen
	  using the libexec logic. This removes the last reason for VM to
	  still know much about exec() and vm/exec.c is gone.

Further Implementation:
        . All segments are based at 0 and have a 4 GB limit.
        . The kernel is mapped in at the top of the virtual address
          space so as not to constrain the user processes.
        . Processes do not use segments from the LDT at all; there are
          no segments in the LDT any more, so no LLDT is needed.
        . The Minix segments T/D/S are gone and so none of the
          user-space or in-kernel copy functions use them. The copy
          functions use a process endpoint of NONE to realize it's
          a physical address, virtual otherwise.
        . The umap call only makes sense to translate a virtual address
          to a physical address now.
        . Segments-related calls like newmap and alloc_segments are gone.
        . All segments-related translation in VM is gone (vir2map etc).
        . Initialization in VM is simpler as no moving around is necessary.
        . VM and all other boot processes can be linked wherever they wish
          and will be mapped in at the right location by the kernel and VM
          respectively.

Other changes:
        . The multiboot code is less special: it does not use mb_print
          for its diagnostics any more but uses printf() as normal, saving
          the output into the diagnostics buffer, only printing to the
          screen using the direct print functions if a panic() occurs.
        . The multiboot code uses the flexible 'free memory map list'
          style to receive the list of free memory if available.
        . The kernel determines the memory layout of the processes to
          a degree: it tells VM where the kernel starts and ends and
          where the kernel wants the top of the process to be. VM then
          uses this entire range, i.e. the stack is right at the top,
          and mmap()ped bits of memory are placed below that downwards,
          and the break grows upwards.

Other Consequences:
        . Every process gets its own page table as address spaces
          can't be separated any more by segments.
        . As all segments are 0-based, there is no distinction between
          virtual and linear addresses, nor between userspace and
          kernel addresses.
        . Less work is done when context switching, leading to a net
          performance increase. (8% faster on my machine for 'make servers'.)
	. The layout and configuration of the GDT makes sysenter and syscall
	  possible.
2012-07-15 22:30:15 +02:00
Thomas Veerman
bb226763a0 Make mkfs.mfs cross compilable 2012-06-18 10:54:54 +00:00
Thomas Veerman
2ed11343a6 Remove duplicate sed 2012-06-18 10:54:51 +00:00
Thomas Veerman
9d04c193d2 Import NetBSD xinstall
Also, fix mk files for cross compilation.
2012-06-18 10:54:50 +00:00
Thomas Veerman
0b2db08aec Import NetBSD mktemp 2012-06-18 10:54:48 +00:00
Thomas Veerman
2e2caf5919 Import NetBSD make 2012-06-18 10:54:48 +00:00
Thomas Veerman
357f105029 Import NetBSD flex 2012-06-18 10:54:47 +00:00
Thomas Veerman
4a17663c14 Import NetBSD byacc 2012-06-18 10:54:17 +00:00
Thomas Veerman
5ea9e707be Import NetBSD nawk 2012-06-18 10:53:38 +00:00
Thomas Veerman
f09c2e014f Use MACHINE_ARCH instead of ARCH 2012-06-18 10:53:35 +00:00
Kees Jongenburger
ade7dc8ded Upgrade libddekit and introduce devmand.
Devmand (Device manager daemon) is the daemon that will
dynamically manage services based on events received from
the system.
2012-06-18 09:23:42 +02:00
Ben Gras
b332803b6f release fixes
. make ramdisk buildable without ../etc having pwd.db
	. add cat to release bootstrap cmds
	. support running dynamically linked executables for
	  release bootstrap cmds
	. import netbsd chroot to help
2012-04-17 16:58:58 +02:00
Ben Gras
53f94f8ed4 dynamic executables on ramdisk support
See UPDATING about upgrading clang for dynamic linking.

	. allow executables on ramdisk to be dynamically linked; this means
	  putting a few required shared libraries and ld.elf_so on the ramdisk.
	. this makes the ramdisk (usage) smaller when they are dynamic, but
	  bigger when they're not.
	. also we can safely ditch newroot and call mount directly as that is
	  all newroot does.
	. create proto.common to share a bunch of entries between
	  small/nonsmall cases
2012-04-16 14:06:09 +02:00
Thomas Veerman
8dd0880985 Add a simple utility to send control messages
The svrctl utility allows you to send control messages to VFS or PM.
This way you can retrieve runtime information or alter behavior.
2012-04-13 12:58:41 +00:00
Ben Gras
a2d1372680 Import NetBSD usr.bin/login 2012-04-11 20:02:15 +02:00
Ben Gras
ef01931f76 Import NetBSD file(1) 2012-04-03 01:15:36 +02:00
Ben Gras
4de51eedad import NetBSD su 2012-03-27 02:19:46 +02:00
Ben Gras
6a73e85ad1 retire _PROTOTYPE
. only good for obsolete K&R support
	. also remove a stray ansi.h and the proto cmd
2012-03-25 16:17:10 +02:00
Ben Gras
6af9856d4a libcompat_minix-centric cleanup
remove some old minix-userland-specific stuff

	. /etc/ttytab as a file, and minix-compat function (fftyslot()),
	  replaced by /etc/ttys and new libc functions
	. also remove minix-specific nlist(), cuserid(), fttyslot(), v8 regex
	  functions and <compat/regex.h>
	. and remaining minix-only utilities that use them
	. also unused <compat/pwd.h> and <compat/syslog.h> and
	  redundant <sys/sigcontext.h>
2012-03-16 17:06:24 +01:00
Evgeniy Ivanov
72eca0eb1e update_bootcfg improvements.
- Fix a glitch (don't glob '*' in for).
- update_bootcfg should be a /bin command.
- update_bootcfg should not add menu entries for missing kernels.
2012-03-14 18:19:28 +01:00
Andy Kosela
24f1e18846 Import NetBSD cat(1) 2012-03-14 16:54:30 +01:00
Andy Kosela
7e81b07cc5 Import NetBSD ed(1) 2012-03-14 16:52:31 +01:00
David van Moolenbroek
4aaa5377b3 Import NetBSD du(1) 2012-03-05 22:32:33 +01:00
Ben Gras
957f4181c0 /usr/lib/cpp script to invoke clang -E
. mainly to keep X working
2012-02-17 15:15:31 +00:00
Antoine Leca
3fb8cb760c More cleaning up 2012-02-15 19:04:58 +00:00
Evgeniy Ivanov
9ab3cf37ce updateboot script for netbsd bootloader 2012-02-15 03:51:27 +01:00
David van Moolenbroek
342171fa9e Remove elle(1) editor, due to an incompatible license 2012-02-14 19:13:17 +01:00
Ben Gras
2fe8fb192f Full switch to clang/ELF. Drop ack. Simplify.
There is important information about booting non-ack images in
docs/UPDATING. ack/aout-format images can't be built any more, and
booting clang/ELF-format ones is a little different. Updating to the
new boot monitor is recommended.

Changes in this commit:

	. drop boot monitor -> allowing dropping ack support
	. facility to copy ELF boot files to /boot so that old boot monitor
	  can still boot fairly easily, see UPDATING
	. no more ack-format libraries -> single-case libraries
	. some cleanup of OBJECT_FMT, COMPILER_TYPE, etc cases
	. drop several ack toolchain commands, but not all support
	  commands (e.g. aal is gone but acksize is not yet).
	. a few libc files moved to netbsd libc dir
	. new /bin/date as minix date used code in libc/
	. test compile fix
	. harmonize includes
	. /usr/lib is no longer special: without ack, /usr/lib plays no
	  kind of special bootstrapping role any more and bootstrapping
	  is done exclusively through packages, so releases depend even
	  less on the state of the machine making them now.
	. rename nbsd_lib* to lib*
	. reduce mtree
2012-02-14 14:52:02 +01:00
Thomas Veerman
e35528ae79 New zoneinfo port from NetBSD 2012-01-16 11:03:25 +00:00
Thomas Veerman
e257c999b8 Replace rm and rmdir with NetBSD version 2012-01-16 10:46:14 +00:00
David van Moolenbroek
e7db2d3588 Add fbd -- Faulty Block Device driver
This driver can be loaded as an overlay on top of a real block
device, and can then be used to generate block-level failures for
certain transfer requests. Specifically, a rule-based system allows
the user to introduce (overt and silent) data corruption and errors.

It exposes itself through /dev/fbd, and a file system can be mounted
on top of it. The new fbdctl(8) tool can be used to control the
driver; see ``man fbdctl'' for details. It also comes with a test
set, located in test/fbdtest.
2011-12-11 22:45:46 +01:00
David van Moolenbroek
fd4c2b74f3 Add block device tracing facility
The implementation is in libblockdriver, and works transparently for
all block drivers. The new btrace(8) tool can be used to control block
tracing; see ``man btrace'' for details.
2011-11-25 13:47:21 +01:00
Adriana Szekeres
c30f014a89 gcore command to coredump a process 2011-11-22 22:07:41 +01:00
Ben Gras
f379b3eecb remove ftpd200 2011-11-18 12:19:50 +01:00
Vivek Prakash
79bfef9aab import NetBSD bzip2 and libbz2 2011-11-18 11:11:04 +01:00
Ben Gras
5c00743626 Switch to NetBSD passwd format
Based on work by Vivek Prakash and Gianluca Guida.

See UPDATING about caveats on currently existing accounts.

	. restores netbsd libc pwcache functions
2011-11-14 14:47:42 +00:00
Ben Gras
cedeabb357 mainstream-format fstab format
. detect both formats in /etc/rc
	. generate new format in setup
	. obsoletes /etc/fstab.local: everything can go in /etc/fstab
	. put shutdown/reboot/halt and a copy of /usr/adm/wtmp
	  (/etc/wtmp) on root FS so that we can do shutdown checks before
	  mounting /usr
	. new fstab format makes getfsent() and friends work
2011-11-14 14:47:28 +00:00
Ben Gras
0c3983b25a update/fix manpage support
. add bsd-style MLINKS to minix man set, restoring aliases
	  (e.g. man add64 -> int64)
	. update daily cron script to run makewhatis and restore makewhatis
	  in man Makefile (makedb), restores functionality of man -k
	. netbsd imports of man, mdocml, makewhatis, libutil, apropos
	. update man.conf with manpage locations, restoring man [-s] <section>
	. throws out some obsolete manpages
2011-09-28 15:24:15 +00:00
Ben Gras
bc0a39238e worldstone benchmark script
. also imports seq(1) to help it
	. add -C option to time(1) to print tsc difference
	. increase col width for ministat for tsc numbers
2011-09-24 14:49:55 +00:00
Vivek Prakash
12d9a08f0d import NetBSD mkdir
. add symlink from old location /usr/bin/mkdir for
	  for compatability
2011-08-25 14:35:12 +00:00
Ben Gras
f984dbba70 increase system-wide filename limit to 255
. move mfs-specific struct, constants to mfs/, so
	  mfs-specific, on-disk format structs and consts are
	  fully isolated from generic structs and functions
	. removes de and readfs utils
2011-08-17 16:00:01 +00:00
Arun Thomas
3ca141ab81 Remove legacy swapfs utility 2011-07-27 17:16:17 +02:00
Arun Thomas
93bb4036ce Remove bigmake, binsizes, packman 2011-07-26 13:38:27 +02:00
Arun Thomas
2a9322a117 Remove advent and ttt 2011-07-26 11:37:18 +02:00
Ben Gras
51ffecc181 import elf-only -lcurses
. abandons mixer, gomoku, talk, talkd, top from base system
	. compile top with clang so no ack-compiled program
	  needs -lcurses any more
2011-07-25 11:08:17 +02:00
Vivek Prakash
1ea07af9da Replace uniq with NetBSD version 2011-07-24 21:40:02 +02:00
Arun Thomas
06617e7fdf Import NetBSD mkdep 2011-07-22 17:28:27 +02:00
Arun Thomas
db5c83c918 Remove strings(1) 2011-07-21 19:06:06 +02:00