Commit graph

276 commits

Author SHA1 Message Date
Ben Gras
6410f4b5db VM: some sanitycheck fixes
minor fixes to restore SANITYCHECKS
2012-09-18 13:17:45 +02:00
Ben Gras
053fa581b5 vm: remove stack handling for signals
. moved to the kernel as the handling was only
	  reading it; the kernel may as well write it too
2012-08-29 17:31:38 +02:00
Ben Gras
e4ac80eb60 various warning/errorwarning fixes for gcc47
. warnings (sometimes promoted to errors) in servers/ and kernel/
 . -Os for ext2 boot module to make it small enough
2012-08-27 16:19:18 +02:00
Arun Thomas
7ca1ce7968 VM: Add PTF_READ page table flag 2012-08-17 00:17:52 +02:00
Arun Thomas
fc9b3f9bdd VM: allow for 16KB chunk alignment 2012-08-17 00:17:52 +02:00
Arun Thomas
697f0d097f Rename sys_vmctl_get_cr3_i386 2012-08-12 23:30:54 +02:00
Ben Gras
b3f47f5835 vm: ignore RS pin (pre-allocate) requests for now
. done by RS to reduce/remove dependency on VM for recovery
	. RS has the default stack size of 64MB since the nosegments
	  change, using a huge amount of unused memory to pre-allocate
	. ignore these requests until actually required (i.e. being able
	  to survive VM crashes)

Thanks to pikpik for investigating why RS was so huge.
2012-08-08 15:51:10 +02:00
Ben Gras
b6ea15115c kernel: facility for user-visible memory
. map all objects named usermapped_*.o with globally visible
	  pages; usermapped_glo_*.o with the VM 'global' bit on, i.e.
	  permanently in tlb (very scarce resource!)
	. added kinfo, machine, kmessages and loadinfo for a start
	. modified log, tty to make use of the shared messages struct
2012-07-28 20:57:38 +00:00
Ben Gras
cbcdb838f1 various coverity-inspired fixes
. some strncpy/strcpy to strlcpy conversions
	. new <minix/param.h> to avoid including other minix headers
	  that have colliding definitions with library and commands code,
	  causing parse warnings
	. removed some dead code / assignments
2012-07-16 14:00:56 +02:00
Ben Gras
50e2064049 No more intel/minix segments.
This commit removes all traces of Minix segments (the text/data/stack
memory map abstraction in the kernel) and significance of Intel segments
(hardware segments like CS, DS that add offsets to all addressing before
page table translation). This ultimately simplifies the memory layout
and addressing and makes the same layout possible on non-Intel
architectures.

There are only two types of addresses in the world now: virtual
and physical; even the kernel and processes have the same virtual
address space. Kernel and user processes can be distinguished at a
glance as processes won't use 0xF0000000 and above.

No static pre-allocated memory sizes exist any more.

Changes to booting:
        . The pre_init.c leaves the kernel and modules exactly as
          they were left by the bootloader in physical memory
        . The kernel starts running using physical addressing,
          loaded at a fixed location given in its linker script by the
          bootloader.  All code and data in this phase are linked to
          this fixed low location.
        . It makes a bootstrap pagetable to map itself to a
          fixed high location (also in linker script) and jumps to
          the high address. All code and data then use this high addressing.
        . All code/data symbols linked at the low addresses is prefixed by
          an objcopy step with __k_unpaged_*, so that that code cannot
          reference highly-linked symbols (which aren't valid yet) or vice
          versa (symbols that aren't valid any more).
        . The two addressing modes are separated in the linker script by
          collecting the unpaged_*.o objects and linking them with low
          addresses, and linking the rest high. Some objects are linked
          twice, once low and once high.
        . The bootstrap phase passes a lot of information (e.g. free memory
          list, physical location of the modules, etc.) using the kinfo
          struct.
        . After this bootstrap the low-linked part is freed.
        . The kernel maps in VM into the bootstrap page table so that VM can
          begin executing. Its first job is to make page tables for all other
          boot processes. So VM runs before RS, and RS gets a fully dynamic,
          VM-managed address space. VM gets its privilege info from RS as usual
          but that happens after RS starts running.
        . Both the kernel loading VM and VM organizing boot processes happen
	  using the libexec logic. This removes the last reason for VM to
	  still know much about exec() and vm/exec.c is gone.

Further Implementation:
        . All segments are based at 0 and have a 4 GB limit.
        . The kernel is mapped in at the top of the virtual address
          space so as not to constrain the user processes.
        . Processes do not use segments from the LDT at all; there are
          no segments in the LDT any more, so no LLDT is needed.
        . The Minix segments T/D/S are gone and so none of the
          user-space or in-kernel copy functions use them. The copy
          functions use a process endpoint of NONE to realize it's
          a physical address, virtual otherwise.
        . The umap call only makes sense to translate a virtual address
          to a physical address now.
        . Segments-related calls like newmap and alloc_segments are gone.
        . All segments-related translation in VM is gone (vir2map etc).
        . Initialization in VM is simpler as no moving around is necessary.
        . VM and all other boot processes can be linked wherever they wish
          and will be mapped in at the right location by the kernel and VM
          respectively.

Other changes:
        . The multiboot code is less special: it does not use mb_print
          for its diagnostics any more but uses printf() as normal, saving
          the output into the diagnostics buffer, only printing to the
          screen using the direct print functions if a panic() occurs.
        . The multiboot code uses the flexible 'free memory map list'
          style to receive the list of free memory if available.
        . The kernel determines the memory layout of the processes to
          a degree: it tells VM where the kernel starts and ends and
          where the kernel wants the top of the process to be. VM then
          uses this entire range, i.e. the stack is right at the top,
          and mmap()ped bits of memory are placed below that downwards,
          and the break grows upwards.

Other Consequences:
        . Every process gets its own page table as address spaces
          can't be separated any more by segments.
        . As all segments are 0-based, there is no distinction between
          virtual and linear addresses, nor between userspace and
          kernel addresses.
        . Less work is done when context switching, leading to a net
          performance increase. (8% faster on my machine for 'make servers'.)
	. The layout and configuration of the GDT makes sysenter and syscall
	  possible.
2012-07-15 22:30:15 +02:00
Thomas Veerman
f09c2e014f Use MACHINE_ARCH instead of ARCH 2012-06-18 10:53:35 +00:00
Thomas Veerman
f93afa00e9 Remove MINIXSRCDIR and use NETBSDSRCDIR
NETBSDSRCDIR is used all over the place anyway, and this reduces
our diff with NetBSD a little.
2012-06-18 10:53:35 +00:00
Ben Gras
2bfeeed885 drop segment from safecopy invocations
. all invocations were S or D, so can safely be dropped
	  to prepare for the segmentless world
	. still assign D to the SCP_SEG field in the message
	  to make previous kernels usable
2012-06-16 16:22:51 +00:00
Ben Gras
769af57274 further libexec generalization
. new mode for sys_memset: include process so memset can be
	  done in physical or virtual address space.
	. add a mode to mmap() that lets a process allocate uninitialized
	  memory.
	. this allows an exec()er (RS, VFS, etc.) to request uninitialized
	  memory from VM and selectively clear the ranges that don't come
	  from a file, leaving no uninitialized memory left for the process
	  to see.
	. use callbacks for clearing the process, clearing memory in the
	  process, and copying into the process; so that the libexec code
	  can be used from rs, vfs, and in the future, kernel (to load vm)
	  and vm (to load boot-time processes)
2012-06-07 15:15:02 +02:00
Ben Gras
040362e379 exec() cleanup, generalization, improvement
. make exec() callers (i.e. vfs and rs) determine the
	  memory layout by explicitly reserving regions using
	  mmap() calls on behalf of the exec()ing process,
	  i.e. handling all of the exec logic, thereby eliminating
	  all special exec() knowledge from VM.
	. the new procedure is: clear the exec()ing process
	  first, then call third-party mmap()s to reserve memory, then
	  copy the executable file section contents in, all using callbacks
	  tailored to the caller's way of starting an executable
	. i.e. no more explicit EXEC_NEWMEM-style calls in PM or VM
	  as with rigid 2-section arguments
	. this naturally allows generalizing exec() by simply loading
	  all ELF sections
	. drop/merge of lots of duplicate exec() code into libexec
	. not copying the code sections to vfs and into the executable
	  again is a measurable performance improvement (about 3.3% faster
	  for 'make' in src/servers/)
2012-06-07 15:15:01 +02:00
Ben Gras
ee4016155e vm: add third-party mmap() mode and PROCCTL
these two functions will be used to support all exec() functionality
going into a single library shared by RS and VFS and exec() knowledge
leaving VM.

	. third-party mmap: allow certain processes (VFS, RS) to
	  do mmap() on behalf of another process
	. PROCCTL: used to free and clear a process' address space
2012-06-07 12:43:16 +02:00
Ben Gras
99866db8cd vm: add MAP_FIXED 2012-04-12 15:01:36 +02:00
Ben Gras
b480472a76 vm: full memory view
for user-space processes, increase text segment so it reaches the full
address space, so code can be executed anywhere.
2012-04-12 02:58:39 +02:00
Ben Gras
660e15b2d1 vm: break: allow brk() on any region
use the user-supplied point to lookup which region to perform brk() on,
and if it's a reasonable one, do it, no matter what vm's notion of the
heap region is.
2012-04-12 02:57:59 +02:00
Ben Gras
7336a67dfe retire PUBLIC, PRIVATE and FORWARD 2012-03-25 21:58:14 +02:00
Ben Gras
6a73e85ad1 retire _PROTOTYPE
. only good for obsolete K&R support
	. also remove a stray ansi.h and the proto cmd
2012-03-25 16:17:10 +02:00
David van Moolenbroek
b91295a8d2 vm: if mmap address is given, try that first
Previously, the mmap address (if given) was merely used as a lower
bound, and then possibly overriden with a hint. Now, the mapping is
first tried at the exact given address. If that fails, the start of
the mmap range is used as lower bound (which is then still overridden
by the hint for efficiency).

This allows two pages to be mapped in at predefined addresses, where
the second address is lower than the first. That was not possible.
2012-03-24 19:51:14 +01:00
Ben Gras
6af9856d4a libcompat_minix-centric cleanup
remove some old minix-userland-specific stuff

	. /etc/ttytab as a file, and minix-compat function (fftyslot()),
	  replaced by /etc/ttys and new libc functions
	. also remove minix-specific nlist(), cuserid(), fttyslot(), v8 regex
	  functions and <compat/regex.h>
	. and remaining minix-only utilities that use them
	. also unused <compat/pwd.h> and <compat/syslog.h> and
	  redundant <sys/sigcontext.h>
2012-03-16 17:06:24 +01:00
Arun Thomas
5d958f894f servers: fix clang warnings in iso9660fs and vm 2012-02-17 16:48:40 +01:00
Arun Thomas
ff56906879 Remove obsolete INSTALLFLAGS from makefiles 2012-02-16 23:26:38 +01:00
Antoine Leca
3fb8cb760c More cleaning up 2012-02-15 19:04:58 +00:00
David van Moolenbroek
2c685f34e0 Cut PM out of the adddma/deldma/getdma call path 2012-01-14 00:27:06 +01:00
Ben Gras
4668b84158 vm_remap_ro
- allows shared memory to be mapped in readonly

- ben@minix3.org & tom@minix3.org
2012-01-13 11:30:01 +00:00
Tomas Hruby
88f990e122 VM - vm_unmap() takes SELF as valid argument 2012-01-13 11:30:01 +00:00
Tomas Hruby
974452d4dd VM - clear vminhibit iff it was set 2012-01-13 11:30:01 +00:00
Ben Gras
7cd4002083 vm: clear map cache after kernel requests
. fixes a dirty tlb situation (i.e. random crashes)
	  on some hardware, seemingly new intel architectures
	  (e.g. my desktop i7 machine)
2012-01-11 01:15:35 +01:00
Ben Gras
ef99a7a3dc vm: report cropped region sizes
. when reporting on regions, only report part of the
	  region (addr and length) that really has memory mapped
	  into it
	. to minimise core files
2011-11-26 15:12:17 +00:00
Thomas Veerman
8c9e62b299 Revert to using flushtlb instead of invlpg
Somehow the usage of invlpg causes Minix to crash on Qemu. Reverting
temporarily until we have a proper fix.
2011-09-27 15:15:51 +00:00
Ben Gras
833183a9e1 vm: undo accidental JUNKFREE on 2011-09-19 15:36:46 +00:00
Ben Gras
1dfd43ac27 boot e820 memory detection fixes 2011-09-19 13:36:03 +00:00
Thomas Veerman
d4b72e81b2 Cleanup servers to make GCC/Clang a little happier 2011-09-08 13:57:03 +00:00
Ben Gras
35cc7fbeb8 kernel: invlpg facility
. only use for single-page invalidations initially
	. shows tiny but statistically significant performance
	  improvement; will be more helpful in certain VM debug
	  modes
2011-08-12 13:08:27 +00:00
Ben Gras
d477a9ed82 vm/ipc: only report signals when it matters to ipc
. ipc wants to know about processes that get
	  signals, so that it can break blocking ipc operations
	. doing it for every single signal is wasteful
	  and causes the annoying 'no slot for signals' message
	. this fix tells vm on a per-process basis it (ipc)
	  wants to be notified, i.e. only when it does any ipc calls
	. move ipc config to separate config file while we're at it
2011-08-05 20:52:32 +00:00
Erik van der Kouwe
b1408f61b4 Sanity checks prevent VM panic for mmap, vm_remap and map_phys 2011-07-19 08:58:01 +02:00
Ben Gras
832f7f99fd vm: munmap fix leftover 2011-07-18 12:03:58 +02:00
Ben Gras
85f6d866e6 rename mmap MAP_SHARED to MAP_IPC_SHARED
. MAP_SHARED was used to implement sysv shared memory
	. used to signal shareable memory region to VM
	. assumptions about this situation break when processes
	  use MAP_SHARED for its normal, standardised meaning
2011-07-15 18:10:50 +02:00
Evgeniy Ivanov
ef0a265086 New stat structure.
* VFS and installed MFSes must be in sync before and after this change *

Use struct stat from NetBSD. It requires adding new STAT, FSTAT and LSTAT
syscalls. Libc modification is both backward and forward compatible.

Also new struct stat uses modern field sizes to avoid ABI
incompatibility, when we update uid_t, gid_t and company.
Exceptions are ino_t and off_t in old libc (though paddings added).
2011-07-12 16:39:55 +02:00
Arun Thomas
93ae43f577 boot: Add multiboot support
Not yet fully spec-compliant; work in progress
2011-06-24 17:21:51 +02:00
Gianluca Guida
cc17b27a2b Build NetBSD libc library in world in ELF mode.
3 sets of libraries are built now:
  . ack: all libraries that ack can compile (/usr/lib/i386/)
  . clang+elf: all libraries with minix headers (/usr/lib/)
  . clang+elf: all libraries with netbsd headers (/usr/netbsd/)

Once everything can be compiled with netbsd libraries and headers, the
/usr/netbsd hierarchy will be obsolete and its libraries compiled with
netbsd headers will be installed in /usr/lib, and its headers
in /usr/include. (i.e. minix libc and current minix headers set
will be gone.)

To use the NetBSD libc system (libraries + headers) before
it is the default libc, see:
   http://wiki.minix3.org/en/DevelopersGuide/UsingNetBSDCode
This wiki page also documents the maintenance of the patch
files of minix-specific changes to imported NetBSD code.

Changes in this commit:
  . libsys: Add NBSD compilation and create a safe NBSD-based libc.
  . Port rest of libraries (except libddekit) to new header system.
  . Enable compilation of libddekit with new headers.
  . Enable kernel compilation with new headers.
  . Enable drivers compilation with new headers.
  . Port legacy commands to new headers and libc.
  . Port servers to new headers.
  . Add <sys/sigcontext.h> in compat library.
  . Remove dependency file in tree.
  . Enable compilation of common/lib/libc/atomic in libsys
  . Do not generate RCSID strings in libc.
  . Temporarily disable zoneinfo as they are incompatible with NetBSD format
  . obj-nbsd for .gitignore
  . Procfs: use only integer arithmetic. (Antoine Leca)
  . Increase ramdisk size to create NBSD-based images.
  . Remove INCSYMLINKS handling hack.
  . Add nbsd_include/sys/exec_elf.h
  . Enable ELF compilation with NBSD libc.
  . Add 'make nbsdsrc' in tools to download reference NetBSD sources.
  . Automate minix-port.patch creation.
  . Avoid using fstavfs() as it is *extremely* slow and unneeded.
  . Set err() as PRIVATE to avoid name clash with libc.
  . [NBSD] servers/vm: remove compilation warnings.
  . u32 is not a long in NBSD headers.
  . UPDATING info on netbsd hierarchy
  . commands fixes for netbsd libc
2011-06-24 11:46:30 +02:00
Erik van der Kouwe
c2da8cb535 Globally enable -Wall warnings for GCC 2011-06-08 19:21:03 +00:00
Erik van der Kouwe
0cb22acbda Fix VM -Wall warnings and enable -Werror 2011-06-01 11:30:58 +02:00
Arun Thomas
350b60661a ELF multiboot support 2011-05-04 18:51:43 +02:00
Ben Gras
5f1ab506c8 remove some debugging output from vm/region.c
- don't print verbose process map when legitimate errors occur
2011-03-16 11:57:35 +00:00
Arun Thomas
25a790a631 VM and kernel support for ELF 2011-02-26 23:00:55 +00:00
Ben Gras
c7b2b11ec2 vm: only suspend and resume process with vmctls in pt_writemap in smp mode 2011-02-11 14:56:52 +00:00
Arun Thomas
372b873413 VFS/RS support for ELF 2010-12-10 09:27:56 +00:00
Tomas Hruby
de1bcaa69a VM implicitly aligns mappings to a page
- problem when a PCI BAR is not aligned

- together with  David v. Moolenboek
2010-10-21 17:07:05 +00:00
Ben Gras
ddde360e3e vm - hash table for block cache 2010-10-15 09:10:14 +00:00
Ben Gras
cb2e3a98a2 vm - fix for not placing region outside requested mmap addr 2010-10-15 09:09:29 +00:00
Ben Gras
a3f2df124c vm optimisation - maintain hint for new virtual region position. 2010-10-07 10:04:05 +00:00
Ben Gras
e2570d9b1b vm: optimisation: avl tree for virtual regions
- regions were preivous stored in a linked list, as 'normally'
    there are just 2 or 3 (text, data, stack), but that's slow
    if lots of regions are made with mmap()

  - measurable performance improvement with gcc and clang
2010-10-04 11:41:10 +00:00
Ben Gras
36189370a5 vm: don't check the range of memory during sanity checking
- too many false positives due to memory that is mapped in
    legitimately (acpi, i/o devices)
2010-10-04 11:38:33 +00:00
Tomas Hruby
6513d20744 SMP - Process is stopped when VM modifies the page tables
- RTS_VMINHIBIT flag is used to stop process while VM is fiddling with
  its pagetables

- more generic way of sending synchronous scheduling events among cpus

- do the x-cpu smp sched calls only if the target process is runnable.
  If it is not, it cannot be running and it cannot become runnable
  this CPU holds the BKL
2010-09-15 14:11:12 +00:00
David van Moolenbroek
3eb65448a6 VM: expose secondary cache size 2010-09-14 21:22:56 +00:00
Arun Thomas
de231a713e Move MIN() and MAX() macros to sys/params.h 2010-08-21 13:10:41 +00:00
Arun Thomas
9a21d1a2fd Macros for symbols used in both ASM and C
-The macros take care of prepending the leading underscore when
 necessary.
2010-08-17 16:44:07 +00:00
Erik van der Kouwe
d7c70a41e5 Allow vm to split blocks in memory map (needed for multiboot, contributed by Feiran "Fam" Zheng 2010-07-23 14:22:13 +00:00
Tomas Hruby
f6f20e1269 Overly restrictive assert
- not true if APIC is enabled
2010-07-21 13:46:29 +00:00
Cristiano Giuffrida
0824146978 Disable debug info on trunk. 2010-07-21 08:22:24 +00:00
Cristiano Giuffrida
91a83fe265 Crash recovery and live update support for VM. 2010-07-20 23:03:52 +00:00
Cristiano Giuffrida
0d984b36ef Get rid of static spare pages after VM startup. 2010-07-20 21:59:27 +00:00
Cristiano Giuffrida
40b4e71db2 COW for physically contiguous blocks in VM 2010-07-20 18:57:25 +00:00
Cristiano Giuffrida
1b7d95df29 Generalize interface of some VM functions 2010-07-20 02:08:28 +00:00
Ben Gras
16d0609fad vm: restore sanity checks in NONCONTIGUOUS case. 2010-07-19 18:20:14 +00:00
Ben Gras
068944de36 vm: add MEMPROTECT feature in vm.h. 2010-07-19 18:19:59 +00:00
Ben Gras
77267ca28b vm: add missing USE() macros in utility.c. 2010-07-19 18:19:38 +00:00
Ben Gras
5e2c84b119 vm: allow readonly pages when sanity checking when they're supposed to be rw. 2010-07-19 18:19:16 +00:00
Ben Gras
8c925134f9 vm: NONCONTIGUOUS - try to make physical memory as maximally noncontiguous 2010-07-09 12:22:33 +00:00
Ben Gras
fc8f0b9ee2 vm: better fix for clang 'unused expression' warning 2010-07-05 17:34:26 +00:00
Ben Gras
86139e8bf2 vm: last commit should not have included wiping out these calls
(this was done temporarily to make vm work with clang; vm crashes when
the 64-bit arithmetic functions are used when compiled with clang.)
2010-07-05 14:01:34 +00:00
Ben Gras
b4bea1bfcb vm: fixes for clang warnings 2010-07-05 13:58:57 +00:00
Cristiano Giuffrida
9e4312453d Improve memory pinning. 2010-07-01 08:54:25 +00:00
Cristiano Giuffrida
06700d05d1 Give RS a page table. 2010-06-28 21:53:37 +00:00
Ben Gras
6e47e2996f sep ${.OBJDIR} fixes by Antoine Leica. 2010-06-27 23:47:23 +00:00
Arun Thomas
c0c8d25799 Rename mkfiles from minix.*.mk to bsd.*.mk
Makes things easier for pkgsrc
2010-06-25 18:29:09 +00:00
Cristiano Giuffrida
5c7d64b981 No malloc() in VM. 2010-06-25 12:48:56 +00:00
Erik van der Kouwe
498d7d8a4c Don't use kernel responses in servers 2010-06-24 07:37:26 +00:00
Tomas Hruby
8214e5e284 Removed racy COW optimization
- enabling writing in COW once phys block is reference only once is racy if VM
  is preemptible. original memory location may get overwritten before COW copies
  the memory

- problem when DEBUG_RACE is on and a big problem for SMP
2010-06-18 12:46:18 +00:00
Tomas Hruby
1207fcc6f0 int to endpoint_t conversions in mmap 2010-06-09 09:14:53 +00:00
Ben Gras
31adc0a3c4 vm: junkfree feature that fills freed pages with a recognizable pattern. 2010-06-08 00:59:48 +00:00
Arun Thomas
b641afc78a VM: Remove legacy non-paging code paths 2010-06-05 14:39:40 +00:00
Arun Thomas
007104d60e GCC build fixes/updates
-Set stack sizes for boot image processes
-Increase RS stack size
-Reduce ramdisk size
-HARDWARE task should use kernel stack
-Minor asm tweaks for leading underscores
2010-05-26 18:45:55 +00:00
Erik van der Kouwe
1f11a57141 Oops, last commit included more than was intended 2010-05-20 08:07:47 +00:00
Erik van der Kouwe
5f15ec05b2 More system processes, this was not enough for the release script to run on some configurations 2010-05-20 08:05:07 +00:00
Ben Gras
c5c25e7abc kernel/vm: change pde table info from single buffer to explicit per-process.
makes code in kernel more readable, and allows better sanity checking on
using the pde info.
2010-05-12 08:31:05 +00:00
Tomas Hruby
6e25ad8b0a Use of all NIL_* defines converted to NULL 2010-05-10 13:26:00 +00:00
Ben Gras
d5a0af826a vm: use arch_map2str to print pagefault info, to properly display code addrs 2010-05-08 17:25:54 +00:00
Ben Gras
b6bb75963b vm: remove leftover diag print 2010-05-05 15:26:48 +00:00
Ben Gras
f78d8e74fd secondary cache feature in vm.
A new call to vm lets processes yield a part of their memory to vm,
together with an id, getting newly allocated memory in return. vm is
allowed to forget about it if it runs out of memory. processes can ask
for it back using the same id. (These two operations are normally
combined in a single call.)

It can be used as a as-big-as-memory-will-allow block cache for
filesystems, which is how mfs now uses it.
2010-05-05 11:35:04 +00:00
Cristiano Giuffrida
0164957abb Unified crash recovery and live update.
RS CHANGES:
- Crash recovery is now implemented like live update. Two instances are kept
side by side and the dead version is live updated into the new one. The endpoint
doesn't change and the failure is not exposed (by default) to other system
services.
- The new instance can be created reactively (when a crash is detected) or
proactively. In the latter case, RS can be instructed to keep a replica of
the system service to perform a hot swap when the service fails. The flag
SF_USE_REPL is set in that case.
- The new flag SF_USE_REPL is supported for services in the boot image and
dynamically started services through the RS interface (i.e. -p option in the
service utility).
- Fixed a free unallocated memory bug for core system services.
2010-04-27 11:17:30 +00:00
Tomas Hruby
f51eea4b32 Changed pagefault delivery to VM
this patch changes the way pagefaults are delivered to VM. It adopts
the same model as the out-of-quantum messages sent by kernel to a
scheduler.

- everytime a userspace pagefault occurs, kernel creates a message
  which is sent to VM on behalf of the faulting process

- the process is blocked on delivery to VM in the standard IPC code
  instead of waiting in a spacial in-kernel queue (stack) and is not
  runnable until VM tell kernel that the pagefault is resolved and is
  free to clear the RTS_PAGEFAULT flag.

- VM does not need call kernel and poll the pagefault information
  which saves many (1/2?) calls and kernel calls that return "no more
  data"

- VM notification by kernel does not need to use signals

- each entry in proc table is by 12 bytes smaller (~3k save)
2010-04-26 23:21:26 +00:00
Ben Gras
e0792d72d7 vm: util.S not used currently; leave it out. 2010-04-13 15:02:32 +00:00
Ben Gras
5c17d5e02f vm: include no-caching bits in PTF_ALLFLAGS for flags sanity check. 2010-04-13 11:08:08 +00:00
Ben Gras
1f1f8d2207 vm: don't force physical addresses to be nonzero. 2010-04-13 11:01:40 +00:00
Kees van Reeuwijk
bc314bda91 Remove the types Dev_t, _mnx_Gui, _mnx_Uid, and similar.
Use ANSI-style function declarations where necessary.
2010-04-13 10:58:41 +00:00
Ben Gras
5f7c37bb84 vm: remove assert, map in of phys addr 0 is legit sometimes. 2010-04-13 10:39:46 +00:00
Ben Gras
27fc7ab1f3 vm: use assert() instead of vm_assert(); remove vm_assert(). 2010-04-12 12:37:28 +00:00
Ben Gras
c78250332d let vm use physically fragmented memory for allocations.
map_copy_ph_block is replaced by map_clone_ph_block, which can
replace a single physical block by multiple physical blocks.

also,
 . merge map_mem.c with region.c, as they manipulate the same
   data structures
 . NOTRUNNABLE removed as sanity check
 . use direct functions for ALLOC_MEM and FREE_MEM again
 . add some checks to shared memory mapping code
 . fix for data structure integrity when using shared memory
 . fix sanity checks
2010-04-12 11:25:24 +00:00
Arun Thomas
4ed3a0cf3a Convert kernel over to bsdmake 2010-04-01 22:22:33 +00:00
Cristiano Giuffrida
ef95bf1bb9 Print stacktrace when a system service fails or when a core dump has to be generated for a user process. 2010-03-22 22:46:29 +00:00
Arun Thomas
436d6012a3 Convert drivers/ and servers/ over to bsdmake
-Move libdriver to lib/
-Install all boot image services on filesystem to aid restartability
2010-03-22 21:25:22 +00:00
Tomas Hruby
12ef495cac atomicity fix when enabling paging
- before enabling paging VM asks kernel to resize its segments. This
  may cause kernel to segfault if APIC is used and an interrupt
  happens between this and paging enabled. As these are 2 separate
  vmctl calls it is not atomic. This patch fixes this problem. VM does
  not ask kernel to resize the segments in a separate call anymore.
  The new segments limit is part of the "enable paging" call. It
  generalizes this call in such a way that more information can be
  passed as need be or the information may be completely different if
  another architecture requires this.
2010-03-22 07:42:52 +00:00
Ben Gras
ec30f25d0c VM: fix kernel mappings for children of non-paged parents. 2010-03-18 17:17:31 +00:00
Cristiano Giuffrida
cb176df60f New RS and new signal handling for system processes.
UPDATING INFO:
20100317:
        /usr/src/etc/system.conf updated to ignore default kernel calls: copy
        it (or merge it) to /etc/system.conf.
        The hello driver (/dev/hello) added to the distribution:
        # cd /usr/src/commands/scripts && make clean install
        # cd /dev && MAKEDEV hello

KERNEL CHANGES:
- Generic signal handling support. The kernel no longer assumes PM as a signal
manager for every process. The signal manager of a given process can now be
specified in its privilege slot. When a signal has to be delivered, the kernel
performs the lookup and forwards the signal to the appropriate signal manager.
PM is the default signal manager for user processes, RS is the default signal
manager for system processes. To enable ptrace()ing for system processes, it
is sufficient to change the default signal manager to PM. This will temporarily
disable crash recovery, though.
- sys_exit() is now split into sys_exit() (i.e. exit() for system processes,
which generates a self-termination signal), and sys_clear() (i.e. used by PM
to ask the kernel to clear a process slot when a process exits).
- Added a new kernel call (i.e. sys_update()) to swap two process slots and
implement live update.

PM CHANGES:
- Posix signal handling is no longer allowed for system processes. System
signals are split into two fixed categories: termination and non-termination
signals. When a non-termination signaled is processed, PM transforms the signal
into an IPC message and delivers the message to the system process. When a
termination signal is processed, PM terminates the process.
- PM no longer assumes itself as the signal manager for system processes. It now
makes sure that every system signal goes through the kernel before being
actually processes. The kernel will then dispatch the signal to the appropriate
signal manager which may or may not be PM.

SYSLIB CHANGES:
- Simplified SEF init and LU callbacks.
- Added additional predefined SEF callbacks to debug crash recovery and
live update.
- Fixed a temporary ack in the SEF init protocol. SEF init reply is now
completely synchronous.
- Added SEF signal event type to provide a uniform interface for system
processes to deal with signals. A sef_cb_signal_handler() callback is
available for system processes to handle every received signal. A
sef_cb_signal_manager() callback is used by signal managers to process
system signals on behalf of the kernel.
- Fixed a few bugs with memory mapping and DS.

VM CHANGES:
- Page faults and memory requests coming from the kernel are now implemented
using signals.
- Added a new VM call to swap two process slots and implement live update.
- The call is used by RS at update time and in turn invokes the kernel call
sys_update().

RS CHANGES:
- RS has been reworked with a better functional decomposition.
- Better kernel call masks. com.h now defines the set of very basic kernel calls
every system service is allowed to use. This makes system.conf simpler and
easier to maintain. In addition, this guarantees a higher level of isolation
for system libraries that use one or more kernel calls internally (e.g. printf).
- RS is the default signal manager for system processes. By default, RS
intercepts every signal delivered to every system process. This makes crash
recovery possible before bringing PM and friends in the loop.
- RS now supports fast rollback when something goes wrong while initializing
the new version during a live update.
- Live update is now implemented by keeping the two versions side-by-side and
swapping the process slots when the old version is ready to update.
- Crash recovery is now implemented by keeping the two versions side-by-side
and cleaning up the old version only when the recovery process is complete.

DS CHANGES:
- Fixed a bug when the process doing ds_publish() or ds_delete() is not known
by DS.
- Fixed the completely broken support for strings. String publishing is now
implemented in the system library and simply wraps publishing of memory ranges.
Ideally, we should adopt a similar approach for other data types as well.
- Test suite fixed.

DRIVER CHANGES:
- The hello driver has been added to the Minix distribution to demonstrate basic
live update and crash recovery functionalities.
- Other drivers have been adapted to conform the new SEF interface.
2010-03-17 01:15:29 +00:00
Thomas Veerman
bef0e3eb63 - Add support for the ucontext system calls (getcontext, setcontext,
swapcontext, and makecontext).
- Fix VM to not erroneously think the stack segment and data segment have
  collided when a user-space thread invokes brk().
- Add test51 to test ucontext functionality.
- Add man pages for ucontext system calls.
2010-03-12 15:58:41 +00:00
Arun Thomas
1f9ce647cf Move archtypes.h, fpu.h, and stackframe.h
Move archtypes.h to include/ dir, since several servers require it. Move
fpu.h and stackframe.h to arch-specific header directory. Make source
files and makefiles aware of the new header locations.
2010-03-09 09:41:14 +00:00
Arun Thomas
2a8fabf4ad Include directory reorg and makefile updates.
-Convert the include directory over to using bsdmake
 syntax
-Update/add mkfiles
-Modify install(1) so that it can create symlinks
-Update makefiles to use new install(1) options
-Rename /usr/include/ibm to /usr/include/i386
-Create /usr/include/machine symlink to arch header files
-Move vm_i386.h to its new home in the /usr/include/i386
-Update source files to #include the header files at their
 new homes.
-Add new gnu-includes target for building GCC headers
2010-03-08 11:04:59 +00:00
Ben Gras
35a108b911 panic() cleanup.
this change
   - makes panic() variadic, doing full printf() formatting -
     no more NO_NUM, and no more separate printf() statements
     needed to print extra info (or something in hex) before panicing
   - unifies panic() - same panic() name and usage for everyone -
     vm, kernel and rest have different names/syntax currently
     in order to implement their own luxuries, but no longer
   - throws out the 1st argument, to make source less noisy.
     the panic() in syslib retrieves the server name from the kernel
     so it should be clear enough who is panicing; e.g.
         panic("sigaction failed: %d", errno);
     looks like:
         at_wini(73130): panic: sigaction failed: 0
         syslib:panic.c: stacktrace: 0x74dc 0x2025 0x100a
   - throws out report() - printf() is more convenient and powerful
   - harmonizes/fixes the use of panic() - there were a few places
     that used printf-style formatting (didn't work) and newlines
     (messes up the formatting) in panic()
   - throws out a few per-server panic() functions
   - cleans up a tie-in of tty with panic()

merging printf() and panic() statements to be done incrementally.
2010-03-05 15:05:11 +00:00
Tomas Hruby
975efeac31 Typo in VM server 2010-03-02 10:53:17 +00:00
Arun Thomas
b706112487 Incorporate bsdmake into buildsystem and reorganize libs 2010-02-16 14:41:33 +00:00
Ben Gras
35b471ad94 removal of unused vm<->vfs code. 2010-02-03 13:35:17 +00:00
Kees van Reeuwijk
2fe20e5aaf Fix a complaint about a value return from a void function.
Propagate error codes a bit more.
Some code cleanup.
2010-01-29 14:14:22 +00:00
Kees van Reeuwijk
477b616fe8 Fixed a number of complaints about missing return statements.
Some cases were fixed by declaring the function void, others were fixed
by adding a return <value> statement, thereby avoiding potentially
incorrect behavior (usually in error handling).
Some enum correctness in boot.c.
2010-01-28 13:17:07 +00:00
Kees van Reeuwijk
9a755c3a1f Removed unused code in the ethernet driver that was left from an old implementation
Removed/rewritten the use of uninitialized variables in error messages.
2010-01-20 17:02:55 +00:00
David van Moolenbroek
61bb82a44b VM information interface 2010-01-19 21:00:20 +00:00
Cristiano Giuffrida
c5b309ff07 Merge of Wu's GSOC 09 branch (src.20090525.r4372.wu)
Main changes:
- COW optimization for safecopy.
- safemap, a grant-based interface for sharing memory regions between processes.
- Integration with safemap and complete rework of DS, supporting new data types
  natively (labels, memory ranges, memory mapped ranges).
- For further information:
  http://wiki.minix3.org/en/SummerOfCode2009/MemoryGrants

Additional changes not included in the original Wu's branch:
- Fixed unhandled case in VM when using COW optimization for safecopy in case
  of a block that has already been shared as SMAP.
- Better interface and naming scheme for sys_saferevmap and ds_retrieve_map
  calls.
- Better input checking in syslib: check for page alignment when creating
  memory mapping grants.
- DS notifies subscribers when an entry is deleted.
- Documented the behavior of indirect grants in case of memory mapping.
- Test suite in /usr/src/test/safeperf|safecopy|safemap|ds/* reworked
  and extended.
- Minor fixes and general cleanup.
- TO-DO: Grant ids should be generated and managed the way endpoints are to make
sure grant slots are never misreused.
2010-01-14 15:24:16 +00:00
Tomas Hruby
d96360e4d4 Uninitialized variable fix in VM to kernel protocol
- index must be initialized to 0 otherwise bad things happen like the mappings
  for local APIC are not correct after turning paging on.
2010-01-14 11:30:02 +00:00
Cristiano Giuffrida
d1fd04e72a Initialization protocol for system services.
SYSLIB CHANGES:
- SEF framework now supports a new SEF Init request type from RS. 3 different
callbacks are available (init_fresh, init_lu, init_restart) to specify
initialization code when a service starts fresh, starts after a live update,
or restarts.

SYSTEM SERVICE CHANGES:
- Initialization code for system services is now enclosed in a callback SEF will
automatically call at init time. The return code of the callback will
tell RS whether the initialization completed successfully.
- Each init callback can access information passed by RS to initialize. As of
now, each system service has access to the public entries of RS's system process
table to gather all the information required to initialize. This design
eliminates many existing or potential races at boot time and provides a uniform
initialization interface to system services. The same interface will be reused
for the upcoming publish/subscribe model to handle dynamic 
registration / deregistration of system services.

VM CHANGES:
- Uniform privilege management for all system services. Every service uses the
same call mask format. For boot services, VM copies the call mask from init
data. For dynamic services, VM still receives the call mask via rs_set_priv
call that will be soon replaced by the upcoming publish/subscribe model.

RS CHANGES:
- The system process table has been reorganized and split into private entries
and public entries. Only the latter ones are exposed to system services.
- VM call masks are now entirely configured in rs/table.c
- RS has now its own slot in the system process table. Only kernel tasks and
user processes not included in the boot image are now left out from the system
process table.
- RS implements the initialization protocol for system services.
- For services in the boot image, RS blocks till initialization is complete and
panics when failure is reported back. Services are initialized in their order of
appearance in the boot image priv table and RS blocks to implements synchronous
initialization for every system service having the flag SF_SYNCH_BOOT set.
- For services started dynamically, the initialization protocol is implemented
as though it were the first ping for the service. In this case, if the
system service fails to report back (or reports failure), RS brings the service
down rather than trying to restart it.
2010-01-08 01:20:42 +00:00
David van Moolenbroek
ac9ab099c8 General cleanup:
- clean up kernel section of minix/com.h somewhat
- remove ALLOCMEM and VM_ALLOCMEM calls
- remove non-safecopy and minix-vmd support from Inet
- remove SYS_VIRVCOPY and SYS_PHYSVCOPY calls
- remove obsolete segment encoding in SYS_SAFECOPY*
- remove DEVCTL call, svrctl(FSDEVUNMAP), map_driverX
- remove declarations of unimplemented svrctl requests
- remove everything related to swapping to disk
- remove floppysetup.sh
- remove traces of rescue device
- update DESCRIBE.sh with new devices
- some other small changes
2010-01-05 19:39:27 +00:00
David van Moolenbroek
be992434e7 VM: make munmap(2) round length up, not down (reported by Althaf K Backer) 2010-01-05 09:40:07 +00:00
David van Moolenbroek
ac9a5829a2 suppress kernel/VM memory debugging information 2009-12-29 21:35:12 +00:00
Cristiano Giuffrida
1f5841c8ed Basic System Event Framework (SEF) with ping and live update.
SYSLIB CHANGES:
- SEF must be used by every system process and is thereby part of the system
library.
- The framework provides a receive() interface (sef_receive) for system
processes to automatically catch known system even messages and process them.
- SEF provides a default behavior for each type of system event, but allows
system processes to register callbacks to override the default behavior.
- Custom (local to the process) or predefined (provided by SEF) callback
implementations can be registered to SEF.
- SEF currently includes support for 2 types of system events:
  1. SEF Ping. The event occurs every time RS sends a ping to figure out
  whether a system process is still alive. The default callback implementation
  provided by SEF is to notify RS back to let it know the process is alive
  and kicking.
  2. SEF Live update. The event occurs every time RS sends a prepare to update
  message to let a system process know an update is available and to prepare
  for it. The live update support is very basic for now. SEF only deals with
  verifying if the prepare state can be supported by the process, dumping the
  state for debugging purposes, and providing an event-driven programming
  model to the process to react to state changes check-in when ready to update.
- SEF should be extended in the future to integrate support for more types of
system events. Ideally, all the cross-cutting concerns should be integrated into
SEF to avoid duplicating code and ease extensibility. Examples include:
  * PM notify messages primarily used at shutdown.
  * SYSTEM notify messages primarily used for signals.
  * CLOCK notify messages used for system alarms.
  * Debug messages. IS could still be in charge of fkey handling but would
  forward the debug message to the target process (e.g. PM, if the user
  requested debug information about PM). SEF would then catch the message and
  do nothing unless the process has registered an appropriate callback to
  deal with the event. This simplifies the programming model to print debug
  information, avoids duplicating code, and reduces the effort to print
  debug information.

SYSTEM PROCESSES CHANGES:
- Every system process registers SEF callbacks it needs to override the default
system behavior and calls sef_startup() right after being started.
- sef_startup() does almost nothing now, but will be extended in the future to
support callbacks of its own to let RS control and synchronize with every
system process at initialization time.
- Every system process calls sef_receive() now rather than receive() directly,
to let SEF handle predefined system events.

RS CHANGES:
- RS supports a basic single-component live update protocol now, as follows:
  * When an update command is issued (via "service update *"), RS notifies the
  target system process to prepare for a specific update state.
  * If the process doesn't respond back in time, the update is aborted.
  * When the process responds back, RS kills it and marks it for refreshing.
  * The process is then automatically restarted as for a buggy process and can
  start running again.
  * Live update is currently prototyped as a controlled failure.
2009-12-21 14:12:21 +00:00
Ben Gras
8d800b3df7 Make VM fix up memory for kernel that crosses region boundaries
too.

Add a test to make this happen (triggers a vm panic before this commit).
2009-12-08 13:35:52 +00:00
Ben Gras
f0db9bb328 - map in as much memory as is necessary in 4MB chunks to
let boot processes run with segments
 - allow segment-only processes to fork() by copying them
   and giving them an identity page table
2009-12-07 12:10:44 +00:00
Tomas Hruby
ec1fec6c3f A debug print with no meaning removed from VM. 2009-12-03 10:53:56 +00:00
David van Moolenbroek
bdb85248d4 VM: don't send arbitrary status values to kernel 2009-11-22 13:06:18 +00:00
Tomas Hruby
f98bea8f67 The rest of the r5641 commit 2009-11-11 17:02:45 +00:00
Tomas Hruby
ae75f9d4e5 Removal of the executable flag from files that cannot be executed
- 755 -> 644
2009-11-09 10:26:00 +00:00
Ben Gras
7e73260cf5 - enable remembering of device memory ranges set by PCI and
told to kernel
  - makes VM ask the kernel if a certain process is allowed
    to map in a range of physical memory (VM rounds it to page
    boundaries afterwards - but it's impossible to map anything
    smaller otherwise so I assume this is safe, i.e. there won't
    be anything else in that page; certainly no regular memory)
  - VM permission check cleanup (no more hardcoded calls, less
    hardcoded logic, more readable main loop), a loose end left
    by GQ
  - remove do_copy warning, as the ipc server triggers this but
    it's no more harmful than the special cases already excluded
    explicitly (VFS, PM, etc).
2009-11-03 11:12:23 +00:00
David van Moolenbroek
f89388c241 Kernel, servers: remove unused proto.h definitions 2009-10-31 14:11:50 +00:00
Ben Gras
0e43db1375 disable address space randomisation feature in normal mode. 2009-09-29 12:10:30 +00:00
Ben Gras
856ec906bd assert overammbitious. 2009-09-28 01:23:11 +00:00
Ben Gras
7a45f80bd5 threw out sanity checks before pagetable rewrites, triggering false
page table inconsistencies.

fix handle_memory logic. again.
2009-09-27 12:44:36 +00:00
Ben Gras
67602ce85e field for sanity check feature. 2009-09-27 12:37:37 +00:00
Ben Gras
5715637aa2 new prototypes. 2009-09-27 12:37:16 +00:00
Ben Gras
5cf59f9225 verbose printing and sanity checking functions. 2009-09-27 12:36:48 +00:00
Ben Gras
a02d9aae0a re-enable munmap(). 2009-09-25 19:01:27 +00:00
Ben Gras
e114ae8195 unnecessarily verbose message. 2009-09-25 17:59:38 +00:00
Ben Gras
aa1c90fdd7 correct logic error in region.c. 2009-09-25 17:39:39 +00:00
Ben Gras
f236d1dd9c throw out the debugging stuff again. 2009-09-23 16:26:05 +00:00
Ben Gras
af77f4e964 debug output in SANITYCHECKS mode 2009-09-23 16:02:27 +00:00
Ben Gras
f87a91ce95 missing level-- in out-of-memory case. 2009-09-23 15:19:36 +00:00
Ben Gras
74616a057b temporarily disable munmap() to mask memory problem. 2009-09-23 13:34:04 +00:00
Ben Gras
4c11d7e6f5 - simplify findhole() for use for 1 page only
- do WMF_FREE
 - added WMF_VERIFY to check page table contents
 - randomize address space usage in vm self
2009-09-23 13:33:01 +00:00
Ben Gras
0b713b0de5 unmap VR_DIRECT mappings too (fix for X) 2009-09-22 11:51:17 +00:00