Commit graph

55 commits

Author SHA1 Message Date
Ben Gras
0022ee2cb5 VM: ARM/I386 PDE/PTE macro unification
. further reduces special cases in pagetable.c

Change-Id: Ifd7dd9ee10ca618a47bb4a372467205fbe6da90e
2013-03-14 13:55:04 +00:00
Ben Gras
718a9ef472 VM: pagetable.c: harmonize x86/ARM findhole() code
ARM needs to be able to find N consecutive free slots; use same
code for x86.

Change-Id: Ic79677961c8adfca2aeb5385962942ae0d76867c
2013-03-13 12:09:32 +00:00
Ben Gras
90d777f053 VM: pagequeue data structure
. data structure that automatically keeps a set
	  of pages in reserve, to replace sparepages and
	  possibly re-used in the future for similar situations,
	  e.g. if in-filesystem-cache block eviction is
	  implemented and FS asks for a new block

Change-Id: I149d46c14b9c8e75df16cb94e08907f008c339a6
2013-03-13 12:09:32 +00:00
Lionel Sambuc
8e4736f2df Removing obsolete _MINIX define
Change-Id: Id33ac7e973d1c0e249b690fe44a597474fac6076
2013-02-26 09:44:20 +00:00
Lionel Sambuc
e4fa9802cb ARM: Enable caches
First round, some more optimizations are possible and should be
activated.

Change-Id: I3b7dee7c82fbffd823a08bec1c5d5ebcf769f92f
2013-02-18 09:08:26 +01:00
Ben Gras
d1df256de9 VM: slight pagedir mapping generalization
. allow any number of pde's used for pagedir mapping
	. allows >1024 NR_PROCS on x86, >64 on ARM
	. allows NR_PROCS to be the same in both cases
	. also cleanup: allocating spare PDE's is not necessary
	  throw that function out

Change-Id: Ibb8f8cf6e7db6a4d6384b6911d1a3f3f5e5d8256
2013-02-10 21:50:34 +01:00
Ben Gras
b7ea9f3fd1 vm: fix sanity checks on arm
The 'polarity' of the RW bit is inversed on ARM, causing one
of the sanity check compensations to fail. ARM now runs basic
stuff with sanity checks passing.

Change-Id: Iee28ab63e430e759f204eeb204b24c301d5ea3c9
2013-02-01 16:45:56 +00:00
Ben Gras
08cb986d3f vm: fix kernel-requested mappings for arm
. make vm tell kernel virtual locations of mappings
	. makes _minix_kerninfo feature work
	. fix for mappings being larger than what 1 pde can address
	  (e.g. devices memory requested on arm)
	. still requires a special case for devices memory for the
	  kernel, which has to switch to virtual addressing

Change-Id: I2e94090aa432346fa4da0edeba72f0b7406c2ad7
2013-01-29 17:57:04 +00:00
Thomas Veerman
b3250450fe VM: ARM needs more spare pages 2013-01-25 17:07:02 +00:00
Kees Jongenburger
fce5ff6513 vm: Fix sparepage resource leak.
Fix a sparepage resource leak by preventing to call
vm_getsparepage twice.

Change-Id: I06557d47b90a7cca74e0a86921c2579f9618685d
2013-01-07 09:12:09 +01:00
Ben Gras
29edcad310 vm: replace phys avl by array
. make vm be able to use malloc() by overriding brk()
   and minix_mmap() functions
 . phys regions can then be malloc()ed and free()d instead
   of being in an avl tree, which is slightly faster
 . 'offset' field in phys_region can go too (offset is implied
   by position in array) but leads to bigger code changes
2012-12-26 16:14:41 +00:00
Ben Gras
6cf98dbe00 vm: reduce noise in merged pagetable.c 2012-11-09 19:00:46 +01:00
Ben Gras
3771a0833d vm: merge i386 and arm pagetable code 2012-11-09 18:46:03 +01:00
Ben Gras
b1da7fafd0 vm: fix a null dereference on out-of-memory
. also make other out-of-memory conditions less fatal
	. add a test case for a user program using all the memory
	  it can
	. remove some diagnostic prints for situations that are normal
	  when running out of memory so running the test isn't noisy
2012-11-09 18:36:51 +01:00
Ben Gras
fe6e291f59 vm, kernel, top: report memory usage of vm, kernel 2012-09-18 23:43:52 +02:00
Ben Gras
0d1f2e6be2 VM: simplify slab allocator
. only keep a list of non-empty, non-full pages with slab objects
	. simplifies alloc/free operations and reduces list management overhead
2012-09-18 13:17:50 +02:00
Ben Gras
6d7b770761 VM: static data structure for mem allocation
. allocate physical memory using a fixed, pre-allocated bitmap so there
   are no call cycles and it's avilable earlier
2012-09-18 13:17:48 +02:00
Ben Gras
6410f4b5db VM: some sanitycheck fixes
minor fixes to restore SANITYCHECKS
2012-09-18 13:17:45 +02:00
Arun Thomas
7ca1ce7968 VM: Add PTF_READ page table flag 2012-08-17 00:17:52 +02:00
Arun Thomas
697f0d097f Rename sys_vmctl_get_cr3_i386 2012-08-12 23:30:54 +02:00
Ben Gras
b6ea15115c kernel: facility for user-visible memory
. map all objects named usermapped_*.o with globally visible
	  pages; usermapped_glo_*.o with the VM 'global' bit on, i.e.
	  permanently in tlb (very scarce resource!)
	. added kinfo, machine, kmessages and loadinfo for a start
	. modified log, tty to make use of the shared messages struct
2012-07-28 20:57:38 +00:00
Ben Gras
cbcdb838f1 various coverity-inspired fixes
. some strncpy/strcpy to strlcpy conversions
	. new <minix/param.h> to avoid including other minix headers
	  that have colliding definitions with library and commands code,
	  causing parse warnings
	. removed some dead code / assignments
2012-07-16 14:00:56 +02:00
Ben Gras
50e2064049 No more intel/minix segments.
This commit removes all traces of Minix segments (the text/data/stack
memory map abstraction in the kernel) and significance of Intel segments
(hardware segments like CS, DS that add offsets to all addressing before
page table translation). This ultimately simplifies the memory layout
and addressing and makes the same layout possible on non-Intel
architectures.

There are only two types of addresses in the world now: virtual
and physical; even the kernel and processes have the same virtual
address space. Kernel and user processes can be distinguished at a
glance as processes won't use 0xF0000000 and above.

No static pre-allocated memory sizes exist any more.

Changes to booting:
        . The pre_init.c leaves the kernel and modules exactly as
          they were left by the bootloader in physical memory
        . The kernel starts running using physical addressing,
          loaded at a fixed location given in its linker script by the
          bootloader.  All code and data in this phase are linked to
          this fixed low location.
        . It makes a bootstrap pagetable to map itself to a
          fixed high location (also in linker script) and jumps to
          the high address. All code and data then use this high addressing.
        . All code/data symbols linked at the low addresses is prefixed by
          an objcopy step with __k_unpaged_*, so that that code cannot
          reference highly-linked symbols (which aren't valid yet) or vice
          versa (symbols that aren't valid any more).
        . The two addressing modes are separated in the linker script by
          collecting the unpaged_*.o objects and linking them with low
          addresses, and linking the rest high. Some objects are linked
          twice, once low and once high.
        . The bootstrap phase passes a lot of information (e.g. free memory
          list, physical location of the modules, etc.) using the kinfo
          struct.
        . After this bootstrap the low-linked part is freed.
        . The kernel maps in VM into the bootstrap page table so that VM can
          begin executing. Its first job is to make page tables for all other
          boot processes. So VM runs before RS, and RS gets a fully dynamic,
          VM-managed address space. VM gets its privilege info from RS as usual
          but that happens after RS starts running.
        . Both the kernel loading VM and VM organizing boot processes happen
	  using the libexec logic. This removes the last reason for VM to
	  still know much about exec() and vm/exec.c is gone.

Further Implementation:
        . All segments are based at 0 and have a 4 GB limit.
        . The kernel is mapped in at the top of the virtual address
          space so as not to constrain the user processes.
        . Processes do not use segments from the LDT at all; there are
          no segments in the LDT any more, so no LLDT is needed.
        . The Minix segments T/D/S are gone and so none of the
          user-space or in-kernel copy functions use them. The copy
          functions use a process endpoint of NONE to realize it's
          a physical address, virtual otherwise.
        . The umap call only makes sense to translate a virtual address
          to a physical address now.
        . Segments-related calls like newmap and alloc_segments are gone.
        . All segments-related translation in VM is gone (vir2map etc).
        . Initialization in VM is simpler as no moving around is necessary.
        . VM and all other boot processes can be linked wherever they wish
          and will be mapped in at the right location by the kernel and VM
          respectively.

Other changes:
        . The multiboot code is less special: it does not use mb_print
          for its diagnostics any more but uses printf() as normal, saving
          the output into the diagnostics buffer, only printing to the
          screen using the direct print functions if a panic() occurs.
        . The multiboot code uses the flexible 'free memory map list'
          style to receive the list of free memory if available.
        . The kernel determines the memory layout of the processes to
          a degree: it tells VM where the kernel starts and ends and
          where the kernel wants the top of the process to be. VM then
          uses this entire range, i.e. the stack is right at the top,
          and mmap()ped bits of memory are placed below that downwards,
          and the break grows upwards.

Other Consequences:
        . Every process gets its own page table as address spaces
          can't be separated any more by segments.
        . As all segments are 0-based, there is no distinction between
          virtual and linear addresses, nor between userspace and
          kernel addresses.
        . Less work is done when context switching, leading to a net
          performance increase. (8% faster on my machine for 'make servers'.)
	. The layout and configuration of the GDT makes sysenter and syscall
	  possible.
2012-07-15 22:30:15 +02:00
Thomas Veerman
f09c2e014f Use MACHINE_ARCH instead of ARCH 2012-06-18 10:53:35 +00:00
Ben Gras
040362e379 exec() cleanup, generalization, improvement
. make exec() callers (i.e. vfs and rs) determine the
	  memory layout by explicitly reserving regions using
	  mmap() calls on behalf of the exec()ing process,
	  i.e. handling all of the exec logic, thereby eliminating
	  all special exec() knowledge from VM.
	. the new procedure is: clear the exec()ing process
	  first, then call third-party mmap()s to reserve memory, then
	  copy the executable file section contents in, all using callbacks
	  tailored to the caller's way of starting an executable
	. i.e. no more explicit EXEC_NEWMEM-style calls in PM or VM
	  as with rigid 2-section arguments
	. this naturally allows generalizing exec() by simply loading
	  all ELF sections
	. drop/merge of lots of duplicate exec() code into libexec
	. not copying the code sections to vfs and into the executable
	  again is a measurable performance improvement (about 3.3% faster
	  for 'make' in src/servers/)
2012-06-07 15:15:01 +02:00
Ben Gras
7336a67dfe retire PUBLIC, PRIVATE and FORWARD 2012-03-25 21:58:14 +02:00
Arun Thomas
5d958f894f servers: fix clang warnings in iso9660fs and vm 2012-02-17 16:48:40 +01:00
Tomas Hruby
974452d4dd VM - clear vminhibit iff it was set 2012-01-13 11:30:01 +00:00
Ben Gras
7cd4002083 vm: clear map cache after kernel requests
. fixes a dirty tlb situation (i.e. random crashes)
	  on some hardware, seemingly new intel architectures
	  (e.g. my desktop i7 machine)
2012-01-11 01:15:35 +01:00
Thomas Veerman
8c9e62b299 Revert to using flushtlb instead of invlpg
Somehow the usage of invlpg causes Minix to crash on Qemu. Reverting
temporarily until we have a proper fix.
2011-09-27 15:15:51 +00:00
Ben Gras
35cc7fbeb8 kernel: invlpg facility
. only use for single-page invalidations initially
	. shows tiny but statistically significant performance
	  improvement; will be more helpful in certain VM debug
	  modes
2011-08-12 13:08:27 +00:00
Gianluca Guida
cc17b27a2b Build NetBSD libc library in world in ELF mode.
3 sets of libraries are built now:
  . ack: all libraries that ack can compile (/usr/lib/i386/)
  . clang+elf: all libraries with minix headers (/usr/lib/)
  . clang+elf: all libraries with netbsd headers (/usr/netbsd/)

Once everything can be compiled with netbsd libraries and headers, the
/usr/netbsd hierarchy will be obsolete and its libraries compiled with
netbsd headers will be installed in /usr/lib, and its headers
in /usr/include. (i.e. minix libc and current minix headers set
will be gone.)

To use the NetBSD libc system (libraries + headers) before
it is the default libc, see:
   http://wiki.minix3.org/en/DevelopersGuide/UsingNetBSDCode
This wiki page also documents the maintenance of the patch
files of minix-specific changes to imported NetBSD code.

Changes in this commit:
  . libsys: Add NBSD compilation and create a safe NBSD-based libc.
  . Port rest of libraries (except libddekit) to new header system.
  . Enable compilation of libddekit with new headers.
  . Enable kernel compilation with new headers.
  . Enable drivers compilation with new headers.
  . Port legacy commands to new headers and libc.
  . Port servers to new headers.
  . Add <sys/sigcontext.h> in compat library.
  . Remove dependency file in tree.
  . Enable compilation of common/lib/libc/atomic in libsys
  . Do not generate RCSID strings in libc.
  . Temporarily disable zoneinfo as they are incompatible with NetBSD format
  . obj-nbsd for .gitignore
  . Procfs: use only integer arithmetic. (Antoine Leca)
  . Increase ramdisk size to create NBSD-based images.
  . Remove INCSYMLINKS handling hack.
  . Add nbsd_include/sys/exec_elf.h
  . Enable ELF compilation with NBSD libc.
  . Add 'make nbsdsrc' in tools to download reference NetBSD sources.
  . Automate minix-port.patch creation.
  . Avoid using fstavfs() as it is *extremely* slow and unneeded.
  . Set err() as PRIVATE to avoid name clash with libc.
  . [NBSD] servers/vm: remove compilation warnings.
  . u32 is not a long in NBSD headers.
  . UPDATING info on netbsd hierarchy
  . commands fixes for netbsd libc
2011-06-24 11:46:30 +02:00
Erik van der Kouwe
0cb22acbda Fix VM -Wall warnings and enable -Werror 2011-06-01 11:30:58 +02:00
Arun Thomas
25a790a631 VM and kernel support for ELF 2011-02-26 23:00:55 +00:00
Ben Gras
c7b2b11ec2 vm: only suspend and resume process with vmctls in pt_writemap in smp mode 2011-02-11 14:56:52 +00:00
Tomas Hruby
6513d20744 SMP - Process is stopped when VM modifies the page tables
- RTS_VMINHIBIT flag is used to stop process while VM is fiddling with
  its pagetables

- more generic way of sending synchronous scheduling events among cpus

- do the x-cpu smp sched calls only if the target process is runnable.
  If it is not, it cannot be running and it cannot become runnable
  this CPU holds the BKL
2010-09-15 14:11:12 +00:00
Arun Thomas
9a21d1a2fd Macros for symbols used in both ASM and C
-The macros take care of prepending the leading underscore when
 necessary.
2010-08-17 16:44:07 +00:00
Tomas Hruby
f6f20e1269 Overly restrictive assert
- not true if APIC is enabled
2010-07-21 13:46:29 +00:00
Cristiano Giuffrida
91a83fe265 Crash recovery and live update support for VM. 2010-07-20 23:03:52 +00:00
Cristiano Giuffrida
0d984b36ef Get rid of static spare pages after VM startup. 2010-07-20 21:59:27 +00:00
Cristiano Giuffrida
1b7d95df29 Generalize interface of some VM functions 2010-07-20 02:08:28 +00:00
Ben Gras
5e2c84b119 vm: allow readonly pages when sanity checking when they're supposed to be rw. 2010-07-19 18:19:16 +00:00
Ben Gras
b4bea1bfcb vm: fixes for clang warnings 2010-07-05 13:58:57 +00:00
Arun Thomas
c0c8d25799 Rename mkfiles from minix.*.mk to bsd.*.mk
Makes things easier for pkgsrc
2010-06-25 18:29:09 +00:00
Cristiano Giuffrida
5c7d64b981 No malloc() in VM. 2010-06-25 12:48:56 +00:00
Erik van der Kouwe
1f11a57141 Oops, last commit included more than was intended 2010-05-20 08:07:47 +00:00
Erik van der Kouwe
5f15ec05b2 More system processes, this was not enough for the release script to run on some configurations 2010-05-20 08:05:07 +00:00
Ben Gras
c5c25e7abc kernel/vm: change pde table info from single buffer to explicit per-process.
makes code in kernel more readable, and allows better sanity checking on
using the pde info.
2010-05-12 08:31:05 +00:00
Tomas Hruby
f51eea4b32 Changed pagefault delivery to VM
this patch changes the way pagefaults are delivered to VM. It adopts
the same model as the out-of-quantum messages sent by kernel to a
scheduler.

- everytime a userspace pagefault occurs, kernel creates a message
  which is sent to VM on behalf of the faulting process

- the process is blocked on delivery to VM in the standard IPC code
  instead of waiting in a spacial in-kernel queue (stack) and is not
  runnable until VM tell kernel that the pagefault is resolved and is
  free to clear the RTS_PAGEFAULT flag.

- VM does not need call kernel and poll the pagefault information
  which saves many (1/2?) calls and kernel calls that return "no more
  data"

- VM notification by kernel does not need to use signals

- each entry in proc table is by 12 bytes smaller (~3k save)
2010-04-26 23:21:26 +00:00
Ben Gras
e0792d72d7 vm: util.S not used currently; leave it out. 2010-04-13 15:02:32 +00:00