Commit graph

214 commits

Author SHA1 Message Date
Ben Gras
3bc6d7df06 impove memory accounting
. the total amount of memory in the system didn't include the memory
	  used by the boot-time modules and some dynamic allocation by the
	  kernel at boot time (to map in VM). especially apparent on our
	  ARM board with 'only' 512MB of memory and a huge ramdisk.
	. also: *add* the VM loaded module to the freelist after it has
	  been allocated for & mapped in instead of cutting it *out* of the
	  freelist! so we get a few more MB free..

Change-Id: If37ac32b21c9d38610830e21421264da4f20bc4f
2013-02-11 19:31:57 +01:00
Ben Gras
d1df256de9 VM: slight pagedir mapping generalization
. allow any number of pde's used for pagedir mapping
	. allows >1024 NR_PROCS on x86, >64 on ARM
	. allows NR_PROCS to be the same in both cases
	. also cleanup: allocating spare PDE's is not necessary
	  throw that function out

Change-Id: Ibb8f8cf6e7db6a4d6384b6911d1a3f3f5e5d8256
2013-02-10 21:50:34 +01:00
Ben Gras
298b41b523 libexec: detect short files
if an exec() fails partway through reading in the sections, the target
process is already gone and a defunct process remains. sanity checking
the binary beforehand helps that.

test10 mutilates binaries and exec()s them on purpose; making an exec()
fail cleanly in such cases seems like acceptable behaviour.

fixes test10 on ARM.

Change-Id: I1ed9bb200ce469d4d349073cadccad5503b2fcb0
2013-02-04 12:04:35 +01:00
Ben Gras
b7ea9f3fd1 vm: fix sanity checks on arm
The 'polarity' of the RW bit is inversed on ARM, causing one
of the sanity check compensations to fail. ARM now runs basic
stuff with sanity checks passing.

Change-Id: Iee28ab63e430e759f204eeb204b24c301d5ea3c9
2013-02-01 16:45:56 +00:00
Ben Gras
08cb986d3f vm: fix kernel-requested mappings for arm
. make vm tell kernel virtual locations of mappings
	. makes _minix_kerninfo feature work
	. fix for mappings being larger than what 1 pde can address
	  (e.g. devices memory requested on arm)
	. still requires a special case for devices memory for the
	  kernel, which has to switch to virtual addressing

Change-Id: I2e94090aa432346fa4da0edeba72f0b7406c2ad7
2013-01-29 17:57:04 +00:00
Thomas Veerman
b3250450fe VM: ARM needs more spare pages 2013-01-25 17:07:02 +00:00
Thomas Veerman
dad3b4e8c8 ARM: Provide linker script for VM 2013-01-25 17:07:01 +00:00
Lionel Sambuc
b1c4ba4ab6 ARM updates
Due to the ABI we are using we have to use the earm architecture
moniker for the build system to behave correctly. This involves
then some headers to move around.

There is also a few related Makefile updates as well as minor
source code corrections.
2013-01-17 10:03:58 +01:00
Lionel Sambuc
0ac803292e Fix warnings trivial warnings.
Fix warnings about:
 . Unused variables
 . format mismatch in printf/scanf format string and arguments
 . Missing parenthesis around assignment as truth values
 . Clang warnings anout unknown GCC pragma
2013-01-14 11:44:31 +01:00
Ben Gras
ecf9b40841 vm: fix region reporting bug
. logic was backwards, causing no reporting ever
	. this broke coredumps ('Warning: Program has too many regions')
2013-01-09 19:50:52 +00:00
Ben Gras
b258ab0e66 vm: restore stacktrace on SIGSEGV 2013-01-08 15:47:37 +00:00
Kees Jongenburger
fce5ff6513 vm: Fix sparepage resource leak.
Fix a sparepage resource leak by preventing to call
vm_getsparepage twice.

Change-Id: I06557d47b90a7cca74e0a86921c2579f9618685d
2013-01-07 09:12:09 +01:00
Ben Gras
29edcad310 vm: replace phys avl by array
. make vm be able to use malloc() by overriding brk()
   and minix_mmap() functions
 . phys regions can then be malloc()ed and free()d instead
   of being in an avl tree, which is slightly faster
 . 'offset' field in phys_region can go too (offset is implied
   by position in array) but leads to bigger code changes
2012-12-26 16:14:41 +00:00
Ben Gras
6cf98dbe00 vm: reduce noise in merged pagetable.c 2012-11-09 19:00:46 +01:00
Ben Gras
3771a0833d vm: merge i386 and arm pagetable code 2012-11-09 18:46:03 +01:00
Ben Gras
b1da7fafd0 vm: fix a null dereference on out-of-memory
. also make other out-of-memory conditions less fatal
	. add a test case for a user program using all the memory
	  it can
	. remove some diagnostic prints for situations that are normal
	  when running out of memory so running the test isn't noisy
2012-11-09 18:36:51 +01:00
Ben Gras
196021cd82 drop safemap code 2012-10-30 13:55:42 +01:00
Ben Gras
aefc6db005 vm: fix potential null deref 2012-10-24 19:47:47 +02:00
David van Moolenbroek
46fca87456 VM: munmap fix 2012-10-13 19:08:28 +02:00
Ben Gras
d343041caa VM: make mapping types explicit
Introduce explicit abstractions for different mapping types,
handling the instantiation, forking, pagefaults and freeing of
anonymous memory, direct physical mappings, shared memory and
physically contiguous anonymous memory as separate types, making
region.c more generic.

Also some other genericification like merging the 3 munmap cases
into one.

COW and SMAP safemap code is still implicit in region.c.
2012-10-12 14:52:01 +02:00
Ben Gras
fd4ddef49a VM: munmap used by VM for itself is no longer used 2012-10-12 14:51:35 +02:00
Arun Thomas
471a03a362 ARM support for kernel and vm 2012-10-07 21:38:03 -04:00
Ben Gras
2d72cbec41 SYSENTER/SYSCALL support
. add cpufeature detection of both
	. use it for both ipc and kernelcall traps, using a register
	  for call number
	. SYSENTER/SYSCALL does not save any context, therefore userland
	  has to save it
	. to accomodate multiple kernel entry/exit types, the entry
	  type is recorded in the process struct. hitherto all types
	  were interrupt (soft int, exception, hard int); now SYSENTER/SYSCALL
	  is new, with the difference that context is not fully restored
	  from proc struct when running the process again. this can't be
	  done as some information is missing.
	. complication: cases in which the kernel has to fully change
	  process context (i.e. sigreturn). in that case the exit type
	  is changed from SYSENTER/SYSEXIT to soft-int (i.e. iret) and
	  context is fully restored from the proc struct. this does mean
	  the PC and SP must change, as the sysenter/sysexit userland code
	  will otherwise try to restore its own context. this is true in the
	  sigreturn case.
	. override all usage by setting libc_ipc=1
2012-09-24 15:53:43 +02:00
Ben Gras
8a3b6ca3bb remove unused <tools.h> 2012-09-20 12:24:22 +02:00
Ben Gras
b16aacc7f3 vm: fix failed alloc condition 2012-09-19 22:24:56 +02:00
Ben Gras
bc4c07f4f1 coverity appeasement - redundant check 2012-09-19 17:19:57 +02:00
Ben Gras
25817b0854 vm: change NO_MEM to a more impossible value
fixes an assert() firing when starting X. thanks to the report by pikpik.

	. NO_MEM was 0, which is actually an existing piece
	  of physical memory. it can't be allocated because it's reserved
	  for bios data (by the kernel), but it can be mapped in (e.g.
	  by X), causing sanity check disaster.
	. NONCONTIGUOUS is also obsolete as all allocations are single-page
	  now, i.e. NONCONTIGUOUS is really the default and only mode.
2012-09-19 15:31:36 +02:00
Ben Gras
fe6e291f59 vm, kernel, top: report memory usage of vm, kernel 2012-09-18 23:43:52 +02:00
Ben Gras
aa82e375c6 VM: remove dead code 2012-09-18 18:40:57 +02:00
Ben Gras
d526f1a0db some coverity fixes. 2012-09-18 15:11:51 +02:00
Ben Gras
ddf1981004 VM: restore >4k secondary cache functionality
. by storing length in the yielded blocks node again
2012-09-18 13:17:52 +02:00
Ben Gras
ed1af3c86c VM: full munmap
complete munmap implementation; single-page references made
a general munmap() implementation possible to write cleanly.

	. memory: let the MIOCRAMSIZE ioctl set the imgrd device
	  size (but only to 0)
	. let the ramdisk command set sizes to 0
	. use this command to set /dev/imgrd to 0 after mounting /usr
	  in /etc/rc, so the boot time ramdisk is freed (about 4MB
	  currently)
2012-09-18 13:17:52 +02:00
Ben Gras
16c3870b2e VM: abstract datastructures a bit
. a little less duplication in region.c
2012-09-18 13:17:51 +02:00
Ben Gras
0d1f2e6be2 VM: simplify slab allocator
. only keep a list of non-empty, non-full pages with slab objects
	. simplifies alloc/free operations and reduces list management overhead
2012-09-18 13:17:50 +02:00
Ben Gras
19e6dad47b VM: only single page chunks
. only reference single pages in process data structures
   to simplify page faults, copy-on-write, etc.
 . this breaks the secondary cache for objects that are
   not one-page-sized; restored in a next commit
2012-09-18 13:17:49 +02:00
Ben Gras
6d7b770761 VM: static data structure for mem allocation
. allocate physical memory using a fixed, pre-allocated bitmap so there
   are no call cycles and it's avilable earlier
2012-09-18 13:17:48 +02:00
Ben Gras
2cb560297c VM: remove unused dma memory support functions from vm
. unused calls / data structures
2012-09-18 13:17:47 +02:00
Ben Gras
8821c73a9e VM: forget about 'holes'
. unused data structures and code
2012-09-18 13:17:46 +02:00
Ben Gras
6410f4b5db VM: some sanitycheck fixes
minor fixes to restore SANITYCHECKS
2012-09-18 13:17:45 +02:00
Ben Gras
053fa581b5 vm: remove stack handling for signals
. moved to the kernel as the handling was only
	  reading it; the kernel may as well write it too
2012-08-29 17:31:38 +02:00
Ben Gras
e4ac80eb60 various warning/errorwarning fixes for gcc47
. warnings (sometimes promoted to errors) in servers/ and kernel/
 . -Os for ext2 boot module to make it small enough
2012-08-27 16:19:18 +02:00
Arun Thomas
7ca1ce7968 VM: Add PTF_READ page table flag 2012-08-17 00:17:52 +02:00
Arun Thomas
fc9b3f9bdd VM: allow for 16KB chunk alignment 2012-08-17 00:17:52 +02:00
Arun Thomas
697f0d097f Rename sys_vmctl_get_cr3_i386 2012-08-12 23:30:54 +02:00
Ben Gras
b3f47f5835 vm: ignore RS pin (pre-allocate) requests for now
. done by RS to reduce/remove dependency on VM for recovery
	. RS has the default stack size of 64MB since the nosegments
	  change, using a huge amount of unused memory to pre-allocate
	. ignore these requests until actually required (i.e. being able
	  to survive VM crashes)

Thanks to pikpik for investigating why RS was so huge.
2012-08-08 15:51:10 +02:00
Ben Gras
b6ea15115c kernel: facility for user-visible memory
. map all objects named usermapped_*.o with globally visible
	  pages; usermapped_glo_*.o with the VM 'global' bit on, i.e.
	  permanently in tlb (very scarce resource!)
	. added kinfo, machine, kmessages and loadinfo for a start
	. modified log, tty to make use of the shared messages struct
2012-07-28 20:57:38 +00:00
Ben Gras
cbcdb838f1 various coverity-inspired fixes
. some strncpy/strcpy to strlcpy conversions
	. new <minix/param.h> to avoid including other minix headers
	  that have colliding definitions with library and commands code,
	  causing parse warnings
	. removed some dead code / assignments
2012-07-16 14:00:56 +02:00
Ben Gras
50e2064049 No more intel/minix segments.
This commit removes all traces of Minix segments (the text/data/stack
memory map abstraction in the kernel) and significance of Intel segments
(hardware segments like CS, DS that add offsets to all addressing before
page table translation). This ultimately simplifies the memory layout
and addressing and makes the same layout possible on non-Intel
architectures.

There are only two types of addresses in the world now: virtual
and physical; even the kernel and processes have the same virtual
address space. Kernel and user processes can be distinguished at a
glance as processes won't use 0xF0000000 and above.

No static pre-allocated memory sizes exist any more.

Changes to booting:
        . The pre_init.c leaves the kernel and modules exactly as
          they were left by the bootloader in physical memory
        . The kernel starts running using physical addressing,
          loaded at a fixed location given in its linker script by the
          bootloader.  All code and data in this phase are linked to
          this fixed low location.
        . It makes a bootstrap pagetable to map itself to a
          fixed high location (also in linker script) and jumps to
          the high address. All code and data then use this high addressing.
        . All code/data symbols linked at the low addresses is prefixed by
          an objcopy step with __k_unpaged_*, so that that code cannot
          reference highly-linked symbols (which aren't valid yet) or vice
          versa (symbols that aren't valid any more).
        . The two addressing modes are separated in the linker script by
          collecting the unpaged_*.o objects and linking them with low
          addresses, and linking the rest high. Some objects are linked
          twice, once low and once high.
        . The bootstrap phase passes a lot of information (e.g. free memory
          list, physical location of the modules, etc.) using the kinfo
          struct.
        . After this bootstrap the low-linked part is freed.
        . The kernel maps in VM into the bootstrap page table so that VM can
          begin executing. Its first job is to make page tables for all other
          boot processes. So VM runs before RS, and RS gets a fully dynamic,
          VM-managed address space. VM gets its privilege info from RS as usual
          but that happens after RS starts running.
        . Both the kernel loading VM and VM organizing boot processes happen
	  using the libexec logic. This removes the last reason for VM to
	  still know much about exec() and vm/exec.c is gone.

Further Implementation:
        . All segments are based at 0 and have a 4 GB limit.
        . The kernel is mapped in at the top of the virtual address
          space so as not to constrain the user processes.
        . Processes do not use segments from the LDT at all; there are
          no segments in the LDT any more, so no LLDT is needed.
        . The Minix segments T/D/S are gone and so none of the
          user-space or in-kernel copy functions use them. The copy
          functions use a process endpoint of NONE to realize it's
          a physical address, virtual otherwise.
        . The umap call only makes sense to translate a virtual address
          to a physical address now.
        . Segments-related calls like newmap and alloc_segments are gone.
        . All segments-related translation in VM is gone (vir2map etc).
        . Initialization in VM is simpler as no moving around is necessary.
        . VM and all other boot processes can be linked wherever they wish
          and will be mapped in at the right location by the kernel and VM
          respectively.

Other changes:
        . The multiboot code is less special: it does not use mb_print
          for its diagnostics any more but uses printf() as normal, saving
          the output into the diagnostics buffer, only printing to the
          screen using the direct print functions if a panic() occurs.
        . The multiboot code uses the flexible 'free memory map list'
          style to receive the list of free memory if available.
        . The kernel determines the memory layout of the processes to
          a degree: it tells VM where the kernel starts and ends and
          where the kernel wants the top of the process to be. VM then
          uses this entire range, i.e. the stack is right at the top,
          and mmap()ped bits of memory are placed below that downwards,
          and the break grows upwards.

Other Consequences:
        . Every process gets its own page table as address spaces
          can't be separated any more by segments.
        . As all segments are 0-based, there is no distinction between
          virtual and linear addresses, nor between userspace and
          kernel addresses.
        . Less work is done when context switching, leading to a net
          performance increase. (8% faster on my machine for 'make servers'.)
	. The layout and configuration of the GDT makes sysenter and syscall
	  possible.
2012-07-15 22:30:15 +02:00
Thomas Veerman
f09c2e014f Use MACHINE_ARCH instead of ARCH 2012-06-18 10:53:35 +00:00
Thomas Veerman
f93afa00e9 Remove MINIXSRCDIR and use NETBSDSRCDIR
NETBSDSRCDIR is used all over the place anyway, and this reduces
our diff with NetBSD a little.
2012-06-18 10:53:35 +00:00