Commit graph

229 commits

Author SHA1 Message Date
Ben Gras
0cfff08e56 libexec: mmap support, prealloc variants
In libexec, split the memory allocation method into cleared and
non-cleared. Cleared gives zeroed memory, non-cleared gives 'junk'
memory (that will be overwritten anyway, and so needn't be cleared)
that is faster to get.

Also introduce the 'memmap' method that can be used, if available,
to map code and data from executables into a process using the
third-party mmap() mode.

Change-Id: I26694fd3c21deb8b97e01ed675dfc14719b0672b
2013-04-24 10:18:16 +00:00
Ben Gras
49eb1f4806 vm: new secondary cache code
Primary purpose of change: to support the mmap implementation, VM must
know both (a) about some block metadata for FS cache blocks, i.e.
inode numbers and inode offsets where applicable; and (b) know about
*all* cache blocks, i.e.  also of the FS primary caches and not just
the blocks that spill into the secondary one. This changes the
interface and VM data structures.

This change is only for the interface (libminixfs) and VM data
structures; the filesystem code is unmodified, so although the
secondary cache will be used as normal, blocks will not be annotated
with inode information until the FS is modified to provide this
information. Until it is modified, mmap of files will fail gracefully
on such filesystems.

This is indicated to VFS/VM by returning ENOSYS for REQ_PEEK.

Change-Id: I1d2df6c485e6c5e89eb28d9055076cc02629594e
2013-04-24 10:18:16 +00:00
Ben Gras
7421728360 VM: memtype fix
Memory types in VM are described by methods. Each mapped region has
a type, and all pages instantiated get that type on creation.
Individual page types has to be able to change though. This commit
changes the code to use the memory types of the individual pages,
where appropriate, instead of just the higher-level region, in case
it has changed. This is needed to e.g. support future copy-on-write
MAP_PRIVATE mmap modes.

Change-Id: I5523db14ac036ec774a54392fb67f9acb8725731
2013-04-24 10:18:15 +00:00
Ben Gras
9e88c40e40 VM: 64-bit mmap()
Some (backwards-compatible) changes in mmap() call message fields
that allow for a 64-bit offset. minix_mmap() takes an off_t and
minix_mmap64() takes a u64_t. Some mmap() work in VM goes into a
separate function, using the new fields, so that that can be re-used
when files are to be mapped (future commit).

Change-Id: Ifb77a90b593dd3c33cf81b396068e4da1ec5fb1c
2013-04-24 10:18:15 +00:00
Ben Gras
adf2032bc0 vm: remove secondary cache code
This commit removes the secondary cache code implementation from
VM and its usage from libminixfs. It is to be replaced by a new
implementation.

Change-Id: I8fa3af06330e7604c7e0dd4cbe39d3ce353a05b1
2013-04-24 10:18:10 +00:00
Ben Gras
32a4e0d84d many new tests
. test70: regression test for m_out vfs race condition

The following tests use testcache.c to generate test i/o
patterns, generate random write data and verify the reads.

	. test71: blackbox full-stack test of FS operation, testing
	  using the regular VFS interface crazy i/o patterns
	  with various working set sizes, triggering only
	  primary cache, also secondary cache, and finally
	  disk i/o and verifying contents all the time
	. test72: unit test of libminixfs, implementing
	  functions it needs from -lsys and -lblockdriver
	  and the client in order to simulate a working
	  cache client and backend environment.
	. test73: blackbox test of secondary vm cache in isolation

Change-Id: I1287e9753182b8719e634917ad158e3c1e079ceb
2013-04-19 16:21:48 +00:00
Thomas Cort
e67fc5771d libc: add clock_getres()/clock_gettime() system calls.
In order to make it more clear that ticks should be used for timers
and realtime should be used for timestamps / displaying the date/time,
getuptime() was renamed to getticks() and getuptime2() was renamed to
getuptime().

Servers, drivers, libraries, tests, etc that use getuptime()/getuptime2()
have been updated. In instances where a realtime was calculated, the
calculation was changed to use realtime.

System calls clock_getres() and clock_gettime() were added to PM/libc.
2013-04-04 15:04:53 +02:00
Ben Gras
0022ee2cb5 VM: ARM/I386 PDE/PTE macro unification
. further reduces special cases in pagetable.c

Change-Id: Ifd7dd9ee10ca618a47bb4a372467205fbe6da90e
2013-03-14 13:55:04 +00:00
Ben Gras
718a9ef472 VM: pagetable.c: harmonize x86/ARM findhole() code
ARM needs to be able to find N consecutive free slots; use same
code for x86.

Change-Id: Ic79677961c8adfca2aeb5385962942ae0d76867c
2013-03-13 12:09:32 +00:00
Ben Gras
90d777f053 VM: pagequeue data structure
. data structure that automatically keeps a set
	  of pages in reserve, to replace sparepages and
	  possibly re-used in the future for similar situations,
	  e.g. if in-filesystem-cache block eviction is
	  implemented and FS asks for a new block

Change-Id: I149d46c14b9c8e75df16cb94e08907f008c339a6
2013-03-13 12:09:32 +00:00
Ben Gras
449ed17833 VM: shared memory pagefault fix
. if there is no memory there, it's not writable; this
	  check bug by the shared memory's writable() method causes
	  pagefaults not to be handled  at all in certain situations,
	  triggering an assert() in pt_writemap()
	. added some assert()s to catch this and similar situations
	  in the future

Change-Id: Ife89bfab4f9a3aa7bf4e33dfb0b13b89dcd5bb94
2013-03-07 10:40:22 +01:00
Lionel Sambuc
8f3fbf7cc1 Cleanup: Remove minix.bootprog.mk
The build system distinction between "bootprog" and "service" is
meaningless as boot programs are standard services.

As minix.service.mk simply imports minix.bootprog.mk, reduce confusion
by removing minix.bootprog.mk and placing the rules in minix.service.mk.

Change-Id: I4056b1e574bed59a8c890239b41b1a7c7cad63e8
2013-03-06 11:56:56 +01:00
Lionel Sambuc
8e4736f2df Removing obsolete _MINIX define
Change-Id: Id33ac7e973d1c0e249b690fe44a597474fac6076
2013-02-26 09:44:20 +00:00
Lionel Sambuc
e4fa9802cb ARM: Enable caches
First round, some more optimizations are possible and should be
activated.

Change-Id: I3b7dee7c82fbffd823a08bec1c5d5ebcf769f92f
2013-02-18 09:08:26 +01:00
Kees Jongenburger
f240e1eaf5 vm:Display boot process name upon failure to load or execute.
Change-Id: I80fdaca09ae255622083b16dee72715f009dc0ee
2013-02-16 11:20:45 +01:00
Ben Gras
3bc6d7df06 impove memory accounting
. the total amount of memory in the system didn't include the memory
	  used by the boot-time modules and some dynamic allocation by the
	  kernel at boot time (to map in VM). especially apparent on our
	  ARM board with 'only' 512MB of memory and a huge ramdisk.
	. also: *add* the VM loaded module to the freelist after it has
	  been allocated for & mapped in instead of cutting it *out* of the
	  freelist! so we get a few more MB free..

Change-Id: If37ac32b21c9d38610830e21421264da4f20bc4f
2013-02-11 19:31:57 +01:00
Ben Gras
d1df256de9 VM: slight pagedir mapping generalization
. allow any number of pde's used for pagedir mapping
	. allows >1024 NR_PROCS on x86, >64 on ARM
	. allows NR_PROCS to be the same in both cases
	. also cleanup: allocating spare PDE's is not necessary
	  throw that function out

Change-Id: Ibb8f8cf6e7db6a4d6384b6911d1a3f3f5e5d8256
2013-02-10 21:50:34 +01:00
Ben Gras
298b41b523 libexec: detect short files
if an exec() fails partway through reading in the sections, the target
process is already gone and a defunct process remains. sanity checking
the binary beforehand helps that.

test10 mutilates binaries and exec()s them on purpose; making an exec()
fail cleanly in such cases seems like acceptable behaviour.

fixes test10 on ARM.

Change-Id: I1ed9bb200ce469d4d349073cadccad5503b2fcb0
2013-02-04 12:04:35 +01:00
Ben Gras
b7ea9f3fd1 vm: fix sanity checks on arm
The 'polarity' of the RW bit is inversed on ARM, causing one
of the sanity check compensations to fail. ARM now runs basic
stuff with sanity checks passing.

Change-Id: Iee28ab63e430e759f204eeb204b24c301d5ea3c9
2013-02-01 16:45:56 +00:00
Ben Gras
08cb986d3f vm: fix kernel-requested mappings for arm
. make vm tell kernel virtual locations of mappings
	. makes _minix_kerninfo feature work
	. fix for mappings being larger than what 1 pde can address
	  (e.g. devices memory requested on arm)
	. still requires a special case for devices memory for the
	  kernel, which has to switch to virtual addressing

Change-Id: I2e94090aa432346fa4da0edeba72f0b7406c2ad7
2013-01-29 17:57:04 +00:00
Thomas Veerman
b3250450fe VM: ARM needs more spare pages 2013-01-25 17:07:02 +00:00
Thomas Veerman
dad3b4e8c8 ARM: Provide linker script for VM 2013-01-25 17:07:01 +00:00
Lionel Sambuc
b1c4ba4ab6 ARM updates
Due to the ABI we are using we have to use the earm architecture
moniker for the build system to behave correctly. This involves
then some headers to move around.

There is also a few related Makefile updates as well as minor
source code corrections.
2013-01-17 10:03:58 +01:00
Lionel Sambuc
0ac803292e Fix warnings trivial warnings.
Fix warnings about:
 . Unused variables
 . format mismatch in printf/scanf format string and arguments
 . Missing parenthesis around assignment as truth values
 . Clang warnings anout unknown GCC pragma
2013-01-14 11:44:31 +01:00
Ben Gras
ecf9b40841 vm: fix region reporting bug
. logic was backwards, causing no reporting ever
	. this broke coredumps ('Warning: Program has too many regions')
2013-01-09 19:50:52 +00:00
Ben Gras
b258ab0e66 vm: restore stacktrace on SIGSEGV 2013-01-08 15:47:37 +00:00
Kees Jongenburger
fce5ff6513 vm: Fix sparepage resource leak.
Fix a sparepage resource leak by preventing to call
vm_getsparepage twice.

Change-Id: I06557d47b90a7cca74e0a86921c2579f9618685d
2013-01-07 09:12:09 +01:00
Ben Gras
29edcad310 vm: replace phys avl by array
. make vm be able to use malloc() by overriding brk()
   and minix_mmap() functions
 . phys regions can then be malloc()ed and free()d instead
   of being in an avl tree, which is slightly faster
 . 'offset' field in phys_region can go too (offset is implied
   by position in array) but leads to bigger code changes
2012-12-26 16:14:41 +00:00
Ben Gras
6cf98dbe00 vm: reduce noise in merged pagetable.c 2012-11-09 19:00:46 +01:00
Ben Gras
3771a0833d vm: merge i386 and arm pagetable code 2012-11-09 18:46:03 +01:00
Ben Gras
b1da7fafd0 vm: fix a null dereference on out-of-memory
. also make other out-of-memory conditions less fatal
	. add a test case for a user program using all the memory
	  it can
	. remove some diagnostic prints for situations that are normal
	  when running out of memory so running the test isn't noisy
2012-11-09 18:36:51 +01:00
Ben Gras
196021cd82 drop safemap code 2012-10-30 13:55:42 +01:00
Ben Gras
aefc6db005 vm: fix potential null deref 2012-10-24 19:47:47 +02:00
David van Moolenbroek
46fca87456 VM: munmap fix 2012-10-13 19:08:28 +02:00
Ben Gras
d343041caa VM: make mapping types explicit
Introduce explicit abstractions for different mapping types,
handling the instantiation, forking, pagefaults and freeing of
anonymous memory, direct physical mappings, shared memory and
physically contiguous anonymous memory as separate types, making
region.c more generic.

Also some other genericification like merging the 3 munmap cases
into one.

COW and SMAP safemap code is still implicit in region.c.
2012-10-12 14:52:01 +02:00
Ben Gras
fd4ddef49a VM: munmap used by VM for itself is no longer used 2012-10-12 14:51:35 +02:00
Arun Thomas
471a03a362 ARM support for kernel and vm 2012-10-07 21:38:03 -04:00
Ben Gras
2d72cbec41 SYSENTER/SYSCALL support
. add cpufeature detection of both
	. use it for both ipc and kernelcall traps, using a register
	  for call number
	. SYSENTER/SYSCALL does not save any context, therefore userland
	  has to save it
	. to accomodate multiple kernel entry/exit types, the entry
	  type is recorded in the process struct. hitherto all types
	  were interrupt (soft int, exception, hard int); now SYSENTER/SYSCALL
	  is new, with the difference that context is not fully restored
	  from proc struct when running the process again. this can't be
	  done as some information is missing.
	. complication: cases in which the kernel has to fully change
	  process context (i.e. sigreturn). in that case the exit type
	  is changed from SYSENTER/SYSEXIT to soft-int (i.e. iret) and
	  context is fully restored from the proc struct. this does mean
	  the PC and SP must change, as the sysenter/sysexit userland code
	  will otherwise try to restore its own context. this is true in the
	  sigreturn case.
	. override all usage by setting libc_ipc=1
2012-09-24 15:53:43 +02:00
Ben Gras
8a3b6ca3bb remove unused <tools.h> 2012-09-20 12:24:22 +02:00
Ben Gras
b16aacc7f3 vm: fix failed alloc condition 2012-09-19 22:24:56 +02:00
Ben Gras
bc4c07f4f1 coverity appeasement - redundant check 2012-09-19 17:19:57 +02:00
Ben Gras
25817b0854 vm: change NO_MEM to a more impossible value
fixes an assert() firing when starting X. thanks to the report by pikpik.

	. NO_MEM was 0, which is actually an existing piece
	  of physical memory. it can't be allocated because it's reserved
	  for bios data (by the kernel), but it can be mapped in (e.g.
	  by X), causing sanity check disaster.
	. NONCONTIGUOUS is also obsolete as all allocations are single-page
	  now, i.e. NONCONTIGUOUS is really the default and only mode.
2012-09-19 15:31:36 +02:00
Ben Gras
fe6e291f59 vm, kernel, top: report memory usage of vm, kernel 2012-09-18 23:43:52 +02:00
Ben Gras
aa82e375c6 VM: remove dead code 2012-09-18 18:40:57 +02:00
Ben Gras
d526f1a0db some coverity fixes. 2012-09-18 15:11:51 +02:00
Ben Gras
ddf1981004 VM: restore >4k secondary cache functionality
. by storing length in the yielded blocks node again
2012-09-18 13:17:52 +02:00
Ben Gras
ed1af3c86c VM: full munmap
complete munmap implementation; single-page references made
a general munmap() implementation possible to write cleanly.

	. memory: let the MIOCRAMSIZE ioctl set the imgrd device
	  size (but only to 0)
	. let the ramdisk command set sizes to 0
	. use this command to set /dev/imgrd to 0 after mounting /usr
	  in /etc/rc, so the boot time ramdisk is freed (about 4MB
	  currently)
2012-09-18 13:17:52 +02:00
Ben Gras
16c3870b2e VM: abstract datastructures a bit
. a little less duplication in region.c
2012-09-18 13:17:51 +02:00
Ben Gras
0d1f2e6be2 VM: simplify slab allocator
. only keep a list of non-empty, non-full pages with slab objects
	. simplifies alloc/free operations and reduces list management overhead
2012-09-18 13:17:50 +02:00
Ben Gras
19e6dad47b VM: only single page chunks
. only reference single pages in process data structures
   to simplify page faults, copy-on-write, etc.
 . this breaks the secondary cache for objects that are
   not one-page-sized; restored in a next commit
2012-09-18 13:17:49 +02:00