Commit graph

1378 commits

Author SHA1 Message Date
Ben Gras
4f3b015903 fix some warnings
Change-Id: I1f3fa582d8e3ddb3c981784dddb13e740102f4f3
2013-05-31 20:57:39 +00:00
Ben Gras
1cc6f4295d vm: handle disappearing process case
Change-Id: Id96759883e4cdb175c79dcef7ef5ff254612101f
2013-05-31 17:31:55 +00:00
Ben Gras
49b9165251 vm: mmap support
. test74 for mmap functionality
	. vm: add a mem_file memory type that specifies an mmap()ped
	  memory range, backed by a file
	. add fdref, an object that keeps track of FD references within
	  VM per process and so knows how to de-duplicate the use of FD's
	  by various mmap()ped ranges; there can be many more than there can
	  be FD's
	. turned off for now, enable with 'filemap=1' as boot option

Change-Id: I640b1126cdaa522a0560301cf6732b7661555672
2013-05-31 15:42:01 +00:00
Ben Gras
f369157d95 pfs, vfs: increase various limits
. pipes in pfs
	. vnodes in vfs
	. thread stack sizes in vfs

Change-Id: Ib27dedd42f57a90821a5b950cd7ea25cb2b42f3f
2013-05-31 15:42:00 +00:00
Ben Gras
33a7ac7557 vfs: mmap support
. libc: add vfs_mmap, a way for vfs to initiate mmap()s.
	  This is a good special case to have as vfs is a slightly
	  different client from regular user processes. It doesn't do it
	  for itself, and has the dev & inode info already so the callback
	  to VFS for the lookup isn't necessary. So it has different info
	  to have to give to VM.
	. libc: also add minix_mmap64() that accepts a 64-bit offset, even
	  though our off_t is still 32 bit now.
	. On exec() time, try to mmap() in the executable if available.
	  (It is not yet available in this commit.)
	. To support mmap(), add do_vm_call that allows VM to lookup
	  (to ino+dev), do i/o from and close FD's on behalf of other
	  processes.

Change-Id: I831551e45a6781c74313c450eb9c967a68505932
2013-05-31 15:42:00 +00:00
Ben Gras
924eb29565 fix inet warnings
Change-Id: Ia6e761d2c649f03a49b2646387f859d6c2a646cb
2013-05-31 14:30:52 +00:00
Ben Gras
2d2a1a077d panic: declare as printf-style-checked
. and related fixes

Change-Id: I5131ac57dc53d8aec8d421a34c5ceea383404d7a
2013-05-31 13:35:25 +00:00
Ben Gras
5507a12d7c vfs: who_p fix
Change-Id: I0e04b6460907f5e67f6c90b2038d296d66b9a414
2013-05-31 09:28:38 +00:00
Ben Gras
4fb3945025 vm: a bit more informative about failed pagefaults
Change-Id: I2b72dfb9291670cb837dfdb279f519892575d4a6
2013-05-29 15:25:45 +00:00
Lionel Sambuc
f0d23d912d servers/pm: Fix EFAULT check for SETGROUPS
The previous test would return EFAULT as soon as the group pointer
was NULL, while it is sensible when the count is also 0.

In that case, the SETGROUP syscall is expected to clear all the
group entries as the new set is empty.

Change-Id: I07b7e1d1f023a52e3035d53f7d9b42b660e039e8
2013-05-28 21:07:04 +02:00
Kees Jongenburger
ffea6706f0 arm:also use 1MB sections for mapping AM335X device memory.
Change-Id: Idc0b285fcbabe8ec4c0be9a600b6a720c0bd3ffc
2013-05-24 15:47:04 +02:00
Kees Jongenburger
1e1ff96aea arm:vm caching fix.
Improve reliability by using write trough cache.
2013-05-16 20:39:20 +02:00
Kees Jongenburger
a3f6529ee2 arm:vm header cleanup. 2013-05-16 20:39:19 +02:00
Ben Gras
4ebb889e7a libsys: panic hook feature
. vfs: use it to dump threads stacks

Change-Id: I7ae3521fc153a407505f11049629e6d4142cf7c7
2013-05-07 17:18:40 +00:00
Ben Gras
85fd078707 tty: non-overlapping code for FKEY_CONTROL
. to decode it in kernel/debug.c

Change-Id: I0d2cc66e87d97a362fa549b364b4d1b0e1225e66
2013-05-01 21:36:43 +00:00
Kees Jongenburger
e6bac75a8b ARM:Rename ARM_BIG_PAGE to ARM_SECTION.
The natural term to use when talking about MINIX big pages on ARM
is SECTION. A section is a level 1 page table entry pointing to
a 1MB area.

Change-Id: I9bd27ca99bc772126c31c27a537b1415db20c4a6
2013-04-29 11:42:26 +02:00
Ben Gras
f497d9b9c4 ext2: annotate cache blocks with inode metadata
. allows mmap() to work on it

Change-Id: I33af8b86ebb408d971478d00ed2caacf15afc7a5
2013-04-26 15:39:07 +00:00
Ben Gras
f0cc010614 libminixfs, mfs, ext2: may re-evaluate cache size
libminixfs may now be informed of changes to the block usage on the
filesystem. if the net change becomes big enough, libminixfs may
resize the cache based on the new usage.

	. update the 2 FSes to provide this information to libminixfs

Change-Id: I158815a11da801fd5572a8de89c9e6c039b82650
2013-04-26 13:57:08 +00:00
Ben Gras
44f34e53d5 VFS: Implement REQ_BPEEK.
This commit introduces a new request type called REQ_BPEEK. It
requests minor device blocks from the FS.  Analogously to REQ_PEEK,
it requests the filesystem to get the requested blocks into its
cache, without actually copying the result anywhere.

Change-Id: If1d06645b0e17553a64b3167091e9d12efeb3d6f
2013-04-24 10:18:16 +00:00
Ben Gras
0cfff08e56 libexec: mmap support, prealloc variants
In libexec, split the memory allocation method into cleared and
non-cleared. Cleared gives zeroed memory, non-cleared gives 'junk'
memory (that will be overwritten anyway, and so needn't be cleared)
that is faster to get.

Also introduce the 'memmap' method that can be used, if available,
to map code and data from executables into a process using the
third-party mmap() mode.

Change-Id: I26694fd3c21deb8b97e01ed675dfc14719b0672b
2013-04-24 10:18:16 +00:00
Ben Gras
6fa5ce74ce MFS: annotate cache blocks with inode metadata
. use lmfs_* cache functions that provide the cache with inode
	  metadata whenever applicable, i.e. tell the cache code which
	  inode number and in-inode offset a particular cache block
	  corresponds to.
	. needed for mmap implementation

Change-Id: Ic7d3c0c49029880f86a31368278722e907bc2896
2013-04-24 10:18:16 +00:00
Ben Gras
49eb1f4806 vm: new secondary cache code
Primary purpose of change: to support the mmap implementation, VM must
know both (a) about some block metadata for FS cache blocks, i.e.
inode numbers and inode offsets where applicable; and (b) know about
*all* cache blocks, i.e.  also of the FS primary caches and not just
the blocks that spill into the secondary one. This changes the
interface and VM data structures.

This change is only for the interface (libminixfs) and VM data
structures; the filesystem code is unmodified, so although the
secondary cache will be used as normal, blocks will not be annotated
with inode information until the FS is modified to provide this
information. Until it is modified, mmap of files will fail gracefully
on such filesystems.

This is indicated to VFS/VM by returning ENOSYS for REQ_PEEK.

Change-Id: I1d2df6c485e6c5e89eb28d9055076cc02629594e
2013-04-24 10:18:16 +00:00
Ben Gras
7421728360 VM: memtype fix
Memory types in VM are described by methods. Each mapped region has
a type, and all pages instantiated get that type on creation.
Individual page types has to be able to change though. This commit
changes the code to use the memory types of the individual pages,
where appropriate, instead of just the higher-level region, in case
it has changed. This is needed to e.g. support future copy-on-write
MAP_PRIVATE mmap modes.

Change-Id: I5523db14ac036ec774a54392fb67f9acb8725731
2013-04-24 10:18:15 +00:00
Ben Gras
9e88c40e40 VM: 64-bit mmap()
Some (backwards-compatible) changes in mmap() call message fields
that allow for a 64-bit offset. minix_mmap() takes an off_t and
minix_mmap64() takes a u64_t. Some mmap() work in VM goes into a
separate function, using the new fields, so that that can be re-used
when files are to be mapped (future commit).

Change-Id: Ifb77a90b593dd3c33cf81b396068e4da1ec5fb1c
2013-04-24 10:18:15 +00:00
Ben Gras
75c5fe4c26 filesystems: return ENOSYS for REQ_PEEK
The filesystems already implement REQ_PEEK, but do not fully
use the new filesystem cache code yet. (Because it isn't committed
yet..) REQ_PEEK should be disabled for them until they do.

This indicates to VFS that they are not annotating their cache
blocks (in VM) with inode number/offset info, and therefore mmap()
shouldn't succeed on any of their files. (Most importantly exec()
won't fallback elegantly otherwise.)

Change-Id: Ic57ee422864b4bbc031eadba32973270907b02fd
2013-04-24 10:18:10 +00:00
Ben Gras
adf2032bc0 vm: remove secondary cache code
This commit removes the secondary cache code implementation from
VM and its usage from libminixfs. It is to be replaced by a new
implementation.

Change-Id: I8fa3af06330e7604c7e0dd4cbe39d3ce353a05b1
2013-04-24 10:18:10 +00:00
Xiaoguang Sun
20e6c9329f Change function prototype to use endpoint_t instead of int 2013-04-23 17:15:15 +02:00
Ben Gras
32a4e0d84d many new tests
. test70: regression test for m_out vfs race condition

The following tests use testcache.c to generate test i/o
patterns, generate random write data and verify the reads.

	. test71: blackbox full-stack test of FS operation, testing
	  using the regular VFS interface crazy i/o patterns
	  with various working set sizes, triggering only
	  primary cache, also secondary cache, and finally
	  disk i/o and verifying contents all the time
	. test72: unit test of libminixfs, implementing
	  functions it needs from -lsys and -lblockdriver
	  and the client in order to simulate a working
	  cache client and backend environment.
	. test73: blackbox test of secondary vm cache in isolation

Change-Id: I1287e9753182b8719e634917ad158e3c1e079ceb
2013-04-19 16:21:48 +00:00
Ben Gras
072d916c1c vfs: fix null deref, pfs: add fchmod()
. vfs read_only() assumes vnode->v_vmnt is non-NULL, but it can
	  be NULL sometimes
	. e.g. fchmod() on UDS triggered NULL deref; add a check and
	  add REQ_CHMOD to pfs so unix domain sockets can be fchmod()ded
	. add to test56

Change-Id: I83c840f101b647516897cc99fcf472116d762012
2013-04-19 17:06:56 +02:00
Ben Gras
cef94e096e vfs: make m_out non-global
m_out is shared between threads as the reply message, and it can happen
results get overwritten by another thread before the reply is sent. This
change

	. makes m_out local to the message handling function,
	  declared on the stack of the caller
	. forces callers of reply() to give it a message, or
	  declare the reply message has no significant fields except
	  for the return code by calling replycode()

Change-Id: Id06300083a63c72c00f34f86a5c7d96e4bbdf9f6
2013-04-12 23:40:38 +00:00
Antoine Leca
9131e98a7d utimens(2) system call
Variant of utime(2) with struct timespec (with ns precision)
instead of time_t values; also allows for tv_nsec members
the values UTIME_NOW (force update to current time) or
UTIME_OMIT (allow to set either atim or mtim independently.)

Provides a superset of utimes(2), futimes(2), lutimes(2),
and futimens(2).
Provides the same subset of utimensat(2) as does NetBSD 6.
Also import utimens() and lutimeNS() from NetBSD-current.
2013-04-12 18:55:39 +00:00
Antoine Leca
4069cef7f9 Subsecond timestamps support for FS
Expand REQ_UTIME to include tv_nsec members
(as in struct timespec) in addition to tv_sec==time_t

Designed with help from David van Moolenbroek
2013-04-12 11:11:59 +02:00
Michael W. Bombardieri
a806c5feb1 make minix lwip make explicit use of 'int'
makes lwip use "unsigned int" instead of "unsigned" since this is
more obvious (i.e. type is not implied).

Change-Id: I852eb80484516e1235241d55be3e15174fa24109
2013-04-11 22:37:30 +02:00
Thomas Cort
f05ad905a8 inet: silence message about exceptions
. inet: silence message about exceptions not being implemented
	  for select(UDP)

This message generates a lot of noise with openntpd. Hide it unless DEBUG is
turned on.

Change-Id: I1527a9ca2583601d6087456062b4f675c80dd711
2013-04-04 15:06:18 +02:00
Thomas Cort
15b3d77268 libc: add adjtime() system call.
Implement the adjtime() system call and add a test for it to test69.
Additionally, install the adjtime.2 and clock_*.2 man pages.
2013-04-04 15:04:54 +02:00
Thomas Cort
516fec97d9 libc: add clock_settime() system call.
This also adds the sys_settime() kernel call which allows for the adjusting
of the clock named realtime in the kernel. The existing sys_stime()
function is still needed for a separate job (setting the boottime). The
boottime is set in the readclock driver. The sys_settime() interface is
meant to be flexible and will support both clock_settime() and adjtime()
when adjtime() is implemented later.

settimeofday() was adjusted to use the clock_settime() interface.

One side note discovered during testing: uptime(1) (part of the last(1)),
uses wtmp to determine boottime (not Minix's times(2)). This leads `uptime`
to report odd results when you set the time to a time prior to boottime.
This isn't a new bug introduced by my changes. It's been there for a while.
2013-04-04 15:04:54 +02:00
Thomas Cort
e67fc5771d libc: add clock_getres()/clock_gettime() system calls.
In order to make it more clear that ticks should be used for timers
and realtime should be used for timestamps / displaying the date/time,
getuptime() was renamed to getticks() and getuptime2() was renamed to
getuptime().

Servers, drivers, libraries, tests, etc that use getuptime()/getuptime2()
have been updated. In instances where a realtime was calculated, the
calculation was changed to use realtime.

System calls clock_getres() and clock_gettime() were added to PM/libc.
2013-04-04 15:04:53 +02:00
Tomas Hruby
a1fa603ed5 lwip (inet's alternative) is compiled by default 2013-03-25 16:51:25 +01:00
Tomas Hruby
9582cbffc5 LWIP - avoid an assert when a driver restarts 2013-03-25 16:51:25 +01:00
Tomas Hruby
00f08c4868 LWIP - avoid crash when an unknown driver come up 2013-03-25 16:51:25 +01:00
Tomas Hruby
f6b5201f64 LWIP - fixed TCP panic when shutting down
- listening sockets do not handle some callbacks
2013-03-25 16:51:25 +01:00
Ambarisha B
2d9009b0ab LWIP - fixed makefiles
- Modified libnetsock/Makefile and servers/lwip/Makefile to
  include ipvX headers

Signed-off-by: Tomas Hruby <tom@minix3.org>
2013-03-25 16:51:25 +01:00
Thomas Veerman
6ee180f5f7 VFS: wikify README
Change-Id: I746f7c8ddabd1e047b8d536df14586c5b1594d55
2013-03-21 15:20:34 +00:00
Ben Gras
4f9139778d vfs: coredump fix: write zeroes for missing memory 2013-03-20 20:05:31 +00:00
David van Moolenbroek
50e46307de Move MINIX reboot definitions into minix/reboot.h
Also fix a buffer overflow in commands/reboot/sh_wall.c.

Change-Id: I3a61057c4f0221d1700e14d44520b4ad17f1dbe1
2013-03-20 16:50:01 +00:00
Ben Gras
0022ee2cb5 VM: ARM/I386 PDE/PTE macro unification
. further reduces special cases in pagetable.c

Change-Id: Ifd7dd9ee10ca618a47bb4a372467205fbe6da90e
2013-03-14 13:55:04 +00:00
Ben Gras
718a9ef472 VM: pagetable.c: harmonize x86/ARM findhole() code
ARM needs to be able to find N consecutive free slots; use same
code for x86.

Change-Id: Ic79677961c8adfca2aeb5385962942ae0d76867c
2013-03-13 12:09:32 +00:00
Ben Gras
90d777f053 VM: pagequeue data structure
. data structure that automatically keeps a set
	  of pages in reserve, to replace sparepages and
	  possibly re-used in the future for similar situations,
	  e.g. if in-filesystem-cache block eviction is
	  implemented and FS asks for a new block

Change-Id: I149d46c14b9c8e75df16cb94e08907f008c339a6
2013-03-13 12:09:32 +00:00
Thomas Veerman
76ddef10da UDS: terminate canonical path string
When you provided a string with junk after the terminating nul to a
UNIX domain socket and used bind(2), the canonical path function would
not properly terminate the new string. This caused VFS to return
ENAMETOOLONG on an otherwise valid path name.

Test case is added to test56.

Change-Id: I883b6be23d9e4ea13c3cee28cbb3726343df037f
2013-03-08 15:42:32 +00:00
Lionel Sambuc
fbd82e76a4 Let the build system manage compilation flags
Do not hardcode warning and optimisation flags, otherwise the
main options (i.e. DBG, CPPFLAGS) will not work as expected.

You can still provide specific default by using DBG?=<value>.

Doing so leaves the opportunity to override the setting from the
commandline, while the default value from the build system is
then ignored for that particular package.

When crosscompiling, and using build.sh, adding -V DBG=<value> has
this same effect as make DBG=<value>.

Change-Id: Ic610e4d33b945acad64571e1431f1814291e2d84
2013-03-08 09:41:49 +01:00
Ben Gras
a9f55a2e46 VFS, FSes: add REQ_PEEK request type
REQ_PEEK behaves just like REQ_READ except that it does not copy
data anywhere, just obtains the blocks from the FS into the cache.

To be used by the future mmap implementation.

Change-Id: I1b56de304f0a7152b69a72c8962d04258adb44f9
2013-03-07 10:57:38 +00:00
Thomas Veerman
1ba514e19c UDS: check connection state better
Select(2)ing on UNIX domain sockets was not working properly because
connection state wasn't properly checked/propagated. So selecting for
a read descriptor and closing the write descriptor on the other end
didn't cause select to return. Similarly, read(2) kept blocking while
it should return an error when the other end closed the socket.

Change-Id: I3f5bb52af1a6b03313d508bf915fc838357ba450
2013-03-07 10:49:21 +00:00
Ben Gras
449ed17833 VM: shared memory pagefault fix
. if there is no memory there, it's not writable; this
	  check bug by the shared memory's writable() method causes
	  pagefaults not to be handled  at all in certain situations,
	  triggering an assert() in pt_writemap()
	. added some assert()s to catch this and similar situations
	  in the future

Change-Id: Ife89bfab4f9a3aa7bf4e33dfb0b13b89dcd5bb94
2013-03-07 10:40:22 +01:00
Lionel Sambuc
8f3fbf7cc1 Cleanup: Remove minix.bootprog.mk
The build system distinction between "bootprog" and "service" is
meaningless as boot programs are standard services.

As minix.service.mk simply imports minix.bootprog.mk, reduce confusion
by removing minix.bootprog.mk and placing the rules in minix.service.mk.

Change-Id: I4056b1e574bed59a8c890239b41b1a7c7cad63e8
2013-03-06 11:56:56 +01:00
Thomas Veerman
49ad4e8888 Spring cleanup
Remove old versions of system calls and system calls that don't have
a libc api interface anymore (dup, dup2, creat).

VFS still contains support for old system call numbers for the new stat
system calls (i.e., 65, 66, 67) to keep supporting old binaries built for
MINIX 3.2.1 (prior to the release).

Change-Id: I721779b58a50c7eeae20669de24658d55d69b25b
2013-03-06 09:56:08 +00:00
Thomas Veerman
473547c777 VFS: implement pipe2
Change-Id: Iedc8042dd73a903456b25ba665d12577f5589ca2
2013-02-28 10:08:53 +00:00
Thomas Veerman
fa78dc389f socket: implement SOCK_CLOEXEC and SOCK_NONBLOCK
Change-Id: I3fa36fa999c82a192d402cb4d913bd397e106e53
2013-02-28 10:08:53 +00:00
Thomas Veerman
fd610ba1b0 VFS: add ability to open files O_CLOEXEC
.adjust libc to make use of it (undo __minix diff)

Change-Id: I90a1aa219fcd1b12b6bc60e72176f326eac8184a
2013-02-28 10:08:53 +00:00
Lionel Sambuc
f640210005 Removing obsolete _NBSD_LIBC define
Change-Id: Ia6ce84ccdf36cf6f64540b990baaa7d85c53533d
2013-02-26 09:44:24 +00:00
Lionel Sambuc
813d4c6a36 Removing obsolete _MINIX_SOURCE define
Change-Id: I924d5af294cd5daf17c73f71803f060cea3c3d8b
2013-02-26 09:44:21 +00:00
Lionel Sambuc
8e4736f2df Removing obsolete _MINIX define
Change-Id: Id33ac7e973d1c0e249b690fe44a597474fac6076
2013-02-26 09:44:20 +00:00
Thomas Veerman
e176fa00ef RS: refresh service upon init failure
When a service fails to initialize, RS exits the service. When injecting
faults this is undesired behavior. With this patch, we're going to assume
that when starting services with the -b flag (no binary exponential
offset), we don't want to exit the service but simply restart the
initialization.

Change-Id: Ie8b9c89e16fe4df8a89ec30ec678a216b4ec5fd0
2013-02-22 16:06:25 +00:00
Thomas Veerman
2b90964e33 VFS: don't garbage collect if file is already closed 2013-02-21 10:29:08 +00:00
Thomas Veerman
cfcce207c1 VFS: prevent unmapping drivers that don't support reopening
libchardriver does not support DEV_REOPEN and will return ERESTART
when you do try it. This made VFS unhappy and concluded erroneously
that the driver was EDEADEPT.
2013-02-21 10:29:08 +00:00
Lionel Sambuc
e4fa9802cb ARM: Enable caches
First round, some more optimizations are possible and should be
activated.

Change-Id: I3b7dee7c82fbffd823a08bec1c5d5ebcf769f92f
2013-02-18 09:08:26 +01:00
Lionel Sambuc
c3ae1bdfcd Switch to disable bin. exp. wait time in RS 2013-02-16 19:35:19 +01:00
Kees Jongenburger
f240e1eaf5 vm:Display boot process name upon failure to load or execute.
Change-Id: I80fdaca09ae255622083b16dee72715f009dc0ee
2013-02-16 11:20:45 +01:00
Ben Gras
3bc6d7df06 impove memory accounting
. the total amount of memory in the system didn't include the memory
	  used by the boot-time modules and some dynamic allocation by the
	  kernel at boot time (to map in VM). especially apparent on our
	  ARM board with 'only' 512MB of memory and a huge ramdisk.
	. also: *add* the VM loaded module to the freelist after it has
	  been allocated for & mapped in instead of cutting it *out* of the
	  freelist! so we get a few more MB free..

Change-Id: If37ac32b21c9d38610830e21421264da4f20bc4f
2013-02-11 19:31:57 +01:00
Ben Gras
d1df256de9 VM: slight pagedir mapping generalization
. allow any number of pde's used for pagedir mapping
	. allows >1024 NR_PROCS on x86, >64 on ARM
	. allows NR_PROCS to be the same in both cases
	. also cleanup: allocating spare PDE's is not necessary
	  throw that function out

Change-Id: Ibb8f8cf6e7db6a4d6384b6911d1a3f3f5e5d8256
2013-02-10 21:50:34 +01:00
Ben Gras
298b41b523 libexec: detect short files
if an exec() fails partway through reading in the sections, the target
process is already gone and a defunct process remains. sanity checking
the binary beforehand helps that.

test10 mutilates binaries and exec()s them on purpose; making an exec()
fail cleanly in such cases seems like acceptable behaviour.

fixes test10 on ARM.

Change-Id: I1ed9bb200ce469d4d349073cadccad5503b2fcb0
2013-02-04 12:04:35 +01:00
Ben Gras
b7ea9f3fd1 vm: fix sanity checks on arm
The 'polarity' of the RW bit is inversed on ARM, causing one
of the sanity check compensations to fail. ARM now runs basic
stuff with sanity checks passing.

Change-Id: Iee28ab63e430e759f204eeb204b24c301d5ea3c9
2013-02-01 16:45:56 +00:00
Ben Gras
08cb986d3f vm: fix kernel-requested mappings for arm
. make vm tell kernel virtual locations of mappings
	. makes _minix_kerninfo feature work
	. fix for mappings being larger than what 1 pde can address
	  (e.g. devices memory requested on arm)
	. still requires a special case for devices memory for the
	  kernel, which has to switch to virtual addressing

Change-Id: I2e94090aa432346fa4da0edeba72f0b7406c2ad7
2013-01-29 17:57:04 +00:00
Thomas Veerman
06e2adbeaa VFS: fix select again
Change-Id: Ia5e26cdbfe38e3fb293dd57269a76b15c1fe236b
2013-01-25 17:42:36 +00:00
Thomas Veerman
b3250450fe VM: ARM needs more spare pages 2013-01-25 17:07:02 +00:00
Thomas Veerman
dad3b4e8c8 ARM: Provide linker script for VM 2013-01-25 17:07:01 +00:00
Thomas Veerman
0968a93748 PFS: verify nrbytes 2013-01-25 17:03:06 +00:00
Thomas Veerman
b180f32ab3 VFS/PFS: remove remnants of file position in pipes 2013-01-23 11:14:34 +00:00
Thomas Veerman
306f3ccd6f VFS: fix select bug on pipes 2013-01-23 11:14:34 +00:00
Lionel Sambuc
b1c4ba4ab6 ARM updates
Due to the ABI we are using we have to use the earm architecture
moniker for the build system to behave correctly. This involves
then some headers to move around.

There is also a few related Makefile updates as well as minor
source code corrections.
2013-01-17 10:03:58 +01:00
Lionel Sambuc
0ac803292e Fix warnings trivial warnings.
Fix warnings about:
 . Unused variables
 . format mismatch in printf/scanf format string and arguments
 . Missing parenthesis around assignment as truth values
 . Clang warnings anout unknown GCC pragma
2013-01-14 11:44:31 +01:00
Lionel Sambuc
22e69f558e Cleaning usage of MAJOR/MINOR
Change-Id: I4d7718c51967930375a5f0657b61f869745cf644
2013-01-14 11:36:26 +01:00
Lionel Sambuc
f14fb60209 Libraries updates and cleanup
* Updating common/lib
 * Updating lib/csu
 * Updating lib/libc
 * Updating libexec/ld.elf_so
 * Corrected test on __minix in featuretest to actually follow the
   meaning of the comment.
 * Cleaned up _REENTRANT-related defintions.
 * Disabled -D_REENTRANT for libfetch
 * Removing some unneeded __NBSD_LIBC defines and tests

Change-Id: Ic1394baef74d11b9f86b312f5ff4bbc3cbf72ce2
2013-01-14 11:36:26 +01:00
Thomas Veerman
bdfef53dbf VFS: initialize variables 2013-01-11 12:46:44 +00:00
Thomas Veerman
aa521228a5 VFS: Coverity appeasements 2013-01-11 09:42:01 +00:00
Thomas Veerman
ea8ff9284a Add stack trace dumps for VFS over serial 2013-01-11 09:18:36 +00:00
Thomas Veerman
625f4ae4a3 VFS: add documentation about internal working 2013-01-11 09:18:36 +00:00
Thomas Veerman
23c5f56e32 VFS: change locking to ease concurrent FSes
This patch uses stricter locking for REQ_LINK, REQ_MKDIR, REQ_MKNOD,
REQ_RENAME, REQ_RMDIR, REQ_SLINK and REQ_UNLINK. For all requests, VFS
locks the directory in which we add or remove an inode with VNODE_WRITE.
I.e., the operations have exclusive access to that directory.

Furthermore, REQ_CHOWN, REQ_CHMOD, and REQ_FTRUNC now lock the vmnt
VMNT_READ; VMNT_WRITE was unnecessary.
2013-01-11 09:18:35 +00:00
Thomas Veerman
3de8d1cf6e VFS/PFS: remove notion of position in pipes
Because pipes have no file position. VFS maintained (file) offsets into a
buffer internal to PFS and stored them in vnodes for simplicity, mixing
the responsibilities of filp and vnode objects.

With this patch PFS ignores the position field in REQ_READ and REQ_WRITE
requests making VFS' job a lot simpler.
2013-01-11 09:18:35 +00:00
Thomas Veerman
7c8b3ddfed VFS: fix locking bugs
.sync and fsync used unnecessarily restrictive locking type
.fsync violated locking order by obtaining a vmnt lock after a filp lock
.fsync contained a TOCTOU bug
.new_node violated locking rules (didn't upgrade lock upon file creation)
.do_pipe used unnecessarily restrictive locking type
.always lock pipes exclusively; even a read operation might require to do
 a write on a vnode object (update pipe size)
.when opening a file with O_TRUNC, upgrade vnode lock when truncating
.utime used unnecessarily restrictive locking type
.path parsing:
  .always acquire VMNT_WRITE or VMNT_EXCL on vmnt and downgrade to
   VMNT_READ if that was what was actually requested. This prevents the
   following deadlock scenario:
   thread A:
     lock_vmnt(vmp, TLL_READSER);
     lock_vnode(vp, TLL_READSER);
     upgrade_vmnt_lock(vmp, TLL_WRITE);

   thread B:
     lock_vmnt(vmp, TLL_READ);
     lock_vnode(vp, TLL_READSER);

   thread A will be stuck in upgrade_vmnt_lock and thread B is stuck in
   lock_vnode. This happens when, for example, thread A tries create a
   new node (open.c:new_node) and thread B tries to do eat_path to
   change dir (stadir.c:do_chdir). When the path is being resolved, a
   vnode is always locked with VNODE_OPCL (TLL_READSER) and then
   downgraded to VNODE_READ if read-only is actually requested. Thread
   A locks the vmnt with VMNT_WRITE (TLL_READSER) which still allows
   VMNT_READ locks. Thread B can't acquire a lock on the vnode because
   thread A has it; Thread A can't upgrade its vmnt lock to VMNT_WRITE
   (TLL_WRITE) because thread B has a VMNT_READ lock on it.

   By serializing vmnt locks during path parsing, thread B can only
   acquire a lock on vmp when thread A has completely finished its
   operation.
2013-01-11 09:18:35 +00:00
Ben Gras
ecf9b40841 vm: fix region reporting bug
. logic was backwards, causing no reporting ever
	. this broke coredumps ('Warning: Program has too many regions')
2013-01-09 19:50:52 +00:00
Ben Gras
b258ab0e66 vm: restore stacktrace on SIGSEGV 2013-01-08 15:47:37 +00:00
Kees Jongenburger
c0c581a635 vfs:fix for variable 'rfp' set but not used.
mount.c: In function 'mount_pfs':
mount.c:395:17: error: variable 'rfp' set but not used [-Werror=unused-but-set-variable]

Change-Id: I2f22590ab4e3a4a1678e9096626ebca53d2660e6
2013-01-07 09:12:27 +01:00
Kees Jongenburger
fce5ff6513 vm: Fix sparepage resource leak.
Fix a sparepage resource leak by preventing to call
vm_getsparepage twice.

Change-Id: I06557d47b90a7cca74e0a86921c2579f9618685d
2013-01-07 09:12:09 +01:00
Ben Gras
29edcad310 vm: replace phys avl by array
. make vm be able to use malloc() by overriding brk()
   and minix_mmap() functions
 . phys regions can then be malloc()ed and free()d instead
   of being in an avl tree, which is slightly faster
 . 'offset' field in phys_region can go too (offset is implied
   by position in array) but leads to bigger code changes
2012-12-26 16:14:41 +00:00
Ben Gras
8aeac26999 vfs: fix clobbering fd_nr
dumpcore: fd_nr can be in use as blocking fd but will then be clobbered
by common_open, causing disaster for exiting unpause().
2012-12-11 12:00:57 +01:00
Lionel Sambuc
456a46e6bf Removing useless minix/types.h header
Change-Id: If5f922279b87f075f301b64c7786caa18b434c2a
2012-12-07 13:58:06 +01:00
David van Moolenbroek
3027cf8694 RS: do not zero process name for boot processes
This bug was preventing services with IPC restrictions from being
started before the boot processes are edited from /etc/rc.
2012-12-06 13:24:30 +00:00
David van Moolenbroek
766047123a VFS: fix off-by-one in get_name() 2012-11-30 12:24:47 +00:00
Thomas Veerman
179261a9b6 mtab: support moving mount points
Also fix canonical_path function; it fails to parse some paths
2012-11-29 10:50:51 +00:00
Thomas Veerman
d9f4f71916 Implement dynamic mtab support
With this patch /etc/mtab becomes obsolete.
2012-11-26 15:20:18 +00:00
Thomas Veerman
de83b2a9d9 VFS: change 'last_dir' to match locking assumption
new_node makes the assumption that when it does last_dir on a path, a
successive advance would not yield a lock on a vmnt, because last_dir
already locked the vmnt. This is true except when last_dir resolves
to a directory on the parent vmnt of the file that was the result of
advance. For example,
 # cd /
 # echo foo > home
where home is on a different (sub) partition than / is (default
install). last_dir would resolve to / and advance would resolve to
/home.

With this change, last_dir resolves to the root node on the /home
partition, making the assumption valid again.
2012-11-26 15:20:18 +00:00
Ben Gras
bcbee7d9e1 RS: fix for fix 2012-11-22 16:25:07 +01:00
Erik van der Kouwe
57c748b968 Remove ability to pass commands to bootloader 2012-11-22 19:16:17 +01:00
Erik van der Kouwe
22fa466268 Restore poweroff to some of it's former glory (on QEMU, at least) 2012-11-21 20:28:37 +01:00
Ben Gras
a89ec8bc3b can't get_block(NO_DEV) any more
. 'anonymous' cache blocks (retrieved with NO_DEV as dev
	  parameter) were used to implement read()s from holes in
	  inodes that should return zeroes
	. this is an awkward special case in the cache code though
	  and there's a more direct way to implement the same functionality:
	  instead of copying from a new, anonymous, zero block, to
	  the user target buffer, simply sys_safememset the user target
	  buffer directly. as this was the only use of this feature,
	  this is all that's needed to simplify the cache code a little.
2012-11-16 16:37:44 +01:00
Ben Gras
2d43bf5807 RS: invoke a shell explicitly for scripts
. don't rely on the scripts to be executable
	  (and they aren't any more)
2012-11-16 16:34:14 +01:00
Ben Gras
d27f8afc45 iso9660fs: initialize buffer cache 2012-11-16 11:09:03 +00:00
Tomas Hruby
dedb53fb10 ipc.h - IPC defined as functions again
- CHOOSETRAP define makes impossible to use some common words
  like send, receive and notify in any other context, for
  instance as members or structures

- any reasonable compiler inlines the static inline functions so
  no extra function call overhead is introduced by this change

- this gets us back to the situation before the SYSCALL/SYSENTER
  change. It is not perfect, but it used to work and still does.
2012-11-15 16:51:59 +01:00
Lionel Sambuc
9152e1c5a7 Upgrading build system to new NetBSD revision
The tested targets are the followgin ones:
 * tools
 * distribution
 * sets
 * release

The remaining NetBSD targets have not been disabled nor tested
*at all*. Try them at your own risk, they may reboot the earth.

For all compliant Makefiles, objects and generated files are put in
MAKEOBJDIR, which means you can now keep objects between two branch
switching. Same for DESTDIR, please refer to build.sh options.

Regarding new or modifications of Makefiles a few things:
 * Read share/mk/bsd.README
 * If you add a subdirectory, add a Makefile in it, and have it called
   by the parent through the SUBDIR variable.
 * Do not add arbitrary inclusion which crosses to another branch of
   the hierarchy; If you can't do without it, put a comment on why.
   If possible, do not use inclusion at all.
 * Use as much as possible the infrastructure, it is here to make
   life easier, do not fight it.

Sets and package are now used to track files.
We have one set called "minix", composed of one package called "minix-sys"
2012-11-15 16:07:29 +01:00
David van Moolenbroek
7dd286e6b8 VFS: do not save device node for new regular files
The VFS/FS protocol does not require the file server to supply a
special device node number in response to a REQ_CREATE request, as
this call creates only regular files. Therefore, VFS should not
erroneously save this piece of information from the REQ_CREATE reply
either.
2012-11-15 14:29:59 +00:00
Thomas Veerman
14e470be81 VFS: fix TOCTOU bug in sync 2012-11-14 13:24:53 +00:00
Thomas Veerman
ed23a7a7d2 VFS: fix reboot panic with mounted FUSE FS
Upon reboot VFS semi-exits all processes and unmounts the file system.
However, upon unmount, exiting FUSE file systems might need service from
the file system (due to libc). As the FUSE process is halfway the exit
procedure, it doesn't have a valid root directory and working directory.
Trying to do system calls then triggers a sanity check in VFS.

This fix first exits normal processes which should then allow for
unmounting FUSE file systems. Then VFS exits all processes including
File Servers and unmounts the rest of the file system.
2012-11-14 13:18:16 +00:00
Thomas Veerman
badec36b33 VFS: fix deadlock when out of worker threads
There is a deadlock vulnerability when there are no worker threads
available and all of them blocked on a worker thread that's waiting for a
reply from a driver or a reply from an FS that needs to make a back call. In
these cases the deadlock resolver thread should kick in, but didn't in all
cases. Moreover, POSIX calls from File Servers weren't handled properly
anymore, which also could lead to deadlocks.
2012-11-14 13:12:37 +00:00
Ben Gras
6cf98dbe00 vm: reduce noise in merged pagetable.c 2012-11-09 19:00:46 +01:00
Ben Gras
3771a0833d vm: merge i386 and arm pagetable code 2012-11-09 18:46:03 +01:00
Ben Gras
b1da7fafd0 vm: fix a null dereference on out-of-memory
. also make other out-of-memory conditions less fatal
	. add a test case for a user program using all the memory
	  it can
	. remove some diagnostic prints for situations that are normal
	  when running out of memory so running the test isn't noisy
2012-11-09 18:36:51 +01:00
David van Moolenbroek
ff84d11216 PM: do not panic if sys_sigsend returns ENOMEM 2012-11-09 16:11:02 +00:00
Ben Gras
196021cd82 drop safemap code 2012-10-30 13:55:42 +01:00
Ben Gras
aefc6db005 vm: fix potential null deref 2012-10-24 19:47:47 +02:00
Ben Gras
bd3cde4571 Move primary cache code to libminixfs.
Add primary cache management feature to libminixfs as mfs and ext2
currently do separately, remove cache code from mfs and ext2, and make
them use the libminixfs interface. This makes all fields of the buf
struct private to libminixfs and FS clients aren't supposed to access
them at all. Only the opaque 'void *data' field (the FS block contents,
used to be called bp) is to be accessed by the FS client.

The main purpose is to implement the interface to the 2ndary vm cache
just once, get rid of some code duplication, and add a little
abstraction to reduce the code inertia of the whole caching business.

Some minor sanity checking and prohibition done by mfs in this code
as removed from the generic primary cache code as a result:
        - checking all inodes are not in use when allocating/resizing
          the cache
        - checking readonly filesystems aren't written to
        - checking the superblock isn't written to on mounted filesystems

The minixfslib code relies on fs_blockstats() in the client filesystem to
return some FS usage information.
2012-10-23 19:48:38 +02:00
David van Moolenbroek
46fca87456 VM: munmap fix 2012-10-13 19:08:28 +02:00
Ben Gras
d343041caa VM: make mapping types explicit
Introduce explicit abstractions for different mapping types,
handling the instantiation, forking, pagefaults and freeing of
anonymous memory, direct physical mappings, shared memory and
physically contiguous anonymous memory as separate types, making
region.c more generic.

Also some other genericification like merging the 3 munmap cases
into one.

COW and SMAP safemap code is still implicit in region.c.
2012-10-12 14:52:01 +02:00
Ben Gras
fd4ddef49a VM: munmap used by VM for itself is no longer used 2012-10-12 14:51:35 +02:00
Arun Thomas
471a03a362 ARM support for kernel and vm 2012-10-07 21:38:03 -04:00
Arne Welzel
e35c4f78d2 VFS: fix check_bsf() locking
The check_bsf() macro uses assert(mutex_trylock(&bsf_lock)) and
assumes bsf_lock is locked afterwards. This breaks when compiling
with NOASSERTS="yes". Also: macro to function transition.
2012-09-28 14:57:34 +02:00
Arne Welzel
7e1074732b VFS: resolve unused parameter if NOASSERTS="yes"
If VFS is compiled with NOASSERTS="yes", ctty_opcl() does not
use the op parameter. Change to "non-assert()" sanity check.
2012-09-28 14:57:32 +02:00
Ben Gras
2cdbb3041d procfs: make ipc vectors available 2012-09-26 17:14:14 +02:00
Ben Gras
2d72cbec41 SYSENTER/SYSCALL support
. add cpufeature detection of both
	. use it for both ipc and kernelcall traps, using a register
	  for call number
	. SYSENTER/SYSCALL does not save any context, therefore userland
	  has to save it
	. to accomodate multiple kernel entry/exit types, the entry
	  type is recorded in the process struct. hitherto all types
	  were interrupt (soft int, exception, hard int); now SYSENTER/SYSCALL
	  is new, with the difference that context is not fully restored
	  from proc struct when running the process again. this can't be
	  done as some information is missing.
	. complication: cases in which the kernel has to fully change
	  process context (i.e. sigreturn). in that case the exit type
	  is changed from SYSENTER/SYSEXIT to soft-int (i.e. iret) and
	  context is fully restored from the proc struct. this does mean
	  the PC and SP must change, as the sysenter/sysexit userland code
	  will otherwise try to restore its own context. this is true in the
	  sigreturn case.
	. override all usage by setting libc_ipc=1
2012-09-24 15:53:43 +02:00
Ben Gras
8a3b6ca3bb remove unused <tools.h> 2012-09-20 12:24:22 +02:00
Ben Gras
b16aacc7f3 vm: fix failed alloc condition 2012-09-19 22:24:56 +02:00
Ben Gras
bc4c07f4f1 coverity appeasement - redundant check 2012-09-19 17:19:57 +02:00
Ben Gras
60014efb3e vfs: pm_dumpcore: always clean up process
. whenever this function is called, pm will expect
	  the process to be cleaned up
	. so don't abort the process entirely on error
	. fixes a later 'forking on top of in-use child' vfs panic
2012-09-19 17:13:17 +02:00
Ben Gras
25817b0854 vm: change NO_MEM to a more impossible value
fixes an assert() firing when starting X. thanks to the report by pikpik.

	. NO_MEM was 0, which is actually an existing piece
	  of physical memory. it can't be allocated because it's reserved
	  for bios data (by the kernel), but it can be mapped in (e.g.
	  by X), causing sanity check disaster.
	. NONCONTIGUOUS is also obsolete as all allocations are single-page
	  now, i.e. NONCONTIGUOUS is really the default and only mode.
2012-09-19 15:31:36 +02:00
Ben Gras
fe6e291f59 vm, kernel, top: report memory usage of vm, kernel 2012-09-18 23:43:52 +02:00
Ben Gras
aa82e375c6 VM: remove dead code 2012-09-18 18:40:57 +02:00
Ben Gras
d526f1a0db some coverity fixes. 2012-09-18 15:11:51 +02:00
Ben Gras
ddf1981004 VM: restore >4k secondary cache functionality
. by storing length in the yielded blocks node again
2012-09-18 13:17:52 +02:00
Ben Gras
ed1af3c86c VM: full munmap
complete munmap implementation; single-page references made
a general munmap() implementation possible to write cleanly.

	. memory: let the MIOCRAMSIZE ioctl set the imgrd device
	  size (but only to 0)
	. let the ramdisk command set sizes to 0
	. use this command to set /dev/imgrd to 0 after mounting /usr
	  in /etc/rc, so the boot time ramdisk is freed (about 4MB
	  currently)
2012-09-18 13:17:52 +02:00
Ben Gras
16c3870b2e VM: abstract datastructures a bit
. a little less duplication in region.c
2012-09-18 13:17:51 +02:00
Ben Gras
0d1f2e6be2 VM: simplify slab allocator
. only keep a list of non-empty, non-full pages with slab objects
	. simplifies alloc/free operations and reduces list management overhead
2012-09-18 13:17:50 +02:00
Ben Gras
19e6dad47b VM: only single page chunks
. only reference single pages in process data structures
   to simplify page faults, copy-on-write, etc.
 . this breaks the secondary cache for objects that are
   not one-page-sized; restored in a next commit
2012-09-18 13:17:49 +02:00
Ben Gras
6d7b770761 VM: static data structure for mem allocation
. allocate physical memory using a fixed, pre-allocated bitmap so there
   are no call cycles and it's avilable earlier
2012-09-18 13:17:48 +02:00
Ben Gras
2cb560297c VM: remove unused dma memory support functions from vm
. unused calls / data structures
2012-09-18 13:17:47 +02:00
Ben Gras
8821c73a9e VM: forget about 'holes'
. unused data structures and code
2012-09-18 13:17:46 +02:00
Ben Gras
6410f4b5db VM: some sanitycheck fixes
minor fixes to restore SANITYCHECKS
2012-09-18 13:17:45 +02:00
Thomas Veerman
c087a60ed2 VFS: fix GCC compilation error 2012-09-17 15:29:38 +00:00
Thomas Veerman
edefb7b35f PM: don't deliver signals to VM 2012-09-17 11:01:46 +00:00
Thomas Veerman
3881e732a9 VFS: panic when unmount_all fails 2012-09-17 11:01:46 +00:00
Thomas Veerman
992799b91f VFS: make all IPC asynchronous
By decoupling synchronous drivers from VFS, we are a big step closer to
supporting driver crashes under all circumstances. That is, VFS can't
become stuck on IPC with a synchronous driver (e.g., INET) and can
recover from crashing block drivers during open/close/ioctl or during
communication with an FS.

In order to maintain serialized communication with a synchronous driver,
the communication is wrapped by a mutex on a per driver basis (not major
numbers as there can be multiple majors with identical endpoints). Majors
that share a driver endpoint point to a single mutex object.

In order to support crashes from block drivers, the file reopen tactic
had to be changed; first reopen files associated with the crashed
driver, then send the new driver endpoint to FSes. This solves a
deadlock between the FS and the block driver;
  - VFS would send REQ_NEW_DRIVER to an FS, but he FS only receives it
    after retrying the current request to the newly started driver.
  - The block driver would refuse the retried request until all files
    had been reopened.
  - VFS would reopen files only after getting a reply from the initial
    REQ_NEW_DRIVER.

When a character special driver crashes, all associated files have to
be marked invalid and closed (or reopened if flagged as such). However,
they can only be closed if a thread holds exclusive access to it. To
obtain exclusive access, the worker thread (which handles the new driver
endpoint event from DS) schedules a new job to garbage collect invalid
files. This way, we can signal the worker thread that was talking to the
crashed driver and will release exclusive access to a file associated
with the crashed driver and prevent the garbage collecting worker thread
from dead locking on that file.

Also, when a character special driver crashes, RS will unmap the driver
and remap it upon restart. During unmapping, associated files are marked
invalid instead of waiting for an endpoint up event from DS, as that
event might come later than new read/write/select requests and thus
cause confusion in the freshly started driver.

When locking a filp, the usage counters are no longer checked. The usage
counter can legally go down to zero during filp invalidation while there
are locks pending.

DS events are handled by a separate worker thread instead of the main
thread as reopening files could lead to another crash and a stuck thread.
An additional worker thread is then necessary to unlock it.

Finally, with everything asynchronous a race condition in do_select
surfaced. A select entry was only marked in use after succesfully sending
initial select requests to drivers and having to wait. When multiple
select() calls were handled there was opportunity that these entries
were overwritten. This had as effect that some select results were
ignored (and select() remained blocking instead if returning) or do_select
tried to access filps that were not present (because thrown away by
secondary select()). This bug manifested itself with sendrecs, but was
very hard to reproduce. However, it became awfully easy to trigger with
asynsends only.
2012-09-17 11:01:45 +00:00
Sébastien Boisvert
373cb6526c IPC server: do not loop to find syscall handler
Instead of using a loop to find a matching ipc (inter process
communication) system call type, the offset in the call table can be
simply calculated in constant time.

Also, when the interprocess communication server receives an ipc
system call from a process, ipc should tell VM to watch the process
only once. This patch fixes that also.

(Patch and commit message slightly edited by committer.)
2012-09-10 19:20:03 +02:00