The natural term to use when talking about MINIX big pages on ARM
is SECTION. A section is a level 1 page table entry pointing to
a 1MB area.
Change-Id: I9bd27ca99bc772126c31c27a537b1415db20c4a6
libminixfs may now be informed of changes to the block usage on the
filesystem. if the net change becomes big enough, libminixfs may
resize the cache based on the new usage.
. update the 2 FSes to provide this information to libminixfs
Change-Id: I158815a11da801fd5572a8de89c9e6c039b82650
This commit introduces a new request type called REQ_BPEEK. It
requests minor device blocks from the FS. Analogously to REQ_PEEK,
it requests the filesystem to get the requested blocks into its
cache, without actually copying the result anywhere.
Change-Id: If1d06645b0e17553a64b3167091e9d12efeb3d6f
In libexec, split the memory allocation method into cleared and
non-cleared. Cleared gives zeroed memory, non-cleared gives 'junk'
memory (that will be overwritten anyway, and so needn't be cleared)
that is faster to get.
Also introduce the 'memmap' method that can be used, if available,
to map code and data from executables into a process using the
third-party mmap() mode.
Change-Id: I26694fd3c21deb8b97e01ed675dfc14719b0672b
. use lmfs_* cache functions that provide the cache with inode
metadata whenever applicable, i.e. tell the cache code which
inode number and in-inode offset a particular cache block
corresponds to.
. needed for mmap implementation
Change-Id: Ic7d3c0c49029880f86a31368278722e907bc2896
Primary purpose of change: to support the mmap implementation, VM must
know both (a) about some block metadata for FS cache blocks, i.e.
inode numbers and inode offsets where applicable; and (b) know about
*all* cache blocks, i.e. also of the FS primary caches and not just
the blocks that spill into the secondary one. This changes the
interface and VM data structures.
This change is only for the interface (libminixfs) and VM data
structures; the filesystem code is unmodified, so although the
secondary cache will be used as normal, blocks will not be annotated
with inode information until the FS is modified to provide this
information. Until it is modified, mmap of files will fail gracefully
on such filesystems.
This is indicated to VFS/VM by returning ENOSYS for REQ_PEEK.
Change-Id: I1d2df6c485e6c5e89eb28d9055076cc02629594e
Memory types in VM are described by methods. Each mapped region has
a type, and all pages instantiated get that type on creation.
Individual page types has to be able to change though. This commit
changes the code to use the memory types of the individual pages,
where appropriate, instead of just the higher-level region, in case
it has changed. This is needed to e.g. support future copy-on-write
MAP_PRIVATE mmap modes.
Change-Id: I5523db14ac036ec774a54392fb67f9acb8725731
Some (backwards-compatible) changes in mmap() call message fields
that allow for a 64-bit offset. minix_mmap() takes an off_t and
minix_mmap64() takes a u64_t. Some mmap() work in VM goes into a
separate function, using the new fields, so that that can be re-used
when files are to be mapped (future commit).
Change-Id: Ifb77a90b593dd3c33cf81b396068e4da1ec5fb1c
The filesystems already implement REQ_PEEK, but do not fully
use the new filesystem cache code yet. (Because it isn't committed
yet..) REQ_PEEK should be disabled for them until they do.
This indicates to VFS that they are not annotating their cache
blocks (in VM) with inode number/offset info, and therefore mmap()
shouldn't succeed on any of their files. (Most importantly exec()
won't fallback elegantly otherwise.)
Change-Id: Ic57ee422864b4bbc031eadba32973270907b02fd
This commit removes the secondary cache code implementation from
VM and its usage from libminixfs. It is to be replaced by a new
implementation.
Change-Id: I8fa3af06330e7604c7e0dd4cbe39d3ce353a05b1
. test70: regression test for m_out vfs race condition
The following tests use testcache.c to generate test i/o
patterns, generate random write data and verify the reads.
. test71: blackbox full-stack test of FS operation, testing
using the regular VFS interface crazy i/o patterns
with various working set sizes, triggering only
primary cache, also secondary cache, and finally
disk i/o and verifying contents all the time
. test72: unit test of libminixfs, implementing
functions it needs from -lsys and -lblockdriver
and the client in order to simulate a working
cache client and backend environment.
. test73: blackbox test of secondary vm cache in isolation
Change-Id: I1287e9753182b8719e634917ad158e3c1e079ceb
. vfs read_only() assumes vnode->v_vmnt is non-NULL, but it can
be NULL sometimes
. e.g. fchmod() on UDS triggered NULL deref; add a check and
add REQ_CHMOD to pfs so unix domain sockets can be fchmod()ded
. add to test56
Change-Id: I83c840f101b647516897cc99fcf472116d762012
m_out is shared between threads as the reply message, and it can happen
results get overwritten by another thread before the reply is sent. This
change
. makes m_out local to the message handling function,
declared on the stack of the caller
. forces callers of reply() to give it a message, or
declare the reply message has no significant fields except
for the return code by calling replycode()
Change-Id: Id06300083a63c72c00f34f86a5c7d96e4bbdf9f6
Variant of utime(2) with struct timespec (with ns precision)
instead of time_t values; also allows for tv_nsec members
the values UTIME_NOW (force update to current time) or
UTIME_OMIT (allow to set either atim or mtim independently.)
Provides a superset of utimes(2), futimes(2), lutimes(2),
and futimens(2).
Provides the same subset of utimensat(2) as does NetBSD 6.
Also import utimens() and lutimeNS() from NetBSD-current.
makes lwip use "unsigned int" instead of "unsigned" since this is
more obvious (i.e. type is not implied).
Change-Id: I852eb80484516e1235241d55be3e15174fa24109
. inet: silence message about exceptions not being implemented
for select(UDP)
This message generates a lot of noise with openntpd. Hide it unless DEBUG is
turned on.
Change-Id: I1527a9ca2583601d6087456062b4f675c80dd711
This also adds the sys_settime() kernel call which allows for the adjusting
of the clock named realtime in the kernel. The existing sys_stime()
function is still needed for a separate job (setting the boottime). The
boottime is set in the readclock driver. The sys_settime() interface is
meant to be flexible and will support both clock_settime() and adjtime()
when adjtime() is implemented later.
settimeofday() was adjusted to use the clock_settime() interface.
One side note discovered during testing: uptime(1) (part of the last(1)),
uses wtmp to determine boottime (not Minix's times(2)). This leads `uptime`
to report odd results when you set the time to a time prior to boottime.
This isn't a new bug introduced by my changes. It's been there for a while.
In order to make it more clear that ticks should be used for timers
and realtime should be used for timestamps / displaying the date/time,
getuptime() was renamed to getticks() and getuptime2() was renamed to
getuptime().
Servers, drivers, libraries, tests, etc that use getuptime()/getuptime2()
have been updated. In instances where a realtime was calculated, the
calculation was changed to use realtime.
System calls clock_getres() and clock_gettime() were added to PM/libc.
. data structure that automatically keeps a set
of pages in reserve, to replace sparepages and
possibly re-used in the future for similar situations,
e.g. if in-filesystem-cache block eviction is
implemented and FS asks for a new block
Change-Id: I149d46c14b9c8e75df16cb94e08907f008c339a6
When you provided a string with junk after the terminating nul to a
UNIX domain socket and used bind(2), the canonical path function would
not properly terminate the new string. This caused VFS to return
ENAMETOOLONG on an otherwise valid path name.
Test case is added to test56.
Change-Id: I883b6be23d9e4ea13c3cee28cbb3726343df037f
Do not hardcode warning and optimisation flags, otherwise the
main options (i.e. DBG, CPPFLAGS) will not work as expected.
You can still provide specific default by using DBG?=<value>.
Doing so leaves the opportunity to override the setting from the
commandline, while the default value from the build system is
then ignored for that particular package.
When crosscompiling, and using build.sh, adding -V DBG=<value> has
this same effect as make DBG=<value>.
Change-Id: Ic610e4d33b945acad64571e1431f1814291e2d84
REQ_PEEK behaves just like REQ_READ except that it does not copy
data anywhere, just obtains the blocks from the FS into the cache.
To be used by the future mmap implementation.
Change-Id: I1b56de304f0a7152b69a72c8962d04258adb44f9
Select(2)ing on UNIX domain sockets was not working properly because
connection state wasn't properly checked/propagated. So selecting for
a read descriptor and closing the write descriptor on the other end
didn't cause select to return. Similarly, read(2) kept blocking while
it should return an error when the other end closed the socket.
Change-Id: I3f5bb52af1a6b03313d508bf915fc838357ba450
. if there is no memory there, it's not writable; this
check bug by the shared memory's writable() method causes
pagefaults not to be handled at all in certain situations,
triggering an assert() in pt_writemap()
. added some assert()s to catch this and similar situations
in the future
Change-Id: Ife89bfab4f9a3aa7bf4e33dfb0b13b89dcd5bb94
The build system distinction between "bootprog" and "service" is
meaningless as boot programs are standard services.
As minix.service.mk simply imports minix.bootprog.mk, reduce confusion
by removing minix.bootprog.mk and placing the rules in minix.service.mk.
Change-Id: I4056b1e574bed59a8c890239b41b1a7c7cad63e8
Remove old versions of system calls and system calls that don't have
a libc api interface anymore (dup, dup2, creat).
VFS still contains support for old system call numbers for the new stat
system calls (i.e., 65, 66, 67) to keep supporting old binaries built for
MINIX 3.2.1 (prior to the release).
Change-Id: I721779b58a50c7eeae20669de24658d55d69b25b
When a service fails to initialize, RS exits the service. When injecting
faults this is undesired behavior. With this patch, we're going to assume
that when starting services with the -b flag (no binary exponential
offset), we don't want to exit the service but simply restart the
initialization.
Change-Id: Ie8b9c89e16fe4df8a89ec30ec678a216b4ec5fd0
libchardriver does not support DEV_REOPEN and will return ERESTART
when you do try it. This made VFS unhappy and concluded erroneously
that the driver was EDEADEPT.
. the total amount of memory in the system didn't include the memory
used by the boot-time modules and some dynamic allocation by the
kernel at boot time (to map in VM). especially apparent on our
ARM board with 'only' 512MB of memory and a huge ramdisk.
. also: *add* the VM loaded module to the freelist after it has
been allocated for & mapped in instead of cutting it *out* of the
freelist! so we get a few more MB free..
Change-Id: If37ac32b21c9d38610830e21421264da4f20bc4f
. allow any number of pde's used for pagedir mapping
. allows >1024 NR_PROCS on x86, >64 on ARM
. allows NR_PROCS to be the same in both cases
. also cleanup: allocating spare PDE's is not necessary
throw that function out
Change-Id: Ibb8f8cf6e7db6a4d6384b6911d1a3f3f5e5d8256
if an exec() fails partway through reading in the sections, the target
process is already gone and a defunct process remains. sanity checking
the binary beforehand helps that.
test10 mutilates binaries and exec()s them on purpose; making an exec()
fail cleanly in such cases seems like acceptable behaviour.
fixes test10 on ARM.
Change-Id: I1ed9bb200ce469d4d349073cadccad5503b2fcb0
The 'polarity' of the RW bit is inversed on ARM, causing one
of the sanity check compensations to fail. ARM now runs basic
stuff with sanity checks passing.
Change-Id: Iee28ab63e430e759f204eeb204b24c301d5ea3c9
. make vm tell kernel virtual locations of mappings
. makes _minix_kerninfo feature work
. fix for mappings being larger than what 1 pde can address
(e.g. devices memory requested on arm)
. still requires a special case for devices memory for the
kernel, which has to switch to virtual addressing
Change-Id: I2e94090aa432346fa4da0edeba72f0b7406c2ad7
Due to the ABI we are using we have to use the earm architecture
moniker for the build system to behave correctly. This involves
then some headers to move around.
There is also a few related Makefile updates as well as minor
source code corrections.
Fix warnings about:
. Unused variables
. format mismatch in printf/scanf format string and arguments
. Missing parenthesis around assignment as truth values
. Clang warnings anout unknown GCC pragma
* Updating common/lib
* Updating lib/csu
* Updating lib/libc
* Updating libexec/ld.elf_so
* Corrected test on __minix in featuretest to actually follow the
meaning of the comment.
* Cleaned up _REENTRANT-related defintions.
* Disabled -D_REENTRANT for libfetch
* Removing some unneeded __NBSD_LIBC defines and tests
Change-Id: Ic1394baef74d11b9f86b312f5ff4bbc3cbf72ce2
This patch uses stricter locking for REQ_LINK, REQ_MKDIR, REQ_MKNOD,
REQ_RENAME, REQ_RMDIR, REQ_SLINK and REQ_UNLINK. For all requests, VFS
locks the directory in which we add or remove an inode with VNODE_WRITE.
I.e., the operations have exclusive access to that directory.
Furthermore, REQ_CHOWN, REQ_CHMOD, and REQ_FTRUNC now lock the vmnt
VMNT_READ; VMNT_WRITE was unnecessary.
Because pipes have no file position. VFS maintained (file) offsets into a
buffer internal to PFS and stored them in vnodes for simplicity, mixing
the responsibilities of filp and vnode objects.
With this patch PFS ignores the position field in REQ_READ and REQ_WRITE
requests making VFS' job a lot simpler.
.sync and fsync used unnecessarily restrictive locking type
.fsync violated locking order by obtaining a vmnt lock after a filp lock
.fsync contained a TOCTOU bug
.new_node violated locking rules (didn't upgrade lock upon file creation)
.do_pipe used unnecessarily restrictive locking type
.always lock pipes exclusively; even a read operation might require to do
a write on a vnode object (update pipe size)
.when opening a file with O_TRUNC, upgrade vnode lock when truncating
.utime used unnecessarily restrictive locking type
.path parsing:
.always acquire VMNT_WRITE or VMNT_EXCL on vmnt and downgrade to
VMNT_READ if that was what was actually requested. This prevents the
following deadlock scenario:
thread A:
lock_vmnt(vmp, TLL_READSER);
lock_vnode(vp, TLL_READSER);
upgrade_vmnt_lock(vmp, TLL_WRITE);
thread B:
lock_vmnt(vmp, TLL_READ);
lock_vnode(vp, TLL_READSER);
thread A will be stuck in upgrade_vmnt_lock and thread B is stuck in
lock_vnode. This happens when, for example, thread A tries create a
new node (open.c:new_node) and thread B tries to do eat_path to
change dir (stadir.c:do_chdir). When the path is being resolved, a
vnode is always locked with VNODE_OPCL (TLL_READSER) and then
downgraded to VNODE_READ if read-only is actually requested. Thread
A locks the vmnt with VMNT_WRITE (TLL_READSER) which still allows
VMNT_READ locks. Thread B can't acquire a lock on the vnode because
thread A has it; Thread A can't upgrade its vmnt lock to VMNT_WRITE
(TLL_WRITE) because thread B has a VMNT_READ lock on it.
By serializing vmnt locks during path parsing, thread B can only
acquire a lock on vmp when thread A has completely finished its
operation.
mount.c: In function 'mount_pfs':
mount.c:395:17: error: variable 'rfp' set but not used [-Werror=unused-but-set-variable]
Change-Id: I2f22590ab4e3a4a1678e9096626ebca53d2660e6
. make vm be able to use malloc() by overriding brk()
and minix_mmap() functions
. phys regions can then be malloc()ed and free()d instead
of being in an avl tree, which is slightly faster
. 'offset' field in phys_region can go too (offset is implied
by position in array) but leads to bigger code changes
new_node makes the assumption that when it does last_dir on a path, a
successive advance would not yield a lock on a vmnt, because last_dir
already locked the vmnt. This is true except when last_dir resolves
to a directory on the parent vmnt of the file that was the result of
advance. For example,
# cd /
# echo foo > home
where home is on a different (sub) partition than / is (default
install). last_dir would resolve to / and advance would resolve to
/home.
With this change, last_dir resolves to the root node on the /home
partition, making the assumption valid again.
. 'anonymous' cache blocks (retrieved with NO_DEV as dev
parameter) were used to implement read()s from holes in
inodes that should return zeroes
. this is an awkward special case in the cache code though
and there's a more direct way to implement the same functionality:
instead of copying from a new, anonymous, zero block, to
the user target buffer, simply sys_safememset the user target
buffer directly. as this was the only use of this feature,
this is all that's needed to simplify the cache code a little.
- CHOOSETRAP define makes impossible to use some common words
like send, receive and notify in any other context, for
instance as members or structures
- any reasonable compiler inlines the static inline functions so
no extra function call overhead is introduced by this change
- this gets us back to the situation before the SYSCALL/SYSENTER
change. It is not perfect, but it used to work and still does.
The tested targets are the followgin ones:
* tools
* distribution
* sets
* release
The remaining NetBSD targets have not been disabled nor tested
*at all*. Try them at your own risk, they may reboot the earth.
For all compliant Makefiles, objects and generated files are put in
MAKEOBJDIR, which means you can now keep objects between two branch
switching. Same for DESTDIR, please refer to build.sh options.
Regarding new or modifications of Makefiles a few things:
* Read share/mk/bsd.README
* If you add a subdirectory, add a Makefile in it, and have it called
by the parent through the SUBDIR variable.
* Do not add arbitrary inclusion which crosses to another branch of
the hierarchy; If you can't do without it, put a comment on why.
If possible, do not use inclusion at all.
* Use as much as possible the infrastructure, it is here to make
life easier, do not fight it.
Sets and package are now used to track files.
We have one set called "minix", composed of one package called "minix-sys"
The VFS/FS protocol does not require the file server to supply a
special device node number in response to a REQ_CREATE request, as
this call creates only regular files. Therefore, VFS should not
erroneously save this piece of information from the REQ_CREATE reply
either.
Upon reboot VFS semi-exits all processes and unmounts the file system.
However, upon unmount, exiting FUSE file systems might need service from
the file system (due to libc). As the FUSE process is halfway the exit
procedure, it doesn't have a valid root directory and working directory.
Trying to do system calls then triggers a sanity check in VFS.
This fix first exits normal processes which should then allow for
unmounting FUSE file systems. Then VFS exits all processes including
File Servers and unmounts the rest of the file system.
There is a deadlock vulnerability when there are no worker threads
available and all of them blocked on a worker thread that's waiting for a
reply from a driver or a reply from an FS that needs to make a back call. In
these cases the deadlock resolver thread should kick in, but didn't in all
cases. Moreover, POSIX calls from File Servers weren't handled properly
anymore, which also could lead to deadlocks.
