The main purpose of this patch is to fix handling of unpause calls
from PM while another call is ongoing. The solution to this problem
sparked a full revision of the threading model, consisting of a large
number of related changes:
- all active worker threads are now always associated with a process,
and every process has at most one active thread working for it;
- the process lock is always held by a process's worker thread;
- a process can now have both normal work and postponed PM work
associated to it;
- timer expiry and non-postponed PM work is done from the main thread;
- filp garbage collection is done from a thread associated with VFS;
- reboot calls from PM are now done from a thread associated with PM;
- the DS events handler is protected from starting multiple threads;
- support for a system worker thread has been removed;
- the deadlock recovery thread has been replaced by a parameter to the
worker_start() function; the number of worker threads has
consequently been increased by one;
- saving and restoring of global but per-thread variables is now
centralized in worker_suspend() and worker_resume(); err_code is now
saved and restored in all cases;
- the concept of jobs has been removed, and job_m_in now points to a
message stored in the worker thread structure instead;
- the PM lock has been removed;
- the separate exec lock has been replaced by a lock on the VM
process, which was already being locked for exec calls anyway;
- PM_UNPAUSE is now processed as a postponed PM request, from a thread
associated with the target process;
- the FP_DROP_WORK flag has been removed, since it is no longer more
than just an optimization and only applied to processes operating on
a pipe when getting killed;
- assignment to "fp" now takes place only when obtaining new work in
the main thread or a worker thread, when resuming execution of a
thread, and in the special case of exiting processes during reboot;
- there are no longer special cases where the yield() call is used to
force a thread to run.
Change-Id: I7a97b9b95c2450454a9b5318dfa0e6150d4e6858
The main motivation for this change is that only Loris supports
multithreading, and Loris supports dynamic thread allocation, so the
number of supported threads can be implemented as a bit flag (i.e.,
either 1 or "at least as many as VFS has"). The ABI break obviates the
need to support file system versioning at this time, and several
other aspects are better implemented as flags as well. Other changes:
- replace peek/bpeek test upon mount with FS flag as well;
- mark libsffs as 64-bit file size capable;
- remove old (3.2.1) getdents support.
Change-Id: I313eace9c50ed816656c31cd47d969033d952a03
This is a requirement for implementing calls such as getmntinfo(3).
VFS is now responsible for filling in some of the structure's fields.
Change-Id: I0c1fa78019587efefd2949b3be38cd9a7ddc2ced
This commit introduces a new request type called REQ_BPEEK. It
requests minor device blocks from the FS. Analogously to REQ_PEEK,
it requests the filesystem to get the requested blocks into its
cache, without actually copying the result anywhere.
Change-Id: If1d06645b0e17553a64b3167091e9d12efeb3d6f
REQ_PEEK behaves just like REQ_READ except that it does not copy
data anywhere, just obtains the blocks from the FS into the cache.
To be used by the future mmap implementation.
Change-Id: I1b56de304f0a7152b69a72c8962d04258adb44f9
Remove old versions of system calls and system calls that don't have
a libc api interface anymore (dup, dup2, creat).
VFS still contains support for old system call numbers for the new stat
system calls (i.e., 65, 66, 67) to keep supporting old binaries built for
MINIX 3.2.1 (prior to the release).
Change-Id: I721779b58a50c7eeae20669de24658d55d69b25b
The VFS/FS protocol does not require the file server to supply a
special device node number in response to a REQ_CREATE request, as
this call creates only regular files. Therefore, VFS should not
erroneously save this piece of information from the REQ_CREATE reply
either.
. sys_vircopy always uses D for both src and dst
. sys_physcopy uses PHYS_SEG if and only if corresponding
endpoint is NONE, so we can derive the mode (PHYS_SEG or D)
from the endpoint arg in the kernel, dropping the seg args
. fields in msg still filled in for backwards compatability,
using same NONE-logic in the library
According to POSIX the st_size field of struct stat is undefined for
fifos and anonymous pipes. Thus we can do anything we want. We save a
copy by not being accurate on pipe sizes.
By making m_in job local (i.e., each job has its own copy of m_in instead
of refering to the global m_in) we don't have to store and restore m_in
on every thread yield. This reduces overhead. Moreover, remove the
assumption that m_in is preserved. Do_XXX functions have to copy the
system call parameters as soon as possible and only pass those copies to
other functions.
Furthermore, this patch cleans up some code and uses better types in a lot
of places.
Using sendrec directly only results in problems. While it is not
clear whether using fs_sendrec is the best option, it is at least
an improvement.
Also remove some legacy cruft.
* VFS and installed MFSes must be in sync before and after this change *
Use struct stat from NetBSD. It requires adding new STAT, FSTAT and LSTAT
syscalls. Libc modification is both backward and forward compatible.
Also new struct stat uses modern field sizes to avoid ABI
incompatibility, when we update uid_t, gid_t and company.
Exceptions are ino_t and off_t in old libc (though paddings added).
file descriptor passing, PFS does some back calls to VFS. For example, to
verify the validity of a path provided by a process and to tell VFS it must
copy file descriptors from one process to another.
this change
- makes panic() variadic, doing full printf() formatting -
no more NO_NUM, and no more separate printf() statements
needed to print extra info (or something in hex) before panicing
- unifies panic() - same panic() name and usage for everyone -
vm, kernel and rest have different names/syntax currently
in order to implement their own luxuries, but no longer
- throws out the 1st argument, to make source less noisy.
the panic() in syslib retrieves the server name from the kernel
so it should be clear enough who is panicing; e.g.
panic("sigaction failed: %d", errno);
looks like:
at_wini(73130): panic: sigaction failed: 0
syslib:panic.c: stacktrace: 0x74dc 0x2025 0x100a
- throws out report() - printf() is more convenient and powerful
- harmonizes/fixes the use of panic() - there were a few places
that used printf-style formatting (didn't work) and newlines
(messes up the formatting) in panic()
- throws out a few per-server panic() functions
- cleans up a tie-in of tty with panic()
merging printf() and panic() statements to be done incrementally.
- Make open(2) more POSIX compliant
- Add a test case for dangling symlinks and open() syscall with O_CREAT and
O_EXCL on a symlink.
- Update open(2) man page to reflect change.
SYSLIB CHANGES:
- SEF must be used by every system process and is thereby part of the system
library.
- The framework provides a receive() interface (sef_receive) for system
processes to automatically catch known system even messages and process them.
- SEF provides a default behavior for each type of system event, but allows
system processes to register callbacks to override the default behavior.
- Custom (local to the process) or predefined (provided by SEF) callback
implementations can be registered to SEF.
- SEF currently includes support for 2 types of system events:
1. SEF Ping. The event occurs every time RS sends a ping to figure out
whether a system process is still alive. The default callback implementation
provided by SEF is to notify RS back to let it know the process is alive
and kicking.
2. SEF Live update. The event occurs every time RS sends a prepare to update
message to let a system process know an update is available and to prepare
for it. The live update support is very basic for now. SEF only deals with
verifying if the prepare state can be supported by the process, dumping the
state for debugging purposes, and providing an event-driven programming
model to the process to react to state changes check-in when ready to update.
- SEF should be extended in the future to integrate support for more types of
system events. Ideally, all the cross-cutting concerns should be integrated into
SEF to avoid duplicating code and ease extensibility. Examples include:
* PM notify messages primarily used at shutdown.
* SYSTEM notify messages primarily used for signals.
* CLOCK notify messages used for system alarms.
* Debug messages. IS could still be in charge of fkey handling but would
forward the debug message to the target process (e.g. PM, if the user
requested debug information about PM). SEF would then catch the message and
do nothing unless the process has registered an appropriate callback to
deal with the event. This simplifies the programming model to print debug
information, avoids duplicating code, and reduces the effort to print
debug information.
SYSTEM PROCESSES CHANGES:
- Every system process registers SEF callbacks it needs to override the default
system behavior and calls sef_startup() right after being started.
- sef_startup() does almost nothing now, but will be extended in the future to
support callbacks of its own to let RS control and synchronize with every
system process at initialization time.
- Every system process calls sef_receive() now rather than receive() directly,
to let SEF handle predefined system events.
RS CHANGES:
- RS supports a basic single-component live update protocol now, as follows:
* When an update command is issued (via "service update *"), RS notifies the
target system process to prepare for a specific update state.
* If the process doesn't respond back in time, the update is aborted.
* When the process responds back, RS kills it and marks it for refreshing.
* The process is then automatically restarted as for a buggy process and can
start running again.
* Live update is currently prototyped as a controlled failure.
- Revise VFS-FS protocol and update VFS/MFS/ISOFS accordingly.
- Clean up MFS by removing old, dead code (backwards compatibility is broken by
the new VFS-FS protocol, anyway) and rewrite other parts. Also, make sure all
functions have proper banners and prototypes.
- VFS should always provide a (syntactically) valid path to the FS; no need for
the FS to do sanity checks when leaving/entering mount points.
- Fix several bugs in MFS:
- Several path lookup bugs in MFS.
- A link can be too big for the path buffer.
- A mountpoint can become inaccessible when the creation of a new inode
fails, because the inode already exists and is a mountpoint.
- Introduce support for supplemental groups.
- Add test 46 to test supplemental group functionality (and removed obsolete
suppl. tests from test 2).
- Clean up VFS (not everything is done yet).
- ISOFS now opens device read-only. This makes the -r flag in the mount command
unnecessary (but will still report to be mounted read-write).
- Introduce PipeFS. PipeFS is a new FS that handles all anonymous and
named pipes. However, named pipes still reside on the (M)FS, as they are part
of the file system on disk. To make this work VFS now has a concept of
'mapped' inodes, which causes read, write, truncate and stat requests to be
redirected to the mapped FS, and all other requests to the original FS.
- Changed VFS-FS protocol to only store OK or negative error code in
m_type field of reply messages.
- Changed VFS to treat nonzero positive replies from FS as requests.
- Added backwards compatibility to VFS and MFS.
No protection of global data structures is provided in VFS, so many
VFS calls cannot be made safely by FS servers during many FS calls.
Use with caution (or, preferably, not at all).
now used for printing diagnostic messages through the kernel message
buffer. this lets processes print diagnostics without sending messages
to tty and log directly, simplifying the message protocol a lot and
reducing difficulties with deadlocks and other situations in which
diagnostics are blackholed (e.g. grants don't work). this makes
DIAGNOSTICS(_S), ASYN_DIAGNOSTICS and DIAG_REPL obsolete, although tty
and log still accept the codes for 'old' binaries. This also simplifies
diagnostics in several servers and drivers - only tty needs its own
kputc() now.
. simplifications in vfs, and some effort to get the vnode references
right (consistent) even during shutdown. m_mounted_on is now NULL
for root filesystems (!) (the original and new root), a less awkward
special case than 'm_mounted_on == m_root_node'. root now has exactly
one reference, to root, if no files are open, just like all other
filesystems. m_driver_e is unused.
bugfixes:
SYSTEM:
. removed
rc->p_priv->s_flags = 0;
for the priv struct shared by all user processes in get_priv(). this
should only be done once. doing a SYS_PRIV_USER in sys_privctl()
caused the flags of all user processes to be reset, so they were no
longer PREEMPTIBLE. this happened when RS executed a policy script.
(this broke test1 in the test set)
VFS/MFS:
. chown can change the mode of a file, and chmod arguments are only
part of the full file mode so the full filemode is slightly magic.
changed these calls so that the final modes are returned to VFS, so
that the vnode can be kept up-to-date.
(this broke test11 in the test set)
MFS:
. lookup() checked for sizeof(string) instead of sizeof(user_path),
truncating long path names
(caught by test 23)
. truncate functions neglected to update ctime
(this broke test16)
VFS:
. corner case of an empty filename lookup caused fields of a request
not to be filled in in the lookup functions, not making it clear
that the lookup had failed, causing messages to garbage processes,
causing strange failures.
(caught by test 30)
. trust v_size in vnode when doing reads or writes on non-special
files, truncating i/o where necessary; this is necessary for pipes,
as MFS can't tell when a pipe has been truncated without it being
told explicitly each time.
when the last reader/writer on a pipe closes, tell FS about
the new size using truncate_vn().
(this broke test 25, among others)
. permission check for chdir() had disappeared; added a
forbidden() call
(caught by test 23)
new code, shouldn't change anything:
. introduced RTS_SET, RTS_UNSET, and RTS_ISSET macro's, and their
LOCK variants. These macros set and clear the p_rts_flags field,
causing a lot of duplicated logic like
old_flags = rp->p_rts_flags; /* save value of the flags */
rp->p_rts_flags &= ~NO_PRIV;
if (old_flags != 0 && rp->p_rts_flags == 0) lock_enqueue(rp);
to change into the simpler
RTS_LOCK_UNSET(rp, NO_PRIV);
so the macros take care of calling dequeue() and enqueue() (or lock_*()),
as the case may be). This makes the code a bit more readable and a
bit less fragile.
. removed return code from do_clocktick in CLOCK as it currently
never replies
. removed some debug code from VFS
. fixed grant debug message in device.c
preemptive checks, tests, changes:
. added return code checks of receive() to SYSTEM and CLOCK
. O_TRUNC should never arrive at MFS (added sanity check and removed
O_TRUNC code)
. user_path declared with PATH_MAX+1 to let it be null-terminated
. checks in MFS to see if strings passed by VFS are null-terminated
IS:
. static irq name table thrown out
. changed umount() and mount() to call 'service', so that it can include
a custom label, so that umount() works again (RS slot gets freed now).
merged umount() and mount() into one file to encode keep this label
knowledge in one file.
. removed obsolete RS_PID field and RS_RESCUE rescue command
. added label to RS_START struct
. vfs no longer does kill of fs process on unmount (which was failing
due to RS_PID request not working)
. don't assume that if error wasn't one of three errors, that no error
occured in vfs/request.c
mfs changes:
. added checks to copy statements to truncate copies at buffer sizes
(left in debug code for now)
. added checks for null-terminatedness, if less than NAME_MAX was copied
. added checks for copy function success
is changes:
. dump rs label
drivers.conf changes:
. added acl for mfs so that mfs can be started with 'service start',
so that a custom label can be provided
. corrected device match for unmount (otherwise unmount would
proceed with bogus mount slot, often sending messages to 0 (PM))
. added some sanity checking to fs process number
. made fs_sendrec PRIVATE to request.c
