SYSLIB CHANGES:
- SEF framework now supports a new SEF Init request type from RS. 3 different
callbacks are available (init_fresh, init_lu, init_restart) to specify
initialization code when a service starts fresh, starts after a live update,
or restarts.
SYSTEM SERVICE CHANGES:
- Initialization code for system services is now enclosed in a callback SEF will
automatically call at init time. The return code of the callback will
tell RS whether the initialization completed successfully.
- Each init callback can access information passed by RS to initialize. As of
now, each system service has access to the public entries of RS's system process
table to gather all the information required to initialize. This design
eliminates many existing or potential races at boot time and provides a uniform
initialization interface to system services. The same interface will be reused
for the upcoming publish/subscribe model to handle dynamic
registration / deregistration of system services.
VM CHANGES:
- Uniform privilege management for all system services. Every service uses the
same call mask format. For boot services, VM copies the call mask from init
data. For dynamic services, VM still receives the call mask via rs_set_priv
call that will be soon replaced by the upcoming publish/subscribe model.
RS CHANGES:
- The system process table has been reorganized and split into private entries
and public entries. Only the latter ones are exposed to system services.
- VM call masks are now entirely configured in rs/table.c
- RS has now its own slot in the system process table. Only kernel tasks and
user processes not included in the boot image are now left out from the system
process table.
- RS implements the initialization protocol for system services.
- For services in the boot image, RS blocks till initialization is complete and
panics when failure is reported back. Services are initialized in their order of
appearance in the boot image priv table and RS blocks to implements synchronous
initialization for every system service having the flag SF_SYNCH_BOOT set.
- For services started dynamically, the initialization protocol is implemented
as though it were the first ping for the service. In this case, if the
system service fails to report back (or reports failure), RS brings the service
down rather than trying to restart it.
- clean up kernel section of minix/com.h somewhat
- remove ALLOCMEM and VM_ALLOCMEM calls
- remove non-safecopy and minix-vmd support from Inet
- remove SYS_VIRVCOPY and SYS_PHYSVCOPY calls
- remove obsolete segment encoding in SYS_SAFECOPY*
- remove DEVCTL call, svrctl(FSDEVUNMAP), map_driverX
- remove declarations of unimplemented svrctl requests
- remove everything related to swapping to disk
- remove floppysetup.sh
- remove traces of rescue device
- update DESCRIBE.sh with new devices
- some other small changes
SYSLIB CHANGES:
- SEF must be used by every system process and is thereby part of the system
library.
- The framework provides a receive() interface (sef_receive) for system
processes to automatically catch known system even messages and process them.
- SEF provides a default behavior for each type of system event, but allows
system processes to register callbacks to override the default behavior.
- Custom (local to the process) or predefined (provided by SEF) callback
implementations can be registered to SEF.
- SEF currently includes support for 2 types of system events:
1. SEF Ping. The event occurs every time RS sends a ping to figure out
whether a system process is still alive. The default callback implementation
provided by SEF is to notify RS back to let it know the process is alive
and kicking.
2. SEF Live update. The event occurs every time RS sends a prepare to update
message to let a system process know an update is available and to prepare
for it. The live update support is very basic for now. SEF only deals with
verifying if the prepare state can be supported by the process, dumping the
state for debugging purposes, and providing an event-driven programming
model to the process to react to state changes check-in when ready to update.
- SEF should be extended in the future to integrate support for more types of
system events. Ideally, all the cross-cutting concerns should be integrated into
SEF to avoid duplicating code and ease extensibility. Examples include:
* PM notify messages primarily used at shutdown.
* SYSTEM notify messages primarily used for signals.
* CLOCK notify messages used for system alarms.
* Debug messages. IS could still be in charge of fkey handling but would
forward the debug message to the target process (e.g. PM, if the user
requested debug information about PM). SEF would then catch the message and
do nothing unless the process has registered an appropriate callback to
deal with the event. This simplifies the programming model to print debug
information, avoids duplicating code, and reduces the effort to print
debug information.
SYSTEM PROCESSES CHANGES:
- Every system process registers SEF callbacks it needs to override the default
system behavior and calls sef_startup() right after being started.
- sef_startup() does almost nothing now, but will be extended in the future to
support callbacks of its own to let RS control and synchronize with every
system process at initialization time.
- Every system process calls sef_receive() now rather than receive() directly,
to let SEF handle predefined system events.
RS CHANGES:
- RS supports a basic single-component live update protocol now, as follows:
* When an update command is issued (via "service update *"), RS notifies the
target system process to prepare for a specific update state.
* If the process doesn't respond back in time, the update is aborted.
* When the process responds back, RS kills it and marks it for refreshing.
* The process is then automatically restarted as for a buggy process and can
start running again.
* Live update is currently prototyped as a controlled failure.
- Revise VFS-FS protocol and update VFS/MFS/ISOFS accordingly.
- Clean up MFS by removing old, dead code (backwards compatibility is broken by
the new VFS-FS protocol, anyway) and rewrite other parts. Also, make sure all
functions have proper banners and prototypes.
- VFS should always provide a (syntactically) valid path to the FS; no need for
the FS to do sanity checks when leaving/entering mount points.
- Fix several bugs in MFS:
- Several path lookup bugs in MFS.
- A link can be too big for the path buffer.
- A mountpoint can become inaccessible when the creation of a new inode
fails, because the inode already exists and is a mountpoint.
- Introduce support for supplemental groups.
- Add test 46 to test supplemental group functionality (and removed obsolete
suppl. tests from test 2).
- Clean up VFS (not everything is done yet).
- ISOFS now opens device read-only. This makes the -r flag in the mount command
unnecessary (but will still report to be mounted read-write).
- Introduce PipeFS. PipeFS is a new FS that handles all anonymous and
named pipes. However, named pipes still reside on the (M)FS, as they are part
of the file system on disk. To make this work VFS now has a concept of
'mapped' inodes, which causes read, write, truncate and stat requests to be
redirected to the mapped FS, and all other requests to the original FS.
KERNEL CHANGES:
- The kernel only knows about privileges of kernel tasks and the root system
process (now RS).
- Kernel tasks and the root system process are the only processes that are made
schedulable by the kernel at startup. All the other processes in the boot image
don't get their privileges set at startup and are inhibited from running by the
RTS_NO_PRIV flag.
- Removed the assumption on the ordering of processes in the boot image table.
System processes can now appear in any order in the boot image table.
- Privilege ids can now be assigned both statically or dynamically. The kernel
assigns static privilege ids to kernel tasks and the root system process. Each
id is directly derived from the process number.
- User processes now all share the static privilege id of the root user
process (now INIT).
- sys_privctl split: we have more calls now to let RS set privileges for system
processes. SYS_PRIV_ALLOW / SYS_PRIV_DISALLOW are only used to flip the
RTS_NO_PRIV flag and allow / disallow a process from running. SYS_PRIV_SET_SYS /
SYS_PRIV_SET_USER are used to set privileges for a system / user process.
- boot image table flags split: PROC_FULLVM is the only flag that has been
moved out of the privilege flags and is still maintained in the boot image
table. All the other privilege flags are out of the kernel now.
RS CHANGES:
- RS is the only user-space process who gets to run right after in-kernel
startup.
- RS uses the boot image table from the kernel and three additional boot image
info table (priv table, sys table, dev table) to complete the initialization
of the system.
- RS checks that the entries in the priv table match the entries in the boot
image table to make sure that every process in the boot image gets schedulable.
- RS only uses static privilege ids to set privileges for system services in
the boot image.
- RS includes basic memory management support to allocate the boot image buffer
dynamically during initialization. The buffer shall contain the executable
image of all the system services we would like to restart after a crash.
- First step towards decoupling between resource provisioning and resource
requirements in RS: RS must know what resources it needs to restart a process
and what resources it has currently available. This is useful to tradeoff
reliability and resource consumption. When required resources are missing, the
process cannot be restarted. In that case, in the future, a system flag will
tell RS what to do. For example, if CORE_PROC is set, RS should trigger a
system-wide panic because the system can no longer function correctly without
a core system process.
PM CHANGES:
- The process tree built at initialization time is changed to have INIT as root
with pid 0, RS child of INIT and all the system services children of RS. This
is required to make RS in control of all the system services.
- PM no longer registers labels for system services in the boot image. This is
now part of RS's initialization process.
- allow PM to tell sys_runctl() whether to use delay call feature
- only use this feature in PM for delivering signals - not for exits
- do better error checking in PM on sys_runctl() calls
- rename SIGKREADY to SIGNDELAY
o Support for ptrace T_ATTACH/T_DETACH and T_SYSCALL
o PM signal handling logic should now work properly, even with debuggers
being present
o Asynchronous PM/VFS protocol, full IPC support for senda(), and
AMF_NOREPLY senda() flag
DETAILS
Process stop and delay call handling of PM:
o Added sys_runctl() kernel call with sys_stop() and sys_resume()
aliases, for PM to stop and resume a process
o Added exception for sending/syscall-traced processes to sys_runctl(),
and matching SIGKREADY pseudo-signal to PM
o Fixed PM signal logic to deal with requests from a process after
stopping it (so-called "delay calls"), using the SIGKREADY facility
o Fixed various PM panics due to race conditions with delay calls versus
VFS calls
o Removed special PRIO_STOP priority value
o Added SYS_LOCK RTS kernel flag, to stop an individual process from
running while modifying its process structure
Signal and debugger handling in PM:
o Fixed debugger signals being dropped if a second signal arrives when
the debugger has not retrieved the first one
o Fixed debugger signals being sent to the debugger more than once
o Fixed debugger signals unpausing process in VFS; removed PM_UNPAUSE_TR
protocol message
o Detached debugger signals from general signal logic and from being
blocked on VFS calls, meaning that even VFS can now be traced
o Fixed debugger being unable to receive more than one pending signal in
one process stop
o Fixed signal delivery being delayed needlessly when multiple signals
are pending
o Fixed wait test for tracer, which was returning for children that were
not waited for
o Removed second parallel pending call from PM to VFS for any process
o Fixed process becoming runnable between exec() and debugger trap
o Added support for notifying the debugger before the parent when a
debugged child exits
o Fixed debugger death causing child to remain stopped forever
o Fixed consistently incorrect use of _NSIG
Extensions to ptrace():
o Added T_ATTACH and T_DETACH ptrace request, to attach and detach a
debugger to and from a process
o Added T_SYSCALL ptrace request, to trace system calls
o Added T_SETOPT ptrace request, to set trace options
o Added TO_TRACEFORK trace option, to attach automatically to children
of a traced process
o Added TO_ALTEXEC trace option, to send SIGSTOP instead of SIGTRAP upon
a successful exec() of the tracee
o Extended T_GETUSER ptrace support to allow retrieving a process's priv
structure
o Removed T_STOP ptrace request again, as it does not help implementing
debuggers properly
o Added MINIX3-specific ptrace test (test42)
o Added proper manual page for ptrace(2)
Asynchronous PM/VFS interface:
o Fixed asynchronous messages not being checked when receive() is called
with an endpoint other than ANY
o Added AMF_NOREPLY senda() flag, preventing such messages from
satisfying the receive part of a sendrec()
o Added asynsend3() that takes optional flags; asynsend() is now a
#define passing in 0 as third parameter
o Made PM/VFS protocol asynchronous; reintroduced tell_fs()
o Made PM_BASE request/reply number range unique
o Hacked in a horrible temporary workaround into RS to deal with newly
revealed RS-PM-VFS race condition triangle until VFS is asynchronous
System signal handling:
o Fixed shutdown logic of device drivers; removed old SIGKSTOP signal
o Removed is-superuser check from PM's do_procstat() (aka getsigset())
o Added sigset macros to allow system processes to deal with the full
signal set, rather than just the POSIX subset
Miscellaneous PM fixes:
o Split do_getset into do_get and do_set, merging common code and making
structure clearer
o Fixed setpriority() being able to put to sleep processes using an
invalid parameter, or revive zombie processes
o Made find_proc() global; removed obsolete proc_from_pid()
o Cleanup here and there
Also included:
o Fixed false-positive boot order kernel warning
o Removed last traces of old NOTIFY_FROM code
THINGS OF POSSIBLE INTEREST
o It should now be possible to run PM at any priority, even lower than
user processes
o No assumptions are made about communication speed between PM and VFS,
although communication must be FIFO
o A debugger will now receive incoming debuggee signals at kill time
only; the process may not yet be fully stopped
o A first step has been made towards making the SYSTEM task preemptible
addr and taddr don't have to be defined any more, so that <sys/mman.h>
can be included for proper prototypes of munmap() and friends.
- rename our GETPID to MINIX_GETPID to avoid a name conflict with
other sources
- PM needs its own munmap() and munmap_text() to avoid sending messages
to VM at the startup phase. It *does* want to do that, but only
after initialising. So they're called again with unmap_ok set to 1
later.
- getnuid(), getngid() implementation
o Don't call vm_willexit() more than once upon normal process exit
o Correct two cases of indenting of the no-discussion-possible kind
o Perform slightly stricter ptrace(2) checks:
- process calling ptrace must be target process's parent
- process must call wait/waitpid before using ptrace on stopped child
- no ptrace on zombies
o Allow user processes to use ptrace(T_STOP) to stop an active child
- ipc checking code in kernel didn't properly catch the
sendrec() to self case; added special case check
- triggered by PM using stock panic() - needs its own _exit()
reported by Joren l'Ami.
now used for printing diagnostic messages through the kernel message
buffer. this lets processes print diagnostics without sending messages
to tty and log directly, simplifying the message protocol a lot and
reducing difficulties with deadlocks and other situations in which
diagnostics are blackholed (e.g. grants don't work). this makes
DIAGNOSTICS(_S), ASYN_DIAGNOSTICS and DIAG_REPL obsolete, although tty
and log still accept the codes for 'old' binaries. This also simplifies
diagnostics in several servers and drivers - only tty needs its own
kputc() now.
. simplifications in vfs, and some effort to get the vnode references
right (consistent) even during shutdown. m_mounted_on is now NULL
for root filesystems (!) (the original and new root), a less awkward
special case than 'm_mounted_on == m_root_node'. root now has exactly
one reference, to root, if no files are open, just like all other
filesystems. m_driver_e is unused.
mainly in the kernel and headers. This split based on work by
Ingmar Alting <iaalting@cs.vu.nl> done for his Minix PowerPC architecture
port.
. kernel does not program the interrupt controller directly, do any
other architecture-dependent operations, or contain assembly any more,
but uses architecture-dependent functions in arch/$(ARCH)/.
. architecture-dependent constants and types defined in arch/$(ARCH)/include.
. <ibm/portio.h> moved to <minix/portio.h>, as they have become, for now,
architecture-independent functions.
. int86, sdevio, readbios, and iopenable are now i386-specific kernel calls
and live in arch/i386/do_* now.
. i386 arch now supports even less 86 code; e.g. mpx86.s and klib86.s have
gone, and 'machine.protected' is gone (and always taken to be 1 in i386).
If 86 support is to return, it should be a new architecture.
. prototypes for the architecture-dependent functions defined in
kernel/arch/$(ARCH)/*.c but used in kernel/ are in kernel/proto.h
. /etc/make.conf included in makefiles and shell scripts that need to
know the building architecture; it defines ARCH=<arch>, currently only
i386.
. some basic per-architecture build support outside of the kernel (lib)
. in clock.c, only dequeue a process if it was ready
. fixes for new include files
files deleted:
. mpx/klib.s - only for choosing between mpx/klib86 and -386
. klib86.s - only for 86
i386-specific files files moved (or arch-dependent stuff moved) to arch/i386/:
. mpx386.s (entry point)
. klib386.s
. sconst.h
. exception.c
. protect.c
. protect.h
. i8269.c
as living processes before they are cleaned up (fixes
wait()/waitpid() hanging forever on previously-ZOMBIE processes)
. stop processes from running using sys_nice() with PRIO_STOP
when a handled signal is delivered, before computing
stack locations for sys_sigsend(). (fixes race condition
when runnable processes get signals, and e.g. get scheduled
before FS sends a reply to unpause(), which can make the
signal stack location wrong.)
library to the memory driver. Always put output from within TTY directly on
the console. Removed second include of driver.h from tty.c. Made tty_inrepcode
bigger. First step to move PM and FS calls that are not regular (API)
system calls out of callnr.h (renumbered them, and removed them from the
table.c files). Imported the Minix-vmd uname implementation. This provides
a more stable ABI than the current implementation. Added a bit of security
checking. Unfortunately not nearly enough to get a secure system. Fixed a
bug related to the sizes of the programs in the image (in PM patch_mem_chunks).
instead of keeping a running total of enqueued processes
(because somehow the load average was broken)
. added SI_KPROC_TAB to get a copy of kernel process table from PM, for
a top implementation
. fixed arg to sys_nice() to make it an endpoint, not a slot number
pm: fixed rebooting by making a copy of the monitor code from the user
process. this is necessary because that process is dead by the time
sys_abort() is called.
also added more info to the "can't reply" panic.
'who', indicating caller number in pm and fs and some other servers, has
been removed in favour of 'who_e' (endpoint) and 'who_p' (proc nr.).
In both PM and FS, isokendpt() convert endpoints to process slot
numbers, returning OK if it was a valid and consistent endpoint number.
okendpt() does the same but panic()s if it doesn't succeed. (In PM,
this is pm_isok..)
pm and fs keep their own records of process endpoints in their proc tables,
which are needed to make kernel calls about those processes.
message field names have changed.
fs drivers are endpoints.
fs now doesn't try to get out of driver deadlock, as the protocol isn't
supposed to let that happen any more. (A warning is printed if ELOCKED
is detected though.)
fproc[].fp_task (indicating which driver the process is suspended on)
became an int.
PM and FS now get endpoint numbers of initial boot processes from the
kernel. These happen to be the same as the old proc numbers, to let
user processes reach them with the old numbers, but FS and PM don't know
that. All new processes after INIT, even after the generation number
wraps around, get endpoint numbers with generation 1 and higher, so
the first instances of the boot processes are the only processes ever
to have endpoint numbers in the old proc number range.
More return code checks of sys_* functions have been added.
IS has become endpoint-aware. Ditched the 'text' and 'data' fields
in the kernel dump (which show locations, not sizes, so aren't terribly
useful) in favour of the endpoint number. Proc number is still visible.
Some other dumps (e.g. dmap, rs) show endpoint numbers now too which got
the formatting changed.
PM reading segments using rw_seg() has changed - it uses other fields
in the message now instead of encoding the segment and process number and
fd in the fd field. For that it uses _read_pm() and _write_pm() which to
_taskcall()s directly in pm/misc.c.
PM now sys_exit()s itself on panic(), instead of sys_abort().
RS also talks in endpoints instead of process numbers.
Implemented by changing write_map to accept a WMAP_FREE flag. In that
case, it doesn't update the datablock (creating indirect zones as
necessary) pointer, but it frees the datablock if present. Also it
frees the single and double indirect blocks if unused.
This makes the implementation of truncate_inode() simpler.
truncate_inode() now accepts a truncation length which makes
implementing truncate() and ftruncate() simple.
This also allowed implementing the F_FREESP fcntl().
. new_node() now returns inode of parent dir as argument that
has to be put_node()d again by the caller of new_node().
it can also return the name of the last component as last_dir()
did.
. advance() takes a pointer to a pointer of an inode as the
parent now. This parent can change, in which case the old
one is put_node()d and a new one is put there.
. eat_path() replaced by more flexible parse_path()
. last_dir() replaced by call to parse_path().
. do_slink(), do_readlink(), do_lstat() and slink_traverse() added.
Also added some truncate()/ftruncate()-introduction related changes.
(e.g. renamed truncate() to truncate_inode().)
New Shift-F6 dump for RS server at IS.
New getnpid, getnproc, getpproc library calls at PM.
New reincarnation server (basic functionality is there now).
enforced. If a call is denied, this will be kprinted. Please report any such
errors, so that I can adjust the mask before returning errors instead of
warnings.
Wrote CMOS driver. All CMOS code from FS has been removed. Currently the
driver only supports get time calls. Set time is left out as an exercise
for the book readers ... startup scripts were updated because the CMOS driver
is needed early on. (IS got same treatment.) Don't forget to run MAKEDEV cmos
in /dev/, otherwise the driver cannot be loaded.
The user needs to set label=... to choose the driver of his or her choice.
This driver will be mapped onto the controller that is set in controller=...
Minor cleanup of kernel source code (boot image table now is static).
This was caused by a change in the shared driver code. Not log's fault.
Renamed #definitions of driver process numbers, e.g., TTY now is TTY_PROC_NR.
All known (special) processes now have consistent naming scheme. Kernel tasks
don't follow this scheme.
Kernel no longer keeps track of user alarms on behalf of the PM.
Instead, the PM maintains its own list of watchdog timers, and
uses one synchronous alarm (at the kernel) to get notifications
for expired user timers.
Not a really good solution (as it might not catch situations in which this
is caused by another bug), but the forrest of checks necessary might be worse
than this quick fix - because when looking for the cause, I found some other
cases in which the PM would panic as well. See info in bug 2 for details.
Another fix is to delay notification of PM by SYSTASK of signals delivered
internally until after the reply (e.g. of exec()), because the reply would
be messed up otherwise (receiving the notify instead of reply). This caused
SIGTRAP not to be delivered properly with traced processes.
variable. Parsing worked, but future requests for the variable (such as
by the sysenv command) returned truncated data. This caused the system
(e.g. setup script) to think the amount of memory was tiny, and caused
the enabling of swapspace, while it is unnecessary.
need /tmp any more since 16MB root device; increase to 3.0.5 to make new
CD with working FXP driver. (not tagged 3.0.5 yet as at driver bios-copy
workaround hasn't been done.)
* Removed some variants of the SYS_GETINFO calls from the kernel;
replaced them with new PM and utils libary functionality. Fixed
bugs in utils library that used old get_kenv() variant.
* Implemented a buffer in the kernel to gather random data.
Memory driver periodically checks this for /dev/random.
A better random algorithm can now be implemented in the driver.
Removed SYS_RANDOM; the SYS_GETINFO call is used instead.
* Remove SYS_KMALLOC from the kernel. Memory allocation can now
be done at the process manager with new 'other' library functions.
names. All system processes can now either use panic() or report() from
libutils, or redefine their own function. Assertions are done via the standard
<assert.h> functionality.
using phys_copy to copy zeroes there for every kb), which is a big
optimisation in some cases
fixed a bug that was introduced when function keys became notifies
The call works. Permission check, restriction of outstanding notifications
to be added. Low level code to make it work from within interrupt handlers
will be added as well.
