Commit graph

80 commits

Author SHA1 Message Date
Erik van der Kouwe
23284ee7bd User-space scheduling for system processes 2010-07-01 08:32:33 +00:00
Cristiano Giuffrida
06700d05d1 Give RS a page table. 2010-06-28 21:53:37 +00:00
Arun Thomas
4c10a31440 Remove legacy MM, FS, and FS_PROC_NR macros 2010-06-08 13:58:01 +00:00
Arun Thomas
007104d60e GCC build fixes/updates
-Set stack sizes for boot image processes
-Increase RS stack size
-Reduce ramdisk size
-HARDWARE task should use kernel stack
-Minor asm tweaks for leading underscores
2010-05-26 18:45:55 +00:00
Tomas Hruby
451a6890d6 scheduling - time quantum in miliseconds
- Currently the cpu time quantum is timer-ticks based. Thus the
  remaining quantum is decreased only if the processes is interrupted
  by a timer tick. As processes block a lot this typically does not
  happen for normal user processes. Also the quantum depends on the
  frequency of the timer.

- This change makes the quantum miliseconds based. Internally the
  miliseconds are translated into cpu cycles. Everytime userspace
  execution is interrupted by kernel the cycles just consumed by the
  current process are deducted from the remaining quantum.

- It makes the quantum system timer frequency independent.

- The boot processes quantum is loosely derived from the tick-based
  quantas and 60Hz timer and subject to future change

- the 64bit arithmetics is a little ugly, will be changes once we have
  compiler support for 64bit integers (soon)
2010-05-25 08:06:14 +00:00
Tomas Hruby
dcc81d73e8 boot image - no need for entry point
- removes the initial_pc from struct boot_image. It is always set
  to 0 and RS uses a.out headers.
2010-05-18 13:51:46 +00:00
Tomas Hruby
b09bcf6779 Scheduling server (by Bjorn Swift)
In this second phase, scheduling is moved from PM to its own
scheduler (see r6557 for phase one). In the next phase we hope to a)
include useful information in the "out of quantum" message and b)
create some simple scheduling policy that makes use of that
information.

When the system starts up, PM will iterate over its process table and
ask SCHED to take over scheduling unprivileged processes. This is
done by sending a SCHEDULING_START message to SCHED. This message
includes the processes endpoint, the parent's endpoint and its nice
level. The scheduler adds this process to its schedproc table, issues
a schedctl, and returns its own endpoint to PM - as the endpoint of
the effective scheduler. When a process terminates, a SCHEDULING_STOP
message is sent to the scheduler.

The reason for this effective endpoint is for future compatibility.
Some day, we may have a scheduler that, instead of scheduling the
process itself, forwards the SCHEDULING_START message on to another
scheduler.

PM has information on who schedules whom. As such, scheduling
messages from user-land are sent through PM. An example is when
processes change their priority, using nice(). In that case, a
getsetpriority message is sent to PM, which then sends a
SCHEDULING_SET_NICE to the process's effective scheduler.

When a process is forked through PM, it inherits its parent's
scheduler, but is spawned with an empty quantum. As before, a request
to fork a process flows through VM before returning to PM, which then
wakes up the child process. This flow has been modified slightly so
that PM notifies the scheduler of the new process, before waking up
the child process. If the scheduler fails to take over scheduling,
the child process is torn down and the fork fails with an erroneous
value.

Process priority is entirely decided upon using nice levels. PM
stores a copy of each process's nice level and when a child is
forked, its parent's nice level is sent in the SCHEDULING_START
message. How this level is mapped to a priority queue is up to the
scheduler. It should be noted that the nice level is used to
determine the max_priority and the parent could have been in a lower
priority when it was spawned. To prevent a CPU intensive process from
hawking the CPU by continuously forking children that get scheduled
in the max_priority, the scheduler should determine in which queue
the parent is currently scheduled, and schedule the child in that
same queue.

Other fixes: The USER_Q in kernel/proc.h was incorrectly defined as
NR_SCHED_QUEUES/2. That results in a "off by one" error when
converting priority->nice->priority for nice=0. This also had the
side effect that if someone were to set the MAX_USER_Q to something
else than 0, then USER_Q would be off.
2010-05-18 13:39:04 +00:00
Cristiano Giuffrida
45db6482e8 Prioritized NOTIFY messages for reliable asynchonrous delivery of system events. 2010-03-22 23:44:55 +00:00
Tomas Hruby
ebba20a65d No CLOCK task
- no kernel tasks are runnable

- clock initialization moved to the end of main()

- the rest of the body of clock_task() is moved to bsp_timer_int_handler() as
  for now we are going to handle this on the bootstrap cpu. A change later is
  possible.
2010-02-09 15:22:43 +00:00
Tomas Hruby
728f0f0c49 Removal of the system task
* Userspace change to use the new kernel calls

	- _taskcall(SYSTASK...) changed to _kernel_call(...)

	- int 32 reused for the kernel calls

	- _do_kernel_call() to make the trap to kernel

	- kernel_call() to make the actuall kernel call from C using
	  _do_kernel_call()

	- unlike ipc call the kernel call always succeeds as kernel is
	  always available, however, kernel may return an error

* Kernel side implementation of kernel calls

	- the SYSTEm task does not run, only the proc table entry is
	  preserved

	- every data_copy(SYSTEM is no data_copy(KERNEL

	- "locking" is an empty operation now as everything runs in
	  kernel

	- sys_task() is replaced by kernel_call() which copies the
	  message into kernel, dispatches the call to its handler and
	  finishes by either copying the results back to userspace (if
	  need be) or by suspending the process because of VM

	- suspended processes are later made runnable once the memory
	  issue is resolved, picked up by the scheduler and only at
	  this time the call is resumed (in fact restarted) which does
	  not need to copy the message from userspace as the message
	  is already saved in the process structure.

	- no ned for the vmrestart queue, the scheduler will restart
	  the system calls

	- no special case in do_vmctl(), all requests remove the
	  RTS_VMREQUEST flag
2010-02-09 15:20:09 +00:00
Thomas Veerman
958b25be50 - Introduce support for sticky bit.
- Revise VFS-FS protocol and update VFS/MFS/ISOFS accordingly.
- Clean up MFS by removing old, dead code (backwards compatibility is broken by
  the new VFS-FS protocol, anyway) and rewrite other parts. Also, make sure all
  functions have proper banners and prototypes.
- VFS should always provide a (syntactically) valid path to the FS; no need for
  the FS to do sanity checks when leaving/entering mount points.
- Fix several bugs in MFS:
  - Several path lookup bugs in MFS.
  - A link can be too big for the path buffer.
  - A mountpoint can become inaccessible when the creation of a new inode
    fails, because the inode already exists and is a mountpoint.
- Introduce support for supplemental groups.
- Add test 46 to test supplemental group functionality (and removed obsolete
  suppl. tests from test 2).
- Clean up VFS (not everything is done yet).
- ISOFS now opens device read-only. This makes the -r flag in the mount command
  unnecessary (but will still report to be mounted read-write).
- Introduce PipeFS. PipeFS is a new FS that handles all anonymous and
  named pipes. However, named pipes still reside on the (M)FS, as they are part
  of the file system on disk. To make this work VFS now has a concept of
  'mapped' inodes, which causes read, write, truncate and stat requests to be
  redirected to the mapped FS, and all other requests to the original FS.
2009-12-20 20:27:14 +00:00
Cristiano Giuffrida
f4574783dc Rewrite of boot process
KERNEL CHANGES:
- The kernel only knows about privileges of kernel tasks and the root system
process (now RS).
- Kernel tasks and the root system process are the only processes that are made
schedulable by the kernel at startup. All the other processes in the boot image
don't get their privileges set at startup and are inhibited from running by the
RTS_NO_PRIV flag.
- Removed the assumption on the ordering of processes in the boot image table.
System processes can now appear in any order in the boot image table.
- Privilege ids can now be assigned both statically or dynamically. The kernel
assigns static privilege ids to kernel tasks and the root system process. Each
id is directly derived from the process number.
- User processes now all share the static privilege id of the root user
process (now INIT).
- sys_privctl split: we have more calls now to let RS set privileges for system
processes. SYS_PRIV_ALLOW / SYS_PRIV_DISALLOW are only used to flip the
RTS_NO_PRIV flag and allow / disallow a process from running. SYS_PRIV_SET_SYS /
SYS_PRIV_SET_USER are used to set privileges for a system / user process.
- boot image table flags split: PROC_FULLVM is the only flag that has been
moved out of the privilege flags and is still maintained in the boot image
table. All the other privilege flags are out of the kernel now.

RS CHANGES:
- RS is the only user-space process who gets to run right after in-kernel
startup.
- RS uses the boot image table from the kernel and three additional boot image
info table (priv table, sys table, dev table) to complete the initialization
of the system.
- RS checks that the entries in the priv table match the entries in the boot
image table to make sure that every process in the boot image gets schedulable.
- RS only uses static privilege ids to set privileges for system services in
the boot image.
- RS includes basic memory management support to allocate the boot image buffer
dynamically during initialization. The buffer shall contain the executable
image of all the system services we would like to restart after a crash.
- First step towards decoupling between resource provisioning and resource
requirements in RS: RS must know what resources it needs to restart a process
and what resources it has currently available. This is useful to tradeoff
reliability and resource consumption. When required resources are missing, the
process cannot be restarted. In that case, in the future, a system flag will
tell RS what to do. For example, if CORE_PROC is set, RS should trigger a
system-wide panic because the system can no longer function correctly without
a core system process.

PM CHANGES:
- The process tree built at initialization time is changed to have INIT as root
with pid 0, RS child of INIT and all the system services children of RS. This
is required to make RS in control of all the system services.
- PM no longer registers labels for system services in the boot image. This is
now part of RS's initialization process.
2009-12-11 00:08:19 +00:00
Tomas Hruby
cb9faaebfd No need for a special idle queue
- as the idle task is never placed on any run queue, we don't need any special
  idle queue.

- one more queue available for user processes
2009-11-12 08:47:25 +00:00
Tomas Hruby
ad4dcaab71 Idle task never runs
- idle task becomes a pseudo task which is never scheduled. It is never put on
  any run queue and never enters userspace. An entry for this task still remains
  in the process table for time accounting

- Instead of panicing if there is not process to schedule, pick_proc() returns
  NULL which is a signal to put the cpu in an idle state and set everything in
  such a way that after receiving and interrupt it looks like idle task was
  preempted

- idle task is set non-preemptible to avoid handling in the timer interrupt code
  which make userspace scheduling simpler as idle task does not need to be
  handled as a special case.
2009-11-12 08:42:18 +00:00
Tomas Hruby
ae75f9d4e5 Removal of the executable flag from files that cannot be executed
- 755 -> 644
2009-11-09 10:26:00 +00:00
Ben Gras
cd8b915ed9 Primary goal for these changes is:
- no longer have kernel have its own page table that is loaded
    on every kernel entry (trap, interrupt, exception). the primary
    purpose is to reduce the number of required reloads.
Result:
  - kernel can only access memory of process that was running when
    kernel was entered
  - kernel must be mapped into every process page table, so traps to
    kernel keep working
Problem:
  - kernel must often access memory of arbitrary processes (e.g. send
    arbitrary processes messages); this can't happen directly any more;
    usually because that process' page table isn't loaded at all, sometimes
    because that memory isn't mapped in at all, sometimes because it isn't
    mapped in read-write.
So:
  - kernel must be able to map in memory of any process, in its own
    address space.
Implementation:
  - VM and kernel share a range of memory in which addresses of
    all page tables of all processes are available. This has two purposes:
      . Kernel has to know what data to copy in order to map in a range
      . Kernel has to know where to write the data in order to map it in
    That last point is because kernel has to write in the currently loaded
    page table.
  - Processes and kernel are separated through segments; kernel segments
    haven't changed.
  - The kernel keeps the process whose page table is currently loaded
    in 'ptproc.'
  - If it wants to map in a range of memory, it writes the value of the
    page directory entry for that range into the page directory entry
    in the currently loaded map. There is a slot reserved for such
    purposes. The kernel can then access this memory directly.
  - In order to do this, its segment has been increased (and the
    segments of processes start where it ends).
  - In the pagefault handler, detect if the kernel is doing
    'trappable' memory access (i.e. a pagefault isn't a fatal
     error) and if so,
       - set the saved instruction pointer to phys_copy_fault,
	 breaking out of phys_copy
       - set the saved eax register to the address of the page
	 fault, both for sanity checking and for checking in
	 which of the two ranges that phys_copy was called
	 with the fault occured
  - Some boot-time processes do not have their own page table,
    and are mapped in with the kernel, and separated with
    segments. The kernel detects this using HASPT. If such a
    process has to be scheduled, any page table will work and
    no page table switch is done.

Major changes in kernel are
  - When accessing user processes memory, kernel no longer
    explicitly checks before it does so if that memory is OK.
    It simply makes the mapping (if necessary), tries to do the
    operation, and traps the pagefault if that memory isn't present;
    if that happens, the copy function returns EFAULT.
    So all of the CHECKRANGE_OR_SUSPEND macros are gone.
  - Kernel no longer has to copy/read and parse page tables.
  - A message copying optimisation: when messages are copied, and
    the recipient isn't mapped in, they are copied into a buffer
    in the kernel. This is done in QueueMess. The next time
    the recipient is scheduled, this message is copied into
    its memory. This happens in schedcheck().
    This eliminates the mapping/copying step for messages, and makes
    it easier to deliver messages. This eliminates soft_notify.
  - Kernel no longer creates a page table at all, so the vm_setbuf
    and pagetable writing in memory.c is gone.

Minor changes in kernel are
  - ipc_stats thrown out, wasn't used
  - misc flags all renamed to MF_*
  - NOREC_* macros to enter and leave functions that should not
    be called recursively; just sanity checks really
  - code to fully decode segment selectors and descriptors
    to print on exceptions
  - lots of vmassert()s added, only executed if DEBUG_VMASSERT is 1
2009-09-21 14:31:52 +00:00
David van Moolenbroek
b8b8f537bd IPC privileges fixes
Kernel:
o Remove s_ipc_sendrec, instead using s_ipc_to for all send primitives
o Centralize s_ipc_to bit manipulation,
  - disallowing assignment of bits pointing to unused priv structs;
  - preventing send-to-self by not setting bit for own priv struct;
  - preserving send mask matrix symmetry in all cases
o Add IPC send mask checks to SENDA, which were missing entirely somehow
o Slightly improve IPC stats accounting for SENDA
o Remove SYSTEM from user processes' send mask
o Half-fix the dependency between boot image order and process numbers,
  - correcting the table order of the boot processes;
  - documenting the order requirement needed for proper send masks;
  - warning at boot time if the order is violated

RS:
o Add support in /etc/drivers.conf for servers that talk to user processes,
  - disallowing IPC to user processes if no "ipc" field is present
  - adding a special "USER" label to explicitly allow IPC to user processes
o Always apply IPC masks when specified; remove -i flag from service(8)
o Use kernel send mask symmetry to delay adding IPC permissions for labels
  that do not exist yet, adding them to that label's process upon creation
o Add VM to ipc permissions list for rtl8139 and fxp in drivers.conf

Left to future fixes:
o Removal of the table order vs process numbers dependency altogether,
  possibly using per-process send list structures as used for SYSTEM calls
o Proper assignment of send masks to boot processes;
  some of the assigned (~0) masks are much wider than necessary
o Proper assignment of IPC send masks for many more servers in drivers.conf
o Removal of the debugging warning about the now legitimate case where RS's
  add_forward_ipc cannot find the IPC destination's label yet
2009-07-02 16:25:31 +00:00
Ben Gras
e3ca89c0be more sanity checking. sanity checking disabled by default.
give every process a full pagetable by default now.

first step to disabling kernel page table code (processes
might not have page tables -> no address translation).
2009-05-12 11:35:01 +00:00
Ben Gras
3cc092ff06 . new kernel call sysctl for generic unprivileged system operations;
now used for printing diagnostic messages through the kernel message
   buffer. this lets processes print diagnostics without sending messages
   to tty and log directly, simplifying the message protocol a lot and
   reducing difficulties with deadlocks and other situations in which
   diagnostics are blackholed (e.g. grants don't work). this makes
   DIAGNOSTICS(_S), ASYN_DIAGNOSTICS and DIAG_REPL obsolete, although tty
   and log still accept the codes for 'old' binaries. This also simplifies
   diagnostics in several servers and drivers - only tty needs its own
   kputc() now.
 . simplifications in vfs, and some effort to get the vnode references
   right (consistent) even during shutdown. m_mounted_on is now NULL
   for root filesystems (!) (the original and new root), a less awkward
   special case than 'm_mounted_on == m_root_node'. root now has exactly
   one reference, to root, if no files are open, just like all other
   filesystems. m_driver_e is unused.
2009-01-26 17:43:59 +00:00
Ben Gras
b61687fb1b . VM needs a higher priority than VFS, PM etc
. introduce FULLVM flag: MEMORY and the initial MFS
   get their own full address spaces, making their stacks
   and heaps not preallocated (well, freed after VM has
   initialized it) and letting them allocate more dynamically.
   MEMORY in particular needs this to map in physical memory
   using its own page table, without having to allocate.
2008-12-11 14:21:47 +00:00
Ben Gras
c078ec0331 Basic VM and other minor improvements.
Not complete, probably not fully debugged or optimized.
2008-11-19 12:26:10 +00:00
Philip Homburg
f6872f8323 Added ipc_stats_target. 2008-02-22 12:25:44 +00:00
Philip Homburg
da4bb9144d Removed ECHO. 2007-04-23 13:46:26 +00:00
Ben Gras
75f8ceb70e let tty do sys_physcopy; needed for loadfont. 2007-03-22 16:15:33 +00:00
Ben Gras
6f77685609 Split of architecture-dependent and -independent functions for i386,
mainly in the kernel and headers. This split based on work by
Ingmar Alting <iaalting@cs.vu.nl> done for his Minix PowerPC architecture
port.

 . kernel does not program the interrupt controller directly, do any
   other architecture-dependent operations, or contain assembly any more,
   but uses architecture-dependent functions in arch/$(ARCH)/.
 . architecture-dependent constants and types defined in arch/$(ARCH)/include.
 . <ibm/portio.h> moved to <minix/portio.h>, as they have become, for now,
   architecture-independent functions.
 . int86, sdevio, readbios, and iopenable are now i386-specific kernel calls
   and live in arch/i386/do_* now.
 . i386 arch now supports even less 86 code; e.g. mpx86.s and klib86.s have
   gone, and 'machine.protected' is gone (and always taken to be 1 in i386).
   If 86 support is to return, it should be a new architecture.
 . prototypes for the architecture-dependent functions defined in
   kernel/arch/$(ARCH)/*.c but used in kernel/ are in kernel/proto.h
 . /etc/make.conf included in makefiles and shell scripts that need to
   know the building architecture; it defines ARCH=<arch>, currently only
   i386.
 . some basic per-architecture build support outside of the kernel (lib)
 . in clock.c, only dequeue a process if it was ready
 . fixes for new include files

files deleted:
 . mpx/klib.s - only for choosing between mpx/klib86 and -386
 . klib86.s - only for 86

i386-specific files files moved (or arch-dependent stuff moved) to arch/i386/:
 . mpx386.s (entry point)
 . klib386.s
 . sconst.h
 . exception.c
 . protect.c
 . protect.h
 . i8269.c
2006-12-22 15:22:27 +00:00
Ben Gras
7195fe3325 System statistical and call profiling
support by Rogier Meurs <rogier@meurs.org>.
2006-10-30 15:53:38 +00:00
Ben Gras
fa0ba56bc9 Merge of VFS by Balasz Gerofi with Minix trunk. 2006-10-25 13:40:36 +00:00
Philip Homburg
4ba5826ba6 Improved access checks in system.c. Grant drivers and FS the rights they need. 2006-08-10 10:56:16 +00:00
Ben Gras
ec4223d96a More stack for system. 2006-07-25 09:38:24 +00:00
Ben Gras
2e89eb5270 Change allowed kernel call bitmap to an array of bitmaps to allow
any number of kernel calls.

Allowed kernel calls are stored in table.c for every image process as a
variably-sized array of allowed calls. This is used to fill the bitmap
of size determined at compile time by the number of kernel calls. This
filling is done by main.c. There is a special call called SYS_ALL_CALLS
which fills the bitmap of allowed calls completely, if that is the only
entry in the array.
2006-06-20 09:56:06 +00:00
Philip Homburg
15b8fe54a8 Better initialization of the memory map of processes that are part of the
image. Removed NO_MAP flag.
2006-05-11 14:49:46 +00:00
Jorrit Herder
021e3234d8 Jorrit's ... "progress?" 2006-03-10 16:10:05 +00:00
Philip Homburg
7034c0795d Moved PCI driver to the ramdisk. 2006-02-15 14:21:56 +00:00
Philip Homburg
c3a88d15d8 Initial root filesystem is now on a ramdisk that is part of the image. 2006-02-15 11:18:21 +00:00
Ben Gras
0a02bfb056 Moved ipc debug out of conditional.
Gave pci server sys_getinfo privilege.
2006-02-06 15:16:14 +00:00
Philip Homburg
38a16399f8 Store resource lists for drivers. Limited checks to enforce those lists. 2006-01-27 13:21:12 +00:00
Philip Homburg
eec836014a PCI support in a separate driver 2005-12-02 14:45:38 +00:00
Philip Homburg
b85736bd7b Grant tty permission to use SYS_VM_MAP and SYS_IOPENABLE. 2005-11-09 15:48:41 +00:00
Philip Homburg
0b7fcbfd70 Only the memory driver should be allowed access to SYS_IOPENABLE 2005-11-04 17:06:49 +00:00
Jorrit Herder
86e3333454 New error codes for IPC errors.
Made VM code debug output disappear.
2005-10-20 20:25:32 +00:00
Jorrit Herder
b16a88692a New deadlock check at proc.c.
New exit checks (stop IPC).
2005-10-12 15:08:23 +00:00
Jorrit Herder
32c05f45c6 Minor bug fixes in proc.c and utility.c;
Removed debug dumps on panic from main.c;
Return error on exit when other process does IPC to/from exiting process.
2005-10-05 09:51:50 +00:00
Ben Gras
71b6f31d4e Let TTY call SYS_ABORT 2005-10-05 08:45:55 +00:00
Ben Gras
15c6f7182b Al's comment fixes 2005-10-02 19:02:05 +00:00
Philip Homburg
9bee3f4b08 IOPL, VM, and serial debug output (disabled). 2005-09-30 12:54:59 +00:00
Ben Gras
911ff6a873 Formatting fixes for the book 2005-08-29 16:47:18 +00:00
Jorrit Herder
7bf400a709 *** empty log message *** 2005-08-23 11:31:32 +00:00
Jorrit Herder
10e4e98947 Renamed SM to RS (Reincarnation Server). 2005-08-23 11:28:16 +00:00
Jorrit Herder
5ecb45c346 Improved scheduling: new way to catch infinite loops. 2005-08-23 10:25:01 +00:00
Ben Gras
d11b2e4b8c Al's double-blank-line removal request 2005-08-22 15:23:47 +00:00