Commit graph

65 commits

Author SHA1 Message Date
Lionel Sambuc
a16ccbaaf8 Message type for VFS_UMOUNT
Change-Id: Ia3b8e818b45a31178215cad77fa0fa3e5325f18a
2014-07-28 17:05:31 +02:00
Lionel Sambuc
ecc9010c4b Message type for VFS_MOUNT
Change-Id: I4114f5a1aa4f9efe49a12996722499b695c17230
2014-07-28 17:05:30 +02:00
Ben Gras
565f13088f make vfs & filesystems use failable copying
Change the kernel to add features to vircopy and safecopies so that
transparent copy fixing won't happen to avoid deadlocks, and such copies
fail with EFAULT.

Transparently making copying work from filesystems (as normally done by
the kernel & VM when copying fails because of missing/readonly memory)
is problematic as it can happen that, for file-mapped ranges, that that
same filesystem that is blocked on the copy request is needed to satisfy
the memory range, leading to deadlock. Dito for VFS itself, if done with
a blocking call.

This change makes the copying done from a filesystem fail in such cases
with EFAULT by VFS adding the CPF_TRY flag to the grants. If a FS call
fails with EFAULT, VFS will then request the range to be made available
to VM after the FS is unblocked, allowing it to be used to satisfy the
range if need be in another VFS thread.

Similarly, for datacopies that VFS itself does, it uses the failable
vircopy variant and callers use a wrapper that talk to VM if necessary
to get the copy to work.

	. kernel: add CPF_TRY flag to safecopies
	. kernel: only request writable ranges to VM for the
	  target buffer when copying fails
	. do copying in VFS TRY-first
	. some fixes in VM to build SANITYCHECK mode
	. add regression test for the cases where
	  - a FS system call needs memory mapped in a process that the
	    FS itself must map.
	  - such a range covers more than one file-mapped region.
	. add 'try' mode to vircopy, physcopy
	. add flags field to copy kernel call messages
	. if CP_FLAG_TRY is set, do not transparently try
	  to fix memory ranges
	. for use by VFS when accessing user buffers to avoid
	  deadlock
	. remove some obsolete backwards compatability assignments
        . VFS: let thread scheduling work for VM requests too
          Allows VFS to make calls to VM while suspending and resuming
          the currently running thread. Does currently not work for the
          main thread.
        . VM: add fix memory range call for use by VFS

Change-Id: I295794269cea51a3163519a9cfe5901301d90b32
2014-07-28 17:05:14 +02:00
Lionel Sambuc
37598dccf1 Aligning dev_t to 64bits.
Change-Id: I630f72f8530dd4aaf05c35ca23683ae12c9f8328
2014-03-02 12:28:32 +01:00
David van Moolenbroek
24ec0d73b5 Clean up interface to PM and VFS
- introduce new call numbers, names, and field aliases;
- initialize request messages to zero for all ABI calls;
- format callnr.h in the same way as com.h;
- redo call tables in both servers;
- remove param.h namespace pollution in the servers;
- make brk(2) go to VM directly, rather than through PM;
- remove obsolete BRK, UTIME, and WAIT calls;
- clean up path copying routine in VFS;
- move remaining system calls from libminlib to libc;
- correct some errno-related mistakes in libc routines.

Change-Id: I2d8ec5d061cd7e0b30c51ffd77aa72ebf84e2565
2014-03-01 09:05:01 +01:00
David van Moolenbroek
2c8310fce6 VFS: store m_out as part of worker thread state
There is no need to pass pointers around when there is a structure
available that already stores other similar state, such as m_in.

Change-Id: I3164c5c55c71f443688103d1f0756c086eb05974
2014-03-01 09:05:00 +01:00
Ben Gras
6c8f7fc3ba netbsd dirent.h, import sys/sys *.h, mfs cleanup
. add all sys/sys headers not already present to help compiling
	. take netbsd dirent.h and struct dirent; main result is
	  introducing d_type and d_namlen that have to be set by getdents()
	  in all FS code implementing it
	. d_off is gone
	. alignment of the struct has become 8 bytes instead of 4
	. remove _MAX_BLOCK_SIZE, _MIN_BLOCK_SIZE, _STATIC_BLOCK_SIZE
	. libminlib: cleanup unused yet duplicate code
	. mfs: throw out the long-broken v1, v2 support
	. new test for dirent contents filled by getdents()

Change-Id: I1459755c7ba5e5d1c9396d3a587ce6e63ddc283e
2014-03-01 09:04:59 +01:00
David van Moolenbroek
6b3f4dc157 Input infrastructure, INPUT server, PCKBD driver
This commit separates the low-level keyboard driver from TTY, putting
it in a separate driver (PCKBD). The commit also separates management
of raw input devices from TTY, and puts it in a separate server
(INPUT). All keyboard and mouse input from hardware is sent by drivers
to the INPUT server, which either sends it to a process that has
opened a raw input device, or otherwise forwards it to TTY for
standard processing.

Design by Dirk Vogt. Prototype by Uli Kastlunger.

Additional changes made to the prototype:

- the event communication is now based on USB HID codes; all input
  drivers have to use USB codes to describe events;
- all TTY keymaps have been converted to USB format, with the effect
  that a single keymap covers all keys; there is no (static) escaped
  keymap anymore;
- further keymap tweaks now allow remapping of literally all keys;
- input device renumbering and protocol rewrite;
- INPUT server rewrite, with added support for cancel and select;
- PCKBD reimplementation, including PC/AT-to-USB translation;
- support for manipulating keyboard LEDs has been added;
- keyboard and mouse multiplexer devices have been added to INPUT,
  primarily so that an X server need only open two devices;
- a new "libinputdriver" library abstracts away protocol details from
  input drivers, and should be used by all future input drivers;
- both INPUT and PCKBD can be restarted;
- TTY is now scheduled by KERNEL, so that it won't be punished for
  running a lot; without this, simply running "yes" on the console
  kills the system;
- the KIOCBELL IOCTL has been moved to /dev/console;
- support for the SCANCODES termios setting has been removed;
- obsolete keymap compression has been removed;
- the obsolete Olivetti M24 keymap has been removed.

Change-Id: I3a672fb8c4fd566734e4b46d3994b4b7fc96d578
2014-03-01 09:04:55 +01:00
David van Moolenbroek
723e51327f VFS: worker thread model overhaul
The main purpose of this patch is to fix handling of unpause calls
from PM while another call is ongoing. The solution to this problem
sparked a full revision of the threading model, consisting of a large
number of related changes:

- all active worker threads are now always associated with a process,
  and every process has at most one active thread working for it;
- the process lock is always held by a process's worker thread;
- a process can now have both normal work and postponed PM work
  associated to it;
- timer expiry and non-postponed PM work is done from the main thread;
- filp garbage collection is done from a thread associated with VFS;
- reboot calls from PM are now done from a thread associated with PM;
- the DS events handler is protected from starting multiple threads;
- support for a system worker thread has been removed;
- the deadlock recovery thread has been replaced by a parameter to the
  worker_start() function; the number of worker threads has
  consequently been increased by one;
- saving and restoring of global but per-thread variables is now
  centralized in worker_suspend() and worker_resume(); err_code is now
  saved and restored in all cases;
- the concept of jobs has been removed, and job_m_in now points to a
  message stored in the worker thread structure instead;
- the PM lock has been removed;
- the separate exec lock has been replaced by a lock on the VM
  process, which was already being locked for exec calls anyway;
- PM_UNPAUSE is now processed as a postponed PM request, from a thread
  associated with the target process;
- the FP_DROP_WORK flag has been removed, since it is no longer more
  than just an optimization and only applied to processes operating on
  a pipe when getting killed;
- assignment to "fp" now takes place only when obtaining new work in
  the main thread or a worker thread, when resuming execution of a
  thread, and in the special case of exiting processes during reboot;
- there are no longer special cases where the yield() call is used to
  force a thread to run.

Change-Id: I7a97b9b95c2450454a9b5318dfa0e6150d4e6858
2014-02-18 11:25:03 +01:00
David van Moolenbroek
cc810ee4d9 VFS/FS: replace protocol version with flag field
The main motivation for this change is that only Loris supports
multithreading, and Loris supports dynamic thread allocation, so the
number of supported threads can be implemented as a bit flag (i.e.,
either 1 or "at least as many as VFS has"). The ABI break obviates the
need to support file system versioning at this time, and several
other aspects are better implemented as flags as well. Other changes:

- replace peek/bpeek test upon mount with FS flag as well;
- mark libsffs as 64-bit file size capable;
- remove old (3.2.1) getdents support.

Change-Id: I313eace9c50ed816656c31cd47d969033d952a03
2014-02-18 11:25:02 +01:00
David van Moolenbroek
266239fe64 Implement support for getvfsstat(2)
Change-Id: I99b697919d411c57105de561105beefc7d1d309a
2014-02-18 11:25:02 +01:00
David van Moolenbroek
7113bcb896 Redo mount(2)/umount(2) ABI
- pass in file system type through mount(2), and return this type in
  statvfs structures as generated by [f]statvfs(2);
- align mount flags field with NetBSD's, splitting out service flags
  which are not to be passed to VFS;
- remove limitation of mount ABI to 16-byte labels, so that labels
  can be made larger in the future;
- introduce new m11 message union type for mount(2) as side effect.

Change-Id: I88b7710e297e00a5e4582ada5243d3d5c2801fd9
2014-02-18 11:25:01 +01:00
Thomas Veerman
c66fd312d4 VFS-FS protocol: add versioning
Change-Id: Ice6fbfd4b535b7435653fa08b27a3378d1cfdbf8
2014-02-18 11:25:00 +01:00
Ben Gras
44f34e53d5 VFS: Implement REQ_BPEEK.
This commit introduces a new request type called REQ_BPEEK. It
requests minor device blocks from the FS.  Analogously to REQ_PEEK,
it requests the filesystem to get the requested blocks into its
cache, without actually copying the result anywhere.

Change-Id: If1d06645b0e17553a64b3167091e9d12efeb3d6f
2013-04-24 10:18:16 +00:00
Ben Gras
cef94e096e vfs: make m_out non-global
m_out is shared between threads as the reply message, and it can happen
results get overwritten by another thread before the reply is sent. This
change

	. makes m_out local to the message handling function,
	  declared on the stack of the caller
	. forces callers of reply() to give it a message, or
	  declare the reply message has no significant fields except
	  for the return code by calling replycode()

Change-Id: Id06300083a63c72c00f34f86a5c7d96e4bbdf9f6
2013-04-12 23:40:38 +00:00
Kees Jongenburger
c0c581a635 vfs:fix for variable 'rfp' set but not used.
mount.c: In function 'mount_pfs':
mount.c:395:17: error: variable 'rfp' set but not used [-Werror=unused-but-set-variable]

Change-Id: I2f22590ab4e3a4a1678e9096626ebca53d2660e6
2013-01-07 09:12:27 +01:00
Thomas Veerman
d9f4f71916 Implement dynamic mtab support
With this patch /etc/mtab becomes obsolete.
2012-11-26 15:20:18 +00:00
Thomas Veerman
ed23a7a7d2 VFS: fix reboot panic with mounted FUSE FS
Upon reboot VFS semi-exits all processes and unmounts the file system.
However, upon unmount, exiting FUSE file systems might need service from
the file system (due to libc). As the FUSE process is halfway the exit
procedure, it doesn't have a valid root directory and working directory.
Trying to do system calls then triggers a sanity check in VFS.

This fix first exits normal processes which should then allow for
unmounting FUSE file systems. Then VFS exits all processes including
File Servers and unmounts the rest of the file system.
2012-11-14 13:18:16 +00:00
Thomas Veerman
badec36b33 VFS: fix deadlock when out of worker threads
There is a deadlock vulnerability when there are no worker threads
available and all of them blocked on a worker thread that's waiting for a
reply from a driver or a reply from an FS that needs to make a back call. In
these cases the deadlock resolver thread should kick in, but didn't in all
cases. Moreover, POSIX calls from File Servers weren't handled properly
anymore, which also could lead to deadlocks.
2012-11-14 13:12:37 +00:00
Thomas Veerman
3881e732a9 VFS: panic when unmount_all fails 2012-09-17 11:01:46 +00:00
Thomas Veerman
77dbd766c1 VFS: Use safe string copy functions 2012-07-16 10:57:43 +00:00
Thomas Veerman
db8198d99d VFS: use S_IS* macros 2012-04-27 08:49:38 +00:00
Thomas Veerman
8f55767619 VFS: make m_in job local
By making m_in job local (i.e., each job has its own copy of m_in instead
of refering to the global m_in) we don't have to store and restore m_in
on every thread yield. This reduces overhead. Moreover, remove the
assumption that m_in is preserved. Do_XXX functions have to copy the
system call parameters as soon as possible and only pass those copies to
other functions.

Furthermore, this patch cleans up some code and uses better types in a lot
of places.
2012-04-13 12:50:38 +00:00
Ben Gras
7336a67dfe retire PUBLIC, PRIVATE and FORWARD 2012-03-25 21:58:14 +02:00
Ben Gras
6a73e85ad1 retire _PROTOTYPE
. only good for obsolete K&R support
	. also remove a stray ansi.h and the proto cmd
2012-03-25 16:17:10 +02:00
Thomas Veerman
1efb51b1de VFS: improve crashed FS resource cleanup
When VFS detects that an FS has crashed and tries to clean up
resources, it marks fairly late in the process that a vmnt is not
to be used again (to send requests to). This allows a thread to
become blocked on a vmnt after all blocked threads were stopped, but
before it finds out it shouldn't try to send to that vmnt.
2012-02-22 13:54:35 +00:00
Thomas Veerman
80c4685324 VFS: replace VFS with AVFS 2012-02-13 16:53:21 +00:00
David van Moolenbroek
db087efac4 VFS/FS: REQ_NEW_DRIVER now provides a label 2011-11-30 19:05:26 +01:00
David van Moolenbroek
28f2a169da VFS: bugfixes for handling block-special files:
- on driver restarts, reopen devices on a per-file basis, not per-mount
- do not assume that there is just one vnode per block-special device
- update block-special files in the uncommon mounting success paths, too
- upon mount, sync but also invalidate affected buffers on the root FS
- upon unmount, check whether a vnode is in use before updating it
2011-03-25 10:56:43 +00:00
Dirk Vogt
5e1e763506 removed unneeded global var 2010-11-24 16:30:13 +00:00
Dirk Vogt
9ed280d1ec decouple file system server start/termination from mount/umount 2010-11-23 19:34:56 +00:00
Thomas Veerman
13ef7f1f38 Prepare VFS to support back calls from PFS. For security reasons and to support
file descriptor passing, PFS does some back calls to VFS. For example, to
verify the validity of a path provided by a process and to tell VFS it must
copy file descriptors from one process to another.
2010-08-30 13:44:07 +00:00
Thomas Veerman
c8cfcab5db - Make sure there's space left in the vmnt table for another mount point.
- Increase mount point limit.
2010-08-17 10:02:50 +00:00
Ben Gras
19b790eb53 vfs: don't use a mountpoint if it's in use for anything else.
(this avoids data structure confusion if a mountpoint is reused as
a mountpoint until that's properly fixed.)
2010-06-11 11:41:56 +00:00
Tomas Hruby
6e25ad8b0a Use of all NIL_* defines converted to NULL 2010-05-10 13:26:00 +00:00
Kees van Reeuwijk
86a23c1fbd Remove U16_t and most other similar types. Rewrite functions to ansi-style
declaration if necessary.
2010-04-21 11:05:22 +00:00
Kees van Reeuwijk
bc314bda91 Remove the types Dev_t, _mnx_Gui, _mnx_Uid, and similar.
Use ANSI-style function declarations where necessary.
2010-04-13 10:58:41 +00:00
Cristiano Giuffrida
48c6bb79f4 Driver refactory for live update and crash recovery.
SYSLIB CHANGES:
- DS calls to publish / retrieve labels consider endpoints instead of u32_t.

VFS CHANGES:
- mapdriver() only adds an entry in the dmap table in VFS.
- dev_up() is only executed upon reception of a driver up event.

INET CHANGES:
- INET no longer searches for existing drivers instances at startup.
- A newtwork driver is (re)initialized upon reception of a driver up event.
- Networking startup is now race-free by design. No need to waste 5 seconds
at startup any more.

DRIVER CHANGES:
- Every driver publishes driver up events when starting for the first time or
in case of restart when recovery actions must be taken in the upper layers.
- Driver up events are published by drivers through DS. 
- For regular drivers, VFS is normally the only subscriber, but not necessarily.
For instance, when the filter driver is in use, it must subscribe to driver
up events to initiate recovery.
- For network drivers, inet is the only subscriber for now.
- Every VFS driver is statically linked with libdriver, every network driver
is statically linked with libnetdriver.

DRIVER LIBRARIES CHANGES:
- Libdriver is extended to provide generic receive() and ds_publish() interfaces
for VFS drivers.
- driver_receive() is a wrapper for sef_receive() also used in driver_task()
to discard spurious messages that were meant to be delivered to a previous
version of the driver.
- driver_receive_mq() is the same as driver_receive() but integrates support
for queued messages.
- driver_announce() publishes a driver up event for VFS drivers and marks
the driver as initialized and expecting a DEV_OPEN message.
- Libnetdriver is introduced to provide similar receive() and ds_publish()
interfaces for network drivers (netdriver_announce() and netdriver_receive()).
- Network drivers all support live update with no state transfer now.

KERNEL CHANGES:
- Added kernel call statectl for state management. Used by driver_announce() to
unblock eventual callers sendrecing to the driver.
2010-04-08 13:41:35 +00:00
Kees van Reeuwijk
fc7dced1fa Fix printfs with too few or too many parms, remove unused vars, fix incorrect flag tests, other code cleanup. 2010-04-01 13:25:05 +00:00
Ben Gras
35a108b911 panic() cleanup.
this change
   - makes panic() variadic, doing full printf() formatting -
     no more NO_NUM, and no more separate printf() statements
     needed to print extra info (or something in hex) before panicing
   - unifies panic() - same panic() name and usage for everyone -
     vm, kernel and rest have different names/syntax currently
     in order to implement their own luxuries, but no longer
   - throws out the 1st argument, to make source less noisy.
     the panic() in syslib retrieves the server name from the kernel
     so it should be clear enough who is panicing; e.g.
         panic("sigaction failed: %d", errno);
     looks like:
         at_wini(73130): panic: sigaction failed: 0
         syslib:panic.c: stacktrace: 0x74dc 0x2025 0x100a
   - throws out report() - printf() is more convenient and powerful
   - harmonizes/fixes the use of panic() - there were a few places
     that used printf-style formatting (didn't work) and newlines
     (messes up the formatting) in panic()
   - throws out a few per-server panic() functions
   - cleans up a tie-in of tty with panic()

merging printf() and panic() statements to be done incrementally.
2010-03-05 15:05:11 +00:00
Cristiano Giuffrida
c5b309ff07 Merge of Wu's GSOC 09 branch (src.20090525.r4372.wu)
Main changes:
- COW optimization for safecopy.
- safemap, a grant-based interface for sharing memory regions between processes.
- Integration with safemap and complete rework of DS, supporting new data types
  natively (labels, memory ranges, memory mapped ranges).
- For further information:
  http://wiki.minix3.org/en/SummerOfCode2009/MemoryGrants

Additional changes not included in the original Wu's branch:
- Fixed unhandled case in VM when using COW optimization for safecopy in case
  of a block that has already been shared as SMAP.
- Better interface and naming scheme for sys_saferevmap and ds_retrieve_map
  calls.
- Better input checking in syslib: check for page alignment when creating
  memory mapping grants.
- DS notifies subscribers when an entry is deleted.
- Documented the behavior of indirect grants in case of memory mapping.
- Test suite in /usr/src/test/safeperf|safecopy|safemap|ds/* reworked
  and extended.
- Minor fixes and general cleanup.
- TO-DO: Grant ids should be generated and managed the way endpoints are to make
sure grant slots are never misreused.
2010-01-14 15:24:16 +00:00
David van Moolenbroek
b31119abf5 Mount updates:
- allow mounting with "none" block device
- allow unmounting by mountpoint
- make VFS aware of file system process labels
- allow m3_ca1 to use the full available message size
- use *printf in u/mount(1), as mount(2) uses it already
- fix reference leaks for some mount error cases in VFS
2010-01-12 23:08:50 +00:00
Thomas Veerman
958b25be50 - Introduce support for sticky bit.
- Revise VFS-FS protocol and update VFS/MFS/ISOFS accordingly.
- Clean up MFS by removing old, dead code (backwards compatibility is broken by
  the new VFS-FS protocol, anyway) and rewrite other parts. Also, make sure all
  functions have proper banners and prototypes.
- VFS should always provide a (syntactically) valid path to the FS; no need for
  the FS to do sanity checks when leaving/entering mount points.
- Fix several bugs in MFS:
  - Several path lookup bugs in MFS.
  - A link can be too big for the path buffer.
  - A mountpoint can become inaccessible when the creation of a new inode
    fails, because the inode already exists and is a mountpoint.
- Introduce support for supplemental groups.
- Add test 46 to test supplemental group functionality (and removed obsolete
  suppl. tests from test 2).
- Clean up VFS (not everything is done yet).
- ISOFS now opens device read-only. This makes the -r flag in the mount command
  unnecessary (but will still report to be mounted read-write).
- Introduce PipeFS. PipeFS is a new FS that handles all anonymous and
  named pipes. However, named pipes still reside on the (M)FS, as they are part
  of the file system on disk. To make this work VFS now has a concept of
  'mapped' inodes, which causes read, write, truncate and stat requests to be
  redirected to the mapped FS, and all other requests to the original FS.
2009-12-20 20:27:14 +00:00
Tomas Hruby
f3e0c5c381 VFS quits gracefully if mount fails and mounted_on remains uninitialized 2009-08-18 13:30:05 +00:00
Ben Gras
dc1238b7b9 make unpause() decrease susp_count, as it shouldn't be decreased
if the process was REVIVING. (susp_count doesn't count those
 processes.) this together with dev_io SELECT suspend side effect
 for asynch. character devices solves the hanging pipe bug. or
 at last vastly improves it.

 added sanity checks, turned off by default.

 made the {NOT_,}{SUSPENDING,REVIVING} constants weirder to
 help sanity checking.
2009-05-08 13:56:41 +00:00
Ben Gras
fd7ef243e4 cleanup of vfs shutdown logic; makes clean unmounts easier (but
needs checking if fp_wd or fp_rd is NULL before use)
2009-04-29 16:59:18 +00:00
Ben Gras
3bb80322d9 suppress more mostly-harmless messages. 2009-03-26 16:11:27 +00:00
Ben Gras
2d1c884e35 suppress these noisy, alarming messages. 2009-03-26 15:56:08 +00:00
Ben Gras
8af5f877bc 2009-03-04 17:44:34 +00:00
Ben Gras
3f6e061948 fix error check 2009-03-04 17:38:27 +00:00