Commit graph

36 commits

Author SHA1 Message Date
Ben Gras
565f13088f make vfs & filesystems use failable copying
Change the kernel to add features to vircopy and safecopies so that
transparent copy fixing won't happen to avoid deadlocks, and such copies
fail with EFAULT.

Transparently making copying work from filesystems (as normally done by
the kernel & VM when copying fails because of missing/readonly memory)
is problematic as it can happen that, for file-mapped ranges, that that
same filesystem that is blocked on the copy request is needed to satisfy
the memory range, leading to deadlock. Dito for VFS itself, if done with
a blocking call.

This change makes the copying done from a filesystem fail in such cases
with EFAULT by VFS adding the CPF_TRY flag to the grants. If a FS call
fails with EFAULT, VFS will then request the range to be made available
to VM after the FS is unblocked, allowing it to be used to satisfy the
range if need be in another VFS thread.

Similarly, for datacopies that VFS itself does, it uses the failable
vircopy variant and callers use a wrapper that talk to VM if necessary
to get the copy to work.

	. kernel: add CPF_TRY flag to safecopies
	. kernel: only request writable ranges to VM for the
	  target buffer when copying fails
	. do copying in VFS TRY-first
	. some fixes in VM to build SANITYCHECK mode
	. add regression test for the cases where
	  - a FS system call needs memory mapped in a process that the
	    FS itself must map.
	  - such a range covers more than one file-mapped region.
	. add 'try' mode to vircopy, physcopy
	. add flags field to copy kernel call messages
	. if CP_FLAG_TRY is set, do not transparently try
	  to fix memory ranges
	. for use by VFS when accessing user buffers to avoid
	  deadlock
	. remove some obsolete backwards compatability assignments
        . VFS: let thread scheduling work for VM requests too
          Allows VFS to make calls to VM while suspending and resuming
          the currently running thread. Does currently not work for the
          main thread.
        . VM: add fix memory range call for use by VFS

Change-Id: I295794269cea51a3163519a9cfe5901301d90b32
2014-07-28 17:05:14 +02:00
Lionel Sambuc
55d9e9954b Extending time_t to 64bits.
Change-Id: Ia96b8bfba19cb8179a0237a7d2122d415c24d73f
2014-03-03 20:45:28 +01:00
David van Moolenbroek
24ec0d73b5 Clean up interface to PM and VFS
- introduce new call numbers, names, and field aliases;
- initialize request messages to zero for all ABI calls;
- format callnr.h in the same way as com.h;
- redo call tables in both servers;
- remove param.h namespace pollution in the servers;
- make brk(2) go to VM directly, rather than through PM;
- remove obsolete BRK, UTIME, and WAIT calls;
- clean up path copying routine in VFS;
- move remaining system calls from libminlib to libc;
- correct some errno-related mistakes in libc routines.

Change-Id: I2d8ec5d061cd7e0b30c51ffd77aa72ebf84e2565
2014-03-01 09:05:01 +01:00
David van Moolenbroek
2c8310fce6 VFS: store m_out as part of worker thread state
There is no need to pass pointers around when there is a structure
available that already stores other similar state, such as m_in.

Change-Id: I3164c5c55c71f443688103d1f0756c086eb05974
2014-03-01 09:05:00 +01:00
David van Moolenbroek
664b726cd3 VFS: further cleanup of device code
- all TTY-related exceptions have now been merged into the regular
  code paths, allowing non-TTY drivers to expose TTY-like devices;
- as part of this, CTTY_MAJOR is now fully managed by VFS instead of
  being an ugly stepchild of the TTY driver;
- device styles have become completely obsolete, support for them has
  been removed throughout the system; same for device flags, which had
  already become useless a while ago;
- device map open/close and I/O function pointers have lost their use,
  thus finally making the VFS device code actually readable;
- the device-unrelated pm_setsid has been moved to misc.c;
- some other small cleanup-related changes.

Change-Id: If90b10d1818e98a12139da3e94a15d250c9933da
2014-03-01 09:04:58 +01:00
David van Moolenbroek
723e51327f VFS: worker thread model overhaul
The main purpose of this patch is to fix handling of unpause calls
from PM while another call is ongoing. The solution to this problem
sparked a full revision of the threading model, consisting of a large
number of related changes:

- all active worker threads are now always associated with a process,
  and every process has at most one active thread working for it;
- the process lock is always held by a process's worker thread;
- a process can now have both normal work and postponed PM work
  associated to it;
- timer expiry and non-postponed PM work is done from the main thread;
- filp garbage collection is done from a thread associated with VFS;
- reboot calls from PM are now done from a thread associated with PM;
- the DS events handler is protected from starting multiple threads;
- support for a system worker thread has been removed;
- the deadlock recovery thread has been replaced by a parameter to the
  worker_start() function; the number of worker threads has
  consequently been increased by one;
- saving and restoring of global but per-thread variables is now
  centralized in worker_suspend() and worker_resume(); err_code is now
  saved and restored in all cases;
- the concept of jobs has been removed, and job_m_in now points to a
  message stored in the worker thread structure instead;
- the PM lock has been removed;
- the separate exec lock has been replaced by a lock on the VM
  process, which was already being locked for exec calls anyway;
- PM_UNPAUSE is now processed as a postponed PM request, from a thread
  associated with the target process;
- the FP_DROP_WORK flag has been removed, since it is no longer more
  than just an optimization and only applied to processes operating on
  a pipe when getting killed;
- assignment to "fp" now takes place only when obtaining new work in
  the main thread or a worker thread, when resuming execution of a
  thread, and in the special case of exiting processes during reboot;
- there are no longer special cases where the yield() call is used to
  force a thread to run.

Change-Id: I7a97b9b95c2450454a9b5318dfa0e6150d4e6858
2014-02-18 11:25:03 +01:00
Ben Gras
33a7ac7557 vfs: mmap support
. libc: add vfs_mmap, a way for vfs to initiate mmap()s.
	  This is a good special case to have as vfs is a slightly
	  different client from regular user processes. It doesn't do it
	  for itself, and has the dev & inode info already so the callback
	  to VFS for the lookup isn't necessary. So it has different info
	  to have to give to VM.
	. libc: also add minix_mmap64() that accepts a 64-bit offset, even
	  though our off_t is still 32 bit now.
	. On exec() time, try to mmap() in the executable if available.
	  (It is not yet available in this commit.)
	. To support mmap(), add do_vm_call that allows VM to lookup
	  (to ino+dev), do i/o from and close FD's on behalf of other
	  processes.

Change-Id: I831551e45a6781c74313c450eb9c967a68505932
2013-05-31 15:42:00 +00:00
Ben Gras
cef94e096e vfs: make m_out non-global
m_out is shared between threads as the reply message, and it can happen
results get overwritten by another thread before the reply is sent. This
change

	. makes m_out local to the message handling function,
	  declared on the stack of the caller
	. forces callers of reply() to give it a message, or
	  declare the reply message has no significant fields except
	  for the return code by calling replycode()

Change-Id: Id06300083a63c72c00f34f86a5c7d96e4bbdf9f6
2013-04-12 23:40:38 +00:00
Antoine Leca
4069cef7f9 Subsecond timestamps support for FS
Expand REQ_UTIME to include tv_nsec members
(as in struct timespec) in addition to tv_sec==time_t

Designed with help from David van Moolenbroek
2013-04-12 11:11:59 +02:00
Thomas Cort
e67fc5771d libc: add clock_getres()/clock_gettime() system calls.
In order to make it more clear that ticks should be used for timers
and realtime should be used for timestamps / displaying the date/time,
getuptime() was renamed to getticks() and getuptime2() was renamed to
getuptime().

Servers, drivers, libraries, tests, etc that use getuptime()/getuptime2()
have been updated. In instances where a realtime was calculated, the
calculation was changed to use realtime.

System calls clock_getres() and clock_gettime() were added to PM/libc.
2013-04-04 15:04:53 +02:00
Ben Gras
e4ac80eb60 various warning/errorwarning fixes for gcc47
. warnings (sometimes promoted to errors) in servers/ and kernel/
 . -Os for ext2 boot module to make it small enough
2012-08-27 16:19:18 +02:00
Thomas Veerman
77dbd766c1 VFS: Use safe string copy functions 2012-07-16 10:57:43 +00:00
Thomas Veerman
b956493367 VFS: fix new signed/unsigned comparisons 2012-04-13 13:00:11 +00:00
Thomas Veerman
8f55767619 VFS: make m_in job local
By making m_in job local (i.e., each job has its own copy of m_in instead
of refering to the global m_in) we don't have to store and restore m_in
on every thread yield. This reduces overhead. Moreover, remove the
assumption that m_in is preserved. Do_XXX functions have to copy the
system call parameters as soon as possible and only pass those copies to
other functions.

Furthermore, this patch cleans up some code and uses better types in a lot
of places.
2012-04-13 12:50:38 +00:00
Ben Gras
7336a67dfe retire PUBLIC, PRIVATE and FORWARD 2012-03-25 21:58:14 +02:00
Thomas Veerman
80c4685324 VFS: replace VFS with AVFS 2012-02-13 16:53:21 +00:00
Thomas Veerman
203937456e Fix off-by-one errors and increase PATH_MAX to 1024
In some places it was assumed that PATH_MAX does not include a
terminating null character.

Increases PATH_MAX to 1024 to get in sync with NetBSD. Required some
rewriting in AVFS to keep memory usage low (the stack in use by a thread
is very small).
2011-09-12 09:00:24 +00:00
Ben Gras
a9d15dd3e4 pm, vfs: don't print something for bogus calls 2011-07-05 13:21:48 +02:00
Thomas Veerman
13ef7f1f38 Prepare VFS to support back calls from PFS. For security reasons and to support
file descriptor passing, PFS does some back calls to VFS. For example, to
verify the validity of a path provided by a process and to tell VFS it must
copy file descriptors from one process to another.
2010-08-30 13:44:07 +00:00
Erik van der Kouwe
498d7d8a4c Don't use kernel responses in servers 2010-06-24 07:37:26 +00:00
Kees van Reeuwijk
826b9590f2 More endpoint_t correctness.
More const correctness.
Other code cleanup.
2010-06-08 14:09:18 +00:00
Arun Thomas
4c10a31440 Remove legacy MM, FS, and FS_PROC_NR macros 2010-06-08 13:58:01 +00:00
Kees van Reeuwijk
fc7dced1fa Fix printfs with too few or too many parms, remove unused vars, fix incorrect flag tests, other code cleanup. 2010-04-01 13:25:05 +00:00
Ben Gras
35a108b911 panic() cleanup.
this change
   - makes panic() variadic, doing full printf() formatting -
     no more NO_NUM, and no more separate printf() statements
     needed to print extra info (or something in hex) before panicing
   - unifies panic() - same panic() name and usage for everyone -
     vm, kernel and rest have different names/syntax currently
     in order to implement their own luxuries, but no longer
   - throws out the 1st argument, to make source less noisy.
     the panic() in syslib retrieves the server name from the kernel
     so it should be clear enough who is panicing; e.g.
         panic("sigaction failed: %d", errno);
     looks like:
         at_wini(73130): panic: sigaction failed: 0
         syslib:panic.c: stacktrace: 0x74dc 0x2025 0x100a
   - throws out report() - printf() is more convenient and powerful
   - harmonizes/fixes the use of panic() - there were a few places
     that used printf-style formatting (didn't work) and newlines
     (messes up the formatting) in panic()
   - throws out a few per-server panic() functions
   - cleans up a tie-in of tty with panic()

merging printf() and panic() statements to be done incrementally.
2010-03-05 15:05:11 +00:00
Thomas Veerman
958b25be50 - Introduce support for sticky bit.
- Revise VFS-FS protocol and update VFS/MFS/ISOFS accordingly.
- Clean up MFS by removing old, dead code (backwards compatibility is broken by
  the new VFS-FS protocol, anyway) and rewrite other parts. Also, make sure all
  functions have proper banners and prototypes.
- VFS should always provide a (syntactically) valid path to the FS; no need for
  the FS to do sanity checks when leaving/entering mount points.
- Fix several bugs in MFS:
  - Several path lookup bugs in MFS.
  - A link can be too big for the path buffer.
  - A mountpoint can become inaccessible when the creation of a new inode
    fails, because the inode already exists and is a mountpoint.
- Introduce support for supplemental groups.
- Add test 46 to test supplemental group functionality (and removed obsolete
  suppl. tests from test 2).
- Clean up VFS (not everything is done yet).
- ISOFS now opens device read-only. This makes the -r flag in the mount command
  unnecessary (but will still report to be mounted read-write).
- Introduce PipeFS. PipeFS is a new FS that handles all anonymous and
  named pipes. However, named pipes still reside on the (M)FS, as they are part
  of the file system on disk. To make this work VFS now has a concept of
  'mapped' inodes, which causes read, write, truncate and stat requests to be
  redirected to the mapped FS, and all other requests to the original FS.
2009-12-20 20:27:14 +00:00
David van Moolenbroek
42f0bf7dda VFS: fetch_name() buffer underflow (reported by John Peace, bug #305) 2009-08-29 08:22:50 +00:00
Ben Gras
746e138036 turn off scary looking debug messages. 2009-05-07 09:57:43 +00:00
Ben Gras
3287b7f7d8 don't hang old binaries 2008-12-11 14:45:49 +00:00
Ben Gras
c078ec0331 Basic VM and other minor improvements.
Not complete, probably not fully debugged or optimized.
2008-11-19 12:26:10 +00:00
Ben Gras
67d1b67805 exit prototype 2007-10-23 14:19:16 +00:00
Ben Gras
e8aec69c7b tweak to panic functions of mfs and vfs.
. print newline
  . when recursive panic detected, don't simply return, confusing
    the caller, but print a diagnostic and exit
  . don't call sys_exit as this may confuse PM; it should be OK
    to call PM exit() nowadays.
2007-10-23 14:17:51 +00:00
Philip Homburg
f46319037b New VFS interface 2007-08-07 12:52:47 +00:00
Ben Gras
41e9fedf87 Mostly bugfixes of bugs triggered by the test set.
bugfixes:
 SYSTEM:
 . removed
        rc->p_priv->s_flags = 0;
   for the priv struct shared by all user processes in get_priv(). this
   should only be done once. doing a SYS_PRIV_USER in sys_privctl()
   caused the flags of all user processes to be reset, so they were no
   longer PREEMPTIBLE. this happened when RS executed a policy script.
   (this broke test1 in the test set)

 VFS/MFS:
 . chown can change the mode of a file, and chmod arguments are only
   part of the full file mode so the full filemode is slightly magic.
   changed these calls so that the final modes are returned to VFS, so
   that the vnode can be kept up-to-date.
   (this broke test11 in the test set)

 MFS:
 . lookup() checked for sizeof(string) instead of sizeof(user_path),
   truncating long path names
   (caught by test 23)
 . truncate functions neglected to update ctime
   (this broke test16)

 VFS:
 . corner case of an empty filename lookup caused fields of a request
   not to be filled in in the lookup functions, not making it clear
   that the lookup had failed, causing messages to garbage processes,
   causing strange failures.
   (caught by test 30)
 . trust v_size in vnode when doing reads or writes on non-special
   files, truncating i/o where necessary; this is necessary for pipes,
   as MFS can't tell when a pipe has been truncated without it being
   told explicitly each time.
   when the last reader/writer on a pipe closes, tell FS about
   the new size using truncate_vn().
   (this broke test 25, among others)
 . permission check for chdir() had disappeared; added a
   forbidden() call
   (caught by test 23)

new code, shouldn't change anything:
 . introduced RTS_SET, RTS_UNSET, and RTS_ISSET macro's, and their
   LOCK variants. These macros set and clear the p_rts_flags field,
   causing a lot of duplicated logic like

       old_flags = rp->p_rts_flags;            /* save value of the flags */
       rp->p_rts_flags &= ~NO_PRIV;
       if (old_flags != 0 && rp->p_rts_flags == 0) lock_enqueue(rp);

   to change into the simpler

       RTS_LOCK_UNSET(rp, NO_PRIV);

   so the macros take care of calling dequeue() and enqueue() (or lock_*()),
   as the case may be). This makes the code a bit more readable and a
   bit less fragile.
 . removed return code from do_clocktick in CLOCK as it currently
   never replies
 . removed some debug code from VFS
 . fixed grant debug message in device.c
 
preemptive checks, tests, changes:
 . added return code checks of receive() to SYSTEM and CLOCK
 . O_TRUNC should never arrive at MFS (added sanity check and removed
   O_TRUNC code)
 . user_path declared with PATH_MAX+1 to let it be null-terminated
 . checks in MFS to see if strings passed by VFS are null-terminated
 
 IS:
 . static irq name table thrown out
2007-02-01 17:50:02 +00:00
Ben Gras
2194bc0310 vfs/mount/rs/service changes:
. changed umount() and mount() to call 'service', so that it can include
   a custom label, so that umount() works again (RS slot gets freed now).
   merged umount() and mount() into one file to encode keep this label
   knowledge in one file.
 . removed obsolete RS_PID field and RS_RESCUE rescue command
 . added label to RS_START struct
 . vfs no longer does kill of fs process on unmount (which was failing
   due to RS_PID request not working)
 . don't assume that if error wasn't one of three errors, that no error
   occured in vfs/request.c
mfs changes:
 . added checks to copy statements to truncate copies at buffer sizes
   (left in debug code for now)
 . added checks for null-terminatedness, if less than NAME_MAX was copied
 . added checks for copy function success
is changes: 
 . dump rs label
drivers.conf changes:
 . added acl for mfs so that mfs can be started with 'service start',
   so that a custom label can be provided
2007-01-22 15:25:41 +00:00
Ben Gras
722f1b2b9f . added checks for buffer sizes in sys_datacopy() functions in mfs,
print debug message if copy is truncated
. increased buffer in lookup() to be PATH_MAX instead of NAME_MAX
. sanity check in fetch_name() in vfs to see if name fits, and
  is null-terminated
. first check i < NAME_MAX, then string[i] in search_dir, as we're
  not supposed to look at string[NAME_MAX]
2007-01-16 14:50:10 +00:00
Ben Gras
fa0ba56bc9 Merge of VFS by Balasz Gerofi with Minix trunk. 2006-10-25 13:40:36 +00:00
Renamed from servers/fs/utility.c (Browse further)