Add primary cache management feature to libminixfs as mfs and ext2
currently do separately, remove cache code from mfs and ext2, and make
them use the libminixfs interface. This makes all fields of the buf
struct private to libminixfs and FS clients aren't supposed to access
them at all. Only the opaque 'void *data' field (the FS block contents,
used to be called bp) is to be accessed by the FS client.
The main purpose is to implement the interface to the 2ndary vm cache
just once, get rid of some code duplication, and add a little
abstraction to reduce the code inertia of the whole caching business.
Some minor sanity checking and prohibition done by mfs in this code
as removed from the generic primary cache code as a result:
- checking all inodes are not in use when allocating/resizing
the cache
- checking readonly filesystems aren't written to
- checking the superblock isn't written to on mounted filesystems
The minixfslib code relies on fs_blockstats() in the client filesystem to
return some FS usage information.
. all invocations were S or D, so can safely be dropped
to prepare for the segmentless world
. still assign D to the SCP_SEG field in the message
to make previous kernels usable
MFS' get_block() must never return a newly acquired block buffer that
is marked dirty from previous use. This patch replaces git-dd59d50,
which assumed a working model where blocks for device NO_DEV would
never be dirty. For at least one scenario, that assumption does not
hold, triggering superblock overwrite warnings. In this patch, blocks
are explicitly marked as clean upon being repurposed. The working
model is now restored to be: the dirty state of a block is relevant
only when its associated device is not set to NO_DEV.
POSIX mandates that a file's modification and change time be left
untouched upon truncate/ftruncate iff the file size does not change.
However, an open(O_TRUNC) call must always update the modification and
change time of the file, even if it was already zero-sized. VFS uses
the file systems' truncate call to implement O_TRUNC. This patch
replaces git-255ae85, which did not take into account the open case.
The size check is now moved into VFS, so that individual file systems
need not check for this case anymore.
On MFS file systems, the stat(2) call now counts indirect blocks as
part of the st_blocks calculation, in addition to proper initial
rounding of the file size. The returned value is now a true upper
bound on the actual number of 512-byte blocks allocated to the file.
As before, it is not accurate for sparse files.
. also implement now-possible fsck -p option
. allows unconditional fsck -p invocation at startup,
only checking each filesystem if not marked clean
. mounting unclean is allowed but is forced readonly
. updating the superblock while mounted is now not
allowed by mfs - must be done (e.g. by fsck.mfs)
on an unmounted fs
. clean flag is unset by mfs on mounting, and set by
mfs on clean unmounting (if clean flag was set at
mount time)
Signed-off-by: Ben Gras <ben@minix3.org>
. use dirty marking hooks to check and warn
when inodes/bufs are marked dirty on a readonly
mounted fs
. add readonly mount checks to restore readonly
mounting
Signed-off-by: Ben Gras <ben@minix3.org>
. No functional change
. Only serves to get hooks to do checks in
. e.g. should things be marked dirty when we are
mounted readonly
Signed-off-by: Ben Gras <ben@minix3.org>
This patch makes PFS, EXT2 and MFS print only once that they're out of
space. After freeing up space and running out of space again, the message
will be printed again also.
This patch separates the character and block driver communication
protocols. The old character protocol remains the same, but a new
block protocol is introduced. The libdriver library is replaced by
two new libraries: libchardriver and libblockdriver. Their exposed
API, and drivers that use them, have been updated accordingly.
Together, libbdev and libblockdriver now completely abstract away
the message format used by the block protocol. As the memory driver
is both a character and a block device driver, it now implements its
own message loop.
The most important semantic change made to the block protocol is that
it is no longer possible to return both partial results and an error
for a single transfer. This simplifies the interaction between the
caller and the driver, as the I/O vector no longer needs to be copied
back. Also, drivers are now no longer supposed to decide based on the
layout of the I/O vector when a transfer should be cut short. Put
simply, transfers are now supposed to either succeed completely, or
result in an error.
After this patch, the state of the various pieces is as follows:
- block protocol: stable
- libbdev API: stable for synchronous communication
- libblockdriver API: needs slight revision (the drvlib/partition API
in particular; the threading API will also change shortly)
- character protocol: needs cleanup
- libchardriver API: needs cleanup accordingly
- driver restarts: largely unsupported until endpoint changes are
reintroduced
As a side effect, this patch eliminates several bugs, hacks, and gcc
-Wall and -W warnings all over the place. It probably introduces a
few new ones, too.
Update warning: this patch changes the protocol between MFS and disk
drivers, so in order to use old/new images, the MFS from the ramdisk
must be used to mount all file systems.
The "bdev" library provides basic primitives for file systems to talk
to block device drivers, hiding the details of the underlying protocol
and interaction model.
This version of libbdev is rather basic. It is planned to support the
following features in the long run:
- asynchronous requests and replies;
- recovery support for underlying block drivers;
- retrying of failed I/O requests.
The commit also changes our block-based file systems (mfs, ext2, isofs)
to make use of libbdev.
In some places it was assumed that PATH_MAX does not include a
terminating null character.
Increases PATH_MAX to 1024 to get in sync with NetBSD. Required some
rewriting in AVFS to keep memory usage low (the stack in use by a thread
is very small).
. move cache size heuristic from mfs there
so mfs and ext2 can share it
. add vfs credentials retrieving function, with
backwards compatability from previous struct
format, to be used by both ext2 and mfs
. fix for ext2 - STATICINIT was fed no.
of bytes instead of no. of elements, overallocating
memory by a megabyte or two for the superblock
. move mfs-specific struct, constants to mfs/, so
mfs-specific, on-disk format structs and consts are
fully isolated from generic structs and functions
. removes de and readfs utils
POSIX truncate specification says "Upon successful completion, if
the *file size is changed*, this function shall mark for update the
st_ctime and st_mtime fields of the file." This patch prevents
changing of the date fields when the size stays the same.
* VFS and installed MFSes must be in sync before and after this change *
Use struct stat from NetBSD. It requires adding new STAT, FSTAT and LSTAT
syscalls. Libc modification is both backward and forward compatible.
Also new struct stat uses modern field sizes to avoid ABI
incompatibility, when we update uid_t, gid_t and company.
Exceptions are ino_t and off_t in old libc (though paddings added).
3 sets of libraries are built now:
. ack: all libraries that ack can compile (/usr/lib/i386/)
. clang+elf: all libraries with minix headers (/usr/lib/)
. clang+elf: all libraries with netbsd headers (/usr/netbsd/)
Once everything can be compiled with netbsd libraries and headers, the
/usr/netbsd hierarchy will be obsolete and its libraries compiled with
netbsd headers will be installed in /usr/lib, and its headers
in /usr/include. (i.e. minix libc and current minix headers set
will be gone.)
To use the NetBSD libc system (libraries + headers) before
it is the default libc, see:
http://wiki.minix3.org/en/DevelopersGuide/UsingNetBSDCode
This wiki page also documents the maintenance of the patch
files of minix-specific changes to imported NetBSD code.
Changes in this commit:
. libsys: Add NBSD compilation and create a safe NBSD-based libc.
. Port rest of libraries (except libddekit) to new header system.
. Enable compilation of libddekit with new headers.
. Enable kernel compilation with new headers.
. Enable drivers compilation with new headers.
. Port legacy commands to new headers and libc.
. Port servers to new headers.
. Add <sys/sigcontext.h> in compat library.
. Remove dependency file in tree.
. Enable compilation of common/lib/libc/atomic in libsys
. Do not generate RCSID strings in libc.
. Temporarily disable zoneinfo as they are incompatible with NetBSD format
. obj-nbsd for .gitignore
. Procfs: use only integer arithmetic. (Antoine Leca)
. Increase ramdisk size to create NBSD-based images.
. Remove INCSYMLINKS handling hack.
. Add nbsd_include/sys/exec_elf.h
. Enable ELF compilation with NBSD libc.
. Add 'make nbsdsrc' in tools to download reference NetBSD sources.
. Automate minix-port.patch creation.
. Avoid using fstavfs() as it is *extremely* slow and unneeded.
. Set err() as PRIVATE to avoid name clash with libc.
. [NBSD] servers/vm: remove compilation warnings.
. u32 is not a long in NBSD headers.
. UPDATING info on netbsd hierarchy
. commands fixes for netbsd libc
Before safecopies, the IO_ENDPT and DL_ENDPT message fields were needed
to know which actual process to copy data from/to, as that process may
not always be the caller. Now that we have full safecopy support, these
fields have become useless for that purpose: the owner of the grant is
*always* the caller. Allowing the caller to supply another endpoint is
in fact dangerous, because the callee may then end up using a grant
from a third party. One could call this a variant of the confused
deputy problem.
From now on, safecopy calls should always use the caller's endpoint as
grant owner. This fully obsoletes the DL_ENDPT field in the
inet/ethernet protocol. IO_ENDPT has other uses besides identifying the
grant owner though. This patch renames IO_ENDPT to USER_ENDPT, not only
because that is a more fitting name (it should never be used for I/O
after all), but also in order to intentionally break any old system
source code outside the base system. If this patch breaks your code,
fixing it is fairly simple:
- DL_ENDPT should be replaced with m_source;
- IO_ENDPT should be replaced with m_source when used for safecopies;
- IO_ENDPT should be replaced with USER_ENDPT for any other use, e.g.
when setting REP_ENDPT, matching requests in CANCEL calls, getting
DEV_SELECT flags, and retrieving of the real user process's endpoint
in DEV_OPEN.
The changes in this patch are binary backward compatible.
