sanchayanmaity/minix
1
0
Fork 0
Code Issues Pull requests Packages Projects Releases Wiki Activity
minix/lib/libminixfs/cache.c

974 lines
25 KiB
C
Raw Normal View History

add -lminixfs with fs support routines . move cache size heuristic from mfs there so mfs and ext2 can share it . add vfs credentials retrieving function, with backwards compatability from previous struct format, to be used by both ext2 and mfs . fix for ext2 - STATICINIT was fed no. of bytes instead of no. of elements, overallocating memory by a megabyte or two for the superblock
2011-09-08 18:49:54 +02:00
#define _SYSTEM
Move primary cache code to libminixfs. Add primary cache management feature to libminixfs as mfs and ext2 currently do separately, remove cache code from mfs and ext2, and make them use the libminixfs interface. This makes all fields of the buf struct private to libminixfs and FS clients aren't supposed to access them at all. Only the opaque 'void *data' field (the FS block contents, used to be called bp) is to be accessed by the FS client. The main purpose is to implement the interface to the 2ndary vm cache just once, get rid of some code duplication, and add a little abstraction to reduce the code inertia of the whole caching business. Some minor sanity checking and prohibition done by mfs in this code as removed from the generic primary cache code as a result: - checking all inodes are not in use when allocating/resizing the cache - checking readonly filesystems aren't written to - checking the superblock isn't written to on mounted filesystems The minixfslib code relies on fs_blockstats() in the client filesystem to return some FS usage information.
2012-10-16 17:40:39 +02:00
#include <assert.h>
#include <errno.h>
#include <math.h>
#include <stdlib.h>
#include <sys/param.h>
vm: new secondary cache code Primary purpose of change: to support the mmap implementation, VM must know both (a) about some block metadata for FS cache blocks, i.e. inode numbers and inode offsets where applicable; and (b) know about *all* cache blocks, i.e. also of the FS primary caches and not just the blocks that spill into the secondary one. This changes the interface and VM data structures. This change is only for the interface (libminixfs) and VM data structures; the filesystem code is unmodified, so although the secondary cache will be used as normal, blocks will not be annotated with inode information until the FS is modified to provide this information. Until it is modified, mmap of files will fail gracefully on such filesystems. This is indicated to VFS/VM by returning ENOSYS for REQ_PEEK. Change-Id: I1d2df6c485e6c5e89eb28d9055076cc02629594e
2013-03-20 20:18:52 +01:00
#include <sys/mman.h>
Move primary cache code to libminixfs. Add primary cache management feature to libminixfs as mfs and ext2 currently do separately, remove cache code from mfs and ext2, and make them use the libminixfs interface. This makes all fields of the buf struct private to libminixfs and FS clients aren't supposed to access them at all. Only the opaque 'void *data' field (the FS block contents, used to be called bp) is to be accessed by the FS client. The main purpose is to implement the interface to the 2ndary vm cache just once, get rid of some code duplication, and add a little abstraction to reduce the code inertia of the whole caching business. Some minor sanity checking and prohibition done by mfs in this code as removed from the generic primary cache code as a result: - checking all inodes are not in use when allocating/resizing the cache - checking readonly filesystems aren't written to - checking the superblock isn't written to on mounted filesystems The minixfslib code relies on fs_blockstats() in the client filesystem to return some FS usage information.
2012-10-16 17:40:39 +02:00
add -lminixfs with fs support routines . move cache size heuristic from mfs there so mfs and ext2 can share it . add vfs credentials retrieving function, with backwards compatability from previous struct format, to be used by both ext2 and mfs . fix for ext2 - STATICINIT was fed no. of bytes instead of no. of elements, overallocating memory by a megabyte or two for the superblock
2011-09-08 18:49:54 +02:00
#include <minix/dmap.h>
Move primary cache code to libminixfs. Add primary cache management feature to libminixfs as mfs and ext2 currently do separately, remove cache code from mfs and ext2, and make them use the libminixfs interface. This makes all fields of the buf struct private to libminixfs and FS clients aren't supposed to access them at all. Only the opaque 'void *data' field (the FS block contents, used to be called bp) is to be accessed by the FS client. The main purpose is to implement the interface to the 2ndary vm cache just once, get rid of some code duplication, and add a little abstraction to reduce the code inertia of the whole caching business. Some minor sanity checking and prohibition done by mfs in this code as removed from the generic primary cache code as a result: - checking all inodes are not in use when allocating/resizing the cache - checking readonly filesystems aren't written to - checking the superblock isn't written to on mounted filesystems The minixfslib code relies on fs_blockstats() in the client filesystem to return some FS usage information.
2012-10-16 17:40:39 +02:00
#include <minix/libminixfs.h>
#include <minix/syslib.h>
Fix a ton of compiler warnings This patch fixes most of current reasons to generate compiler warnings. The changes consist of: - adding missing casts - hiding or unhiding function declarations - including headers where missing - add __UNCONST when assigning a const char * to a char * - adding missing return statements - changing some types from unsigned to signed, as the code seems to want signed ints - converting old-style function definitions to current style (i.e., void func(param1, param2) short param1, param2; {...} to void func (short param1, short param2) {...}) - making the compiler silent about signed vs unsigned comparisons. We have too many of those in the new libc to fix. A number of bugs in the test set were fixed. These bugs were never triggered with our old libc. Consequently, these tests are now forced to link with the new libc or they will generate errors (in particular tests 43 and 55). Most changes in NetBSD libc are limited to moving aroudn "#ifndef __minix" or stuff related to Minix-specific things (code in sys-minix or gen/minix).
2011-11-14 11:07:49 +01:00
#include <minix/sysutil.h>
Move primary cache code to libminixfs. Add primary cache management feature to libminixfs as mfs and ext2 currently do separately, remove cache code from mfs and ext2, and make them use the libminixfs interface. This makes all fields of the buf struct private to libminixfs and FS clients aren't supposed to access them at all. Only the opaque 'void *data' field (the FS block contents, used to be called bp) is to be accessed by the FS client. The main purpose is to implement the interface to the 2ndary vm cache just once, get rid of some code duplication, and add a little abstraction to reduce the code inertia of the whole caching business. Some minor sanity checking and prohibition done by mfs in this code as removed from the generic primary cache code as a result: - checking all inodes are not in use when allocating/resizing the cache - checking readonly filesystems aren't written to - checking the superblock isn't written to on mounted filesystems The minixfslib code relies on fs_blockstats() in the client filesystem to return some FS usage information.
2012-10-16 17:40:39 +02:00
#include <minix/u64.h>
#include <minix/bdev.h>
#define BUFHASH(b) ((b) % nr_bufs)
#define MARKCLEAN lmfs_markclean
#define MINBUFS 6 /* minimal no of bufs for sanity check */
static struct buf *front; /* points to least recently used free block */
static struct buf *rear; /* points to most recently used free block */
static unsigned int bufs_in_use;/* # bufs currently in use (not on free list)*/
static void rm_lru(struct buf *bp);
static void read_block(struct buf *);
static void flushall(dev_t dev);
vm: new secondary cache code Primary purpose of change: to support the mmap implementation, VM must know both (a) about some block metadata for FS cache blocks, i.e. inode numbers and inode offsets where applicable; and (b) know about *all* cache blocks, i.e. also of the FS primary caches and not just the blocks that spill into the secondary one. This changes the interface and VM data structures. This change is only for the interface (libminixfs) and VM data structures; the filesystem code is unmodified, so although the secondary cache will be used as normal, blocks will not be annotated with inode information until the FS is modified to provide this information. Until it is modified, mmap of files will fail gracefully on such filesystems. This is indicated to VFS/VM by returning ENOSYS for REQ_PEEK. Change-Id: I1d2df6c485e6c5e89eb28d9055076cc02629594e
2013-03-20 20:18:52 +01:00
static void freeblock(struct buf *bp);
libminixfs, mfs, ext2: may re-evaluate cache size libminixfs may now be informed of changes to the block usage on the filesystem. if the net change becomes big enough, libminixfs may resize the cache based on the new usage. . update the 2 FSes to provide this information to libminixfs Change-Id: I158815a11da801fd5572a8de89c9e6c039b82650
2013-04-26 11:54:33 +02:00
static void cache_heuristic_check(int major);
Move primary cache code to libminixfs. Add primary cache management feature to libminixfs as mfs and ext2 currently do separately, remove cache code from mfs and ext2, and make them use the libminixfs interface. This makes all fields of the buf struct private to libminixfs and FS clients aren't supposed to access them at all. Only the opaque 'void *data' field (the FS block contents, used to be called bp) is to be accessed by the FS client. The main purpose is to implement the interface to the 2ndary vm cache just once, get rid of some code duplication, and add a little abstraction to reduce the code inertia of the whole caching business. Some minor sanity checking and prohibition done by mfs in this code as removed from the generic primary cache code as a result: - checking all inodes are not in use when allocating/resizing the cache - checking readonly filesystems aren't written to - checking the superblock isn't written to on mounted filesystems The minixfslib code relies on fs_blockstats() in the client filesystem to return some FS usage information.
2012-10-16 17:40:39 +02:00
static int vmcache = 0; /* are we using vm's secondary cache? (initially not) */
static struct buf *buf;
static struct buf **buf_hash; /* the buffer hash table */
static unsigned int nr_bufs;
static int may_use_vmcache;
avoid alloc_contig() for iovec, mfs superblock . initial workaround for assert() firing on iovec size on ARM. likely due to alloc_contig() allocating unusually mapped memory in STATICINIT. . for the same reason use the regular cache i/o functions to read the superblock in mfs - avoid the alloc_contig() that STATICINIT does. Change-Id: I3d8dc635b1cf2666e55b0393feae74cc25b8fed4
2013-10-03 17:22:07 +02:00
static int fs_block_size = PAGE_SIZE; /* raw i/o block size */
Move primary cache code to libminixfs. Add primary cache management feature to libminixfs as mfs and ext2 currently do separately, remove cache code from mfs and ext2, and make them use the libminixfs interface. This makes all fields of the buf struct private to libminixfs and FS clients aren't supposed to access them at all. Only the opaque 'void *data' field (the FS block contents, used to be called bp) is to be accessed by the FS client. The main purpose is to implement the interface to the 2ndary vm cache just once, get rid of some code duplication, and add a little abstraction to reduce the code inertia of the whole caching business. Some minor sanity checking and prohibition done by mfs in this code as removed from the generic primary cache code as a result: - checking all inodes are not in use when allocating/resizing the cache - checking readonly filesystems aren't written to - checking the superblock isn't written to on mounted filesystems The minixfslib code relies on fs_blockstats() in the client filesystem to return some FS usage information.
2012-10-16 17:40:39 +02:00
static int rdwt_err;
add -lminixfs with fs support routines . move cache size heuristic from mfs there so mfs and ext2 can share it . add vfs credentials retrieving function, with backwards compatability from previous struct format, to be used by both ext2 and mfs . fix for ext2 - STATICINIT was fed no. of bytes instead of no. of elements, overallocating memory by a megabyte or two for the superblock
2011-09-08 18:49:54 +02:00
many new tests . test70: regression test for m_out vfs race condition The following tests use testcache.c to generate test i/o patterns, generate random write data and verify the reads. . test71: blackbox full-stack test of FS operation, testing using the regular VFS interface crazy i/o patterns with various working set sizes, triggering only primary cache, also secondary cache, and finally disk i/o and verifying contents all the time . test72: unit test of libminixfs, implementing functions it needs from -lsys and -lblockdriver and the client in order to simulate a working cache client and backend environment. . test73: blackbox test of secondary vm cache in isolation Change-Id: I1287e9753182b8719e634917ad158e3c1e079ceb
2013-04-12 22:54:42 +02:00
static int quiet = 0;
void
lmfs_setquiet(int q) { quiet = q; }
32 to 64 bits fsblkcnt_t and fsfilcnt_t. Change-Id: I432229143c85cd178262b802a76ac606801ac59a
2013-04-18 14:42:15 +02:00
static u32_t fs_bufs_heuristic(int minbufs, u32_t btotal, u64_t bfree,
add -lminixfs with fs support routines . move cache size heuristic from mfs there so mfs and ext2 can share it . add vfs credentials retrieving function, with backwards compatability from previous struct format, to be used by both ext2 and mfs . fix for ext2 - STATICINIT was fed no. of bytes instead of no. of elements, overallocating memory by a megabyte or two for the superblock
2011-09-08 18:49:54 +02:00
int blocksize, dev_t majordev)
{
struct vm_stats_info vsi;
int bufs;
u32_t kbytes_used_fs, kbytes_total_fs, kbcache, kb_fsmax;
32 to 64 bits fsblkcnt_t and fsfilcnt_t. Change-Id: I432229143c85cd178262b802a76ac606801ac59a
2013-04-18 14:42:15 +02:00
u32_t kbytes_remain_mem;
u64_t bused;
add -lminixfs with fs support routines . move cache size heuristic from mfs there so mfs and ext2 can share it . add vfs credentials retrieving function, with backwards compatability from previous struct format, to be used by both ext2 and mfs . fix for ext2 - STATICINIT was fed no. of bytes instead of no. of elements, overallocating memory by a megabyte or two for the superblock
2011-09-08 18:49:54 +02:00
bused = btotal-bfree;
/* set a reasonable cache size; cache at most a certain
* portion of the used FS, and at most a certain %age of remaining
* memory
*/
libminixfs, mfs, ext2: may re-evaluate cache size libminixfs may now be informed of changes to the block usage on the filesystem. if the net change becomes big enough, libminixfs may resize the cache based on the new usage. . update the 2 FSes to provide this information to libminixfs Change-Id: I158815a11da801fd5572a8de89c9e6c039b82650
2013-04-26 11:54:33 +02:00
if(vm_info_stats(&vsi) != OK) {
add -lminixfs with fs support routines . move cache size heuristic from mfs there so mfs and ext2 can share it . add vfs credentials retrieving function, with backwards compatability from previous struct format, to be used by both ext2 and mfs . fix for ext2 - STATICINIT was fed no. of bytes instead of no. of elements, overallocating memory by a megabyte or two for the superblock
2011-09-08 18:49:54 +02:00
bufs = 1024;
libminixfs, mfs, ext2: may re-evaluate cache size libminixfs may now be informed of changes to the block usage on the filesystem. if the net change becomes big enough, libminixfs may resize the cache based on the new usage. . update the 2 FSes to provide this information to libminixfs Change-Id: I158815a11da801fd5572a8de89c9e6c039b82650
2013-04-26 11:54:33 +02:00
if(!quiet)
printf("fslib: heuristic info fail: default to %d bufs\n", bufs);
add -lminixfs with fs support routines . move cache size heuristic from mfs there so mfs and ext2 can share it . add vfs credentials retrieving function, with backwards compatability from previous struct format, to be used by both ext2 and mfs . fix for ext2 - STATICINIT was fed no. of bytes instead of no. of elements, overallocating memory by a megabyte or two for the superblock
2011-09-08 18:49:54 +02:00
return bufs;
}
libminixfs, mfs, ext2: may re-evaluate cache size libminixfs may now be informed of changes to the block usage on the filesystem. if the net change becomes big enough, libminixfs may resize the cache based on the new usage. . update the 2 FSes to provide this information to libminixfs Change-Id: I158815a11da801fd5572a8de89c9e6c039b82650
2013-04-26 11:54:33 +02:00
/* remaining free memory is unused memory plus memory in used for cache,
* as the cache can be evicted
*/
kbytes_remain_mem = (u64_t)(vsi.vsi_free + vsi.vsi_cached) *
vsi.vsi_pagesize / 1024;
add -lminixfs with fs support routines . move cache size heuristic from mfs there so mfs and ext2 can share it . add vfs credentials retrieving function, with backwards compatability from previous struct format, to be used by both ext2 and mfs . fix for ext2 - STATICINIT was fed no. of bytes instead of no. of elements, overallocating memory by a megabyte or two for the superblock
2011-09-08 18:49:54 +02:00
/* check fs usage. */
kbytes_used_fs = div64u(mul64u(bused, blocksize), 1024);
kbytes_total_fs = div64u(mul64u(btotal, blocksize), 1024);
/* heuristic for a desired cache size based on FS usage;
* but never bigger than half of the total filesystem
*/
kb_fsmax = sqrt_approx(kbytes_used_fs)*40;
kb_fsmax = MIN(kb_fsmax, kbytes_total_fs/2);
/* heuristic for a maximum usage - 10% of remaining memory */
kbcache = MIN(kbytes_remain_mem/10, kb_fsmax);
bufs = kbcache * 1024 / blocksize;
/* but we simply need MINBUFS no matter what */
if(bufs < minbufs)
bufs = minbufs;
return bufs;
}
libminixfs, mfs, ext2: may re-evaluate cache size libminixfs may now be informed of changes to the block usage on the filesystem. if the net change becomes big enough, libminixfs may resize the cache based on the new usage. . update the 2 FSes to provide this information to libminixfs Change-Id: I158815a11da801fd5572a8de89c9e6c039b82650
2013-04-26 11:54:33 +02:00
void lmfs_blockschange(dev_t dev, int delta)
{
/* Change the number of allocated blocks by 'delta.'
* Also accumulate the delta since the last cache re-evaluation.
* If it is outside a certain band, ask the cache library to
* re-evaluate the cache size.
*/
static int bitdelta = 0;
bitdelta += delta;
#define BANDKB (10*1024) /* recheck cache every 10MB change */
if(bitdelta*fs_block_size/1024 > BANDKB ||
bitdelta*fs_block_size/1024 < -BANDKB) {
lmfs_cache_reevaluate(dev);
bitdelta = 0;
}
}
Move primary cache code to libminixfs. Add primary cache management feature to libminixfs as mfs and ext2 currently do separately, remove cache code from mfs and ext2, and make them use the libminixfs interface. This makes all fields of the buf struct private to libminixfs and FS clients aren't supposed to access them at all. Only the opaque 'void *data' field (the FS block contents, used to be called bp) is to be accessed by the FS client. The main purpose is to implement the interface to the 2ndary vm cache just once, get rid of some code duplication, and add a little abstraction to reduce the code inertia of the whole caching business. Some minor sanity checking and prohibition done by mfs in this code as removed from the generic primary cache code as a result: - checking all inodes are not in use when allocating/resizing the cache - checking readonly filesystems aren't written to - checking the superblock isn't written to on mounted filesystems The minixfslib code relies on fs_blockstats() in the client filesystem to return some FS usage information.
2012-10-16 17:40:39 +02:00
void
lmfs_markdirty(struct buf *bp)
{
vm: new secondary cache code Primary purpose of change: to support the mmap implementation, VM must know both (a) about some block metadata for FS cache blocks, i.e. inode numbers and inode offsets where applicable; and (b) know about *all* cache blocks, i.e. also of the FS primary caches and not just the blocks that spill into the secondary one. This changes the interface and VM data structures. This change is only for the interface (libminixfs) and VM data structures; the filesystem code is unmodified, so although the secondary cache will be used as normal, blocks will not be annotated with inode information until the FS is modified to provide this information. Until it is modified, mmap of files will fail gracefully on such filesystems. This is indicated to VFS/VM by returning ENOSYS for REQ_PEEK. Change-Id: I1d2df6c485e6c5e89eb28d9055076cc02629594e
2013-03-20 20:18:52 +01:00
bp->lmfs_flags |= VMMC_DIRTY;
Move primary cache code to libminixfs. Add primary cache management feature to libminixfs as mfs and ext2 currently do separately, remove cache code from mfs and ext2, and make them use the libminixfs interface. This makes all fields of the buf struct private to libminixfs and FS clients aren't supposed to access them at all. Only the opaque 'void *data' field (the FS block contents, used to be called bp) is to be accessed by the FS client. The main purpose is to implement the interface to the 2ndary vm cache just once, get rid of some code duplication, and add a little abstraction to reduce the code inertia of the whole caching business. Some minor sanity checking and prohibition done by mfs in this code as removed from the generic primary cache code as a result: - checking all inodes are not in use when allocating/resizing the cache - checking readonly filesystems aren't written to - checking the superblock isn't written to on mounted filesystems The minixfslib code relies on fs_blockstats() in the client filesystem to return some FS usage information.
2012-10-16 17:40:39 +02:00
}
void
lmfs_markclean(struct buf *bp)
{
vm: new secondary cache code Primary purpose of change: to support the mmap implementation, VM must know both (a) about some block metadata for FS cache blocks, i.e. inode numbers and inode offsets where applicable; and (b) know about *all* cache blocks, i.e. also of the FS primary caches and not just the blocks that spill into the secondary one. This changes the interface and VM data structures. This change is only for the interface (libminixfs) and VM data structures; the filesystem code is unmodified, so although the secondary cache will be used as normal, blocks will not be annotated with inode information until the FS is modified to provide this information. Until it is modified, mmap of files will fail gracefully on such filesystems. This is indicated to VFS/VM by returning ENOSYS for REQ_PEEK. Change-Id: I1d2df6c485e6c5e89eb28d9055076cc02629594e
2013-03-20 20:18:52 +01:00
bp->lmfs_flags &= ~VMMC_DIRTY;
Move primary cache code to libminixfs. Add primary cache management feature to libminixfs as mfs and ext2 currently do separately, remove cache code from mfs and ext2, and make them use the libminixfs interface. This makes all fields of the buf struct private to libminixfs and FS clients aren't supposed to access them at all. Only the opaque 'void *data' field (the FS block contents, used to be called bp) is to be accessed by the FS client. The main purpose is to implement the interface to the 2ndary vm cache just once, get rid of some code duplication, and add a little abstraction to reduce the code inertia of the whole caching business. Some minor sanity checking and prohibition done by mfs in this code as removed from the generic primary cache code as a result: - checking all inodes are not in use when allocating/resizing the cache - checking readonly filesystems aren't written to - checking the superblock isn't written to on mounted filesystems The minixfslib code relies on fs_blockstats() in the client filesystem to return some FS usage information.
2012-10-16 17:40:39 +02:00
}
int
lmfs_isclean(struct buf *bp)
{
vm: new secondary cache code Primary purpose of change: to support the mmap implementation, VM must know both (a) about some block metadata for FS cache blocks, i.e. inode numbers and inode offsets where applicable; and (b) know about *all* cache blocks, i.e. also of the FS primary caches and not just the blocks that spill into the secondary one. This changes the interface and VM data structures. This change is only for the interface (libminixfs) and VM data structures; the filesystem code is unmodified, so although the secondary cache will be used as normal, blocks will not be annotated with inode information until the FS is modified to provide this information. Until it is modified, mmap of files will fail gracefully on such filesystems. This is indicated to VFS/VM by returning ENOSYS for REQ_PEEK. Change-Id: I1d2df6c485e6c5e89eb28d9055076cc02629594e
2013-03-20 20:18:52 +01:00
return !(bp->lmfs_flags & VMMC_DIRTY);
Move primary cache code to libminixfs. Add primary cache management feature to libminixfs as mfs and ext2 currently do separately, remove cache code from mfs and ext2, and make them use the libminixfs interface. This makes all fields of the buf struct private to libminixfs and FS clients aren't supposed to access them at all. Only the opaque 'void *data' field (the FS block contents, used to be called bp) is to be accessed by the FS client. The main purpose is to implement the interface to the 2ndary vm cache just once, get rid of some code duplication, and add a little abstraction to reduce the code inertia of the whole caching business. Some minor sanity checking and prohibition done by mfs in this code as removed from the generic primary cache code as a result: - checking all inodes are not in use when allocating/resizing the cache - checking readonly filesystems aren't written to - checking the superblock isn't written to on mounted filesystems The minixfslib code relies on fs_blockstats() in the client filesystem to return some FS usage information.
2012-10-16 17:40:39 +02:00
}
dev_t
lmfs_dev(struct buf *bp)
{
return bp->lmfs_dev;
}
int lmfs_bytes(struct buf *bp)
{
return bp->lmfs_bytes;
}
vm: new secondary cache code Primary purpose of change: to support the mmap implementation, VM must know both (a) about some block metadata for FS cache blocks, i.e. inode numbers and inode offsets where applicable; and (b) know about *all* cache blocks, i.e. also of the FS primary caches and not just the blocks that spill into the secondary one. This changes the interface and VM data structures. This change is only for the interface (libminixfs) and VM data structures; the filesystem code is unmodified, so although the secondary cache will be used as normal, blocks will not be annotated with inode information until the FS is modified to provide this information. Until it is modified, mmap of files will fail gracefully on such filesystems. This is indicated to VFS/VM by returning ENOSYS for REQ_PEEK. Change-Id: I1d2df6c485e6c5e89eb28d9055076cc02629594e
2013-03-20 20:18:52 +01:00
static void
free_unused_blocks(void)
{
struct buf *bp;
int freed = 0, bytes = 0;
printf("libminixfs: freeing; %d blocks in use\n", bufs_in_use);
for(bp = &buf[0]; bp < &buf[nr_bufs]; bp++) {
if(bp->lmfs_bytes > 0 && bp->lmfs_count == 0) {
freed++;
bytes += bp->lmfs_bytes;
freeblock(bp);
}
}
printf("libminixfs: freeing; %d blocks, %d bytes\n", freed, bytes);
}
static void
lmfs_alloc_block(struct buf *bp)
{
libminixfs: allow non-pagesize-multiple FSes The memory-mapped files implementation (mmap() etc.) is implemented with the help of the filesystems using the in-VM FS cache. Filesystems tell it about all cached blocks and their metadata. Metadata is: device offset and, if any (and known), inode number and in-inode offset. VM can then map in requested memory-mapped file blocks, and request them if necessary. A limitation of this system is that filesystem block sizes that are not a multiple of the VM system (and VM hardware) page size are not possible; we can't map blocks in partially. (We can copy, but then the benefits of mapping and sharing the physical pages is gone.) So until before this commit various pieces of caching code assumed page size multiple blocksizes. This isn't strictly necessary as long as mmap() needn't be supported on that FS. This change allows the in-FS cache code (libminixfs) to allocate any-sized blocks, and will not interact with the VM cache for non-pagesize-multiple blocks. In that case it will also signal requestors, by failing 'peek' requests, that mmap() should not be supported on this FS. VM and VFS will then gracefully fail all file-mapping mmap() calls, and exec() will fall back to copying executable blocks instead of mmap()ping executables. As a result, 3 diagnostics that signal file-mapped mmap()s failing (hitherto an unusual occurence) are disabled, as ld.so does file-mapped mmap()s to map in objects it needs. On FSes not supporting it this situation is legitimate and shouldn't cause so much noise. ld.so will revert to its own minix-specific allocate+copy style of starting executables if mmap()s fail. Change-Id: Iecb1c8090f5e0be28da8f5181bb35084eb18f67b
2013-11-19 16:59:52 +01:00
int len;
vm: new secondary cache code Primary purpose of change: to support the mmap implementation, VM must know both (a) about some block metadata for FS cache blocks, i.e. inode numbers and inode offsets where applicable; and (b) know about *all* cache blocks, i.e. also of the FS primary caches and not just the blocks that spill into the secondary one. This changes the interface and VM data structures. This change is only for the interface (libminixfs) and VM data structures; the filesystem code is unmodified, so although the secondary cache will be used as normal, blocks will not be annotated with inode information until the FS is modified to provide this information. Until it is modified, mmap of files will fail gracefully on such filesystems. This is indicated to VFS/VM by returning ENOSYS for REQ_PEEK. Change-Id: I1d2df6c485e6c5e89eb28d9055076cc02629594e
2013-03-20 20:18:52 +01:00
ASSERT(!bp->data);
ASSERT(bp->lmfs_bytes == 0);
libminixfs: allow non-pagesize-multiple FSes The memory-mapped files implementation (mmap() etc.) is implemented with the help of the filesystems using the in-VM FS cache. Filesystems tell it about all cached blocks and their metadata. Metadata is: device offset and, if any (and known), inode number and in-inode offset. VM can then map in requested memory-mapped file blocks, and request them if necessary. A limitation of this system is that filesystem block sizes that are not a multiple of the VM system (and VM hardware) page size are not possible; we can't map blocks in partially. (We can copy, but then the benefits of mapping and sharing the physical pages is gone.) So until before this commit various pieces of caching code assumed page size multiple blocksizes. This isn't strictly necessary as long as mmap() needn't be supported on that FS. This change allows the in-FS cache code (libminixfs) to allocate any-sized blocks, and will not interact with the VM cache for non-pagesize-multiple blocks. In that case it will also signal requestors, by failing 'peek' requests, that mmap() should not be supported on this FS. VM and VFS will then gracefully fail all file-mapping mmap() calls, and exec() will fall back to copying executable blocks instead of mmap()ping executables. As a result, 3 diagnostics that signal file-mapped mmap()s failing (hitherto an unusual occurence) are disabled, as ld.so does file-mapped mmap()s to map in objects it needs. On FSes not supporting it this situation is legitimate and shouldn't cause so much noise. ld.so will revert to its own minix-specific allocate+copy style of starting executables if mmap()s fail. Change-Id: Iecb1c8090f5e0be28da8f5181bb35084eb18f67b
2013-11-19 16:59:52 +01:00
len = roundup(fs_block_size, PAGE_SIZE);
vm: new secondary cache code Primary purpose of change: to support the mmap implementation, VM must know both (a) about some block metadata for FS cache blocks, i.e. inode numbers and inode offsets where applicable; and (b) know about *all* cache blocks, i.e. also of the FS primary caches and not just the blocks that spill into the secondary one. This changes the interface and VM data structures. This change is only for the interface (libminixfs) and VM data structures; the filesystem code is unmodified, so although the secondary cache will be used as normal, blocks will not be annotated with inode information until the FS is modified to provide this information. Until it is modified, mmap of files will fail gracefully on such filesystems. This is indicated to VFS/VM by returning ENOSYS for REQ_PEEK. Change-Id: I1d2df6c485e6c5e89eb28d9055076cc02629594e
2013-03-20 20:18:52 +01:00
if((bp->data = minix_mmap(0, fs_block_size,
PROT_READ|PROT_WRITE, MAP_PREALLOC|MAP_ANON, -1, 0)) == MAP_FAILED) {
free_unused_blocks();
if((bp->data = minix_mmap(0, fs_block_size, PROT_READ|PROT_WRITE,
MAP_PREALLOC|MAP_ANON, -1, 0)) == MAP_FAILED) {
panic("libminixfs: could not allocate block");
}
}
assert(bp->data);
bp->lmfs_bytes = fs_block_size;
bp->lmfs_needsetcache = 1;
}
Move primary cache code to libminixfs. Add primary cache management feature to libminixfs as mfs and ext2 currently do separately, remove cache code from mfs and ext2, and make them use the libminixfs interface. This makes all fields of the buf struct private to libminixfs and FS clients aren't supposed to access them at all. Only the opaque 'void *data' field (the FS block contents, used to be called bp) is to be accessed by the FS client. The main purpose is to implement the interface to the 2ndary vm cache just once, get rid of some code duplication, and add a little abstraction to reduce the code inertia of the whole caching business. Some minor sanity checking and prohibition done by mfs in this code as removed from the generic primary cache code as a result: - checking all inodes are not in use when allocating/resizing the cache - checking readonly filesystems aren't written to - checking the superblock isn't written to on mounted filesystems The minixfslib code relies on fs_blockstats() in the client filesystem to return some FS usage information.
2012-10-16 17:40:39 +02:00
/*===========================================================================*
* lmfs_get_block *
*===========================================================================*/
vm: new secondary cache code Primary purpose of change: to support the mmap implementation, VM must know both (a) about some block metadata for FS cache blocks, i.e. inode numbers and inode offsets where applicable; and (b) know about *all* cache blocks, i.e. also of the FS primary caches and not just the blocks that spill into the secondary one. This changes the interface and VM data structures. This change is only for the interface (libminixfs) and VM data structures; the filesystem code is unmodified, so although the secondary cache will be used as normal, blocks will not be annotated with inode information until the FS is modified to provide this information. Until it is modified, mmap of files will fail gracefully on such filesystems. This is indicated to VFS/VM by returning ENOSYS for REQ_PEEK. Change-Id: I1d2df6c485e6c5e89eb28d9055076cc02629594e
2013-03-20 20:18:52 +01:00
struct buf *lmfs_get_block(register dev_t dev, register block_t block,
int only_search)
{
return lmfs_get_block_ino(dev, block, only_search, VMC_NO_INODE, 0);
}
void minix_munmap_t(void *a, int len)
{
vir_bytes av = (vir_bytes) a;
assert(a);
assert(a != MAP_FAILED);
assert(len > 0);
assert(!(av % PAGE_SIZE));
libminixfs: allow non-pagesize-multiple FSes The memory-mapped files implementation (mmap() etc.) is implemented with the help of the filesystems using the in-VM FS cache. Filesystems tell it about all cached blocks and their metadata. Metadata is: device offset and, if any (and known), inode number and in-inode offset. VM can then map in requested memory-mapped file blocks, and request them if necessary. A limitation of this system is that filesystem block sizes that are not a multiple of the VM system (and VM hardware) page size are not possible; we can't map blocks in partially. (We can copy, but then the benefits of mapping and sharing the physical pages is gone.) So until before this commit various pieces of caching code assumed page size multiple blocksizes. This isn't strictly necessary as long as mmap() needn't be supported on that FS. This change allows the in-FS cache code (libminixfs) to allocate any-sized blocks, and will not interact with the VM cache for non-pagesize-multiple blocks. In that case it will also signal requestors, by failing 'peek' requests, that mmap() should not be supported on this FS. VM and VFS will then gracefully fail all file-mapping mmap() calls, and exec() will fall back to copying executable blocks instead of mmap()ping executables. As a result, 3 diagnostics that signal file-mapped mmap()s failing (hitherto an unusual occurence) are disabled, as ld.so does file-mapped mmap()s to map in objects it needs. On FSes not supporting it this situation is legitimate and shouldn't cause so much noise. ld.so will revert to its own minix-specific allocate+copy style of starting executables if mmap()s fail. Change-Id: Iecb1c8090f5e0be28da8f5181bb35084eb18f67b
2013-11-19 16:59:52 +01:00
len = roundup(len, PAGE_SIZE);
assert(!(len % PAGE_SIZE));
vm: new secondary cache code Primary purpose of change: to support the mmap implementation, VM must know both (a) about some block metadata for FS cache blocks, i.e. inode numbers and inode offsets where applicable; and (b) know about *all* cache blocks, i.e. also of the FS primary caches and not just the blocks that spill into the secondary one. This changes the interface and VM data structures. This change is only for the interface (libminixfs) and VM data structures; the filesystem code is unmodified, so although the secondary cache will be used as normal, blocks will not be annotated with inode information until the FS is modified to provide this information. Until it is modified, mmap of files will fail gracefully on such filesystems. This is indicated to VFS/VM by returning ENOSYS for REQ_PEEK. Change-Id: I1d2df6c485e6c5e89eb28d9055076cc02629594e
2013-03-20 20:18:52 +01:00
if(minix_munmap(a, len) < 0)
panic("libminixfs cache: munmap failed");
}
static void raisecount(struct buf *bp)
{
assert(bufs_in_use >= 0);
ASSERT(bp->lmfs_count >= 0);
bp->lmfs_count++;
if(bp->lmfs_count == 1) bufs_in_use++;
assert(bufs_in_use > 0);
}
static void lowercount(struct buf *bp)
{
assert(bufs_in_use > 0);
ASSERT(bp->lmfs_count > 0);
bp->lmfs_count--;
if(bp->lmfs_count == 0) bufs_in_use--;
assert(bufs_in_use >= 0);
}
static void freeblock(struct buf *bp)
{
ASSERT(bp->lmfs_count == 0);
/* If the block taken is dirty, make it clean by writing it to the disk.
* Avoid hysteresis by flushing all other dirty blocks for the same device.
*/
if (bp->lmfs_dev != NO_DEV) {
if (!lmfs_isclean(bp)) flushall(bp->lmfs_dev);
assert(bp->lmfs_bytes == fs_block_size);
bp->lmfs_dev = NO_DEV;
}
/* Fill in block's parameters and add it to the hash chain where it goes. */
MARKCLEAN(bp); /* NO_DEV blocks may be marked dirty */
if(bp->lmfs_bytes > 0) {
assert(bp->data);
minix_munmap_t(bp->data, bp->lmfs_bytes);
bp->lmfs_bytes = 0;
bp->data = NULL;
} else assert(!bp->data);
}
/*===========================================================================*
* lmfs_get_block_ino *
*===========================================================================*/
struct buf *lmfs_get_block_ino(dev_t dev, block_t block, int only_search,
ino_t ino, u64_t ino_off)
Move primary cache code to libminixfs. Add primary cache management feature to libminixfs as mfs and ext2 currently do separately, remove cache code from mfs and ext2, and make them use the libminixfs interface. This makes all fields of the buf struct private to libminixfs and FS clients aren't supposed to access them at all. Only the opaque 'void *data' field (the FS block contents, used to be called bp) is to be accessed by the FS client. The main purpose is to implement the interface to the 2ndary vm cache just once, get rid of some code duplication, and add a little abstraction to reduce the code inertia of the whole caching business. Some minor sanity checking and prohibition done by mfs in this code as removed from the generic primary cache code as a result: - checking all inodes are not in use when allocating/resizing the cache - checking readonly filesystems aren't written to - checking the superblock isn't written to on mounted filesystems The minixfslib code relies on fs_blockstats() in the client filesystem to return some FS usage information.
2012-10-16 17:40:39 +02:00
{
/* Check to see if the requested block is in the block cache. If so, return
* a pointer to it. If not, evict some other block and fetch it (unless
* 'only_search' is 1). All the blocks in the cache that are not in use
* are linked together in a chain, with 'front' pointing to the least recently
* used block and 'rear' to the most recently used block. If 'only_search' is
* 1, the block being requested will be overwritten in its entirety, so it is
* only necessary to see if it is in the cache; if it is not, any free buffer
* will do. It is not necessary to actually read the block in from disk.
* If 'only_search' is PREFETCH, the block need not be read from the disk,
* and the device is not to be marked on the block, so callers can tell if
* the block returned is valid.
* In addition to the LRU chain, there is also a hash chain to link together
* blocks whose block numbers end with the same bit strings, for fast lookup.
*/
int b;
vm: new secondary cache code Primary purpose of change: to support the mmap implementation, VM must know both (a) about some block metadata for FS cache blocks, i.e. inode numbers and inode offsets where applicable; and (b) know about *all* cache blocks, i.e. also of the FS primary caches and not just the blocks that spill into the secondary one. This changes the interface and VM data structures. This change is only for the interface (libminixfs) and VM data structures; the filesystem code is unmodified, so although the secondary cache will be used as normal, blocks will not be annotated with inode information until the FS is modified to provide this information. Until it is modified, mmap of files will fail gracefully on such filesystems. This is indicated to VFS/VM by returning ENOSYS for REQ_PEEK. Change-Id: I1d2df6c485e6c5e89eb28d9055076cc02629594e
2013-03-20 20:18:52 +01:00
static struct buf *bp;
u64_t dev_off = (u64_t) block * fs_block_size;
struct buf *prev_ptr;
Move primary cache code to libminixfs. Add primary cache management feature to libminixfs as mfs and ext2 currently do separately, remove cache code from mfs and ext2, and make them use the libminixfs interface. This makes all fields of the buf struct private to libminixfs and FS clients aren't supposed to access them at all. Only the opaque 'void *data' field (the FS block contents, used to be called bp) is to be accessed by the FS client. The main purpose is to implement the interface to the 2ndary vm cache just once, get rid of some code duplication, and add a little abstraction to reduce the code inertia of the whole caching business. Some minor sanity checking and prohibition done by mfs in this code as removed from the generic primary cache code as a result: - checking all inodes are not in use when allocating/resizing the cache - checking readonly filesystems aren't written to - checking the superblock isn't written to on mounted filesystems The minixfslib code relies on fs_blockstats() in the client filesystem to return some FS usage information.
2012-10-16 17:40:39 +02:00
assert(buf_hash);
assert(buf);
assert(nr_bufs > 0);
ASSERT(fs_block_size > 0);
can't get_block(NO_DEV) any more . 'anonymous' cache blocks (retrieved with NO_DEV as dev parameter) were used to implement read()s from holes in inodes that should return zeroes . this is an awkward special case in the cache code though and there's a more direct way to implement the same functionality: instead of copying from a new, anonymous, zero block, to the user target buffer, simply sys_safememset the user target buffer directly. as this was the only use of this feature, this is all that's needed to simplify the cache code a little.
2012-11-12 19:15:10 +01:00
assert(dev != NO_DEV);
vm: new secondary cache code Primary purpose of change: to support the mmap implementation, VM must know both (a) about some block metadata for FS cache blocks, i.e. inode numbers and inode offsets where applicable; and (b) know about *all* cache blocks, i.e. also of the FS primary caches and not just the blocks that spill into the secondary one. This changes the interface and VM data structures. This change is only for the interface (libminixfs) and VM data structures; the filesystem code is unmodified, so although the secondary cache will be used as normal, blocks will not be annotated with inode information until the FS is modified to provide this information. Until it is modified, mmap of files will fail gracefully on such filesystems. This is indicated to VFS/VM by returning ENOSYS for REQ_PEEK. Change-Id: I1d2df6c485e6c5e89eb28d9055076cc02629594e
2013-03-20 20:18:52 +01:00
if((ino_off % fs_block_size)) {
printf("cache: unaligned lmfs_get_block_ino ino_off %llu\n",
ino_off);
util_stacktrace();
}
/* Search the hash chain for (dev, block). */
can't get_block(NO_DEV) any more . 'anonymous' cache blocks (retrieved with NO_DEV as dev parameter) were used to implement read()s from holes in inodes that should return zeroes . this is an awkward special case in the cache code though and there's a more direct way to implement the same functionality: instead of copying from a new, anonymous, zero block, to the user target buffer, simply sys_safememset the user target buffer directly. as this was the only use of this feature, this is all that's needed to simplify the cache code a little.
2012-11-12 19:15:10 +01:00
b = BUFHASH(block);
bp = buf_hash[b];
while (bp != NULL) {
if (bp->lmfs_blocknr == block && bp->lmfs_dev == dev) {
vm: new secondary cache code Primary purpose of change: to support the mmap implementation, VM must know both (a) about some block metadata for FS cache blocks, i.e. inode numbers and inode offsets where applicable; and (b) know about *all* cache blocks, i.e. also of the FS primary caches and not just the blocks that spill into the secondary one. This changes the interface and VM data structures. This change is only for the interface (libminixfs) and VM data structures; the filesystem code is unmodified, so although the secondary cache will be used as normal, blocks will not be annotated with inode information until the FS is modified to provide this information. Until it is modified, mmap of files will fail gracefully on such filesystems. This is indicated to VFS/VM by returning ENOSYS for REQ_PEEK. Change-Id: I1d2df6c485e6c5e89eb28d9055076cc02629594e
2013-03-20 20:18:52 +01:00
if(bp->lmfs_flags & VMMC_EVICTED) {
/* We had it but VM evicted it; invalidate it. */
ASSERT(bp->lmfs_count == 0);
ASSERT(!(bp->lmfs_flags & VMMC_BLOCK_LOCKED));
ASSERT(!(bp->lmfs_flags & VMMC_DIRTY));
bp->lmfs_dev = NO_DEV;
bp->lmfs_bytes = 0;
bp->data = NULL;
break;
}
ASSERT(bp->lmfs_needsetcache == 0);
can't get_block(NO_DEV) any more . 'anonymous' cache blocks (retrieved with NO_DEV as dev parameter) were used to implement read()s from holes in inodes that should return zeroes . this is an awkward special case in the cache code though and there's a more direct way to implement the same functionality: instead of copying from a new, anonymous, zero block, to the user target buffer, simply sys_safememset the user target buffer directly. as this was the only use of this feature, this is all that's needed to simplify the cache code a little.
2012-11-12 19:15:10 +01:00
/* Block needed has been found. */
vm: new secondary cache code Primary purpose of change: to support the mmap implementation, VM must know both (a) about some block metadata for FS cache blocks, i.e. inode numbers and inode offsets where applicable; and (b) know about *all* cache blocks, i.e. also of the FS primary caches and not just the blocks that spill into the secondary one. This changes the interface and VM data structures. This change is only for the interface (libminixfs) and VM data structures; the filesystem code is unmodified, so although the secondary cache will be used as normal, blocks will not be annotated with inode information until the FS is modified to provide this information. Until it is modified, mmap of files will fail gracefully on such filesystems. This is indicated to VFS/VM by returning ENOSYS for REQ_PEEK. Change-Id: I1d2df6c485e6c5e89eb28d9055076cc02629594e
2013-03-20 20:18:52 +01:00
if (bp->lmfs_count == 0) {
rm_lru(bp);
ASSERT(!(bp->lmfs_flags & VMMC_BLOCK_LOCKED));
bp->lmfs_flags |= VMMC_BLOCK_LOCKED;
}
raisecount(bp);
can't get_block(NO_DEV) any more . 'anonymous' cache blocks (retrieved with NO_DEV as dev parameter) were used to implement read()s from holes in inodes that should return zeroes . this is an awkward special case in the cache code though and there's a more direct way to implement the same functionality: instead of copying from a new, anonymous, zero block, to the user target buffer, simply sys_safememset the user target buffer directly. as this was the only use of this feature, this is all that's needed to simplify the cache code a little.
2012-11-12 19:15:10 +01:00
ASSERT(bp->lmfs_bytes == fs_block_size);
ASSERT(bp->lmfs_dev == dev);
ASSERT(bp->lmfs_dev != NO_DEV);
vm: new secondary cache code Primary purpose of change: to support the mmap implementation, VM must know both (a) about some block metadata for FS cache blocks, i.e. inode numbers and inode offsets where applicable; and (b) know about *all* cache blocks, i.e. also of the FS primary caches and not just the blocks that spill into the secondary one. This changes the interface and VM data structures. This change is only for the interface (libminixfs) and VM data structures; the filesystem code is unmodified, so although the secondary cache will be used as normal, blocks will not be annotated with inode information until the FS is modified to provide this information. Until it is modified, mmap of files will fail gracefully on such filesystems. This is indicated to VFS/VM by returning ENOSYS for REQ_PEEK. Change-Id: I1d2df6c485e6c5e89eb28d9055076cc02629594e
2013-03-20 20:18:52 +01:00
ASSERT(bp->lmfs_flags & VMMC_BLOCK_LOCKED);
can't get_block(NO_DEV) any more . 'anonymous' cache blocks (retrieved with NO_DEV as dev parameter) were used to implement read()s from holes in inodes that should return zeroes . this is an awkward special case in the cache code though and there's a more direct way to implement the same functionality: instead of copying from a new, anonymous, zero block, to the user target buffer, simply sys_safememset the user target buffer directly. as this was the only use of this feature, this is all that's needed to simplify the cache code a little.
2012-11-12 19:15:10 +01:00
ASSERT(bp->data);
vm: new secondary cache code Primary purpose of change: to support the mmap implementation, VM must know both (a) about some block metadata for FS cache blocks, i.e. inode numbers and inode offsets where applicable; and (b) know about *all* cache blocks, i.e. also of the FS primary caches and not just the blocks that spill into the secondary one. This changes the interface and VM data structures. This change is only for the interface (libminixfs) and VM data structures; the filesystem code is unmodified, so although the secondary cache will be used as normal, blocks will not be annotated with inode information until the FS is modified to provide this information. Until it is modified, mmap of files will fail gracefully on such filesystems. This is indicated to VFS/VM by returning ENOSYS for REQ_PEEK. Change-Id: I1d2df6c485e6c5e89eb28d9055076cc02629594e
2013-03-20 20:18:52 +01:00
if(ino != VMC_NO_INODE) {
if(bp->lmfs_inode == VMC_NO_INODE
|| bp->lmfs_inode != ino
|| bp->lmfs_inode_offset != ino_off) {
bp->lmfs_inode = ino;
bp->lmfs_inode_offset = ino_off;
bp->lmfs_needsetcache = 1;
}
}
can't get_block(NO_DEV) any more . 'anonymous' cache blocks (retrieved with NO_DEV as dev parameter) were used to implement read()s from holes in inodes that should return zeroes . this is an awkward special case in the cache code though and there's a more direct way to implement the same functionality: instead of copying from a new, anonymous, zero block, to the user target buffer, simply sys_safememset the user target buffer directly. as this was the only use of this feature, this is all that's needed to simplify the cache code a little.
2012-11-12 19:15:10 +01:00
return(bp);
} else {
/* This block is not the one sought. */
bp = bp->lmfs_hash; /* move to next block on hash chain */
}
Move primary cache code to libminixfs. Add primary cache management feature to libminixfs as mfs and ext2 currently do separately, remove cache code from mfs and ext2, and make them use the libminixfs interface. This makes all fields of the buf struct private to libminixfs and FS clients aren't supposed to access them at all. Only the opaque 'void *data' field (the FS block contents, used to be called bp) is to be accessed by the FS client. The main purpose is to implement the interface to the 2ndary vm cache just once, get rid of some code duplication, and add a little abstraction to reduce the code inertia of the whole caching business. Some minor sanity checking and prohibition done by mfs in this code as removed from the generic primary cache code as a result: - checking all inodes are not in use when allocating/resizing the cache - checking readonly filesystems aren't written to - checking the superblock isn't written to on mounted filesystems The minixfslib code relies on fs_blockstats() in the client filesystem to return some FS usage information.
2012-10-16 17:40:39 +02:00
}
vm: new secondary cache code Primary purpose of change: to support the mmap implementation, VM must know both (a) about some block metadata for FS cache blocks, i.e. inode numbers and inode offsets where applicable; and (b) know about *all* cache blocks, i.e. also of the FS primary caches and not just the blocks that spill into the secondary one. This changes the interface and VM data structures. This change is only for the interface (libminixfs) and VM data structures; the filesystem code is unmodified, so although the secondary cache will be used as normal, blocks will not be annotated with inode information until the FS is modified to provide this information. Until it is modified, mmap of files will fail gracefully on such filesystems. This is indicated to VFS/VM by returning ENOSYS for REQ_PEEK. Change-Id: I1d2df6c485e6c5e89eb28d9055076cc02629594e
2013-03-20 20:18:52 +01:00
/* Desired block is not on available chain. Find a free block to use. */
if(bp) {
ASSERT(bp->lmfs_flags & VMMC_EVICTED);
} else {
if ((bp = front) == NULL) panic("all buffers in use: %d", nr_bufs);
Move primary cache code to libminixfs. Add primary cache management feature to libminixfs as mfs and ext2 currently do separately, remove cache code from mfs and ext2, and make them use the libminixfs interface. This makes all fields of the buf struct private to libminixfs and FS clients aren't supposed to access them at all. Only the opaque 'void *data' field (the FS block contents, used to be called bp) is to be accessed by the FS client. The main purpose is to implement the interface to the 2ndary vm cache just once, get rid of some code duplication, and add a little abstraction to reduce the code inertia of the whole caching business. Some minor sanity checking and prohibition done by mfs in this code as removed from the generic primary cache code as a result: - checking all inodes are not in use when allocating/resizing the cache - checking readonly filesystems aren't written to - checking the superblock isn't written to on mounted filesystems The minixfslib code relies on fs_blockstats() in the client filesystem to return some FS usage information.
2012-10-16 17:40:39 +02:00
}
vm: new secondary cache code Primary purpose of change: to support the mmap implementation, VM must know both (a) about some block metadata for FS cache blocks, i.e. inode numbers and inode offsets where applicable; and (b) know about *all* cache blocks, i.e. also of the FS primary caches and not just the blocks that spill into the secondary one. This changes the interface and VM data structures. This change is only for the interface (libminixfs) and VM data structures; the filesystem code is unmodified, so although the secondary cache will be used as normal, blocks will not be annotated with inode information until the FS is modified to provide this information. Until it is modified, mmap of files will fail gracefully on such filesystems. This is indicated to VFS/VM by returning ENOSYS for REQ_PEEK. Change-Id: I1d2df6c485e6c5e89eb28d9055076cc02629594e
2013-03-20 20:18:52 +01:00
assert(bp);
Move primary cache code to libminixfs. Add primary cache management feature to libminixfs as mfs and ext2 currently do separately, remove cache code from mfs and ext2, and make them use the libminixfs interface. This makes all fields of the buf struct private to libminixfs and FS clients aren't supposed to access them at all. Only the opaque 'void *data' field (the FS block contents, used to be called bp) is to be accessed by the FS client. The main purpose is to implement the interface to the 2ndary vm cache just once, get rid of some code duplication, and add a little abstraction to reduce the code inertia of the whole caching business. Some minor sanity checking and prohibition done by mfs in this code as removed from the generic primary cache code as a result: - checking all inodes are not in use when allocating/resizing the cache - checking readonly filesystems aren't written to - checking the superblock isn't written to on mounted filesystems The minixfslib code relies on fs_blockstats() in the client filesystem to return some FS usage information.
2012-10-16 17:40:39 +02:00
rm_lru(bp);
/* Remove the block that was just taken from its hash chain. */
b = BUFHASH(bp->lmfs_blocknr);
prev_ptr = buf_hash[b];
if (prev_ptr == bp) {
buf_hash[b] = bp->lmfs_hash;
} else {
/* The block just taken is not on the front of its hash chain. */
while (prev_ptr->lmfs_hash != NULL)
if (prev_ptr->lmfs_hash == bp) {
prev_ptr->lmfs_hash = bp->lmfs_hash; /* found it */
break;
} else {
prev_ptr = prev_ptr->lmfs_hash; /* keep looking */
}
}
vm: new secondary cache code Primary purpose of change: to support the mmap implementation, VM must know both (a) about some block metadata for FS cache blocks, i.e. inode numbers and inode offsets where applicable; and (b) know about *all* cache blocks, i.e. also of the FS primary caches and not just the blocks that spill into the secondary one. This changes the interface and VM data structures. This change is only for the interface (libminixfs) and VM data structures; the filesystem code is unmodified, so although the secondary cache will be used as normal, blocks will not be annotated with inode information until the FS is modified to provide this information. Until it is modified, mmap of files will fail gracefully on such filesystems. This is indicated to VFS/VM by returning ENOSYS for REQ_PEEK. Change-Id: I1d2df6c485e6c5e89eb28d9055076cc02629594e
2013-03-20 20:18:52 +01:00
freeblock(bp);
Move primary cache code to libminixfs. Add primary cache management feature to libminixfs as mfs and ext2 currently do separately, remove cache code from mfs and ext2, and make them use the libminixfs interface. This makes all fields of the buf struct private to libminixfs and FS clients aren't supposed to access them at all. Only the opaque 'void *data' field (the FS block contents, used to be called bp) is to be accessed by the FS client. The main purpose is to implement the interface to the 2ndary vm cache just once, get rid of some code duplication, and add a little abstraction to reduce the code inertia of the whole caching business. Some minor sanity checking and prohibition done by mfs in this code as removed from the generic primary cache code as a result: - checking all inodes are not in use when allocating/resizing the cache - checking readonly filesystems aren't written to - checking the superblock isn't written to on mounted filesystems The minixfslib code relies on fs_blockstats() in the client filesystem to return some FS usage information.
2012-10-16 17:40:39 +02:00
vm: new secondary cache code Primary purpose of change: to support the mmap implementation, VM must know both (a) about some block metadata for FS cache blocks, i.e. inode numbers and inode offsets where applicable; and (b) know about *all* cache blocks, i.e. also of the FS primary caches and not just the blocks that spill into the secondary one. This changes the interface and VM data structures. This change is only for the interface (libminixfs) and VM data structures; the filesystem code is unmodified, so although the secondary cache will be used as normal, blocks will not be annotated with inode information until the FS is modified to provide this information. Until it is modified, mmap of files will fail gracefully on such filesystems. This is indicated to VFS/VM by returning ENOSYS for REQ_PEEK. Change-Id: I1d2df6c485e6c5e89eb28d9055076cc02629594e
2013-03-20 20:18:52 +01:00
bp->lmfs_inode = ino;
bp->lmfs_inode_offset = ino_off;
Move primary cache code to libminixfs. Add primary cache management feature to libminixfs as mfs and ext2 currently do separately, remove cache code from mfs and ext2, and make them use the libminixfs interface. This makes all fields of the buf struct private to libminixfs and FS clients aren't supposed to access them at all. Only the opaque 'void *data' field (the FS block contents, used to be called bp) is to be accessed by the FS client. The main purpose is to implement the interface to the 2ndary vm cache just once, get rid of some code duplication, and add a little abstraction to reduce the code inertia of the whole caching business. Some minor sanity checking and prohibition done by mfs in this code as removed from the generic primary cache code as a result: - checking all inodes are not in use when allocating/resizing the cache - checking readonly filesystems aren't written to - checking the superblock isn't written to on mounted filesystems The minixfslib code relies on fs_blockstats() in the client filesystem to return some FS usage information.
2012-10-16 17:40:39 +02:00
vm: new secondary cache code Primary purpose of change: to support the mmap implementation, VM must know both (a) about some block metadata for FS cache blocks, i.e. inode numbers and inode offsets where applicable; and (b) know about *all* cache blocks, i.e. also of the FS primary caches and not just the blocks that spill into the secondary one. This changes the interface and VM data structures. This change is only for the interface (libminixfs) and VM data structures; the filesystem code is unmodified, so although the secondary cache will be used as normal, blocks will not be annotated with inode information until the FS is modified to provide this information. Until it is modified, mmap of files will fail gracefully on such filesystems. This is indicated to VFS/VM by returning ENOSYS for REQ_PEEK. Change-Id: I1d2df6c485e6c5e89eb28d9055076cc02629594e
2013-03-20 20:18:52 +01:00
bp->lmfs_flags = VMMC_BLOCK_LOCKED;
bp->lmfs_needsetcache = 0;
Move primary cache code to libminixfs. Add primary cache management feature to libminixfs as mfs and ext2 currently do separately, remove cache code from mfs and ext2, and make them use the libminixfs interface. This makes all fields of the buf struct private to libminixfs and FS clients aren't supposed to access them at all. Only the opaque 'void *data' field (the FS block contents, used to be called bp) is to be accessed by the FS client. The main purpose is to implement the interface to the 2ndary vm cache just once, get rid of some code duplication, and add a little abstraction to reduce the code inertia of the whole caching business. Some minor sanity checking and prohibition done by mfs in this code as removed from the generic primary cache code as a result: - checking all inodes are not in use when allocating/resizing the cache - checking readonly filesystems aren't written to - checking the superblock isn't written to on mounted filesystems The minixfslib code relies on fs_blockstats() in the client filesystem to return some FS usage information.
2012-10-16 17:40:39 +02:00
bp->lmfs_dev = dev; /* fill in device number */
bp->lmfs_blocknr = block; /* fill in block number */
vm: new secondary cache code Primary purpose of change: to support the mmap implementation, VM must know both (a) about some block metadata for FS cache blocks, i.e. inode numbers and inode offsets where applicable; and (b) know about *all* cache blocks, i.e. also of the FS primary caches and not just the blocks that spill into the secondary one. This changes the interface and VM data structures. This change is only for the interface (libminixfs) and VM data structures; the filesystem code is unmodified, so although the secondary cache will be used as normal, blocks will not be annotated with inode information until the FS is modified to provide this information. Until it is modified, mmap of files will fail gracefully on such filesystems. This is indicated to VFS/VM by returning ENOSYS for REQ_PEEK. Change-Id: I1d2df6c485e6c5e89eb28d9055076cc02629594e
2013-03-20 20:18:52 +01:00
ASSERT(bp->lmfs_count == 0);
raisecount(bp);
Move primary cache code to libminixfs. Add primary cache management feature to libminixfs as mfs and ext2 currently do separately, remove cache code from mfs and ext2, and make them use the libminixfs interface. This makes all fields of the buf struct private to libminixfs and FS clients aren't supposed to access them at all. Only the opaque 'void *data' field (the FS block contents, used to be called bp) is to be accessed by the FS client. The main purpose is to implement the interface to the 2ndary vm cache just once, get rid of some code duplication, and add a little abstraction to reduce the code inertia of the whole caching business. Some minor sanity checking and prohibition done by mfs in this code as removed from the generic primary cache code as a result: - checking all inodes are not in use when allocating/resizing the cache - checking readonly filesystems aren't written to - checking the superblock isn't written to on mounted filesystems The minixfslib code relies on fs_blockstats() in the client filesystem to return some FS usage information.
2012-10-16 17:40:39 +02:00
b = BUFHASH(bp->lmfs_blocknr);
bp->lmfs_hash = buf_hash[b];
buf_hash[b] = bp; /* add to hash list */
can't get_block(NO_DEV) any more . 'anonymous' cache blocks (retrieved with NO_DEV as dev parameter) were used to implement read()s from holes in inodes that should return zeroes . this is an awkward special case in the cache code though and there's a more direct way to implement the same functionality: instead of copying from a new, anonymous, zero block, to the user target buffer, simply sys_safememset the user target buffer directly. as this was the only use of this feature, this is all that's needed to simplify the cache code a little.
2012-11-12 19:15:10 +01:00
assert(dev != NO_DEV);
Move primary cache code to libminixfs. Add primary cache management feature to libminixfs as mfs and ext2 currently do separately, remove cache code from mfs and ext2, and make them use the libminixfs interface. This makes all fields of the buf struct private to libminixfs and FS clients aren't supposed to access them at all. Only the opaque 'void *data' field (the FS block contents, used to be called bp) is to be accessed by the FS client. The main purpose is to implement the interface to the 2ndary vm cache just once, get rid of some code duplication, and add a little abstraction to reduce the code inertia of the whole caching business. Some minor sanity checking and prohibition done by mfs in this code as removed from the generic primary cache code as a result: - checking all inodes are not in use when allocating/resizing the cache - checking readonly filesystems aren't written to - checking the superblock isn't written to on mounted filesystems The minixfslib code relies on fs_blockstats() in the client filesystem to return some FS usage information.
2012-10-16 17:40:39 +02:00
vm: new secondary cache code Primary purpose of change: to support the mmap implementation, VM must know both (a) about some block metadata for FS cache blocks, i.e. inode numbers and inode offsets where applicable; and (b) know about *all* cache blocks, i.e. also of the FS primary caches and not just the blocks that spill into the secondary one. This changes the interface and VM data structures. This change is only for the interface (libminixfs) and VM data structures; the filesystem code is unmodified, so although the secondary cache will be used as normal, blocks will not be annotated with inode information until the FS is modified to provide this information. Until it is modified, mmap of files will fail gracefully on such filesystems. This is indicated to VFS/VM by returning ENOSYS for REQ_PEEK. Change-Id: I1d2df6c485e6c5e89eb28d9055076cc02629594e
2013-03-20 20:18:52 +01:00
/* Block is not found in our cache, but we do want it
* if it's in the vm cache.
*/
assert(!bp->data);
assert(!bp->lmfs_bytes);
if(vmcache) {
if((bp->data = vm_map_cacheblock(dev, dev_off, ino, ino_off,
&bp->lmfs_flags, fs_block_size)) != MAP_FAILED) {
bp->lmfs_bytes = fs_block_size;
ASSERT(!bp->lmfs_needsetcache);
return bp;
Move primary cache code to libminixfs. Add primary cache management feature to libminixfs as mfs and ext2 currently do separately, remove cache code from mfs and ext2, and make them use the libminixfs interface. This makes all fields of the buf struct private to libminixfs and FS clients aren't supposed to access them at all. Only the opaque 'void *data' field (the FS block contents, used to be called bp) is to be accessed by the FS client. The main purpose is to implement the interface to the 2ndary vm cache just once, get rid of some code duplication, and add a little abstraction to reduce the code inertia of the whole caching business. Some minor sanity checking and prohibition done by mfs in this code as removed from the generic primary cache code as a result: - checking all inodes are not in use when allocating/resizing the cache - checking readonly filesystems aren't written to - checking the superblock isn't written to on mounted filesystems The minixfslib code relies on fs_blockstats() in the client filesystem to return some FS usage information.
2012-10-16 17:40:39 +02:00
}
}
vm: new secondary cache code Primary purpose of change: to support the mmap implementation, VM must know both (a) about some block metadata for FS cache blocks, i.e. inode numbers and inode offsets where applicable; and (b) know about *all* cache blocks, i.e. also of the FS primary caches and not just the blocks that spill into the secondary one. This changes the interface and VM data structures. This change is only for the interface (libminixfs) and VM data structures; the filesystem code is unmodified, so although the secondary cache will be used as normal, blocks will not be annotated with inode information until the FS is modified to provide this information. Until it is modified, mmap of files will fail gracefully on such filesystems. This is indicated to VFS/VM by returning ENOSYS for REQ_PEEK. Change-Id: I1d2df6c485e6c5e89eb28d9055076cc02629594e
2013-03-20 20:18:52 +01:00
bp->data = NULL;
/* Not in the cache; reserve memory for its contents. */
lmfs_alloc_block(bp);
assert(bp->data);
Move primary cache code to libminixfs. Add primary cache management feature to libminixfs as mfs and ext2 currently do separately, remove cache code from mfs and ext2, and make them use the libminixfs interface. This makes all fields of the buf struct private to libminixfs and FS clients aren't supposed to access them at all. Only the opaque 'void *data' field (the FS block contents, used to be called bp) is to be accessed by the FS client. The main purpose is to implement the interface to the 2ndary vm cache just once, get rid of some code duplication, and add a little abstraction to reduce the code inertia of the whole caching business. Some minor sanity checking and prohibition done by mfs in this code as removed from the generic primary cache code as a result: - checking all inodes are not in use when allocating/resizing the cache - checking readonly filesystems aren't written to - checking the superblock isn't written to on mounted filesystems The minixfslib code relies on fs_blockstats() in the client filesystem to return some FS usage information.
2012-10-16 17:40:39 +02:00
if(only_search == PREFETCH) {
/* PREFETCH: don't do i/o. */
bp->lmfs_dev = NO_DEV;
} else if (only_search == NORMAL) {
read_block(bp);
} else if(only_search == NO_READ) {
vm: new secondary cache code Primary purpose of change: to support the mmap implementation, VM must know both (a) about some block metadata for FS cache blocks, i.e. inode numbers and inode offsets where applicable; and (b) know about *all* cache blocks, i.e. also of the FS primary caches and not just the blocks that spill into the secondary one. This changes the interface and VM data structures. This change is only for the interface (libminixfs) and VM data structures; the filesystem code is unmodified, so although the secondary cache will be used as normal, blocks will not be annotated with inode information until the FS is modified to provide this information. Until it is modified, mmap of files will fail gracefully on such filesystems. This is indicated to VFS/VM by returning ENOSYS for REQ_PEEK. Change-Id: I1d2df6c485e6c5e89eb28d9055076cc02629594e
2013-03-20 20:18:52 +01:00
/* This block will be overwritten by new contents. */
Move primary cache code to libminixfs. Add primary cache management feature to libminixfs as mfs and ext2 currently do separately, remove cache code from mfs and ext2, and make them use the libminixfs interface. This makes all fields of the buf struct private to libminixfs and FS clients aren't supposed to access them at all. Only the opaque 'void *data' field (the FS block contents, used to be called bp) is to be accessed by the FS client. The main purpose is to implement the interface to the 2ndary vm cache just once, get rid of some code duplication, and add a little abstraction to reduce the code inertia of the whole caching business. Some minor sanity checking and prohibition done by mfs in this code as removed from the generic primary cache code as a result: - checking all inodes are not in use when allocating/resizing the cache - checking readonly filesystems aren't written to - checking the superblock isn't written to on mounted filesystems The minixfslib code relies on fs_blockstats() in the client filesystem to return some FS usage information.
2012-10-16 17:40:39 +02:00
} else
panic("unexpected only_search value: %d", only_search);
assert(bp->data);
return(bp); /* return the newly acquired block */
}
/*===========================================================================*
* lmfs_put_block *
*===========================================================================*/
void lmfs_put_block(bp, block_type)
register struct buf *bp; /* pointer to the buffer to be released */
int block_type; /* INODE_BLOCK, DIRECTORY_BLOCK, or whatever */
{
/* Return a block to the list of available blocks. Depending on 'block_type'
* it may be put on the front or rear of the LRU chain. Blocks that are
* expected to be needed again shortly (e.g., partially full data blocks)
* go on the rear; blocks that are unlikely to be needed again shortly
* (e.g., full data blocks) go on the front. Blocks whose loss can hurt
* the integrity of the file system (e.g., inode blocks) are written to
* disk immediately if they are dirty.
*/
vm: new secondary cache code Primary purpose of change: to support the mmap implementation, VM must know both (a) about some block metadata for FS cache blocks, i.e. inode numbers and inode offsets where applicable; and (b) know about *all* cache blocks, i.e. also of the FS primary caches and not just the blocks that spill into the secondary one. This changes the interface and VM data structures. This change is only for the interface (libminixfs) and VM data structures; the filesystem code is unmodified, so although the secondary cache will be used as normal, blocks will not be annotated with inode information until the FS is modified to provide this information. Until it is modified, mmap of files will fail gracefully on such filesystems. This is indicated to VFS/VM by returning ENOSYS for REQ_PEEK. Change-Id: I1d2df6c485e6c5e89eb28d9055076cc02629594e
2013-03-20 20:18:52 +01:00
dev_t dev;
u64_t dev_off;
int r;
Move primary cache code to libminixfs. Add primary cache management feature to libminixfs as mfs and ext2 currently do separately, remove cache code from mfs and ext2, and make them use the libminixfs interface. This makes all fields of the buf struct private to libminixfs and FS clients aren't supposed to access them at all. Only the opaque 'void *data' field (the FS block contents, used to be called bp) is to be accessed by the FS client. The main purpose is to implement the interface to the 2ndary vm cache just once, get rid of some code duplication, and add a little abstraction to reduce the code inertia of the whole caching business. Some minor sanity checking and prohibition done by mfs in this code as removed from the generic primary cache code as a result: - checking all inodes are not in use when allocating/resizing the cache - checking readonly filesystems aren't written to - checking the superblock isn't written to on mounted filesystems The minixfslib code relies on fs_blockstats() in the client filesystem to return some FS usage information.
2012-10-16 17:40:39 +02:00
if (bp == NULL) return; /* it is easier to check here than in caller */
vm: new secondary cache code Primary purpose of change: to support the mmap implementation, VM must know both (a) about some block metadata for FS cache blocks, i.e. inode numbers and inode offsets where applicable; and (b) know about *all* cache blocks, i.e. also of the FS primary caches and not just the blocks that spill into the secondary one. This changes the interface and VM data structures. This change is only for the interface (libminixfs) and VM data structures; the filesystem code is unmodified, so although the secondary cache will be used as normal, blocks will not be annotated with inode information until the FS is modified to provide this information. Until it is modified, mmap of files will fail gracefully on such filesystems. This is indicated to VFS/VM by returning ENOSYS for REQ_PEEK. Change-Id: I1d2df6c485e6c5e89eb28d9055076cc02629594e
2013-03-20 20:18:52 +01:00
dev = bp->lmfs_dev;
Move primary cache code to libminixfs. Add primary cache management feature to libminixfs as mfs and ext2 currently do separately, remove cache code from mfs and ext2, and make them use the libminixfs interface. This makes all fields of the buf struct private to libminixfs and FS clients aren't supposed to access them at all. Only the opaque 'void *data' field (the FS block contents, used to be called bp) is to be accessed by the FS client. The main purpose is to implement the interface to the 2ndary vm cache just once, get rid of some code duplication, and add a little abstraction to reduce the code inertia of the whole caching business. Some minor sanity checking and prohibition done by mfs in this code as removed from the generic primary cache code as a result: - checking all inodes are not in use when allocating/resizing the cache - checking readonly filesystems aren't written to - checking the superblock isn't written to on mounted filesystems The minixfslib code relies on fs_blockstats() in the client filesystem to return some FS usage information.
2012-10-16 17:40:39 +02:00
vm: new secondary cache code Primary purpose of change: to support the mmap implementation, VM must know both (a) about some block metadata for FS cache blocks, i.e. inode numbers and inode offsets where applicable; and (b) know about *all* cache blocks, i.e. also of the FS primary caches and not just the blocks that spill into the secondary one. This changes the interface and VM data structures. This change is only for the interface (libminixfs) and VM data structures; the filesystem code is unmodified, so although the secondary cache will be used as normal, blocks will not be annotated with inode information until the FS is modified to provide this information. Until it is modified, mmap of files will fail gracefully on such filesystems. This is indicated to VFS/VM by returning ENOSYS for REQ_PEEK. Change-Id: I1d2df6c485e6c5e89eb28d9055076cc02629594e
2013-03-20 20:18:52 +01:00
dev_off = (u64_t) bp->lmfs_blocknr * fs_block_size;
lowercount(bp);
if (bp->lmfs_count != 0) return; /* block is still in use */
Move primary cache code to libminixfs. Add primary cache management feature to libminixfs as mfs and ext2 currently do separately, remove cache code from mfs and ext2, and make them use the libminixfs interface. This makes all fields of the buf struct private to libminixfs and FS clients aren't supposed to access them at all. Only the opaque 'void *data' field (the FS block contents, used to be called bp) is to be accessed by the FS client. The main purpose is to implement the interface to the 2ndary vm cache just once, get rid of some code duplication, and add a little abstraction to reduce the code inertia of the whole caching business. Some minor sanity checking and prohibition done by mfs in this code as removed from the generic primary cache code as a result: - checking all inodes are not in use when allocating/resizing the cache - checking readonly filesystems aren't written to - checking the superblock isn't written to on mounted filesystems The minixfslib code relies on fs_blockstats() in the client filesystem to return some FS usage information.
2012-10-16 17:40:39 +02:00
/* Put this block back on the LRU chain. */
vm: new secondary cache code Primary purpose of change: to support the mmap implementation, VM must know both (a) about some block metadata for FS cache blocks, i.e. inode numbers and inode offsets where applicable; and (b) know about *all* cache blocks, i.e. also of the FS primary caches and not just the blocks that spill into the secondary one. This changes the interface and VM data structures. This change is only for the interface (libminixfs) and VM data structures; the filesystem code is unmodified, so although the secondary cache will be used as normal, blocks will not be annotated with inode information until the FS is modified to provide this information. Until it is modified, mmap of files will fail gracefully on such filesystems. This is indicated to VFS/VM by returning ENOSYS for REQ_PEEK. Change-Id: I1d2df6c485e6c5e89eb28d9055076cc02629594e
2013-03-20 20:18:52 +01:00
if (dev == DEV_RAM || (block_type & ONE_SHOT)) {
Move primary cache code to libminixfs. Add primary cache management feature to libminixfs as mfs and ext2 currently do separately, remove cache code from mfs and ext2, and make them use the libminixfs interface. This makes all fields of the buf struct private to libminixfs and FS clients aren't supposed to access them at all. Only the opaque 'void *data' field (the FS block contents, used to be called bp) is to be accessed by the FS client. The main purpose is to implement the interface to the 2ndary vm cache just once, get rid of some code duplication, and add a little abstraction to reduce the code inertia of the whole caching business. Some minor sanity checking and prohibition done by mfs in this code as removed from the generic primary cache code as a result: - checking all inodes are not in use when allocating/resizing the cache - checking readonly filesystems aren't written to - checking the superblock isn't written to on mounted filesystems The minixfslib code relies on fs_blockstats() in the client filesystem to return some FS usage information.
2012-10-16 17:40:39 +02:00
/* Block probably won't be needed quickly. Put it on front of chain.
* It will be the next block to be evicted from the cache.
*/
bp->lmfs_prev = NULL;
bp->lmfs_next = front;
if (front == NULL)
rear = bp; /* LRU chain was empty */
else
front->lmfs_prev = bp;
front = bp;
}
else {
/* Block probably will be needed quickly. Put it on rear of chain.
* It will not be evicted from the cache for a long time.
*/
bp->lmfs_prev = rear;
bp->lmfs_next = NULL;
if (rear == NULL)
front = bp;
else
rear->lmfs_next = bp;
rear = bp;
}
vm: new secondary cache code Primary purpose of change: to support the mmap implementation, VM must know both (a) about some block metadata for FS cache blocks, i.e. inode numbers and inode offsets where applicable; and (b) know about *all* cache blocks, i.e. also of the FS primary caches and not just the blocks that spill into the secondary one. This changes the interface and VM data structures. This change is only for the interface (libminixfs) and VM data structures; the filesystem code is unmodified, so although the secondary cache will be used as normal, blocks will not be annotated with inode information until the FS is modified to provide this information. Until it is modified, mmap of files will fail gracefully on such filesystems. This is indicated to VFS/VM by returning ENOSYS for REQ_PEEK. Change-Id: I1d2df6c485e6c5e89eb28d9055076cc02629594e
2013-03-20 20:18:52 +01:00
assert(bp->lmfs_flags & VMMC_BLOCK_LOCKED);
bp->lmfs_flags &= ~VMMC_BLOCK_LOCKED;
/* block has sensible content - if necesary, identify it to VM */
if(vmcache && bp->lmfs_needsetcache && dev != NO_DEV) {
if((r=vm_set_cacheblock(bp->data, dev, dev_off,
bp->lmfs_inode, bp->lmfs_inode_offset,
&bp->lmfs_flags, fs_block_size)) != OK) {
if(r == ENOSYS) {
printf("libminixfs: ENOSYS, disabling VM calls\n");
vmcache = 0;
} else {
panic: declare as printf-style-checked . and related fixes Change-Id: I5131ac57dc53d8aec8d421a34c5ceea383404d7a
2013-02-02 01:55:35 +01:00
panic("libminixfs: setblock of %p dev 0x%x off "
vm: new secondary cache code Primary purpose of change: to support the mmap implementation, VM must know both (a) about some block metadata for FS cache blocks, i.e. inode numbers and inode offsets where applicable; and (b) know about *all* cache blocks, i.e. also of the FS primary caches and not just the blocks that spill into the secondary one. This changes the interface and VM data structures. This change is only for the interface (libminixfs) and VM data structures; the filesystem code is unmodified, so although the secondary cache will be used as normal, blocks will not be annotated with inode information until the FS is modified to provide this information. Until it is modified, mmap of files will fail gracefully on such filesystems. This is indicated to VFS/VM by returning ENOSYS for REQ_PEEK. Change-Id: I1d2df6c485e6c5e89eb28d9055076cc02629594e
2013-03-20 20:18:52 +01:00
"0x%llx failed\n", bp->data, dev, dev_off);
}
}
}
bp->lmfs_needsetcache = 0;
libminixfs, mfs, ext2: may re-evaluate cache size libminixfs may now be informed of changes to the block usage on the filesystem. if the net change becomes big enough, libminixfs may resize the cache based on the new usage. . update the 2 FSes to provide this information to libminixfs Change-Id: I158815a11da801fd5572a8de89c9e6c039b82650
2013-04-26 11:54:33 +02:00
}
void lmfs_cache_reevaluate(dev_t dev)
{
if(bufs_in_use == 0 && dev != NO_DEV) {
/* if the cache isn't in use any more, we could resize it. */
cache_heuristic_check(major(dev));
}
Move primary cache code to libminixfs. Add primary cache management feature to libminixfs as mfs and ext2 currently do separately, remove cache code from mfs and ext2, and make them use the libminixfs interface. This makes all fields of the buf struct private to libminixfs and FS clients aren't supposed to access them at all. Only the opaque 'void *data' field (the FS block contents, used to be called bp) is to be accessed by the FS client. The main purpose is to implement the interface to the 2ndary vm cache just once, get rid of some code duplication, and add a little abstraction to reduce the code inertia of the whole caching business. Some minor sanity checking and prohibition done by mfs in this code as removed from the generic primary cache code as a result: - checking all inodes are not in use when allocating/resizing the cache - checking readonly filesystems aren't written to - checking the superblock isn't written to on mounted filesystems The minixfslib code relies on fs_blockstats() in the client filesystem to return some FS usage information.
2012-10-16 17:40:39 +02:00
}
/*===========================================================================*
* read_block *
*===========================================================================*/
static void read_block(bp)
register struct buf *bp; /* buffer pointer */
{
/* Read or write a disk block. This is the only routine in which actual disk
* I/O is invoked. If an error occurs, a message is printed here, but the error
* is not reported to the caller. If the error occurred while purging a block
* from the cache, it is not clear what the caller could do about it anyway.
*/
int r, op_failed;
u64_t pos;
can't get_block(NO_DEV) any more . 'anonymous' cache blocks (retrieved with NO_DEV as dev parameter) were used to implement read()s from holes in inodes that should return zeroes . this is an awkward special case in the cache code though and there's a more direct way to implement the same functionality: instead of copying from a new, anonymous, zero block, to the user target buffer, simply sys_safememset the user target buffer directly. as this was the only use of this feature, this is all that's needed to simplify the cache code a little.
2012-11-12 19:15:10 +01:00
dev_t dev = bp->lmfs_dev;
Move primary cache code to libminixfs. Add primary cache management feature to libminixfs as mfs and ext2 currently do separately, remove cache code from mfs and ext2, and make them use the libminixfs interface. This makes all fields of the buf struct private to libminixfs and FS clients aren't supposed to access them at all. Only the opaque 'void *data' field (the FS block contents, used to be called bp) is to be accessed by the FS client. The main purpose is to implement the interface to the 2ndary vm cache just once, get rid of some code duplication, and add a little abstraction to reduce the code inertia of the whole caching business. Some minor sanity checking and prohibition done by mfs in this code as removed from the generic primary cache code as a result: - checking all inodes are not in use when allocating/resizing the cache - checking readonly filesystems aren't written to - checking the superblock isn't written to on mounted filesystems The minixfslib code relies on fs_blockstats() in the client filesystem to return some FS usage information.
2012-10-16 17:40:39 +02:00
op_failed = 0;
can't get_block(NO_DEV) any more . 'anonymous' cache blocks (retrieved with NO_DEV as dev parameter) were used to implement read()s from holes in inodes that should return zeroes . this is an awkward special case in the cache code though and there's a more direct way to implement the same functionality: instead of copying from a new, anonymous, zero block, to the user target buffer, simply sys_safememset the user target buffer directly. as this was the only use of this feature, this is all that's needed to simplify the cache code a little.
2012-11-12 19:15:10 +01:00
assert(dev != NO_DEV);
vm: new secondary cache code Primary purpose of change: to support the mmap implementation, VM must know both (a) about some block metadata for FS cache blocks, i.e. inode numbers and inode offsets where applicable; and (b) know about *all* cache blocks, i.e. also of the FS primary caches and not just the blocks that spill into the secondary one. This changes the interface and VM data structures. This change is only for the interface (libminixfs) and VM data structures; the filesystem code is unmodified, so although the secondary cache will be used as normal, blocks will not be annotated with inode information until the FS is modified to provide this information. Until it is modified, mmap of files will fail gracefully on such filesystems. This is indicated to VFS/VM by returning ENOSYS for REQ_PEEK. Change-Id: I1d2df6c485e6c5e89eb28d9055076cc02629594e
2013-03-20 20:18:52 +01:00
ASSERT(bp->lmfs_bytes == fs_block_size);
ASSERT(fs_block_size > 0);
can't get_block(NO_DEV) any more . 'anonymous' cache blocks (retrieved with NO_DEV as dev parameter) were used to implement read()s from holes in inodes that should return zeroes . this is an awkward special case in the cache code though and there's a more direct way to implement the same functionality: instead of copying from a new, anonymous, zero block, to the user target buffer, simply sys_safememset the user target buffer directly. as this was the only use of this feature, this is all that's needed to simplify the cache code a little.
2012-11-12 19:15:10 +01:00
pos = mul64u(bp->lmfs_blocknr, fs_block_size);
vm: new secondary cache code Primary purpose of change: to support the mmap implementation, VM must know both (a) about some block metadata for FS cache blocks, i.e. inode numbers and inode offsets where applicable; and (b) know about *all* cache blocks, i.e. also of the FS primary caches and not just the blocks that spill into the secondary one. This changes the interface and VM data structures. This change is only for the interface (libminixfs) and VM data structures; the filesystem code is unmodified, so although the secondary cache will be used as normal, blocks will not be annotated with inode information until the FS is modified to provide this information. Until it is modified, mmap of files will fail gracefully on such filesystems. This is indicated to VFS/VM by returning ENOSYS for REQ_PEEK. Change-Id: I1d2df6c485e6c5e89eb28d9055076cc02629594e
2013-03-20 20:18:52 +01:00
if(fs_block_size > PAGE_SIZE) {
#define MAXPAGES 20
libminixfs: allow non-pagesize-multiple FSes The memory-mapped files implementation (mmap() etc.) is implemented with the help of the filesystems using the in-VM FS cache. Filesystems tell it about all cached blocks and their metadata. Metadata is: device offset and, if any (and known), inode number and in-inode offset. VM can then map in requested memory-mapped file blocks, and request them if necessary. A limitation of this system is that filesystem block sizes that are not a multiple of the VM system (and VM hardware) page size are not possible; we can't map blocks in partially. (We can copy, but then the benefits of mapping and sharing the physical pages is gone.) So until before this commit various pieces of caching code assumed page size multiple blocksizes. This isn't strictly necessary as long as mmap() needn't be supported on that FS. This change allows the in-FS cache code (libminixfs) to allocate any-sized blocks, and will not interact with the VM cache for non-pagesize-multiple blocks. In that case it will also signal requestors, by failing 'peek' requests, that mmap() should not be supported on this FS. VM and VFS will then gracefully fail all file-mapping mmap() calls, and exec() will fall back to copying executable blocks instead of mmap()ping executables. As a result, 3 diagnostics that signal file-mapped mmap()s failing (hitherto an unusual occurence) are disabled, as ld.so does file-mapped mmap()s to map in objects it needs. On FSes not supporting it this situation is legitimate and shouldn't cause so much noise. ld.so will revert to its own minix-specific allocate+copy style of starting executables if mmap()s fail. Change-Id: Iecb1c8090f5e0be28da8f5181bb35084eb18f67b
2013-11-19 16:59:52 +01:00
vir_bytes blockrem, vaddr = (vir_bytes) bp->data;
int p = 0;
vm: new secondary cache code Primary purpose of change: to support the mmap implementation, VM must know both (a) about some block metadata for FS cache blocks, i.e. inode numbers and inode offsets where applicable; and (b) know about *all* cache blocks, i.e. also of the FS primary caches and not just the blocks that spill into the secondary one. This changes the interface and VM data structures. This change is only for the interface (libminixfs) and VM data structures; the filesystem code is unmodified, so although the secondary cache will be used as normal, blocks will not be annotated with inode information until the FS is modified to provide this information. Until it is modified, mmap of files will fail gracefully on such filesystems. This is indicated to VFS/VM by returning ENOSYS for REQ_PEEK. Change-Id: I1d2df6c485e6c5e89eb28d9055076cc02629594e
2013-03-20 20:18:52 +01:00
static iovec_t iovec[MAXPAGES];
libminixfs: allow non-pagesize-multiple FSes The memory-mapped files implementation (mmap() etc.) is implemented with the help of the filesystems using the in-VM FS cache. Filesystems tell it about all cached blocks and their metadata. Metadata is: device offset and, if any (and known), inode number and in-inode offset. VM can then map in requested memory-mapped file blocks, and request them if necessary. A limitation of this system is that filesystem block sizes that are not a multiple of the VM system (and VM hardware) page size are not possible; we can't map blocks in partially. (We can copy, but then the benefits of mapping and sharing the physical pages is gone.) So until before this commit various pieces of caching code assumed page size multiple blocksizes. This isn't strictly necessary as long as mmap() needn't be supported on that FS. This change allows the in-FS cache code (libminixfs) to allocate any-sized blocks, and will not interact with the VM cache for non-pagesize-multiple blocks. In that case it will also signal requestors, by failing 'peek' requests, that mmap() should not be supported on this FS. VM and VFS will then gracefully fail all file-mapping mmap() calls, and exec() will fall back to copying executable blocks instead of mmap()ping executables. As a result, 3 diagnostics that signal file-mapped mmap()s failing (hitherto an unusual occurence) are disabled, as ld.so does file-mapped mmap()s to map in objects it needs. On FSes not supporting it this situation is legitimate and shouldn't cause so much noise. ld.so will revert to its own minix-specific allocate+copy style of starting executables if mmap()s fail. Change-Id: Iecb1c8090f5e0be28da8f5181bb35084eb18f67b
2013-11-19 16:59:52 +01:00
blockrem = fs_block_size;
while(blockrem > 0) {
vir_bytes chunk = blockrem >= PAGE_SIZE ? PAGE_SIZE : blockrem;
vm: new secondary cache code Primary purpose of change: to support the mmap implementation, VM must know both (a) about some block metadata for FS cache blocks, i.e. inode numbers and inode offsets where applicable; and (b) know about *all* cache blocks, i.e. also of the FS primary caches and not just the blocks that spill into the secondary one. This changes the interface and VM data structures. This change is only for the interface (libminixfs) and VM data structures; the filesystem code is unmodified, so although the secondary cache will be used as normal, blocks will not be annotated with inode information until the FS is modified to provide this information. Until it is modified, mmap of files will fail gracefully on such filesystems. This is indicated to VFS/VM by returning ENOSYS for REQ_PEEK. Change-Id: I1d2df6c485e6c5e89eb28d9055076cc02629594e
2013-03-20 20:18:52 +01:00
iovec[p].iov_addr = vaddr;
libminixfs: allow non-pagesize-multiple FSes The memory-mapped files implementation (mmap() etc.) is implemented with the help of the filesystems using the in-VM FS cache. Filesystems tell it about all cached blocks and their metadata. Metadata is: device offset and, if any (and known), inode number and in-inode offset. VM can then map in requested memory-mapped file blocks, and request them if necessary. A limitation of this system is that filesystem block sizes that are not a multiple of the VM system (and VM hardware) page size are not possible; we can't map blocks in partially. (We can copy, but then the benefits of mapping and sharing the physical pages is gone.) So until before this commit various pieces of caching code assumed page size multiple blocksizes. This isn't strictly necessary as long as mmap() needn't be supported on that FS. This change allows the in-FS cache code (libminixfs) to allocate any-sized blocks, and will not interact with the VM cache for non-pagesize-multiple blocks. In that case it will also signal requestors, by failing 'peek' requests, that mmap() should not be supported on this FS. VM and VFS will then gracefully fail all file-mapping mmap() calls, and exec() will fall back to copying executable blocks instead of mmap()ping executables. As a result, 3 diagnostics that signal file-mapped mmap()s failing (hitherto an unusual occurence) are disabled, as ld.so does file-mapped mmap()s to map in objects it needs. On FSes not supporting it this situation is legitimate and shouldn't cause so much noise. ld.so will revert to its own minix-specific allocate+copy style of starting executables if mmap()s fail. Change-Id: Iecb1c8090f5e0be28da8f5181bb35084eb18f67b
2013-11-19 16:59:52 +01:00
iovec[p].iov_size = chunk;
vaddr += chunk;
blockrem -= chunk;
p++;
vm: new secondary cache code Primary purpose of change: to support the mmap implementation, VM must know both (a) about some block metadata for FS cache blocks, i.e. inode numbers and inode offsets where applicable; and (b) know about *all* cache blocks, i.e. also of the FS primary caches and not just the blocks that spill into the secondary one. This changes the interface and VM data structures. This change is only for the interface (libminixfs) and VM data structures; the filesystem code is unmodified, so although the secondary cache will be used as normal, blocks will not be annotated with inode information until the FS is modified to provide this information. Until it is modified, mmap of files will fail gracefully on such filesystems. This is indicated to VFS/VM by returning ENOSYS for REQ_PEEK. Change-Id: I1d2df6c485e6c5e89eb28d9055076cc02629594e
2013-03-20 20:18:52 +01:00
}
libminixfs: allow non-pagesize-multiple FSes The memory-mapped files implementation (mmap() etc.) is implemented with the help of the filesystems using the in-VM FS cache. Filesystems tell it about all cached blocks and their metadata. Metadata is: device offset and, if any (and known), inode number and in-inode offset. VM can then map in requested memory-mapped file blocks, and request them if necessary. A limitation of this system is that filesystem block sizes that are not a multiple of the VM system (and VM hardware) page size are not possible; we can't map blocks in partially. (We can copy, but then the benefits of mapping and sharing the physical pages is gone.) So until before this commit various pieces of caching code assumed page size multiple blocksizes. This isn't strictly necessary as long as mmap() needn't be supported on that FS. This change allows the in-FS cache code (libminixfs) to allocate any-sized blocks, and will not interact with the VM cache for non-pagesize-multiple blocks. In that case it will also signal requestors, by failing 'peek' requests, that mmap() should not be supported on this FS. VM and VFS will then gracefully fail all file-mapping mmap() calls, and exec() will fall back to copying executable blocks instead of mmap()ping executables. As a result, 3 diagnostics that signal file-mapped mmap()s failing (hitherto an unusual occurence) are disabled, as ld.so does file-mapped mmap()s to map in objects it needs. On FSes not supporting it this situation is legitimate and shouldn't cause so much noise. ld.so will revert to its own minix-specific allocate+copy style of starting executables if mmap()s fail. Change-Id: Iecb1c8090f5e0be28da8f5181bb35084eb18f67b
2013-11-19 16:59:52 +01:00
r = bdev_gather(dev, pos, iovec, p, BDEV_NOFLAGS);
vm: new secondary cache code Primary purpose of change: to support the mmap implementation, VM must know both (a) about some block metadata for FS cache blocks, i.e. inode numbers and inode offsets where applicable; and (b) know about *all* cache blocks, i.e. also of the FS primary caches and not just the blocks that spill into the secondary one. This changes the interface and VM data structures. This change is only for the interface (libminixfs) and VM data structures; the filesystem code is unmodified, so although the secondary cache will be used as normal, blocks will not be annotated with inode information until the FS is modified to provide this information. Until it is modified, mmap of files will fail gracefully on such filesystems. This is indicated to VFS/VM by returning ENOSYS for REQ_PEEK. Change-Id: I1d2df6c485e6c5e89eb28d9055076cc02629594e
2013-03-20 20:18:52 +01:00
} else {
r = bdev_read(dev, pos, bp->data, fs_block_size,
BDEV_NOFLAGS);
}
can't get_block(NO_DEV) any more . 'anonymous' cache blocks (retrieved with NO_DEV as dev parameter) were used to implement read()s from holes in inodes that should return zeroes . this is an awkward special case in the cache code though and there's a more direct way to implement the same functionality: instead of copying from a new, anonymous, zero block, to the user target buffer, simply sys_safememset the user target buffer directly. as this was the only use of this feature, this is all that's needed to simplify the cache code a little.
2012-11-12 19:15:10 +01:00
if (r < 0) {
printf("fs cache: I/O error on device %d/%d, block %u\n",
major(dev), minor(dev), bp->lmfs_blocknr);
op_failed = 1;
} else if (r != (ssize_t) fs_block_size) {
r = END_OF_FILE;
op_failed = 1;
}
Move primary cache code to libminixfs. Add primary cache management feature to libminixfs as mfs and ext2 currently do separately, remove cache code from mfs and ext2, and make them use the libminixfs interface. This makes all fields of the buf struct private to libminixfs and FS clients aren't supposed to access them at all. Only the opaque 'void *data' field (the FS block contents, used to be called bp) is to be accessed by the FS client. The main purpose is to implement the interface to the 2ndary vm cache just once, get rid of some code duplication, and add a little abstraction to reduce the code inertia of the whole caching business. Some minor sanity checking and prohibition done by mfs in this code as removed from the generic primary cache code as a result: - checking all inodes are not in use when allocating/resizing the cache - checking readonly filesystems aren't written to - checking the superblock isn't written to on mounted filesystems The minixfslib code relies on fs_blockstats() in the client filesystem to return some FS usage information.
2012-10-16 17:40:39 +02:00
can't get_block(NO_DEV) any more . 'anonymous' cache blocks (retrieved with NO_DEV as dev parameter) were used to implement read()s from holes in inodes that should return zeroes . this is an awkward special case in the cache code though and there's a more direct way to implement the same functionality: instead of copying from a new, anonymous, zero block, to the user target buffer, simply sys_safememset the user target buffer directly. as this was the only use of this feature, this is all that's needed to simplify the cache code a little.
2012-11-12 19:15:10 +01:00
if (op_failed) {
bp->lmfs_dev = NO_DEV; /* invalidate block */
Move primary cache code to libminixfs. Add primary cache management feature to libminixfs as mfs and ext2 currently do separately, remove cache code from mfs and ext2, and make them use the libminixfs interface. This makes all fields of the buf struct private to libminixfs and FS clients aren't supposed to access them at all. Only the opaque 'void *data' field (the FS block contents, used to be called bp) is to be accessed by the FS client. The main purpose is to implement the interface to the 2ndary vm cache just once, get rid of some code duplication, and add a little abstraction to reduce the code inertia of the whole caching business. Some minor sanity checking and prohibition done by mfs in this code as removed from the generic primary cache code as a result: - checking all inodes are not in use when allocating/resizing the cache - checking readonly filesystems aren't written to - checking the superblock isn't written to on mounted filesystems The minixfslib code relies on fs_blockstats() in the client filesystem to return some FS usage information.
2012-10-16 17:40:39 +02:00
can't get_block(NO_DEV) any more . 'anonymous' cache blocks (retrieved with NO_DEV as dev parameter) were used to implement read()s from holes in inodes that should return zeroes . this is an awkward special case in the cache code though and there's a more direct way to implement the same functionality: instead of copying from a new, anonymous, zero block, to the user target buffer, simply sys_safememset the user target buffer directly. as this was the only use of this feature, this is all that's needed to simplify the cache code a little.
2012-11-12 19:15:10 +01:00
/* Report read errors to interested parties. */
rdwt_err = r;
Move primary cache code to libminixfs. Add primary cache management feature to libminixfs as mfs and ext2 currently do separately, remove cache code from mfs and ext2, and make them use the libminixfs interface. This makes all fields of the buf struct private to libminixfs and FS clients aren't supposed to access them at all. Only the opaque 'void *data' field (the FS block contents, used to be called bp) is to be accessed by the FS client. The main purpose is to implement the interface to the 2ndary vm cache just once, get rid of some code duplication, and add a little abstraction to reduce the code inertia of the whole caching business. Some minor sanity checking and prohibition done by mfs in this code as removed from the generic primary cache code as a result: - checking all inodes are not in use when allocating/resizing the cache - checking readonly filesystems aren't written to - checking the superblock isn't written to on mounted filesystems The minixfslib code relies on fs_blockstats() in the client filesystem to return some FS usage information.
2012-10-16 17:40:39 +02:00
}
vm: new secondary cache code Primary purpose of change: to support the mmap implementation, VM must know both (a) about some block metadata for FS cache blocks, i.e. inode numbers and inode offsets where applicable; and (b) know about *all* cache blocks, i.e. also of the FS primary caches and not just the blocks that spill into the secondary one. This changes the interface and VM data structures. This change is only for the interface (libminixfs) and VM data structures; the filesystem code is unmodified, so although the secondary cache will be used as normal, blocks will not be annotated with inode information until the FS is modified to provide this information. Until it is modified, mmap of files will fail gracefully on such filesystems. This is indicated to VFS/VM by returning ENOSYS for REQ_PEEK. Change-Id: I1d2df6c485e6c5e89eb28d9055076cc02629594e
2013-03-20 20:18:52 +01:00
Move primary cache code to libminixfs. Add primary cache management feature to libminixfs as mfs and ext2 currently do separately, remove cache code from mfs and ext2, and make them use the libminixfs interface. This makes all fields of the buf struct private to libminixfs and FS clients aren't supposed to access them at all. Only the opaque 'void *data' field (the FS block contents, used to be called bp) is to be accessed by the FS client. The main purpose is to implement the interface to the 2ndary vm cache just once, get rid of some code duplication, and add a little abstraction to reduce the code inertia of the whole caching business. Some minor sanity checking and prohibition done by mfs in this code as removed from the generic primary cache code as a result: - checking all inodes are not in use when allocating/resizing the cache - checking readonly filesystems aren't written to - checking the superblock isn't written to on mounted filesystems The minixfslib code relies on fs_blockstats() in the client filesystem to return some FS usage information.
2012-10-16 17:40:39 +02:00
}
/*===========================================================================*
* lmfs_invalidate *
*===========================================================================*/
void lmfs_invalidate(
dev_t device /* device whose blocks are to be purged */
)
{
/* Remove all the blocks belonging to some device from the cache. */
register struct buf *bp;
vm: new secondary cache code Primary purpose of change: to support the mmap implementation, VM must know both (a) about some block metadata for FS cache blocks, i.e. inode numbers and inode offsets where applicable; and (b) know about *all* cache blocks, i.e. also of the FS primary caches and not just the blocks that spill into the secondary one. This changes the interface and VM data structures. This change is only for the interface (libminixfs) and VM data structures; the filesystem code is unmodified, so although the secondary cache will be used as normal, blocks will not be annotated with inode information until the FS is modified to provide this information. Until it is modified, mmap of files will fail gracefully on such filesystems. This is indicated to VFS/VM by returning ENOSYS for REQ_PEEK. Change-Id: I1d2df6c485e6c5e89eb28d9055076cc02629594e
2013-03-20 20:18:52 +01:00
for (bp = &buf[0]; bp < &buf[nr_bufs]; bp++) {
if (bp->lmfs_dev == device) {
assert(bp->data);
assert(bp->lmfs_bytes > 0);
minix_munmap_t(bp->data, bp->lmfs_bytes);
bp->lmfs_dev = NO_DEV;
bp->lmfs_bytes = 0;
bp->data = NULL;
}
}
VM: readd support for forgetting cached FS blocks Not all services involved in block I/O go through VM to access the blocks they need. As a result, the blocks in VM may become stale, possibly causing corruption when the stale copy is restored by a service that does go through VM later on. This patch restores support for forgetting cached blocks that belong to a particular device, and makes the relevant file systems use this functionality 1) when requested by VFS through REQ_FLUSH, and 2) upon unmount. Change-Id: I0758c5ed8fe4b5ba81d432595d2113175776aff8
2013-09-15 18:55:42 +02:00
vm_clear_cache(device);
Move primary cache code to libminixfs. Add primary cache management feature to libminixfs as mfs and ext2 currently do separately, remove cache code from mfs and ext2, and make them use the libminixfs interface. This makes all fields of the buf struct private to libminixfs and FS clients aren't supposed to access them at all. Only the opaque 'void *data' field (the FS block contents, used to be called bp) is to be accessed by the FS client. The main purpose is to implement the interface to the 2ndary vm cache just once, get rid of some code duplication, and add a little abstraction to reduce the code inertia of the whole caching business. Some minor sanity checking and prohibition done by mfs in this code as removed from the generic primary cache code as a result: - checking all inodes are not in use when allocating/resizing the cache - checking readonly filesystems aren't written to - checking the superblock isn't written to on mounted filesystems The minixfslib code relies on fs_blockstats() in the client filesystem to return some FS usage information.
2012-10-16 17:40:39 +02:00
}
/*===========================================================================*
* flushall *
*===========================================================================*/
static void flushall(dev_t dev)
{
/* Flush all dirty blocks for one device. */
register struct buf *bp;
static struct buf **dirty; /* static so it isn't on stack */
static unsigned int dirtylistsize = 0;
int ndirty;
if(dirtylistsize != nr_bufs) {
if(dirtylistsize > 0) {
assert(dirty != NULL);
free(dirty);
}
if(!(dirty = malloc(sizeof(dirty[0])*nr_bufs)))
panic("couldn't allocate dirty buf list");
dirtylistsize = nr_bufs;
}
for (bp = &buf[0], ndirty = 0; bp < &buf[nr_bufs]; bp++) {
vm: new secondary cache code Primary purpose of change: to support the mmap implementation, VM must know both (a) about some block metadata for FS cache blocks, i.e. inode numbers and inode offsets where applicable; and (b) know about *all* cache blocks, i.e. also of the FS primary caches and not just the blocks that spill into the secondary one. This changes the interface and VM data structures. This change is only for the interface (libminixfs) and VM data structures; the filesystem code is unmodified, so although the secondary cache will be used as normal, blocks will not be annotated with inode information until the FS is modified to provide this information. Until it is modified, mmap of files will fail gracefully on such filesystems. This is indicated to VFS/VM by returning ENOSYS for REQ_PEEK. Change-Id: I1d2df6c485e6c5e89eb28d9055076cc02629594e
2013-03-20 20:18:52 +01:00
if (!lmfs_isclean(bp) && bp->lmfs_dev == dev) {
Move primary cache code to libminixfs. Add primary cache management feature to libminixfs as mfs and ext2 currently do separately, remove cache code from mfs and ext2, and make them use the libminixfs interface. This makes all fields of the buf struct private to libminixfs and FS clients aren't supposed to access them at all. Only the opaque 'void *data' field (the FS block contents, used to be called bp) is to be accessed by the FS client. The main purpose is to implement the interface to the 2ndary vm cache just once, get rid of some code duplication, and add a little abstraction to reduce the code inertia of the whole caching business. Some minor sanity checking and prohibition done by mfs in this code as removed from the generic primary cache code as a result: - checking all inodes are not in use when allocating/resizing the cache - checking readonly filesystems aren't written to - checking the superblock isn't written to on mounted filesystems The minixfslib code relies on fs_blockstats() in the client filesystem to return some FS usage information.
2012-10-16 17:40:39 +02:00
dirty[ndirty++] = bp;
}
}
lmfs_rw_scattered(dev, dirty, ndirty, WRITING);
}
/*===========================================================================*
* lmfs_rw_scattered *
*===========================================================================*/
void lmfs_rw_scattered(
dev_t dev, /* major-minor device number */
struct buf **bufq, /* pointer to array of buffers */
int bufqsize, /* number of buffers */
int rw_flag /* READING or WRITING */
)
{
/* Read or write scattered data from a device. */
register struct buf *bp;
int gap;
register int i;
register iovec_t *iop;
avoid alloc_contig() for iovec, mfs superblock . initial workaround for assert() firing on iovec size on ARM. likely due to alloc_contig() allocating unusually mapped memory in STATICINIT. . for the same reason use the regular cache i/o functions to read the superblock in mfs - avoid the alloc_contig() that STATICINIT does. Change-Id: I3d8dc635b1cf2666e55b0393feae74cc25b8fed4
2013-10-03 17:22:07 +02:00
static iovec_t iovec[NR_IOREQS];
Move primary cache code to libminixfs. Add primary cache management feature to libminixfs as mfs and ext2 currently do separately, remove cache code from mfs and ext2, and make them use the libminixfs interface. This makes all fields of the buf struct private to libminixfs and FS clients aren't supposed to access them at all. Only the opaque 'void *data' field (the FS block contents, used to be called bp) is to be accessed by the FS client. The main purpose is to implement the interface to the 2ndary vm cache just once, get rid of some code duplication, and add a little abstraction to reduce the code inertia of the whole caching business. Some minor sanity checking and prohibition done by mfs in this code as removed from the generic primary cache code as a result: - checking all inodes are not in use when allocating/resizing the cache - checking readonly filesystems aren't written to - checking the superblock isn't written to on mounted filesystems The minixfslib code relies on fs_blockstats() in the client filesystem to return some FS usage information.
2012-10-16 17:40:39 +02:00
u64_t pos;
vm: new secondary cache code Primary purpose of change: to support the mmap implementation, VM must know both (a) about some block metadata for FS cache blocks, i.e. inode numbers and inode offsets where applicable; and (b) know about *all* cache blocks, i.e. also of the FS primary caches and not just the blocks that spill into the secondary one. This changes the interface and VM data structures. This change is only for the interface (libminixfs) and VM data structures; the filesystem code is unmodified, so although the secondary cache will be used as normal, blocks will not be annotated with inode information until the FS is modified to provide this information. Until it is modified, mmap of files will fail gracefully on such filesystems. This is indicated to VFS/VM by returning ENOSYS for REQ_PEEK. Change-Id: I1d2df6c485e6c5e89eb28d9055076cc02629594e
2013-03-20 20:18:52 +01:00
int iov_per_block;
libminixfs: fix buf leak . mfs: add asserts to catch similar problems in the future Change-Id: I939230b8d63f100ac071696425b35496b2169bc4
2013-11-05 15:43:27 +01:00
int start_in_use = bufs_in_use, start_bufqsize = bufqsize;
assert(bufqsize >= 0);
if(bufqsize == 0) return;
/* for READING, check all buffers on the list are obtained and held
* (count > 0)
*/
if (rw_flag == READING) {
for(i = 0; i < bufqsize; i++) {
assert(bufq[i] != NULL);
assert(bufq[i]->lmfs_count > 0);
}
/* therefore they are all 'in use' and must be at least this many */
assert(start_in_use >= start_bufqsize);
}
Move primary cache code to libminixfs. Add primary cache management feature to libminixfs as mfs and ext2 currently do separately, remove cache code from mfs and ext2, and make them use the libminixfs interface. This makes all fields of the buf struct private to libminixfs and FS clients aren't supposed to access them at all. Only the opaque 'void *data' field (the FS block contents, used to be called bp) is to be accessed by the FS client. The main purpose is to implement the interface to the 2ndary vm cache just once, get rid of some code duplication, and add a little abstraction to reduce the code inertia of the whole caching business. Some minor sanity checking and prohibition done by mfs in this code as removed from the generic primary cache code as a result: - checking all inodes are not in use when allocating/resizing the cache - checking readonly filesystems aren't written to - checking the superblock isn't written to on mounted filesystems The minixfslib code relies on fs_blockstats() in the client filesystem to return some FS usage information.
2012-10-16 17:40:39 +02:00
vm: new secondary cache code Primary purpose of change: to support the mmap implementation, VM must know both (a) about some block metadata for FS cache blocks, i.e. inode numbers and inode offsets where applicable; and (b) know about *all* cache blocks, i.e. also of the FS primary caches and not just the blocks that spill into the secondary one. This changes the interface and VM data structures. This change is only for the interface (libminixfs) and VM data structures; the filesystem code is unmodified, so although the secondary cache will be used as normal, blocks will not be annotated with inode information until the FS is modified to provide this information. Until it is modified, mmap of files will fail gracefully on such filesystems. This is indicated to VFS/VM by returning ENOSYS for REQ_PEEK. Change-Id: I1d2df6c485e6c5e89eb28d9055076cc02629594e
2013-03-20 20:18:52 +01:00
assert(dev != NO_DEV);
assert(fs_block_size > 0);
libminixfs: allow non-pagesize-multiple FSes The memory-mapped files implementation (mmap() etc.) is implemented with the help of the filesystems using the in-VM FS cache. Filesystems tell it about all cached blocks and their metadata. Metadata is: device offset and, if any (and known), inode number and in-inode offset. VM can then map in requested memory-mapped file blocks, and request them if necessary. A limitation of this system is that filesystem block sizes that are not a multiple of the VM system (and VM hardware) page size are not possible; we can't map blocks in partially. (We can copy, but then the benefits of mapping and sharing the physical pages is gone.) So until before this commit various pieces of caching code assumed page size multiple blocksizes. This isn't strictly necessary as long as mmap() needn't be supported on that FS. This change allows the in-FS cache code (libminixfs) to allocate any-sized blocks, and will not interact with the VM cache for non-pagesize-multiple blocks. In that case it will also signal requestors, by failing 'peek' requests, that mmap() should not be supported on this FS. VM and VFS will then gracefully fail all file-mapping mmap() calls, and exec() will fall back to copying executable blocks instead of mmap()ping executables. As a result, 3 diagnostics that signal file-mapped mmap()s failing (hitherto an unusual occurence) are disabled, as ld.so does file-mapped mmap()s to map in objects it needs. On FSes not supporting it this situation is legitimate and shouldn't cause so much noise. ld.so will revert to its own minix-specific allocate+copy style of starting executables if mmap()s fail. Change-Id: Iecb1c8090f5e0be28da8f5181bb35084eb18f67b
2013-11-19 16:59:52 +01:00
iov_per_block = roundup(fs_block_size, PAGE_SIZE) / PAGE_SIZE;
assert(iov_per_block < NR_IOREQS);
vm: new secondary cache code Primary purpose of change: to support the mmap implementation, VM must know both (a) about some block metadata for FS cache blocks, i.e. inode numbers and inode offsets where applicable; and (b) know about *all* cache blocks, i.e. also of the FS primary caches and not just the blocks that spill into the secondary one. This changes the interface and VM data structures. This change is only for the interface (libminixfs) and VM data structures; the filesystem code is unmodified, so although the secondary cache will be used as normal, blocks will not be annotated with inode information until the FS is modified to provide this information. Until it is modified, mmap of files will fail gracefully on such filesystems. This is indicated to VFS/VM by returning ENOSYS for REQ_PEEK. Change-Id: I1d2df6c485e6c5e89eb28d9055076cc02629594e
2013-03-20 20:18:52 +01:00
Move primary cache code to libminixfs. Add primary cache management feature to libminixfs as mfs and ext2 currently do separately, remove cache code from mfs and ext2, and make them use the libminixfs interface. This makes all fields of the buf struct private to libminixfs and FS clients aren't supposed to access them at all. Only the opaque 'void *data' field (the FS block contents, used to be called bp) is to be accessed by the FS client. The main purpose is to implement the interface to the 2ndary vm cache just once, get rid of some code duplication, and add a little abstraction to reduce the code inertia of the whole caching business. Some minor sanity checking and prohibition done by mfs in this code as removed from the generic primary cache code as a result: - checking all inodes are not in use when allocating/resizing the cache - checking readonly filesystems aren't written to - checking the superblock isn't written to on mounted filesystems The minixfslib code relies on fs_blockstats() in the client filesystem to return some FS usage information.
2012-10-16 17:40:39 +02:00
/* (Shell) sort buffers on lmfs_blocknr. */
gap = 1;
do
gap = 3 * gap + 1;
while (gap <= bufqsize);
while (gap != 1) {
vm: new secondary cache code Primary purpose of change: to support the mmap implementation, VM must know both (a) about some block metadata for FS cache blocks, i.e. inode numbers and inode offsets where applicable; and (b) know about *all* cache blocks, i.e. also of the FS primary caches and not just the blocks that spill into the secondary one. This changes the interface and VM data structures. This change is only for the interface (libminixfs) and VM data structures; the filesystem code is unmodified, so although the secondary cache will be used as normal, blocks will not be annotated with inode information until the FS is modified to provide this information. Until it is modified, mmap of files will fail gracefully on such filesystems. This is indicated to VFS/VM by returning ENOSYS for REQ_PEEK. Change-Id: I1d2df6c485e6c5e89eb28d9055076cc02629594e
2013-03-20 20:18:52 +01:00
int j;
Move primary cache code to libminixfs. Add primary cache management feature to libminixfs as mfs and ext2 currently do separately, remove cache code from mfs and ext2, and make them use the libminixfs interface. This makes all fields of the buf struct private to libminixfs and FS clients aren't supposed to access them at all. Only the opaque 'void *data' field (the FS block contents, used to be called bp) is to be accessed by the FS client. The main purpose is to implement the interface to the 2ndary vm cache just once, get rid of some code duplication, and add a little abstraction to reduce the code inertia of the whole caching business. Some minor sanity checking and prohibition done by mfs in this code as removed from the generic primary cache code as a result: - checking all inodes are not in use when allocating/resizing the cache - checking readonly filesystems aren't written to - checking the superblock isn't written to on mounted filesystems The minixfslib code relies on fs_blockstats() in the client filesystem to return some FS usage information.
2012-10-16 17:40:39 +02:00
gap /= 3;
for (j = gap; j < bufqsize; j++) {
for (i = j - gap;
i >= 0 && bufq[i]->lmfs_blocknr > bufq[i + gap]->lmfs_blocknr;
i -= gap) {
bp = bufq[i];
bufq[i] = bufq[i + gap];
bufq[i + gap] = bp;
}
}
}
/* Set up I/O vector and do I/O. The result of bdev I/O is OK if everything
* went fine, otherwise the error code for the first failed transfer.
*/
while (bufqsize > 0) {
vm: new secondary cache code Primary purpose of change: to support the mmap implementation, VM must know both (a) about some block metadata for FS cache blocks, i.e. inode numbers and inode offsets where applicable; and (b) know about *all* cache blocks, i.e. also of the FS primary caches and not just the blocks that spill into the secondary one. This changes the interface and VM data structures. This change is only for the interface (libminixfs) and VM data structures; the filesystem code is unmodified, so although the secondary cache will be used as normal, blocks will not be annotated with inode information until the FS is modified to provide this information. Until it is modified, mmap of files will fail gracefully on such filesystems. This is indicated to VFS/VM by returning ENOSYS for REQ_PEEK. Change-Id: I1d2df6c485e6c5e89eb28d9055076cc02629594e
2013-03-20 20:18:52 +01:00
int nblocks = 0, niovecs = 0;
int r;
for (iop = iovec; nblocks < bufqsize; nblocks++) {
int p;
libminixfs: allow non-pagesize-multiple FSes The memory-mapped files implementation (mmap() etc.) is implemented with the help of the filesystems using the in-VM FS cache. Filesystems tell it about all cached blocks and their metadata. Metadata is: device offset and, if any (and known), inode number and in-inode offset. VM can then map in requested memory-mapped file blocks, and request them if necessary. A limitation of this system is that filesystem block sizes that are not a multiple of the VM system (and VM hardware) page size are not possible; we can't map blocks in partially. (We can copy, but then the benefits of mapping and sharing the physical pages is gone.) So until before this commit various pieces of caching code assumed page size multiple blocksizes. This isn't strictly necessary as long as mmap() needn't be supported on that FS. This change allows the in-FS cache code (libminixfs) to allocate any-sized blocks, and will not interact with the VM cache for non-pagesize-multiple blocks. In that case it will also signal requestors, by failing 'peek' requests, that mmap() should not be supported on this FS. VM and VFS will then gracefully fail all file-mapping mmap() calls, and exec() will fall back to copying executable blocks instead of mmap()ping executables. As a result, 3 diagnostics that signal file-mapped mmap()s failing (hitherto an unusual occurence) are disabled, as ld.so does file-mapped mmap()s to map in objects it needs. On FSes not supporting it this situation is legitimate and shouldn't cause so much noise. ld.so will revert to its own minix-specific allocate+copy style of starting executables if mmap()s fail. Change-Id: Iecb1c8090f5e0be28da8f5181bb35084eb18f67b
2013-11-19 16:59:52 +01:00
vir_bytes vdata, blockrem;
vm: new secondary cache code Primary purpose of change: to support the mmap implementation, VM must know both (a) about some block metadata for FS cache blocks, i.e. inode numbers and inode offsets where applicable; and (b) know about *all* cache blocks, i.e. also of the FS primary caches and not just the blocks that spill into the secondary one. This changes the interface and VM data structures. This change is only for the interface (libminixfs) and VM data structures; the filesystem code is unmodified, so although the secondary cache will be used as normal, blocks will not be annotated with inode information until the FS is modified to provide this information. Until it is modified, mmap of files will fail gracefully on such filesystems. This is indicated to VFS/VM by returning ENOSYS for REQ_PEEK. Change-Id: I1d2df6c485e6c5e89eb28d9055076cc02629594e
2013-03-20 20:18:52 +01:00
bp = bufq[nblocks];
if (bp->lmfs_blocknr != (block_t) bufq[0]->lmfs_blocknr + nblocks)
break;
if(niovecs >= NR_IOREQS-iov_per_block) break;
vdata = (vir_bytes) bp->data;
libminixfs: allow non-pagesize-multiple FSes The memory-mapped files implementation (mmap() etc.) is implemented with the help of the filesystems using the in-VM FS cache. Filesystems tell it about all cached blocks and their metadata. Metadata is: device offset and, if any (and known), inode number and in-inode offset. VM can then map in requested memory-mapped file blocks, and request them if necessary. A limitation of this system is that filesystem block sizes that are not a multiple of the VM system (and VM hardware) page size are not possible; we can't map blocks in partially. (We can copy, but then the benefits of mapping and sharing the physical pages is gone.) So until before this commit various pieces of caching code assumed page size multiple blocksizes. This isn't strictly necessary as long as mmap() needn't be supported on that FS. This change allows the in-FS cache code (libminixfs) to allocate any-sized blocks, and will not interact with the VM cache for non-pagesize-multiple blocks. In that case it will also signal requestors, by failing 'peek' requests, that mmap() should not be supported on this FS. VM and VFS will then gracefully fail all file-mapping mmap() calls, and exec() will fall back to copying executable blocks instead of mmap()ping executables. As a result, 3 diagnostics that signal file-mapped mmap()s failing (hitherto an unusual occurence) are disabled, as ld.so does file-mapped mmap()s to map in objects it needs. On FSes not supporting it this situation is legitimate and shouldn't cause so much noise. ld.so will revert to its own minix-specific allocate+copy style of starting executables if mmap()s fail. Change-Id: Iecb1c8090f5e0be28da8f5181bb35084eb18f67b
2013-11-19 16:59:52 +01:00
blockrem = fs_block_size;
vm: new secondary cache code Primary purpose of change: to support the mmap implementation, VM must know both (a) about some block metadata for FS cache blocks, i.e. inode numbers and inode offsets where applicable; and (b) know about *all* cache blocks, i.e. also of the FS primary caches and not just the blocks that spill into the secondary one. This changes the interface and VM data structures. This change is only for the interface (libminixfs) and VM data structures; the filesystem code is unmodified, so although the secondary cache will be used as normal, blocks will not be annotated with inode information until the FS is modified to provide this information. Until it is modified, mmap of files will fail gracefully on such filesystems. This is indicated to VFS/VM by returning ENOSYS for REQ_PEEK. Change-Id: I1d2df6c485e6c5e89eb28d9055076cc02629594e
2013-03-20 20:18:52 +01:00
for(p = 0; p < iov_per_block; p++) {
libminixfs: allow non-pagesize-multiple FSes The memory-mapped files implementation (mmap() etc.) is implemented with the help of the filesystems using the in-VM FS cache. Filesystems tell it about all cached blocks and their metadata. Metadata is: device offset and, if any (and known), inode number and in-inode offset. VM can then map in requested memory-mapped file blocks, and request them if necessary. A limitation of this system is that filesystem block sizes that are not a multiple of the VM system (and VM hardware) page size are not possible; we can't map blocks in partially. (We can copy, but then the benefits of mapping and sharing the physical pages is gone.) So until before this commit various pieces of caching code assumed page size multiple blocksizes. This isn't strictly necessary as long as mmap() needn't be supported on that FS. This change allows the in-FS cache code (libminixfs) to allocate any-sized blocks, and will not interact with the VM cache for non-pagesize-multiple blocks. In that case it will also signal requestors, by failing 'peek' requests, that mmap() should not be supported on this FS. VM and VFS will then gracefully fail all file-mapping mmap() calls, and exec() will fall back to copying executable blocks instead of mmap()ping executables. As a result, 3 diagnostics that signal file-mapped mmap()s failing (hitherto an unusual occurence) are disabled, as ld.so does file-mapped mmap()s to map in objects it needs. On FSes not supporting it this situation is legitimate and shouldn't cause so much noise. ld.so will revert to its own minix-specific allocate+copy style of starting executables if mmap()s fail. Change-Id: Iecb1c8090f5e0be28da8f5181bb35084eb18f67b
2013-11-19 16:59:52 +01:00
vir_bytes chunk = blockrem < PAGE_SIZE ? blockrem : PAGE_SIZE;
vm: new secondary cache code Primary purpose of change: to support the mmap implementation, VM must know both (a) about some block metadata for FS cache blocks, i.e. inode numbers and inode offsets where applicable; and (b) know about *all* cache blocks, i.e. also of the FS primary caches and not just the blocks that spill into the secondary one. This changes the interface and VM data structures. This change is only for the interface (libminixfs) and VM data structures; the filesystem code is unmodified, so although the secondary cache will be used as normal, blocks will not be annotated with inode information until the FS is modified to provide this information. Until it is modified, mmap of files will fail gracefully on such filesystems. This is indicated to VFS/VM by returning ENOSYS for REQ_PEEK. Change-Id: I1d2df6c485e6c5e89eb28d9055076cc02629594e
2013-03-20 20:18:52 +01:00
iop->iov_addr = vdata;
libminixfs: allow non-pagesize-multiple FSes The memory-mapped files implementation (mmap() etc.) is implemented with the help of the filesystems using the in-VM FS cache. Filesystems tell it about all cached blocks and their metadata. Metadata is: device offset and, if any (and known), inode number and in-inode offset. VM can then map in requested memory-mapped file blocks, and request them if necessary. A limitation of this system is that filesystem block sizes that are not a multiple of the VM system (and VM hardware) page size are not possible; we can't map blocks in partially. (We can copy, but then the benefits of mapping and sharing the physical pages is gone.) So until before this commit various pieces of caching code assumed page size multiple blocksizes. This isn't strictly necessary as long as mmap() needn't be supported on that FS. This change allows the in-FS cache code (libminixfs) to allocate any-sized blocks, and will not interact with the VM cache for non-pagesize-multiple blocks. In that case it will also signal requestors, by failing 'peek' requests, that mmap() should not be supported on this FS. VM and VFS will then gracefully fail all file-mapping mmap() calls, and exec() will fall back to copying executable blocks instead of mmap()ping executables. As a result, 3 diagnostics that signal file-mapped mmap()s failing (hitherto an unusual occurence) are disabled, as ld.so does file-mapped mmap()s to map in objects it needs. On FSes not supporting it this situation is legitimate and shouldn't cause so much noise. ld.so will revert to its own minix-specific allocate+copy style of starting executables if mmap()s fail. Change-Id: Iecb1c8090f5e0be28da8f5181bb35084eb18f67b
2013-11-19 16:59:52 +01:00
iop->iov_size = chunk;
vm: new secondary cache code Primary purpose of change: to support the mmap implementation, VM must know both (a) about some block metadata for FS cache blocks, i.e. inode numbers and inode offsets where applicable; and (b) know about *all* cache blocks, i.e. also of the FS primary caches and not just the blocks that spill into the secondary one. This changes the interface and VM data structures. This change is only for the interface (libminixfs) and VM data structures; the filesystem code is unmodified, so although the secondary cache will be used as normal, blocks will not be annotated with inode information until the FS is modified to provide this information. Until it is modified, mmap of files will fail gracefully on such filesystems. This is indicated to VFS/VM by returning ENOSYS for REQ_PEEK. Change-Id: I1d2df6c485e6c5e89eb28d9055076cc02629594e
2013-03-20 20:18:52 +01:00
vdata += PAGE_SIZE;
libminixfs: allow non-pagesize-multiple FSes The memory-mapped files implementation (mmap() etc.) is implemented with the help of the filesystems using the in-VM FS cache. Filesystems tell it about all cached blocks and their metadata. Metadata is: device offset and, if any (and known), inode number and in-inode offset. VM can then map in requested memory-mapped file blocks, and request them if necessary. A limitation of this system is that filesystem block sizes that are not a multiple of the VM system (and VM hardware) page size are not possible; we can't map blocks in partially. (We can copy, but then the benefits of mapping and sharing the physical pages is gone.) So until before this commit various pieces of caching code assumed page size multiple blocksizes. This isn't strictly necessary as long as mmap() needn't be supported on that FS. This change allows the in-FS cache code (libminixfs) to allocate any-sized blocks, and will not interact with the VM cache for non-pagesize-multiple blocks. In that case it will also signal requestors, by failing 'peek' requests, that mmap() should not be supported on this FS. VM and VFS will then gracefully fail all file-mapping mmap() calls, and exec() will fall back to copying executable blocks instead of mmap()ping executables. As a result, 3 diagnostics that signal file-mapped mmap()s failing (hitherto an unusual occurence) are disabled, as ld.so does file-mapped mmap()s to map in objects it needs. On FSes not supporting it this situation is legitimate and shouldn't cause so much noise. ld.so will revert to its own minix-specific allocate+copy style of starting executables if mmap()s fail. Change-Id: Iecb1c8090f5e0be28da8f5181bb35084eb18f67b
2013-11-19 16:59:52 +01:00
blockrem -= chunk;
vm: new secondary cache code Primary purpose of change: to support the mmap implementation, VM must know both (a) about some block metadata for FS cache blocks, i.e. inode numbers and inode offsets where applicable; and (b) know about *all* cache blocks, i.e. also of the FS primary caches and not just the blocks that spill into the secondary one. This changes the interface and VM data structures. This change is only for the interface (libminixfs) and VM data structures; the filesystem code is unmodified, so although the secondary cache will be used as normal, blocks will not be annotated with inode information until the FS is modified to provide this information. Until it is modified, mmap of files will fail gracefully on such filesystems. This is indicated to VFS/VM by returning ENOSYS for REQ_PEEK. Change-Id: I1d2df6c485e6c5e89eb28d9055076cc02629594e
2013-03-20 20:18:52 +01:00
iop++;
niovecs++;
}
libminixfs: allow non-pagesize-multiple FSes The memory-mapped files implementation (mmap() etc.) is implemented with the help of the filesystems using the in-VM FS cache. Filesystems tell it about all cached blocks and their metadata. Metadata is: device offset and, if any (and known), inode number and in-inode offset. VM can then map in requested memory-mapped file blocks, and request them if necessary. A limitation of this system is that filesystem block sizes that are not a multiple of the VM system (and VM hardware) page size are not possible; we can't map blocks in partially. (We can copy, but then the benefits of mapping and sharing the physical pages is gone.) So until before this commit various pieces of caching code assumed page size multiple blocksizes. This isn't strictly necessary as long as mmap() needn't be supported on that FS. This change allows the in-FS cache code (libminixfs) to allocate any-sized blocks, and will not interact with the VM cache for non-pagesize-multiple blocks. In that case it will also signal requestors, by failing 'peek' requests, that mmap() should not be supported on this FS. VM and VFS will then gracefully fail all file-mapping mmap() calls, and exec() will fall back to copying executable blocks instead of mmap()ping executables. As a result, 3 diagnostics that signal file-mapped mmap()s failing (hitherto an unusual occurence) are disabled, as ld.so does file-mapped mmap()s to map in objects it needs. On FSes not supporting it this situation is legitimate and shouldn't cause so much noise. ld.so will revert to its own minix-specific allocate+copy style of starting executables if mmap()s fail. Change-Id: Iecb1c8090f5e0be28da8f5181bb35084eb18f67b
2013-11-19 16:59:52 +01:00
assert(p == iov_per_block);
assert(blockrem == 0);
Move primary cache code to libminixfs. Add primary cache management feature to libminixfs as mfs and ext2 currently do separately, remove cache code from mfs and ext2, and make them use the libminixfs interface. This makes all fields of the buf struct private to libminixfs and FS clients aren't supposed to access them at all. Only the opaque 'void *data' field (the FS block contents, used to be called bp) is to be accessed by the FS client. The main purpose is to implement the interface to the 2ndary vm cache just once, get rid of some code duplication, and add a little abstraction to reduce the code inertia of the whole caching business. Some minor sanity checking and prohibition done by mfs in this code as removed from the generic primary cache code as a result: - checking all inodes are not in use when allocating/resizing the cache - checking readonly filesystems aren't written to - checking the superblock isn't written to on mounted filesystems The minixfslib code relies on fs_blockstats() in the client filesystem to return some FS usage information.
2012-10-16 17:40:39 +02:00
}
vm: new secondary cache code Primary purpose of change: to support the mmap implementation, VM must know both (a) about some block metadata for FS cache blocks, i.e. inode numbers and inode offsets where applicable; and (b) know about *all* cache blocks, i.e. also of the FS primary caches and not just the blocks that spill into the secondary one. This changes the interface and VM data structures. This change is only for the interface (libminixfs) and VM data structures; the filesystem code is unmodified, so although the secondary cache will be used as normal, blocks will not be annotated with inode information until the FS is modified to provide this information. Until it is modified, mmap of files will fail gracefully on such filesystems. This is indicated to VFS/VM by returning ENOSYS for REQ_PEEK. Change-Id: I1d2df6c485e6c5e89eb28d9055076cc02629594e
2013-03-20 20:18:52 +01:00
assert(nblocks > 0);
assert(niovecs > 0);
Move primary cache code to libminixfs. Add primary cache management feature to libminixfs as mfs and ext2 currently do separately, remove cache code from mfs and ext2, and make them use the libminixfs interface. This makes all fields of the buf struct private to libminixfs and FS clients aren't supposed to access them at all. Only the opaque 'void *data' field (the FS block contents, used to be called bp) is to be accessed by the FS client. The main purpose is to implement the interface to the 2ndary vm cache just once, get rid of some code duplication, and add a little abstraction to reduce the code inertia of the whole caching business. Some minor sanity checking and prohibition done by mfs in this code as removed from the generic primary cache code as a result: - checking all inodes are not in use when allocating/resizing the cache - checking readonly filesystems aren't written to - checking the superblock isn't written to on mounted filesystems The minixfslib code relies on fs_blockstats() in the client filesystem to return some FS usage information.
2012-10-16 17:40:39 +02:00
pos = mul64u(bufq[0]->lmfs_blocknr, fs_block_size);
if (rw_flag == READING)
vm: new secondary cache code Primary purpose of change: to support the mmap implementation, VM must know both (a) about some block metadata for FS cache blocks, i.e. inode numbers and inode offsets where applicable; and (b) know about *all* cache blocks, i.e. also of the FS primary caches and not just the blocks that spill into the secondary one. This changes the interface and VM data structures. This change is only for the interface (libminixfs) and VM data structures; the filesystem code is unmodified, so although the secondary cache will be used as normal, blocks will not be annotated with inode information until the FS is modified to provide this information. Until it is modified, mmap of files will fail gracefully on such filesystems. This is indicated to VFS/VM by returning ENOSYS for REQ_PEEK. Change-Id: I1d2df6c485e6c5e89eb28d9055076cc02629594e
2013-03-20 20:18:52 +01:00
r = bdev_gather(dev, pos, iovec, niovecs, BDEV_NOFLAGS);
Move primary cache code to libminixfs. Add primary cache management feature to libminixfs as mfs and ext2 currently do separately, remove cache code from mfs and ext2, and make them use the libminixfs interface. This makes all fields of the buf struct private to libminixfs and FS clients aren't supposed to access them at all. Only the opaque 'void *data' field (the FS block contents, used to be called bp) is to be accessed by the FS client. The main purpose is to implement the interface to the 2ndary vm cache just once, get rid of some code duplication, and add a little abstraction to reduce the code inertia of the whole caching business. Some minor sanity checking and prohibition done by mfs in this code as removed from the generic primary cache code as a result: - checking all inodes are not in use when allocating/resizing the cache - checking readonly filesystems aren't written to - checking the superblock isn't written to on mounted filesystems The minixfslib code relies on fs_blockstats() in the client filesystem to return some FS usage information.
2012-10-16 17:40:39 +02:00
else
vm: new secondary cache code Primary purpose of change: to support the mmap implementation, VM must know both (a) about some block metadata for FS cache blocks, i.e. inode numbers and inode offsets where applicable; and (b) know about *all* cache blocks, i.e. also of the FS primary caches and not just the blocks that spill into the secondary one. This changes the interface and VM data structures. This change is only for the interface (libminixfs) and VM data structures; the filesystem code is unmodified, so although the secondary cache will be used as normal, blocks will not be annotated with inode information until the FS is modified to provide this information. Until it is modified, mmap of files will fail gracefully on such filesystems. This is indicated to VFS/VM by returning ENOSYS for REQ_PEEK. Change-Id: I1d2df6c485e6c5e89eb28d9055076cc02629594e
2013-03-20 20:18:52 +01:00
r = bdev_scatter(dev, pos, iovec, niovecs, BDEV_NOFLAGS);
Move primary cache code to libminixfs. Add primary cache management feature to libminixfs as mfs and ext2 currently do separately, remove cache code from mfs and ext2, and make them use the libminixfs interface. This makes all fields of the buf struct private to libminixfs and FS clients aren't supposed to access them at all. Only the opaque 'void *data' field (the FS block contents, used to be called bp) is to be accessed by the FS client. The main purpose is to implement the interface to the 2ndary vm cache just once, get rid of some code duplication, and add a little abstraction to reduce the code inertia of the whole caching business. Some minor sanity checking and prohibition done by mfs in this code as removed from the generic primary cache code as a result: - checking all inodes are not in use when allocating/resizing the cache - checking readonly filesystems aren't written to - checking the superblock isn't written to on mounted filesystems The minixfslib code relies on fs_blockstats() in the client filesystem to return some FS usage information.
2012-10-16 17:40:39 +02:00
/* Harvest the results. The driver may have returned an error, or it
* may have done less than what we asked for.
*/
if (r < 0) {
printf("fs cache: I/O error %d on device %d/%d, block %u\n",
r, major(dev), minor(dev), bufq[0]->lmfs_blocknr);
}
vm: new secondary cache code Primary purpose of change: to support the mmap implementation, VM must know both (a) about some block metadata for FS cache blocks, i.e. inode numbers and inode offsets where applicable; and (b) know about *all* cache blocks, i.e. also of the FS primary caches and not just the blocks that spill into the secondary one. This changes the interface and VM data structures. This change is only for the interface (libminixfs) and VM data structures; the filesystem code is unmodified, so although the secondary cache will be used as normal, blocks will not be annotated with inode information until the FS is modified to provide this information. Until it is modified, mmap of files will fail gracefully on such filesystems. This is indicated to VFS/VM by returning ENOSYS for REQ_PEEK. Change-Id: I1d2df6c485e6c5e89eb28d9055076cc02629594e
2013-03-20 20:18:52 +01:00
for (i = 0; i < nblocks; i++) {
Move primary cache code to libminixfs. Add primary cache management feature to libminixfs as mfs and ext2 currently do separately, remove cache code from mfs and ext2, and make them use the libminixfs interface. This makes all fields of the buf struct private to libminixfs and FS clients aren't supposed to access them at all. Only the opaque 'void *data' field (the FS block contents, used to be called bp) is to be accessed by the FS client. The main purpose is to implement the interface to the 2ndary vm cache just once, get rid of some code duplication, and add a little abstraction to reduce the code inertia of the whole caching business. Some minor sanity checking and prohibition done by mfs in this code as removed from the generic primary cache code as a result: - checking all inodes are not in use when allocating/resizing the cache - checking readonly filesystems aren't written to - checking the superblock isn't written to on mounted filesystems The minixfslib code relies on fs_blockstats() in the client filesystem to return some FS usage information.
2012-10-16 17:40:39 +02:00
bp = bufq[i];
if (r < (ssize_t) fs_block_size) {
/* Transfer failed. */
if (i == 0) {
bp->lmfs_dev = NO_DEV; /* Invalidate block */
}
break;
}
if (rw_flag == READING) {
bp->lmfs_dev = dev; /* validate block */
lmfs_put_block(bp, PARTIAL_DATA_BLOCK);
} else {
MARKCLEAN(bp);
}
r -= fs_block_size;
}
libminixfs: fix buf leak . mfs: add asserts to catch similar problems in the future Change-Id: I939230b8d63f100ac071696425b35496b2169bc4
2013-11-05 15:43:27 +01:00
bufq += i;
bufqsize -= i;
Move primary cache code to libminixfs. Add primary cache management feature to libminixfs as mfs and ext2 currently do separately, remove cache code from mfs and ext2, and make them use the libminixfs interface. This makes all fields of the buf struct private to libminixfs and FS clients aren't supposed to access them at all. Only the opaque 'void *data' field (the FS block contents, used to be called bp) is to be accessed by the FS client. The main purpose is to implement the interface to the 2ndary vm cache just once, get rid of some code duplication, and add a little abstraction to reduce the code inertia of the whole caching business. Some minor sanity checking and prohibition done by mfs in this code as removed from the generic primary cache code as a result: - checking all inodes are not in use when allocating/resizing the cache - checking readonly filesystems aren't written to - checking the superblock isn't written to on mounted filesystems The minixfslib code relies on fs_blockstats() in the client filesystem to return some FS usage information.
2012-10-16 17:40:39 +02:00
if (rw_flag == READING) {
/* Don't bother reading more than the device is willing to
* give at this time. Don't forget to release those extras.
*/
while (bufqsize > 0) {
lmfs_put_block(*bufq++, PARTIAL_DATA_BLOCK);
bufqsize--;
}
}
if (rw_flag == WRITING && i == 0) {
/* We're not making progress, this means we might keep
* looping. Buffers remain dirty if un-written. Buffers are
* lost if invalidate()d or LRU-removed while dirty. This
* is better than keeping unwritable blocks around forever..
*/
break;
}
}
libminixfs: fix buf leak . mfs: add asserts to catch similar problems in the future Change-Id: I939230b8d63f100ac071696425b35496b2169bc4
2013-11-05 15:43:27 +01:00
if(rw_flag == READING) {
assert(start_in_use >= start_bufqsize);
/* READING callers assume all bufs are released. */
assert(start_in_use - start_bufqsize == bufs_in_use);
}
Move primary cache code to libminixfs. Add primary cache management feature to libminixfs as mfs and ext2 currently do separately, remove cache code from mfs and ext2, and make them use the libminixfs interface. This makes all fields of the buf struct private to libminixfs and FS clients aren't supposed to access them at all. Only the opaque 'void *data' field (the FS block contents, used to be called bp) is to be accessed by the FS client. The main purpose is to implement the interface to the 2ndary vm cache just once, get rid of some code duplication, and add a little abstraction to reduce the code inertia of the whole caching business. Some minor sanity checking and prohibition done by mfs in this code as removed from the generic primary cache code as a result: - checking all inodes are not in use when allocating/resizing the cache - checking readonly filesystems aren't written to - checking the superblock isn't written to on mounted filesystems The minixfslib code relies on fs_blockstats() in the client filesystem to return some FS usage information.
2012-10-16 17:40:39 +02:00
}
/*===========================================================================*
* rm_lru *
*===========================================================================*/
static void rm_lru(bp)
struct buf *bp;
{
/* Remove a block from its LRU chain. */
struct buf *next_ptr, *prev_ptr;
next_ptr = bp->lmfs_next; /* successor on LRU chain */
prev_ptr = bp->lmfs_prev; /* predecessor on LRU chain */
if (prev_ptr != NULL)
prev_ptr->lmfs_next = next_ptr;
else
front = next_ptr; /* this block was at front of chain */
if (next_ptr != NULL)
next_ptr->lmfs_prev = prev_ptr;
else
rear = prev_ptr; /* this block was at rear of chain */
}
/*===========================================================================*
* cache_resize *
*===========================================================================*/
static void cache_resize(unsigned int blocksize, unsigned int bufs)
{
struct buf *bp;
assert(blocksize > 0);
assert(bufs >= MINBUFS);
for (bp = &buf[0]; bp < &buf[nr_bufs]; bp++)
if(bp->lmfs_count != 0) panic("change blocksize with buffer in use");
lmfs_buf_pool(bufs);
fs_block_size = blocksize;
}
libminixfs, mfs, ext2: may re-evaluate cache size libminixfs may now be informed of changes to the block usage on the filesystem. if the net change becomes big enough, libminixfs may resize the cache based on the new usage. . update the 2 FSes to provide this information to libminixfs Change-Id: I158815a11da801fd5572a8de89c9e6c039b82650
2013-04-26 11:54:33 +02:00
static void cache_heuristic_check(int major)
Move primary cache code to libminixfs. Add primary cache management feature to libminixfs as mfs and ext2 currently do separately, remove cache code from mfs and ext2, and make them use the libminixfs interface. This makes all fields of the buf struct private to libminixfs and FS clients aren't supposed to access them at all. Only the opaque 'void *data' field (the FS block contents, used to be called bp) is to be accessed by the FS client. The main purpose is to implement the interface to the 2ndary vm cache just once, get rid of some code duplication, and add a little abstraction to reduce the code inertia of the whole caching business. Some minor sanity checking and prohibition done by mfs in this code as removed from the generic primary cache code as a result: - checking all inodes are not in use when allocating/resizing the cache - checking readonly filesystems aren't written to - checking the superblock isn't written to on mounted filesystems The minixfslib code relies on fs_blockstats() in the client filesystem to return some FS usage information.
2012-10-16 17:40:39 +02:00
{
libminixfs, mfs, ext2: may re-evaluate cache size libminixfs may now be informed of changes to the block usage on the filesystem. if the net change becomes big enough, libminixfs may resize the cache based on the new usage. . update the 2 FSes to provide this information to libminixfs Change-Id: I158815a11da801fd5572a8de89c9e6c039b82650
2013-04-26 11:54:33 +02:00
int bufs, d;
32 to 64 bits fsblkcnt_t and fsfilcnt_t. Change-Id: I432229143c85cd178262b802a76ac606801ac59a
2013-04-18 14:42:15 +02:00
u64_t btotal, bfree, bused;
Move primary cache code to libminixfs. Add primary cache management feature to libminixfs as mfs and ext2 currently do separately, remove cache code from mfs and ext2, and make them use the libminixfs interface. This makes all fields of the buf struct private to libminixfs and FS clients aren't supposed to access them at all. Only the opaque 'void *data' field (the FS block contents, used to be called bp) is to be accessed by the FS client. The main purpose is to implement the interface to the 2ndary vm cache just once, get rid of some code duplication, and add a little abstraction to reduce the code inertia of the whole caching business. Some minor sanity checking and prohibition done by mfs in this code as removed from the generic primary cache code as a result: - checking all inodes are not in use when allocating/resizing the cache - checking readonly filesystems aren't written to - checking the superblock isn't written to on mounted filesystems The minixfslib code relies on fs_blockstats() in the client filesystem to return some FS usage information.
2012-10-16 17:40:39 +02:00
fs_blockstats(&btotal, &bfree, &bused);
bufs = fs_bufs_heuristic(10, btotal, bfree,
libminixfs, mfs, ext2: may re-evaluate cache size libminixfs may now be informed of changes to the block usage on the filesystem. if the net change becomes big enough, libminixfs may resize the cache based on the new usage. . update the 2 FSes to provide this information to libminixfs Change-Id: I158815a11da801fd5572a8de89c9e6c039b82650
2013-04-26 11:54:33 +02:00
fs_block_size, major);
/* set the cache to the new heuristic size if the new one
* is more than 10% off from the current one.
*/
d = bufs-nr_bufs;
if(d < 0) d = -d;
if(d*100/nr_bufs > 10) {
cache_resize(fs_block_size, bufs);
}
}
Move primary cache code to libminixfs. Add primary cache management feature to libminixfs as mfs and ext2 currently do separately, remove cache code from mfs and ext2, and make them use the libminixfs interface. This makes all fields of the buf struct private to libminixfs and FS clients aren't supposed to access them at all. Only the opaque 'void *data' field (the FS block contents, used to be called bp) is to be accessed by the FS client. The main purpose is to implement the interface to the 2ndary vm cache just once, get rid of some code duplication, and add a little abstraction to reduce the code inertia of the whole caching business. Some minor sanity checking and prohibition done by mfs in this code as removed from the generic primary cache code as a result: - checking all inodes are not in use when allocating/resizing the cache - checking readonly filesystems aren't written to - checking the superblock isn't written to on mounted filesystems The minixfslib code relies on fs_blockstats() in the client filesystem to return some FS usage information.
2012-10-16 17:40:39 +02:00
libminixfs, mfs, ext2: may re-evaluate cache size libminixfs may now be informed of changes to the block usage on the filesystem. if the net change becomes big enough, libminixfs may resize the cache based on the new usage. . update the 2 FSes to provide this information to libminixfs Change-Id: I158815a11da801fd5572a8de89c9e6c039b82650
2013-04-26 11:54:33 +02:00
/*===========================================================================*
* lmfs_set_blocksize *
*===========================================================================*/
void lmfs_set_blocksize(int new_block_size, int major)
{
cache_resize(new_block_size, MINBUFS);
cache_heuristic_check(major);
Move primary cache code to libminixfs. Add primary cache management feature to libminixfs as mfs and ext2 currently do separately, remove cache code from mfs and ext2, and make them use the libminixfs interface. This makes all fields of the buf struct private to libminixfs and FS clients aren't supposed to access them at all. Only the opaque 'void *data' field (the FS block contents, used to be called bp) is to be accessed by the FS client. The main purpose is to implement the interface to the 2ndary vm cache just once, get rid of some code duplication, and add a little abstraction to reduce the code inertia of the whole caching business. Some minor sanity checking and prohibition done by mfs in this code as removed from the generic primary cache code as a result: - checking all inodes are not in use when allocating/resizing the cache - checking readonly filesystems aren't written to - checking the superblock isn't written to on mounted filesystems The minixfslib code relies on fs_blockstats() in the client filesystem to return some FS usage information.
2012-10-16 17:40:39 +02:00
/* Decide whether to use seconday cache or not.
* Only do this if
* - it's available, and
* - use of it hasn't been disabled for this fs, and
* - our main FS device isn't a memory device
*/
vmcache = 0;
vm: new secondary cache code Primary purpose of change: to support the mmap implementation, VM must know both (a) about some block metadata for FS cache blocks, i.e. inode numbers and inode offsets where applicable; and (b) know about *all* cache blocks, i.e. also of the FS primary caches and not just the blocks that spill into the secondary one. This changes the interface and VM data structures. This change is only for the interface (libminixfs) and VM data structures; the filesystem code is unmodified, so although the secondary cache will be used as normal, blocks will not be annotated with inode information until the FS is modified to provide this information. Until it is modified, mmap of files will fail gracefully on such filesystems. This is indicated to VFS/VM by returning ENOSYS for REQ_PEEK. Change-Id: I1d2df6c485e6c5e89eb28d9055076cc02629594e
2013-03-20 20:18:52 +01:00
libminixfs: allow non-pagesize-multiple FSes The memory-mapped files implementation (mmap() etc.) is implemented with the help of the filesystems using the in-VM FS cache. Filesystems tell it about all cached blocks and their metadata. Metadata is: device offset and, if any (and known), inode number and in-inode offset. VM can then map in requested memory-mapped file blocks, and request them if necessary. A limitation of this system is that filesystem block sizes that are not a multiple of the VM system (and VM hardware) page size are not possible; we can't map blocks in partially. (We can copy, but then the benefits of mapping and sharing the physical pages is gone.) So until before this commit various pieces of caching code assumed page size multiple blocksizes. This isn't strictly necessary as long as mmap() needn't be supported on that FS. This change allows the in-FS cache code (libminixfs) to allocate any-sized blocks, and will not interact with the VM cache for non-pagesize-multiple blocks. In that case it will also signal requestors, by failing 'peek' requests, that mmap() should not be supported on this FS. VM and VFS will then gracefully fail all file-mapping mmap() calls, and exec() will fall back to copying executable blocks instead of mmap()ping executables. As a result, 3 diagnostics that signal file-mapped mmap()s failing (hitherto an unusual occurence) are disabled, as ld.so does file-mapped mmap()s to map in objects it needs. On FSes not supporting it this situation is legitimate and shouldn't cause so much noise. ld.so will revert to its own minix-specific allocate+copy style of starting executables if mmap()s fail. Change-Id: Iecb1c8090f5e0be28da8f5181bb35084eb18f67b
2013-11-19 16:59:52 +01:00
if(may_use_vmcache && !(new_block_size % PAGE_SIZE))
Move primary cache code to libminixfs. Add primary cache management feature to libminixfs as mfs and ext2 currently do separately, remove cache code from mfs and ext2, and make them use the libminixfs interface. This makes all fields of the buf struct private to libminixfs and FS clients aren't supposed to access them at all. Only the opaque 'void *data' field (the FS block contents, used to be called bp) is to be accessed by the FS client. The main purpose is to implement the interface to the 2ndary vm cache just once, get rid of some code duplication, and add a little abstraction to reduce the code inertia of the whole caching business. Some minor sanity checking and prohibition done by mfs in this code as removed from the generic primary cache code as a result: - checking all inodes are not in use when allocating/resizing the cache - checking readonly filesystems aren't written to - checking the superblock isn't written to on mounted filesystems The minixfslib code relies on fs_blockstats() in the client filesystem to return some FS usage information.
2012-10-16 17:40:39 +02:00
vmcache = 1;
}
/*===========================================================================*
* lmfs_buf_pool *
*===========================================================================*/
void lmfs_buf_pool(int new_nr_bufs)
{
/* Initialize the buffer pool. */
register struct buf *bp;
assert(new_nr_bufs >= MINBUFS);
if(nr_bufs > 0) {
assert(buf);
(void) fs_sync();
for (bp = &buf[0]; bp < &buf[nr_bufs]; bp++) {
if(bp->data) {
assert(bp->lmfs_bytes > 0);
vm: new secondary cache code Primary purpose of change: to support the mmap implementation, VM must know both (a) about some block metadata for FS cache blocks, i.e. inode numbers and inode offsets where applicable; and (b) know about *all* cache blocks, i.e. also of the FS primary caches and not just the blocks that spill into the secondary one. This changes the interface and VM data structures. This change is only for the interface (libminixfs) and VM data structures; the filesystem code is unmodified, so although the secondary cache will be used as normal, blocks will not be annotated with inode information until the FS is modified to provide this information. Until it is modified, mmap of files will fail gracefully on such filesystems. This is indicated to VFS/VM by returning ENOSYS for REQ_PEEK. Change-Id: I1d2df6c485e6c5e89eb28d9055076cc02629594e
2013-03-20 20:18:52 +01:00
minix_munmap_t(bp->data, bp->lmfs_bytes);
Move primary cache code to libminixfs. Add primary cache management feature to libminixfs as mfs and ext2 currently do separately, remove cache code from mfs and ext2, and make them use the libminixfs interface. This makes all fields of the buf struct private to libminixfs and FS clients aren't supposed to access them at all. Only the opaque 'void *data' field (the FS block contents, used to be called bp) is to be accessed by the FS client. The main purpose is to implement the interface to the 2ndary vm cache just once, get rid of some code duplication, and add a little abstraction to reduce the code inertia of the whole caching business. Some minor sanity checking and prohibition done by mfs in this code as removed from the generic primary cache code as a result: - checking all inodes are not in use when allocating/resizing the cache - checking readonly filesystems aren't written to - checking the superblock isn't written to on mounted filesystems The minixfslib code relies on fs_blockstats() in the client filesystem to return some FS usage information.
2012-10-16 17:40:39 +02:00
}
}
}
if(buf)
free(buf);
if(!(buf = calloc(sizeof(buf[0]), new_nr_bufs)))
panic("couldn't allocate buf list (%d)", new_nr_bufs);
if(buf_hash)
free(buf_hash);
if(!(buf_hash = calloc(sizeof(buf_hash[0]), new_nr_bufs)))
panic("couldn't allocate buf hash list (%d)", new_nr_bufs);
nr_bufs = new_nr_bufs;
bufs_in_use = 0;
front = &buf[0];
rear = &buf[nr_bufs - 1];
for (bp = &buf[0]; bp < &buf[nr_bufs]; bp++) {
bp->lmfs_blocknr = NO_BLOCK;
bp->lmfs_dev = NO_DEV;
bp->lmfs_next = bp + 1;
bp->lmfs_prev = bp - 1;
bp->data = NULL;
bp->lmfs_bytes = 0;
}
front->lmfs_prev = NULL;
rear->lmfs_next = NULL;
for (bp = &buf[0]; bp < &buf[nr_bufs]; bp++) bp->lmfs_hash = bp->lmfs_next;
buf_hash[0] = front;
}
int lmfs_bufs_in_use(void)
{
return bufs_in_use;
}
int lmfs_nr_bufs(void)
{
return nr_bufs;
}
void lmfs_flushall(void)
{
struct buf *bp;
for(bp = &buf[0]; bp < &buf[nr_bufs]; bp++)
vm: new secondary cache code Primary purpose of change: to support the mmap implementation, VM must know both (a) about some block metadata for FS cache blocks, i.e. inode numbers and inode offsets where applicable; and (b) know about *all* cache blocks, i.e. also of the FS primary caches and not just the blocks that spill into the secondary one. This changes the interface and VM data structures. This change is only for the interface (libminixfs) and VM data structures; the filesystem code is unmodified, so although the secondary cache will be used as normal, blocks will not be annotated with inode information until the FS is modified to provide this information. Until it is modified, mmap of files will fail gracefully on such filesystems. This is indicated to VFS/VM by returning ENOSYS for REQ_PEEK. Change-Id: I1d2df6c485e6c5e89eb28d9055076cc02629594e
2013-03-20 20:18:52 +01:00
if(bp->lmfs_dev != NO_DEV && !lmfs_isclean(bp))
Move primary cache code to libminixfs. Add primary cache management feature to libminixfs as mfs and ext2 currently do separately, remove cache code from mfs and ext2, and make them use the libminixfs interface. This makes all fields of the buf struct private to libminixfs and FS clients aren't supposed to access them at all. Only the opaque 'void *data' field (the FS block contents, used to be called bp) is to be accessed by the FS client. The main purpose is to implement the interface to the 2ndary vm cache just once, get rid of some code duplication, and add a little abstraction to reduce the code inertia of the whole caching business. Some minor sanity checking and prohibition done by mfs in this code as removed from the generic primary cache code as a result: - checking all inodes are not in use when allocating/resizing the cache - checking readonly filesystems aren't written to - checking the superblock isn't written to on mounted filesystems The minixfslib code relies on fs_blockstats() in the client filesystem to return some FS usage information.
2012-10-16 17:40:39 +02:00
flushall(bp->lmfs_dev);
}
int lmfs_fs_block_size(void)
{
return fs_block_size;
}
void lmfs_may_use_vmcache(int ok)
{
may_use_vmcache = ok;
}
void lmfs_reset_rdwt_err(void)
{
rdwt_err = OK;
}
int lmfs_rdwt_err(void)
{
return rdwt_err;
}
vm: new secondary cache code Primary purpose of change: to support the mmap implementation, VM must know both (a) about some block metadata for FS cache blocks, i.e. inode numbers and inode offsets where applicable; and (b) know about *all* cache blocks, i.e. also of the FS primary caches and not just the blocks that spill into the secondary one. This changes the interface and VM data structures. This change is only for the interface (libminixfs) and VM data structures; the filesystem code is unmodified, so although the secondary cache will be used as normal, blocks will not be annotated with inode information until the FS is modified to provide this information. Until it is modified, mmap of files will fail gracefully on such filesystems. This is indicated to VFS/VM by returning ENOSYS for REQ_PEEK. Change-Id: I1d2df6c485e6c5e89eb28d9055076cc02629594e
2013-03-20 20:18:52 +01:00
VFS: Implement REQ_BPEEK. This commit introduces a new request type called REQ_BPEEK. It requests minor device blocks from the FS. Analogously to REQ_PEEK, it requests the filesystem to get the requested blocks into its cache, without actually copying the result anywhere. Change-Id: If1d06645b0e17553a64b3167091e9d12efeb3d6f
2013-03-16 05:09:36 +01:00
int lmfs_do_bpeek(message *m)
{
block_t startblock, b, limitblock;
dev_t dev = m->REQ_DEV2;
u64_t extra, pos = make64(m->REQ_SEEK_POS_LO, m->REQ_SEEK_POS_HI);
size_t len = m->REQ_NBYTES;
struct buf *bp;
assert(m->m_type == REQ_BPEEK);
assert(fs_block_size > 0);
assert(dev != NO_DEV);
libminixfs: allow non-pagesize-multiple FSes The memory-mapped files implementation (mmap() etc.) is implemented with the help of the filesystems using the in-VM FS cache. Filesystems tell it about all cached blocks and their metadata. Metadata is: device offset and, if any (and known), inode number and in-inode offset. VM can then map in requested memory-mapped file blocks, and request them if necessary. A limitation of this system is that filesystem block sizes that are not a multiple of the VM system (and VM hardware) page size are not possible; we can't map blocks in partially. (We can copy, but then the benefits of mapping and sharing the physical pages is gone.) So until before this commit various pieces of caching code assumed page size multiple blocksizes. This isn't strictly necessary as long as mmap() needn't be supported on that FS. This change allows the in-FS cache code (libminixfs) to allocate any-sized blocks, and will not interact with the VM cache for non-pagesize-multiple blocks. In that case it will also signal requestors, by failing 'peek' requests, that mmap() should not be supported on this FS. VM and VFS will then gracefully fail all file-mapping mmap() calls, and exec() will fall back to copying executable blocks instead of mmap()ping executables. As a result, 3 diagnostics that signal file-mapped mmap()s failing (hitherto an unusual occurence) are disabled, as ld.so does file-mapped mmap()s to map in objects it needs. On FSes not supporting it this situation is legitimate and shouldn't cause so much noise. ld.so will revert to its own minix-specific allocate+copy style of starting executables if mmap()s fail. Change-Id: Iecb1c8090f5e0be28da8f5181bb35084eb18f67b
2013-11-19 16:59:52 +01:00
if(!vmcache) { return ENXIO; }
assert(!(fs_block_size % PAGE_SIZE));
VFS: Implement REQ_BPEEK. This commit introduces a new request type called REQ_BPEEK. It requests minor device blocks from the FS. Analogously to REQ_PEEK, it requests the filesystem to get the requested blocks into its cache, without actually copying the result anywhere. Change-Id: If1d06645b0e17553a64b3167091e9d12efeb3d6f
2013-03-16 05:09:36 +01:00
if((extra=(pos % fs_block_size))) {
pos -= extra;
len += extra;
}
len = roundup(len, fs_block_size);
startblock = pos/fs_block_size;
limitblock = startblock + len/fs_block_size;
for(b = startblock; b < limitblock; b++) {
bp = lmfs_get_block(dev, b, NORMAL);
assert(bp);
lmfs_put_block(bp, FULL_DATA_BLOCK);
}
return OK;
}
Reference in a new issue Copy permalink