Commit graph

5451 commits

Author SHA1 Message Date
Antoine Leca
e35821b207 mkfs.mfs: various programs
Split in as many programs as needed to support various variants of MFS
Manual page updated (in mandoc format)
2013-05-13 11:18:15 +02:00
Antoine Leca
49fe5d1617 mkfs reloaded
* restored the zone>block stuff
* generalized the support for more-than-1-block bitmaps to also deal
  with big inodes maps in small blocks
* improved pexit() to printf-like syntax; also replaced pexit() with
  err() or errx() where there are no reference to proto lines
* unified the allocation of blocks to all use alloc_block() instead
  of raw malloc half of the time
* removed the V2_ d2_ etc. prefixes which are obscure and obsolete

While here, also
* used new-form definitions, and closer to KNF style in general
* used often-built-ins such as mem* or strncpy where relevant
* pruned a fair amount of constants which are irrelevant

Thanks to Thomas V. and Lionel who did a good amount of work with the
cross-compilation stuff and made that change to be much smoother.
2013-05-13 11:17:34 +02:00
Ben Gras
717425320f kernel, random: adaptive entropy gathering
kernel: stop gathering timestamps once the bin is full per interrupt

random: once seeded, retrieve new entropy at a lower rate

Change-Id: I4ce6081d39274728d82c6889686d1650cfd5fc2e
2013-05-07 17:19:06 +00:00
Ben Gras
4ebb889e7a libsys: panic hook feature
. vfs: use it to dump threads stacks

Change-Id: I7ae3521fc153a407505f11049629e6d4142cf7c7
2013-05-07 17:18:40 +00:00
Ben Gras
80846c4a79 kernel ipc debug: various fixes
. add receive hooks in the kernel to print asynchronously
	  delivered messages
	. do not rely on MF_REPLY_PEND to decide between calls and errors,
	  as that isn't reliable for asynchronous messages; try both instead
	. add _sendcall() that extract-mfield.sh can then reliably recognize
	  the fields for messages that are sent with just send()
	. add DEBUG_DUMPIPC_NAMES to restrict printed messages to
	  from/to given process names

Change-Id: Ia65eb02a69a2b58e73bf9f009987be06dda774a3
2013-05-01 21:40:23 +00:00
Ben Gras
85fd078707 tty: non-overlapping code for FKEY_CONTROL
. to decode it in kernel/debug.c

Change-Id: I0d2cc66e87d97a362fa549b364b4d1b0e1225e66
2013-05-01 21:36:43 +00:00
Kees Jongenburger
dfbe2f16ee build:Updated proto.dev from mkproto -s /dev
proto.dev was missing some entries and is currently used as base version for
the ARM root file system.
2013-04-29 11:47:40 +02:00
Kees Jongenburger
e6bac75a8b ARM:Rename ARM_BIG_PAGE to ARM_SECTION.
The natural term to use when talking about MINIX big pages on ARM
is SECTION. A section is a level 1 page table entry pointing to
a 1MB area.

Change-Id: I9bd27ca99bc772126c31c27a537b1415db20c4a6
2013-04-29 11:42:26 +02:00
Ben Gras
236a320240 mkfs.mfs: double indirect block support
. allows >4MB files for default 4k block size

Change-Id: I09277fc3704fac1a44af82202c11435dc7ae8d55
2013-04-27 16:25:45 +02:00
Ben Gras
f497d9b9c4 ext2: annotate cache blocks with inode metadata
. allows mmap() to work on it

Change-Id: I33af8b86ebb408d971478d00ed2caacf15afc7a5
2013-04-26 15:39:07 +00:00
Ben Gras
f0cc010614 libminixfs, mfs, ext2: may re-evaluate cache size
libminixfs may now be informed of changes to the block usage on the
filesystem. if the net change becomes big enough, libminixfs may
resize the cache based on the new usage.

	. update the 2 FSes to provide this information to libminixfs

Change-Id: I158815a11da801fd5572a8de89c9e6c039b82650
2013-04-26 13:57:08 +00:00
Kees Jongenburger
988c7e39c0 libelf:use self hosted m4.
Change-Id: Ia836aa6c43d7ccbb02047e63424923fdc74eb42d
2013-04-24 17:22:59 +02:00
Kees Jongenburger
dca81e0581 toproto:convert special execute bits to chars
Convert the special execute bits of the METALOG file mode entries into mkfs.mfs
compatible u and g flags.
2013-04-24 13:22:45 +02:00
Ben Gras
44f34e53d5 VFS: Implement REQ_BPEEK.
This commit introduces a new request type called REQ_BPEEK. It
requests minor device blocks from the FS.  Analogously to REQ_PEEK,
it requests the filesystem to get the requested blocks into its
cache, without actually copying the result anywhere.

Change-Id: If1d06645b0e17553a64b3167091e9d12efeb3d6f
2013-04-24 10:18:16 +00:00
Ben Gras
0cfff08e56 libexec: mmap support, prealloc variants
In libexec, split the memory allocation method into cleared and
non-cleared. Cleared gives zeroed memory, non-cleared gives 'junk'
memory (that will be overwritten anyway, and so needn't be cleared)
that is faster to get.

Also introduce the 'memmap' method that can be used, if available,
to map code and data from executables into a process using the
third-party mmap() mode.

Change-Id: I26694fd3c21deb8b97e01ed675dfc14719b0672b
2013-04-24 10:18:16 +00:00
Ben Gras
6fa5ce74ce MFS: annotate cache blocks with inode metadata
. use lmfs_* cache functions that provide the cache with inode
	  metadata whenever applicable, i.e. tell the cache code which
	  inode number and in-inode offset a particular cache block
	  corresponds to.
	. needed for mmap implementation

Change-Id: Ic7d3c0c49029880f86a31368278722e907bc2896
2013-04-24 10:18:16 +00:00
Ben Gras
49eb1f4806 vm: new secondary cache code
Primary purpose of change: to support the mmap implementation, VM must
know both (a) about some block metadata for FS cache blocks, i.e.
inode numbers and inode offsets where applicable; and (b) know about
*all* cache blocks, i.e.  also of the FS primary caches and not just
the blocks that spill into the secondary one. This changes the
interface and VM data structures.

This change is only for the interface (libminixfs) and VM data
structures; the filesystem code is unmodified, so although the
secondary cache will be used as normal, blocks will not be annotated
with inode information until the FS is modified to provide this
information. Until it is modified, mmap of files will fail gracefully
on such filesystems.

This is indicated to VFS/VM by returning ENOSYS for REQ_PEEK.

Change-Id: I1d2df6c485e6c5e89eb28d9055076cc02629594e
2013-04-24 10:18:16 +00:00
Ben Gras
7421728360 VM: memtype fix
Memory types in VM are described by methods. Each mapped region has
a type, and all pages instantiated get that type on creation.
Individual page types has to be able to change though. This commit
changes the code to use the memory types of the individual pages,
where appropriate, instead of just the higher-level region, in case
it has changed. This is needed to e.g. support future copy-on-write
MAP_PRIVATE mmap modes.

Change-Id: I5523db14ac036ec774a54392fb67f9acb8725731
2013-04-24 10:18:15 +00:00
Ben Gras
9e88c40e40 VM: 64-bit mmap()
Some (backwards-compatible) changes in mmap() call message fields
that allow for a 64-bit offset. minix_mmap() takes an off_t and
minix_mmap64() takes a u64_t. Some mmap() work in VM goes into a
separate function, using the new fields, so that that can be re-used
when files are to be mapped (future commit).

Change-Id: Ifb77a90b593dd3c33cf81b396068e4da1ec5fb1c
2013-04-24 10:18:15 +00:00
Ben Gras
75c5fe4c26 filesystems: return ENOSYS for REQ_PEEK
The filesystems already implement REQ_PEEK, but do not fully
use the new filesystem cache code yet. (Because it isn't committed
yet..) REQ_PEEK should be disabled for them until they do.

This indicates to VFS that they are not annotating their cache
blocks (in VM) with inode number/offset info, and therefore mmap()
shouldn't succeed on any of their files. (Most importantly exec()
won't fallback elegantly otherwise.)

Change-Id: Ic57ee422864b4bbc031eadba32973270907b02fd
2013-04-24 10:18:10 +00:00
Ben Gras
adf2032bc0 vm: remove secondary cache code
This commit removes the secondary cache code implementation from
VM and its usage from libminixfs. It is to be replaced by a new
implementation.

Change-Id: I8fa3af06330e7604c7e0dd4cbe39d3ce353a05b1
2013-04-24 10:18:10 +00:00
Xiaoguang Sun
20e6c9329f Change function prototype to use endpoint_t instead of int 2013-04-23 17:15:15 +02:00
Kees Jongenburger
8a3677def0 build:refactor arm_sdimage.sh as to no longer require root.
There where two reasons for needing root access while making ARM builds. The
first reason was that we needed root permissions to create device nodes and
assign permissions to files. The second reason we needed root was that we used
loop mounted files to copy the build output to the file backed storage.

The first problem is solved by building in unprivileged mode (adding -U) during
compilation. The second problem is solved by using mcopy to put files on the
fat partition and use the mkfs.mfs proto files to create MFS file system.

Future work might be to replace this script by the netbsd makefs tool.

Change-Id: Ibba61d4cad053f7c1ede26169d5214065a4939d8
2013-04-23 11:24:12 +02:00
Kees Jongenburger
fd7bb1786e build:add tool to convert from METALOG to proto files.
Change-Id: I5c4e15b1a2bac5a4ca3a0530f8d6aa2643ac720a
2013-04-21 21:36:30 +02:00
Kees Jongenburger
f8dbcfa7f2 build:add script to sort netbsd sets and sort minix/mi.
Change-Id: I309d8bbcfe2f5b6d5e1ae7626422c4402985ad1d
2013-04-21 21:35:49 +02:00
Kees Jongenburger
fcb66ed1b5 run script:return 1 upon test failure.
Change-Id: If3c9d636a04da7f7b4098b58d29c492c664dab1a
2013-04-21 20:18:11 +02:00
Ben Gras
32a4e0d84d many new tests
. test70: regression test for m_out vfs race condition

The following tests use testcache.c to generate test i/o
patterns, generate random write data and verify the reads.

	. test71: blackbox full-stack test of FS operation, testing
	  using the regular VFS interface crazy i/o patterns
	  with various working set sizes, triggering only
	  primary cache, also secondary cache, and finally
	  disk i/o and verifying contents all the time
	. test72: unit test of libminixfs, implementing
	  functions it needs from -lsys and -lblockdriver
	  and the client in order to simulate a working
	  cache client and backend environment.
	. test73: blackbox test of secondary vm cache in isolation

Change-Id: I1287e9753182b8719e634917ad158e3c1e079ceb
2013-04-19 16:21:48 +00:00
Ben Gras
072d916c1c vfs: fix null deref, pfs: add fchmod()
. vfs read_only() assumes vnode->v_vmnt is non-NULL, but it can
	  be NULL sometimes
	. e.g. fchmod() on UDS triggered NULL deref; add a check and
	  add REQ_CHMOD to pfs so unix domain sockets can be fchmod()ded
	. add to test56

Change-Id: I83c840f101b647516897cc99fcf472116d762012
2013-04-19 17:06:56 +02:00
Kees Jongenburger
04ad4fa1a2 mkfs.mfs:add comment about the overflow check.
Change-Id: I1f09c3a62377164296037f6dc9b7874023123311
2013-04-19 13:10:06 +02:00
Kees Jongenburger
a4eca06cca mkfs.mfs:fix to allow to fill larger file systems.
Allow the number of written blocks to exceed 8 * block_size.

Change-Id: I301ce46a92d86c6220c3af2acee22db26a4b4496
2013-04-19 13:10:06 +02:00
Kees Jongenburger
15bb31ed99 mkfs.mfs:remove MFS V1 related headers.
Change-Id: I46a8128f54112ee0092ef60160d88cadd0038be0
2013-04-19 13:10:06 +02:00
Kees Jongenburger
1beca4b7f5 mkfs.mfs:remove some globals
Remove the gobal inocount, zonecount and blockcount.

Change-Id: I77d120bb79bcf183e0c6b5abed736343af7badf2
2013-04-19 13:10:06 +02:00
Ben Gras
0245723d5f run script: correction, clarity
Change-Id: Ibf583c6277c67646dd90d7c49482daf4e76b78b6
2013-04-19 10:24:50 +00:00
Ben Gras
2a44efb1c0 test run: feature for nicer jenkins interaction
. add -l option to list available tests
	. add -t option to specify tests to run
	. also improve the root check a bit by
	  not relying on an environment var
	. do not print the human-friendly banner & summary
	  if a test list is given to make the test
	  results easy to parse

Change-Id: Id0f87d485240b1924d667af788338ae31c3dc94c
2013-04-18 14:28:03 +00:00
Ben Gras
5977114c42 tests cleanup
. make common.o link with the tests instead of being
	  #included as common.c
	. fix warnings about missing prototypes by declaring functions
	  static
	. reduces some duplicated code

Change-Id: Ic2a765d7f5886add5863190efec3fdd2d2ea2137
2013-04-17 22:00:59 +00:00
Ben Gras
cef94e096e vfs: make m_out non-global
m_out is shared between threads as the reply message, and it can happen
results get overwritten by another thread before the reply is sent. This
change

	. makes m_out local to the message handling function,
	  declared on the stack of the caller
	. forces callers of reply() to give it a message, or
	  declare the reply message has no significant fields except
	  for the return code by calling replycode()

Change-Id: Id06300083a63c72c00f34f86a5c7d96e4bbdf9f6
2013-04-12 23:40:38 +00:00
Michael W. Bombardieri
cb51426c6a rs232: remove unneeded lock()/unlock() functions
Change-Id: Ie1d2a7f460132aa59791cd7d16c9068b79675f1b
2013-04-12 21:37:26 +02:00
Thomas Cort
a0a1440a52 remove utimes() workarounds
With the addition of utimes(), we can remove the workarounds
and use the original NetBSD code for timestamping files.

Also restore use of -p & -r for install while building
2013-04-12 18:55:40 +00:00
Antoine Leca
3ae6c600a9 Add tests for utime*s() functions 2013-04-12 18:55:40 +00:00
Antoine Leca
9131e98a7d utimens(2) system call
Variant of utime(2) with struct timespec (with ns precision)
instead of time_t values; also allows for tv_nsec members
the values UTIME_NOW (force update to current time) or
UTIME_OMIT (allow to set either atim or mtim independently.)

Provides a superset of utimes(2), futimes(2), lutimes(2),
and futimens(2).
Provides the same subset of utimensat(2) as does NetBSD 6.
Also import utimens() and lutimeNS() from NetBSD-current.
2013-04-12 18:55:39 +00:00
Kees Jongenburger
88414c568c cleanup:misc documentation fix. 2013-04-12 20:52:58 +02:00
Kees Jongenburger
99c5247453 arm_sdimage:hide non fatal errors.
Redirect the output of the command that are supposed to fail to
/dev/null.
2013-04-12 20:52:58 +02:00
Kees Jongenburger
9d9456ef7d arm:cleanup add assert. 2013-04-12 20:52:57 +02:00
Kees Jongenburger
1e9492a8b8 arm:cleanup omap serial, remove unused defines. 2013-04-12 20:52:57 +02:00
Kees Jongenburger
037aeb5a13 arm:rs232 fix.
The kernel API for requesting interrupts and the associated callback
have a somewhat strange behaviour. Requesting an interrupts is done
by calling sys_irqsetpolicy using an interrupt and a given id. This
id can be modified by the sys_irqsetpolicy and must be used for
subsequent calls to sys_irqenable/sys_irqdisable. However upon an
incoming call from the kernel NOTIFY_ARG contains the original value
encoded in a set e.g. if  1 << id == true the interrupt was raised.
2013-04-12 20:52:57 +02:00
Kees Jongenburger
aa3db0e0e2 mmc:Change startup message in rc script. 2013-04-12 20:52:57 +02:00
Kees Jongenburger
7015c66bfa arm:increase the amount of interrup vectors
Increase the amount of interrup vectors in preparation for the AM335X.
2013-04-12 20:52:57 +02:00
Kees Jongenburger
390f852bd1 arm:cleanup remove special case memory handling.
We used to load the memory driver above the other modules to allow
the memory driver (ramdisk) to grow. We no longer want or need this
this as we have a working mmc driver.
2013-04-12 20:52:57 +02:00
Lionel Sambuc
08ff44c446 Reducing external/bsd/file differences
Change-Id: I0d025059ba0b200ac1162ea5a764fd756600ecd0
2013-04-12 13:25:18 +02:00
Antoine Leca
c979136e4e Also apply sub-second UTIME to libpuffs 2013-04-12 11:11:59 +02:00