Commit graph

5512 commits

Author SHA1 Message Date
Kees Jongenburger 1e1ff96aea arm:vm caching fix.
Improve reliability by using write trough cache.
2013-05-16 20:39:20 +02:00
Kees Jongenburger a3f6529ee2 arm:vm header cleanup. 2013-05-16 20:39:19 +02:00
Kees Jongenburger b9cb8251bc arm:make no assumptions about TRE and AFE
The bootloader can leave the system control register
in at state that doesn't match our setup. make no assumptions
and configure TRE and AFE.
2013-05-16 20:39:19 +02:00
Kees Jongenburger 84eeb1e681 build:arm enable verbose mode by default. 2013-05-16 20:39:19 +02:00
Kees Jongenburger f6c762b5d3 build:arm disable I and D cache before loading the kernel. 2013-05-16 20:39:19 +02:00
Kees Jongenburger 0aeb7d85d1 build:copy uEnv and cmdline.txt onto OBJ 2013-05-16 20:39:19 +02:00
Kees Jongenburger 7e2c8b516d build:make sdimage more silent.
Change-Id: I9bba37c93c27c599ec914d1f069ca21e6cbfe111
2013-05-16 20:39:18 +02:00
Kees Jongenburger 4ab5b32edf build:add sbin and bin to path.
normal users don't have sbin in their path where sfdisk is
installed.

Change-Id: I7ee3412ca0f6fb208310aa7a0ee8cd88c0f3f80c
2013-05-16 20:39:18 +02:00
Ben Gras 062d5d7811 Revert "virtio_blk: always start if found"
Wrong commit pushed, investigating.

This reverts commit a476662532.
2013-05-16 13:34:17 +00:00
Ben Gras a476662532 virtio_blk: always start if found
Change-Id: Iec70c67e1b675297b5ffb800a9fb263feffa9271
2013-05-16 12:02:48 +00:00
Ben Gras 7a0497feae mk.conf: workaround for pkgin sourcing it
Change-Id: Ie01c5781ff3372d6415bc8878270002988320c56
2013-05-16 11:13:00 +00:00
Antoine Leca e35821b207 mkfs.mfs: various programs
Split in as many programs as needed to support various variants of MFS
Manual page updated (in mandoc format)
2013-05-13 11:18:15 +02:00
Antoine Leca 49fe5d1617 mkfs reloaded
* restored the zone>block stuff
* generalized the support for more-than-1-block bitmaps to also deal
  with big inodes maps in small blocks
* improved pexit() to printf-like syntax; also replaced pexit() with
  err() or errx() where there are no reference to proto lines
* unified the allocation of blocks to all use alloc_block() instead
  of raw malloc half of the time
* removed the V2_ d2_ etc. prefixes which are obscure and obsolete

While here, also
* used new-form definitions, and closer to KNF style in general
* used often-built-ins such as mem* or strncpy where relevant
* pruned a fair amount of constants which are irrelevant

Thanks to Thomas V. and Lionel who did a good amount of work with the
cross-compilation stuff and made that change to be much smoother.
2013-05-13 11:17:34 +02:00
Ben Gras 717425320f kernel, random: adaptive entropy gathering
kernel: stop gathering timestamps once the bin is full per interrupt

random: once seeded, retrieve new entropy at a lower rate

Change-Id: I4ce6081d39274728d82c6889686d1650cfd5fc2e
2013-05-07 17:19:06 +00:00
Ben Gras 4ebb889e7a libsys: panic hook feature
. vfs: use it to dump threads stacks

Change-Id: I7ae3521fc153a407505f11049629e6d4142cf7c7
2013-05-07 17:18:40 +00:00
Ben Gras 80846c4a79 kernel ipc debug: various fixes
. add receive hooks in the kernel to print asynchronously
	  delivered messages
	. do not rely on MF_REPLY_PEND to decide between calls and errors,
	  as that isn't reliable for asynchronous messages; try both instead
	. add _sendcall() that extract-mfield.sh can then reliably recognize
	  the fields for messages that are sent with just send()
	. add DEBUG_DUMPIPC_NAMES to restrict printed messages to
	  from/to given process names

Change-Id: Ia65eb02a69a2b58e73bf9f009987be06dda774a3
2013-05-01 21:40:23 +00:00
Ben Gras 85fd078707 tty: non-overlapping code for FKEY_CONTROL
. to decode it in kernel/debug.c

Change-Id: I0d2cc66e87d97a362fa549b364b4d1b0e1225e66
2013-05-01 21:36:43 +00:00
Kees Jongenburger dfbe2f16ee build:Updated proto.dev from mkproto -s /dev
proto.dev was missing some entries and is currently used as base version for
the ARM root file system.
2013-04-29 11:47:40 +02:00
Kees Jongenburger e6bac75a8b ARM:Rename ARM_BIG_PAGE to ARM_SECTION.
The natural term to use when talking about MINIX big pages on ARM
is SECTION. A section is a level 1 page table entry pointing to
a 1MB area.

Change-Id: I9bd27ca99bc772126c31c27a537b1415db20c4a6
2013-04-29 11:42:26 +02:00
Ben Gras 236a320240 mkfs.mfs: double indirect block support
. allows >4MB files for default 4k block size

Change-Id: I09277fc3704fac1a44af82202c11435dc7ae8d55
2013-04-27 16:25:45 +02:00
Ben Gras f497d9b9c4 ext2: annotate cache blocks with inode metadata
. allows mmap() to work on it

Change-Id: I33af8b86ebb408d971478d00ed2caacf15afc7a5
2013-04-26 15:39:07 +00:00
Ben Gras f0cc010614 libminixfs, mfs, ext2: may re-evaluate cache size
libminixfs may now be informed of changes to the block usage on the
filesystem. if the net change becomes big enough, libminixfs may
resize the cache based on the new usage.

	. update the 2 FSes to provide this information to libminixfs

Change-Id: I158815a11da801fd5572a8de89c9e6c039b82650
2013-04-26 13:57:08 +00:00
Kees Jongenburger 988c7e39c0 libelf:use self hosted m4.
Change-Id: Ia836aa6c43d7ccbb02047e63424923fdc74eb42d
2013-04-24 17:22:59 +02:00
Kees Jongenburger dca81e0581 toproto:convert special execute bits to chars
Convert the special execute bits of the METALOG file mode entries into mkfs.mfs
compatible u and g flags.
2013-04-24 13:22:45 +02:00
Ben Gras 44f34e53d5 VFS: Implement REQ_BPEEK.
This commit introduces a new request type called REQ_BPEEK. It
requests minor device blocks from the FS.  Analogously to REQ_PEEK,
it requests the filesystem to get the requested blocks into its
cache, without actually copying the result anywhere.

Change-Id: If1d06645b0e17553a64b3167091e9d12efeb3d6f
2013-04-24 10:18:16 +00:00
Ben Gras 0cfff08e56 libexec: mmap support, prealloc variants
In libexec, split the memory allocation method into cleared and
non-cleared. Cleared gives zeroed memory, non-cleared gives 'junk'
memory (that will be overwritten anyway, and so needn't be cleared)
that is faster to get.

Also introduce the 'memmap' method that can be used, if available,
to map code and data from executables into a process using the
third-party mmap() mode.

Change-Id: I26694fd3c21deb8b97e01ed675dfc14719b0672b
2013-04-24 10:18:16 +00:00
Ben Gras 6fa5ce74ce MFS: annotate cache blocks with inode metadata
. use lmfs_* cache functions that provide the cache with inode
	  metadata whenever applicable, i.e. tell the cache code which
	  inode number and in-inode offset a particular cache block
	  corresponds to.
	. needed for mmap implementation

Change-Id: Ic7d3c0c49029880f86a31368278722e907bc2896
2013-04-24 10:18:16 +00:00
Ben Gras 49eb1f4806 vm: new secondary cache code
Primary purpose of change: to support the mmap implementation, VM must
know both (a) about some block metadata for FS cache blocks, i.e.
inode numbers and inode offsets where applicable; and (b) know about
*all* cache blocks, i.e.  also of the FS primary caches and not just
the blocks that spill into the secondary one. This changes the
interface and VM data structures.

This change is only for the interface (libminixfs) and VM data
structures; the filesystem code is unmodified, so although the
secondary cache will be used as normal, blocks will not be annotated
with inode information until the FS is modified to provide this
information. Until it is modified, mmap of files will fail gracefully
on such filesystems.

This is indicated to VFS/VM by returning ENOSYS for REQ_PEEK.

Change-Id: I1d2df6c485e6c5e89eb28d9055076cc02629594e
2013-04-24 10:18:16 +00:00
Ben Gras 7421728360 VM: memtype fix
Memory types in VM are described by methods. Each mapped region has
a type, and all pages instantiated get that type on creation.
Individual page types has to be able to change though. This commit
changes the code to use the memory types of the individual pages,
where appropriate, instead of just the higher-level region, in case
it has changed. This is needed to e.g. support future copy-on-write
MAP_PRIVATE mmap modes.

Change-Id: I5523db14ac036ec774a54392fb67f9acb8725731
2013-04-24 10:18:15 +00:00
Ben Gras 9e88c40e40 VM: 64-bit mmap()
Some (backwards-compatible) changes in mmap() call message fields
that allow for a 64-bit offset. minix_mmap() takes an off_t and
minix_mmap64() takes a u64_t. Some mmap() work in VM goes into a
separate function, using the new fields, so that that can be re-used
when files are to be mapped (future commit).

Change-Id: Ifb77a90b593dd3c33cf81b396068e4da1ec5fb1c
2013-04-24 10:18:15 +00:00
Ben Gras 75c5fe4c26 filesystems: return ENOSYS for REQ_PEEK
The filesystems already implement REQ_PEEK, but do not fully
use the new filesystem cache code yet. (Because it isn't committed
yet..) REQ_PEEK should be disabled for them until they do.

This indicates to VFS that they are not annotating their cache
blocks (in VM) with inode number/offset info, and therefore mmap()
shouldn't succeed on any of their files. (Most importantly exec()
won't fallback elegantly otherwise.)

Change-Id: Ic57ee422864b4bbc031eadba32973270907b02fd
2013-04-24 10:18:10 +00:00
Ben Gras adf2032bc0 vm: remove secondary cache code
This commit removes the secondary cache code implementation from
VM and its usage from libminixfs. It is to be replaced by a new
implementation.

Change-Id: I8fa3af06330e7604c7e0dd4cbe39d3ce353a05b1
2013-04-24 10:18:10 +00:00
Xiaoguang Sun 20e6c9329f Change function prototype to use endpoint_t instead of int 2013-04-23 17:15:15 +02:00
Kees Jongenburger 8a3677def0 build:refactor arm_sdimage.sh as to no longer require root.
There where two reasons for needing root access while making ARM builds. The
first reason was that we needed root permissions to create device nodes and
assign permissions to files. The second reason we needed root was that we used
loop mounted files to copy the build output to the file backed storage.

The first problem is solved by building in unprivileged mode (adding -U) during
compilation. The second problem is solved by using mcopy to put files on the
fat partition and use the mkfs.mfs proto files to create MFS file system.

Future work might be to replace this script by the netbsd makefs tool.

Change-Id: Ibba61d4cad053f7c1ede26169d5214065a4939d8
2013-04-23 11:24:12 +02:00
Kees Jongenburger fd7bb1786e build:add tool to convert from METALOG to proto files.
Change-Id: I5c4e15b1a2bac5a4ca3a0530f8d6aa2643ac720a
2013-04-21 21:36:30 +02:00
Kees Jongenburger f8dbcfa7f2 build:add script to sort netbsd sets and sort minix/mi.
Change-Id: I309d8bbcfe2f5b6d5e1ae7626422c4402985ad1d
2013-04-21 21:35:49 +02:00
Kees Jongenburger fcb66ed1b5 run script:return 1 upon test failure.
Change-Id: If3c9d636a04da7f7b4098b58d29c492c664dab1a
2013-04-21 20:18:11 +02:00
Ben Gras 32a4e0d84d many new tests
. test70: regression test for m_out vfs race condition

The following tests use testcache.c to generate test i/o
patterns, generate random write data and verify the reads.

	. test71: blackbox full-stack test of FS operation, testing
	  using the regular VFS interface crazy i/o patterns
	  with various working set sizes, triggering only
	  primary cache, also secondary cache, and finally
	  disk i/o and verifying contents all the time
	. test72: unit test of libminixfs, implementing
	  functions it needs from -lsys and -lblockdriver
	  and the client in order to simulate a working
	  cache client and backend environment.
	. test73: blackbox test of secondary vm cache in isolation

Change-Id: I1287e9753182b8719e634917ad158e3c1e079ceb
2013-04-19 16:21:48 +00:00
Ben Gras 072d916c1c vfs: fix null deref, pfs: add fchmod()
. vfs read_only() assumes vnode->v_vmnt is non-NULL, but it can
	  be NULL sometimes
	. e.g. fchmod() on UDS triggered NULL deref; add a check and
	  add REQ_CHMOD to pfs so unix domain sockets can be fchmod()ded
	. add to test56

Change-Id: I83c840f101b647516897cc99fcf472116d762012
2013-04-19 17:06:56 +02:00
Kees Jongenburger 04ad4fa1a2 mkfs.mfs:add comment about the overflow check.
Change-Id: I1f09c3a62377164296037f6dc9b7874023123311
2013-04-19 13:10:06 +02:00
Kees Jongenburger a4eca06cca mkfs.mfs:fix to allow to fill larger file systems.
Allow the number of written blocks to exceed 8 * block_size.

Change-Id: I301ce46a92d86c6220c3af2acee22db26a4b4496
2013-04-19 13:10:06 +02:00
Kees Jongenburger 15bb31ed99 mkfs.mfs:remove MFS V1 related headers.
Change-Id: I46a8128f54112ee0092ef60160d88cadd0038be0
2013-04-19 13:10:06 +02:00
Kees Jongenburger 1beca4b7f5 mkfs.mfs:remove some globals
Remove the gobal inocount, zonecount and blockcount.

Change-Id: I77d120bb79bcf183e0c6b5abed736343af7badf2
2013-04-19 13:10:06 +02:00
Ben Gras 0245723d5f run script: correction, clarity
Change-Id: Ibf583c6277c67646dd90d7c49482daf4e76b78b6
2013-04-19 10:24:50 +00:00
Ben Gras 2a44efb1c0 test run: feature for nicer jenkins interaction
. add -l option to list available tests
	. add -t option to specify tests to run
	. also improve the root check a bit by
	  not relying on an environment var
	. do not print the human-friendly banner & summary
	  if a test list is given to make the test
	  results easy to parse

Change-Id: Id0f87d485240b1924d667af788338ae31c3dc94c
2013-04-18 14:28:03 +00:00
Ben Gras 5977114c42 tests cleanup
. make common.o link with the tests instead of being
	  #included as common.c
	. fix warnings about missing prototypes by declaring functions
	  static
	. reduces some duplicated code

Change-Id: Ic2a765d7f5886add5863190efec3fdd2d2ea2137
2013-04-17 22:00:59 +00:00
Ben Gras cef94e096e vfs: make m_out non-global
m_out is shared between threads as the reply message, and it can happen
results get overwritten by another thread before the reply is sent. This
change

	. makes m_out local to the message handling function,
	  declared on the stack of the caller
	. forces callers of reply() to give it a message, or
	  declare the reply message has no significant fields except
	  for the return code by calling replycode()

Change-Id: Id06300083a63c72c00f34f86a5c7d96e4bbdf9f6
2013-04-12 23:40:38 +00:00
Michael W. Bombardieri cb51426c6a rs232: remove unneeded lock()/unlock() functions
Change-Id: Ie1d2a7f460132aa59791cd7d16c9068b79675f1b
2013-04-12 21:37:26 +02:00
Thomas Cort a0a1440a52 remove utimes() workarounds
With the addition of utimes(), we can remove the workarounds
and use the original NetBSD code for timestamping files.

Also restore use of -p & -r for install while building
2013-04-12 18:55:40 +00:00
Antoine Leca 3ae6c600a9 Add tests for utime*s() functions 2013-04-12 18:55:40 +00:00