Commit graph

6170 commits

Author SHA1 Message Date
Kees Jongenburger
ad68d15e39 arm:build generate uEnv.txt
Generate uEnv.txt to allow to generate an uEnv containing commands
to boot over the network.

Change-Id: I8bcd831ee474d837411568bf1c1e2c2d96ec931b
2013-05-24 11:16:59 +02:00
Lionel Sambuc
881ff3f6e5 releasetools/arm_sdimage.sh: Improvements
- Added the possibility to define the number of parallel jobs from the
   command line with (example, default is 1):
   $ JOBS=8 ./releasetools/arm_sdimage.sh

 - Generate filesystem filling the reserved partition area, instead of
   being smaller.

 - Added a possibility to specify additional parameter to the build.sh
   script from the command line with (example):
   $ BUILDVARS='-N0 -V MYVAR=yes' ./releasetools/arm_sdimage.sh

Change-Id: I5c34268ebf7d45a8d3408b45b1dc5051d775826c
2013-05-24 11:08:15 +02:00
Lionel Sambuc
e1acf37d77 usr.sbin/mkfs: Allow -b SIZE with proto file
Previously, if a proto file was given, the -b argument would be ignored.

Also added:
 - if the number of inodes is not given, then an estimation is used.
 - sanity checks where moved from one special case to the general case.

Change-Id: I2abfa52bf34206c9087c64b5bfc26af866eb47cb
2013-05-24 11:08:15 +02:00
Lionel Sambuc
154b6d7486 Fix symlink issue when generating an ARM image
A few symlinks were pointing to a nonexistent file named '(null)'
instead of their intended target. This was only seen when using the
arm_sdimage.sh script.

There is two ways of specifying links and directories. the first one
using an entry in one of the mtree files, and the second one in
/etc/Makefile.

Those entries where doubled, and one of them would specify some rights,
while the other had the target.

By removing those entries, I make sure there is only one definition of
these symlinks, which solves the problem I was seeing on an ARM image.

These symlinks are still present on a generated system, as they are
required.

Change-Id: I9ced8860f72d7c4d686a09720de4d8257d6e04fa
2013-05-24 11:08:15 +02:00
Ben Gras
fd5f2edf35 mkfs: only complain about failing stat with -d
mkfs -d is a feature that needs a stat() of the mkfs binary, which
often fails (as mkfs is often not invoked with a full path or from
the same directory). it makes setup look a bit messy as the error
is always printed while installing the system, for each created FS,
even though the situation is harmless. This change only complains
when the stat is actually needed (-d).

Change-Id: I54ac01505aa97c1cbe40456c04a35aed5a7ee953
2013-05-23 12:09:38 +02:00
Ben Gras
d12d57dcd8 arm: recognize instruction fetch pagefault
. an instruction fetch pagefault generates a prefetch
	  abort exception, this should also be a source of
	  a pagefault event
	. the saved_lr argument to the C exception handler was
	  sometimes an address, sometimes a pointer to that address;
	  the kernel sometimes dereferences it, so it should always be
	  an in-kernel address, never a userspace address. fix in mpx.S
	  makes it always a pointer.
	. move dumping of all processes over serial out of
	  the arch-specific arch_system.c
2013-05-21 15:05:06 +02:00
Ben Gras
225ed6d7bd at_wini: enable busmastering in pci if necessary 2013-05-18 15:57:30 +00:00
Ben Gras
e8681e9929 virtio_blk: always start if found
Change-Id: Iec70c67e1b675297b5ffb800a9fb263feffa9271
2013-05-17 11:32:45 +00:00
Kees Jongenburger
1e1ff96aea arm:vm caching fix.
Improve reliability by using write trough cache.
2013-05-16 20:39:20 +02:00
Kees Jongenburger
a3f6529ee2 arm:vm header cleanup. 2013-05-16 20:39:19 +02:00
Kees Jongenburger
b9cb8251bc arm:make no assumptions about TRE and AFE
The bootloader can leave the system control register
in at state that doesn't match our setup. make no assumptions
and configure TRE and AFE.
2013-05-16 20:39:19 +02:00
Kees Jongenburger
84eeb1e681 build:arm enable verbose mode by default. 2013-05-16 20:39:19 +02:00
Kees Jongenburger
f6c762b5d3 build:arm disable I and D cache before loading the kernel. 2013-05-16 20:39:19 +02:00
Kees Jongenburger
0aeb7d85d1 build:copy uEnv and cmdline.txt onto OBJ 2013-05-16 20:39:19 +02:00
Kees Jongenburger
7e2c8b516d build:make sdimage more silent.
Change-Id: I9bba37c93c27c599ec914d1f069ca21e6cbfe111
2013-05-16 20:39:18 +02:00
Kees Jongenburger
4ab5b32edf build:add sbin and bin to path.
normal users don't have sbin in their path where sfdisk is
installed.

Change-Id: I7ee3412ca0f6fb208310aa7a0ee8cd88c0f3f80c
2013-05-16 20:39:18 +02:00
Ben Gras
062d5d7811 Revert "virtio_blk: always start if found"
Wrong commit pushed, investigating.

This reverts commit a476662532.
2013-05-16 13:34:17 +00:00
Ben Gras
a476662532 virtio_blk: always start if found
Change-Id: Iec70c67e1b675297b5ffb800a9fb263feffa9271
2013-05-16 12:02:48 +00:00
Ben Gras
7a0497feae mk.conf: workaround for pkgin sourcing it
Change-Id: Ie01c5781ff3372d6415bc8878270002988320c56
2013-05-16 11:13:00 +00:00
Antoine Leca
e35821b207 mkfs.mfs: various programs
Split in as many programs as needed to support various variants of MFS
Manual page updated (in mandoc format)
2013-05-13 11:18:15 +02:00
Antoine Leca
49fe5d1617 mkfs reloaded
* restored the zone>block stuff
* generalized the support for more-than-1-block bitmaps to also deal
  with big inodes maps in small blocks
* improved pexit() to printf-like syntax; also replaced pexit() with
  err() or errx() where there are no reference to proto lines
* unified the allocation of blocks to all use alloc_block() instead
  of raw malloc half of the time
* removed the V2_ d2_ etc. prefixes which are obscure and obsolete

While here, also
* used new-form definitions, and closer to KNF style in general
* used often-built-ins such as mem* or strncpy where relevant
* pruned a fair amount of constants which are irrelevant

Thanks to Thomas V. and Lionel who did a good amount of work with the
cross-compilation stuff and made that change to be much smoother.
2013-05-13 11:17:34 +02:00
Ben Gras
717425320f kernel, random: adaptive entropy gathering
kernel: stop gathering timestamps once the bin is full per interrupt

random: once seeded, retrieve new entropy at a lower rate

Change-Id: I4ce6081d39274728d82c6889686d1650cfd5fc2e
2013-05-07 17:19:06 +00:00
Ben Gras
4ebb889e7a libsys: panic hook feature
. vfs: use it to dump threads stacks

Change-Id: I7ae3521fc153a407505f11049629e6d4142cf7c7
2013-05-07 17:18:40 +00:00
Ben Gras
80846c4a79 kernel ipc debug: various fixes
. add receive hooks in the kernel to print asynchronously
	  delivered messages
	. do not rely on MF_REPLY_PEND to decide between calls and errors,
	  as that isn't reliable for asynchronous messages; try both instead
	. add _sendcall() that extract-mfield.sh can then reliably recognize
	  the fields for messages that are sent with just send()
	. add DEBUG_DUMPIPC_NAMES to restrict printed messages to
	  from/to given process names

Change-Id: Ia65eb02a69a2b58e73bf9f009987be06dda774a3
2013-05-01 21:40:23 +00:00
Ben Gras
85fd078707 tty: non-overlapping code for FKEY_CONTROL
. to decode it in kernel/debug.c

Change-Id: I0d2cc66e87d97a362fa549b364b4d1b0e1225e66
2013-05-01 21:36:43 +00:00
Kees Jongenburger
dfbe2f16ee build:Updated proto.dev from mkproto -s /dev
proto.dev was missing some entries and is currently used as base version for
the ARM root file system.
2013-04-29 11:47:40 +02:00
Kees Jongenburger
e6bac75a8b ARM:Rename ARM_BIG_PAGE to ARM_SECTION.
The natural term to use when talking about MINIX big pages on ARM
is SECTION. A section is a level 1 page table entry pointing to
a 1MB area.

Change-Id: I9bd27ca99bc772126c31c27a537b1415db20c4a6
2013-04-29 11:42:26 +02:00
Ben Gras
236a320240 mkfs.mfs: double indirect block support
. allows >4MB files for default 4k block size

Change-Id: I09277fc3704fac1a44af82202c11435dc7ae8d55
2013-04-27 16:25:45 +02:00
Ben Gras
f497d9b9c4 ext2: annotate cache blocks with inode metadata
. allows mmap() to work on it

Change-Id: I33af8b86ebb408d971478d00ed2caacf15afc7a5
2013-04-26 15:39:07 +00:00
Ben Gras
f0cc010614 libminixfs, mfs, ext2: may re-evaluate cache size
libminixfs may now be informed of changes to the block usage on the
filesystem. if the net change becomes big enough, libminixfs may
resize the cache based on the new usage.

	. update the 2 FSes to provide this information to libminixfs

Change-Id: I158815a11da801fd5572a8de89c9e6c039b82650
2013-04-26 13:57:08 +00:00
Kees Jongenburger
988c7e39c0 libelf:use self hosted m4.
Change-Id: Ia836aa6c43d7ccbb02047e63424923fdc74eb42d
2013-04-24 17:22:59 +02:00
Kees Jongenburger
dca81e0581 toproto:convert special execute bits to chars
Convert the special execute bits of the METALOG file mode entries into mkfs.mfs
compatible u and g flags.
2013-04-24 13:22:45 +02:00
Ben Gras
44f34e53d5 VFS: Implement REQ_BPEEK.
This commit introduces a new request type called REQ_BPEEK. It
requests minor device blocks from the FS.  Analogously to REQ_PEEK,
it requests the filesystem to get the requested blocks into its
cache, without actually copying the result anywhere.

Change-Id: If1d06645b0e17553a64b3167091e9d12efeb3d6f
2013-04-24 10:18:16 +00:00
Ben Gras
0cfff08e56 libexec: mmap support, prealloc variants
In libexec, split the memory allocation method into cleared and
non-cleared. Cleared gives zeroed memory, non-cleared gives 'junk'
memory (that will be overwritten anyway, and so needn't be cleared)
that is faster to get.

Also introduce the 'memmap' method that can be used, if available,
to map code and data from executables into a process using the
third-party mmap() mode.

Change-Id: I26694fd3c21deb8b97e01ed675dfc14719b0672b
2013-04-24 10:18:16 +00:00
Ben Gras
6fa5ce74ce MFS: annotate cache blocks with inode metadata
. use lmfs_* cache functions that provide the cache with inode
	  metadata whenever applicable, i.e. tell the cache code which
	  inode number and in-inode offset a particular cache block
	  corresponds to.
	. needed for mmap implementation

Change-Id: Ic7d3c0c49029880f86a31368278722e907bc2896
2013-04-24 10:18:16 +00:00
Ben Gras
49eb1f4806 vm: new secondary cache code
Primary purpose of change: to support the mmap implementation, VM must
know both (a) about some block metadata for FS cache blocks, i.e.
inode numbers and inode offsets where applicable; and (b) know about
*all* cache blocks, i.e.  also of the FS primary caches and not just
the blocks that spill into the secondary one. This changes the
interface and VM data structures.

This change is only for the interface (libminixfs) and VM data
structures; the filesystem code is unmodified, so although the
secondary cache will be used as normal, blocks will not be annotated
with inode information until the FS is modified to provide this
information. Until it is modified, mmap of files will fail gracefully
on such filesystems.

This is indicated to VFS/VM by returning ENOSYS for REQ_PEEK.

Change-Id: I1d2df6c485e6c5e89eb28d9055076cc02629594e
2013-04-24 10:18:16 +00:00
Ben Gras
7421728360 VM: memtype fix
Memory types in VM are described by methods. Each mapped region has
a type, and all pages instantiated get that type on creation.
Individual page types has to be able to change though. This commit
changes the code to use the memory types of the individual pages,
where appropriate, instead of just the higher-level region, in case
it has changed. This is needed to e.g. support future copy-on-write
MAP_PRIVATE mmap modes.

Change-Id: I5523db14ac036ec774a54392fb67f9acb8725731
2013-04-24 10:18:15 +00:00
Ben Gras
9e88c40e40 VM: 64-bit mmap()
Some (backwards-compatible) changes in mmap() call message fields
that allow for a 64-bit offset. minix_mmap() takes an off_t and
minix_mmap64() takes a u64_t. Some mmap() work in VM goes into a
separate function, using the new fields, so that that can be re-used
when files are to be mapped (future commit).

Change-Id: Ifb77a90b593dd3c33cf81b396068e4da1ec5fb1c
2013-04-24 10:18:15 +00:00
Ben Gras
75c5fe4c26 filesystems: return ENOSYS for REQ_PEEK
The filesystems already implement REQ_PEEK, but do not fully
use the new filesystem cache code yet. (Because it isn't committed
yet..) REQ_PEEK should be disabled for them until they do.

This indicates to VFS that they are not annotating their cache
blocks (in VM) with inode number/offset info, and therefore mmap()
shouldn't succeed on any of their files. (Most importantly exec()
won't fallback elegantly otherwise.)

Change-Id: Ic57ee422864b4bbc031eadba32973270907b02fd
2013-04-24 10:18:10 +00:00
Ben Gras
adf2032bc0 vm: remove secondary cache code
This commit removes the secondary cache code implementation from
VM and its usage from libminixfs. It is to be replaced by a new
implementation.

Change-Id: I8fa3af06330e7604c7e0dd4cbe39d3ce353a05b1
2013-04-24 10:18:10 +00:00
Xiaoguang Sun
20e6c9329f Change function prototype to use endpoint_t instead of int 2013-04-23 17:15:15 +02:00
Kees Jongenburger
8a3677def0 build:refactor arm_sdimage.sh as to no longer require root.
There where two reasons for needing root access while making ARM builds. The
first reason was that we needed root permissions to create device nodes and
assign permissions to files. The second reason we needed root was that we used
loop mounted files to copy the build output to the file backed storage.

The first problem is solved by building in unprivileged mode (adding -U) during
compilation. The second problem is solved by using mcopy to put files on the
fat partition and use the mkfs.mfs proto files to create MFS file system.

Future work might be to replace this script by the netbsd makefs tool.

Change-Id: Ibba61d4cad053f7c1ede26169d5214065a4939d8
2013-04-23 11:24:12 +02:00
Kees Jongenburger
fd7bb1786e build:add tool to convert from METALOG to proto files.
Change-Id: I5c4e15b1a2bac5a4ca3a0530f8d6aa2643ac720a
2013-04-21 21:36:30 +02:00
Kees Jongenburger
f8dbcfa7f2 build:add script to sort netbsd sets and sort minix/mi.
Change-Id: I309d8bbcfe2f5b6d5e1ae7626422c4402985ad1d
2013-04-21 21:35:49 +02:00
Kees Jongenburger
fcb66ed1b5 run script:return 1 upon test failure.
Change-Id: If3c9d636a04da7f7b4098b58d29c492c664dab1a
2013-04-21 20:18:11 +02:00
Ben Gras
32a4e0d84d many new tests
. test70: regression test for m_out vfs race condition

The following tests use testcache.c to generate test i/o
patterns, generate random write data and verify the reads.

	. test71: blackbox full-stack test of FS operation, testing
	  using the regular VFS interface crazy i/o patterns
	  with various working set sizes, triggering only
	  primary cache, also secondary cache, and finally
	  disk i/o and verifying contents all the time
	. test72: unit test of libminixfs, implementing
	  functions it needs from -lsys and -lblockdriver
	  and the client in order to simulate a working
	  cache client and backend environment.
	. test73: blackbox test of secondary vm cache in isolation

Change-Id: I1287e9753182b8719e634917ad158e3c1e079ceb
2013-04-19 16:21:48 +00:00
Ben Gras
072d916c1c vfs: fix null deref, pfs: add fchmod()
. vfs read_only() assumes vnode->v_vmnt is non-NULL, but it can
	  be NULL sometimes
	. e.g. fchmod() on UDS triggered NULL deref; add a check and
	  add REQ_CHMOD to pfs so unix domain sockets can be fchmod()ded
	. add to test56

Change-Id: I83c840f101b647516897cc99fcf472116d762012
2013-04-19 17:06:56 +02:00
Kees Jongenburger
04ad4fa1a2 mkfs.mfs:add comment about the overflow check.
Change-Id: I1f09c3a62377164296037f6dc9b7874023123311
2013-04-19 13:10:06 +02:00
Kees Jongenburger
a4eca06cca mkfs.mfs:fix to allow to fill larger file systems.
Allow the number of written blocks to exceed 8 * block_size.

Change-Id: I301ce46a92d86c6220c3af2acee22db26a4b4496
2013-04-19 13:10:06 +02:00
Kees Jongenburger
15bb31ed99 mkfs.mfs:remove MFS V1 related headers.
Change-Id: I46a8128f54112ee0092ef60160d88cadd0038be0
2013-04-19 13:10:06 +02:00