Commit graph

89 commits

Author SHA1 Message Date
Ben Gras
50e2064049 No more intel/minix segments.
This commit removes all traces of Minix segments (the text/data/stack
memory map abstraction in the kernel) and significance of Intel segments
(hardware segments like CS, DS that add offsets to all addressing before
page table translation). This ultimately simplifies the memory layout
and addressing and makes the same layout possible on non-Intel
architectures.

There are only two types of addresses in the world now: virtual
and physical; even the kernel and processes have the same virtual
address space. Kernel and user processes can be distinguished at a
glance as processes won't use 0xF0000000 and above.

No static pre-allocated memory sizes exist any more.

Changes to booting:
        . The pre_init.c leaves the kernel and modules exactly as
          they were left by the bootloader in physical memory
        . The kernel starts running using physical addressing,
          loaded at a fixed location given in its linker script by the
          bootloader.  All code and data in this phase are linked to
          this fixed low location.
        . It makes a bootstrap pagetable to map itself to a
          fixed high location (also in linker script) and jumps to
          the high address. All code and data then use this high addressing.
        . All code/data symbols linked at the low addresses is prefixed by
          an objcopy step with __k_unpaged_*, so that that code cannot
          reference highly-linked symbols (which aren't valid yet) or vice
          versa (symbols that aren't valid any more).
        . The two addressing modes are separated in the linker script by
          collecting the unpaged_*.o objects and linking them with low
          addresses, and linking the rest high. Some objects are linked
          twice, once low and once high.
        . The bootstrap phase passes a lot of information (e.g. free memory
          list, physical location of the modules, etc.) using the kinfo
          struct.
        . After this bootstrap the low-linked part is freed.
        . The kernel maps in VM into the bootstrap page table so that VM can
          begin executing. Its first job is to make page tables for all other
          boot processes. So VM runs before RS, and RS gets a fully dynamic,
          VM-managed address space. VM gets its privilege info from RS as usual
          but that happens after RS starts running.
        . Both the kernel loading VM and VM organizing boot processes happen
	  using the libexec logic. This removes the last reason for VM to
	  still know much about exec() and vm/exec.c is gone.

Further Implementation:
        . All segments are based at 0 and have a 4 GB limit.
        . The kernel is mapped in at the top of the virtual address
          space so as not to constrain the user processes.
        . Processes do not use segments from the LDT at all; there are
          no segments in the LDT any more, so no LLDT is needed.
        . The Minix segments T/D/S are gone and so none of the
          user-space or in-kernel copy functions use them. The copy
          functions use a process endpoint of NONE to realize it's
          a physical address, virtual otherwise.
        . The umap call only makes sense to translate a virtual address
          to a physical address now.
        . Segments-related calls like newmap and alloc_segments are gone.
        . All segments-related translation in VM is gone (vir2map etc).
        . Initialization in VM is simpler as no moving around is necessary.
        . VM and all other boot processes can be linked wherever they wish
          and will be mapped in at the right location by the kernel and VM
          respectively.

Other changes:
        . The multiboot code is less special: it does not use mb_print
          for its diagnostics any more but uses printf() as normal, saving
          the output into the diagnostics buffer, only printing to the
          screen using the direct print functions if a panic() occurs.
        . The multiboot code uses the flexible 'free memory map list'
          style to receive the list of free memory if available.
        . The kernel determines the memory layout of the processes to
          a degree: it tells VM where the kernel starts and ends and
          where the kernel wants the top of the process to be. VM then
          uses this entire range, i.e. the stack is right at the top,
          and mmap()ped bits of memory are placed below that downwards,
          and the break grows upwards.

Other Consequences:
        . Every process gets its own page table as address spaces
          can't be separated any more by segments.
        . As all segments are 0-based, there is no distinction between
          virtual and linear addresses, nor between userspace and
          kernel addresses.
        . Less work is done when context switching, leading to a net
          performance increase. (8% faster on my machine for 'make servers'.)
	. The layout and configuration of the GDT makes sysenter and syscall
	  possible.
2012-07-15 22:30:15 +02:00
Thomas Veerman
39dfb2bc55 Don't build ext2 ramdisk for cross compilation
. as mkfs.mfs is now cross compilable, we can remove the ext2 ramdisk
  special case for cross compilation.
2012-06-18 10:54:56 +00:00
Thomas Veerman
791aa4e16f Cross compile MFS ramdisk 2012-06-18 10:54:55 +00:00
Thomas Veerman
9d04c193d2 Import NetBSD xinstall
Also, fix mk files for cross compilation.
2012-06-18 10:54:50 +00:00
Thomas Veerman
be9962e21f Add mk and h files needed for cross compilation 2012-06-18 10:53:37 +00:00
Thomas Veerman
03ea1c1676 Dynamically determine which ramdisk to use
.Also, rename ext2_ramdisk to ramdisk_ext2.
2012-06-18 10:53:36 +00:00
Thomas Veerman
f93afa00e9 Remove MINIXSRCDIR and use NETBSDSRCDIR
NETBSDSRCDIR is used all over the place anyway, and this reduces
our diff with NetBSD a little.
2012-06-18 10:53:35 +00:00
Thomas Veerman
4b94e52787 Fix mk file for cross compilation (2) 2012-06-18 10:53:32 +00:00
Dirk Vogt
7da8d39c06 Fix compiler check 2012-06-18 10:53:27 +00:00
Ben Gras
ad0db93247 no USE_BOOTPARAM 2012-05-31 01:13:27 +02:00
Ben Gras
543adbed3a libarchive import
. clean and nbsd_ports managed import
	. also makes it shared
2012-05-02 13:31:48 +02:00
Ben Gras
4b999f1962 build shared versions of libraries
building defaults to off until clang is updated.

current clang does not handle -shared, necessary to change the ld
invocation to build shared libraries properly. a new clang should be
installed and MKPIC defaults to no unless the newer clang is detected.

changes:

	. mainly small imports of a Makefile or two and small fixes
	  (turning things back on that were turned off in Makefiles)
	. e.g.: dynamic librefuse now depends on dynamic
	  libpuffs, so libpuffs has to be built dynamically too
	  and a make dependency barrier is needed in lib/Makefile
	. all library objects now have a PIC (for .so) and non-PIC
	  version, so everything is built twice.
	. generate PIC versions of the compat (un-RENAMEd) jump files,
	  include function type annotation in generated assembly
	. build progs with -static by default for now
	. also build ld.elf_so
	. also import NetBSD ldd
2012-04-16 05:21:20 +02:00
Ben Gras
e83f7ba2c9 switch to netbsd csu
. file- and functionality-compatible with previous situation
          (FreeBSD csu) (with a crt1.o -> crt0.o symlink in /usr/lib)
        . harmonizes source with netbsd
        . harmonizes linker invocation (e.g. clang) with netbsd
        . helpful to get some arm code in there for the arm port project
2012-04-12 13:26:24 +02:00
David van Moolenbroek
bb4d055fa6 Add libvboxfs: VirtualBox shared folders library 2012-04-09 19:25:18 +02:00
David van Moolenbroek
ef7b484e5c Create SFFS library out of HGFS
This Shared Folders File System library (libsffs) now contains all the
file system logic originally in HGFS. The actual HGFS server code is
now a stub that passes on all the work to libsffs. The libhgfs library
is changed accordingly.
2012-04-09 18:08:26 +02:00
Ben Gras
a5190194a0 build: don't -L/usr/pkg/lib
You might have to update the compiler-rt package! See UPDATING.

	. the purpose of this -L was solely to find compiler-rt, which contains
	  runtime support code for clang-compiled binaries
	. this also makes all other packaged libraries visible, however
	. it is cleaner to isolate the base system from packages, and so
	  compiler-rt puts itself in /usr/pkg/compiler-rt/lib/ too, which the
	  base system henceforth uses exclusively
	. e.g. this solves a link failure when libfetch is installed as a
	  package
	. the new compiler-rt package also puts itself in /usr/pkg/lib for 'old'
	  systems; that is harmless. The benefit of 'new' systems is that the other
	  packages are hidden.
2012-03-27 12:31:30 +02:00
Arun Thomas
ff56906879 Remove obsolete INSTALLFLAGS from makefiles 2012-02-16 23:26:38 +01:00
Ben Gras
39fea0a5b9 install library manpages
. harmonize bsd.lib.mk and bsd.man.mk with netbsd files
	. throw out minix section 3 (library calls) manpages,
	  replaced by netbsd ones that are now installed
2012-02-16 01:48:46 +00:00
Ben Gras
2fe8fb192f Full switch to clang/ELF. Drop ack. Simplify.
There is important information about booting non-ack images in
docs/UPDATING. ack/aout-format images can't be built any more, and
booting clang/ELF-format ones is a little different. Updating to the
new boot monitor is recommended.

Changes in this commit:

	. drop boot monitor -> allowing dropping ack support
	. facility to copy ELF boot files to /boot so that old boot monitor
	  can still boot fairly easily, see UPDATING
	. no more ack-format libraries -> single-case libraries
	. some cleanup of OBJECT_FMT, COMPILER_TYPE, etc cases
	. drop several ack toolchain commands, but not all support
	  commands (e.g. aal is gone but acksize is not yet).
	. a few libc files moved to netbsd libc dir
	. new /bin/date as minix date used code in libc/
	. test compile fix
	. harmonize includes
	. /usr/lib is no longer special: without ack, /usr/lib plays no
	  kind of special bootstrapping role any more and bootstrapping
	  is done exclusively through packages, so releases depend even
	  less on the state of the machine making them now.
	. rename nbsd_lib* to lib*
	. reduce mtree
2012-02-14 14:52:02 +01:00
Thomas Veerman
80c4685324 VFS: replace VFS with AVFS 2012-02-13 16:53:21 +00:00
Ben Gras
21168577a5 add -L for future compiler-rt lib location 2011-12-08 18:07:57 +01:00
David van Moolenbroek
b4d909d415 Split block/character protocols and libdriver
This patch separates the character and block driver communication
protocols. The old character protocol remains the same, but a new
block protocol is introduced. The libdriver library is replaced by
two new libraries: libchardriver and libblockdriver. Their exposed
API, and drivers that use them, have been updated accordingly.
Together, libbdev and libblockdriver now completely abstract away
the message format used by the block protocol. As the memory driver
is both a character and a block device driver, it now implements its
own message loop.

The most important semantic change made to the block protocol is that
it is no longer possible to return both partial results and an error
for a single transfer. This simplifies the interaction between the
caller and the driver, as the I/O vector no longer needs to be copied
back. Also, drivers are now no longer supposed to decide based on the
layout of the I/O vector when a transfer should be cut short. Put
simply, transfers are now supposed to either succeed completely, or
result in an error.

After this patch, the state of the various pieces is as follows:
- block protocol: stable
- libbdev API: stable for synchronous communication
- libblockdriver API: needs slight revision (the drvlib/partition API
  in particular; the threading API will also change shortly)
- character protocol: needs cleanup
- libchardriver API: needs cleanup accordingly
- driver restarts: largely unsupported until endpoint changes are
  reintroduced

As a side effect, this patch eliminates several bugs, hacks, and gcc
-Wall and -W warnings all over the place. It probably introduces a
few new ones, too.

Update warning: this patch changes the protocol between MFS and disk
drivers, so in order to use old/new images, the MFS from the ramdisk
must be used to mount all file systems.
2011-11-23 14:06:37 +01:00
Arun Thomas
3f9ef4ac46 make: set MACHINE_ARCH variable
Also, modify sys.mk so we continue to build with -O
2011-11-18 16:22:25 +01:00
Vivek Prakash
79bfef9aab import NetBSD bzip2 and libbz2 2011-11-18 11:11:04 +01:00
Thomas Veerman
a209c3ae12 Fix a ton of compiler warnings
This patch fixes most of current reasons to generate compiler warnings.
The changes consist of:
 - adding missing casts
 - hiding or unhiding function declarations
 - including headers where missing
 - add __UNCONST when assigning a const char * to a char *
 - adding missing return statements
 - changing some types from unsigned to signed, as the code seems to want
   signed ints
 - converting old-style function definitions to current style (i.e.,
   void func(param1, param2) short param1, param2; {...} to
   void func (short param1, short param2) {...})
 - making the compiler silent about signed vs unsigned comparisons. We
   have too many of those in the new libc to fix.

A number of bugs in the test set were fixed. These bugs were never
triggered with our old libc. Consequently, these tests are now forced to
link with the new libc or they will generate errors (in particular tests 43
and 55).

Most changes in NetBSD libc are limited to moving aroudn "#ifndef __minix"
or stuff related to Minix-specific things (code in sys-minix or gen/minix).
2011-11-14 10:07:49 +00:00
Ben Gras
0c3983b25a update/fix manpage support
. add bsd-style MLINKS to minix man set, restoring aliases
	  (e.g. man add64 -> int64)
	. update daily cron script to run makewhatis and restore makewhatis
	  in man Makefile (makedb), restores functionality of man -k
	. netbsd imports of man, mdocml, makewhatis, libutil, apropos
	. update man.conf with manpage locations, restoring man [-s] <section>
	. throws out some obsolete manpages
2011-09-28 15:24:15 +00:00
Arun Thomas
46581e701a Add MKBOOTPARAM option 2011-09-16 17:31:07 +02:00
Arun Thomas
8a0901c4cb Add MKTRACE 2011-09-07 17:52:48 +02:00
Arun Thomas
4ca68d42a0 Add MKLIVEUPDATE and MKSTATECTL 2011-09-02 16:57:22 +02:00
Arun Thomas
27c0e801f2 Cleaner MKSYSDEBUG option 2011-08-18 15:21:29 +02:00
Thomas Veerman
a6bd3f4a22 Merge AVFS and APFS 2011-08-17 13:40:36 +00:00
Arun Thomas
593e3221fc Rename MKDEBUG to MKSYSDEBUG 2011-08-16 22:30:10 +02:00
Arun Thomas
40592de32d Optionally disable kernel debug code 2011-08-16 17:18:55 +02:00
Ben Gras
cd05a95afc recognize ccc-analyzer as gnu-style compiler
. for clang static analysis
2011-08-10 10:39:45 +00:00
Arun Thomas
86b061078b Build gcov code only if MKCOVERAGE is yes 2011-08-09 10:39:33 +02:00
Arun Thomas
4dece73bf0 bsd.prog.mk: skip 'install -S' rule for GCC/clang 2011-08-08 14:18:25 +02:00
Arun Thomas
862fb8354d minix.ack.mk: use CPPFLAGS for ACK asm files 2011-08-05 14:16:48 +02:00
Arun Thomas
aaefc6f838 Add MKMCONTEXT option 2011-08-02 13:57:31 +02:00
Arun Thomas
f985ecba0d MKEMBED is now MKSMALL 2011-08-01 11:05:17 +02:00
Arun Thomas
ae561b8f12 Add MKAPIC and MKACPI options 2011-07-31 16:22:43 +02:00
Arun Thomas
1a8cf59d04 Add MKWATCHDOG option 2011-07-29 20:37:39 +02:00
Ben Gras
f614d0015a mk: single mk fragment for new libc
. allow commands to use it by setting NEED_NBSDLIBC
	. also add mkfiles to clean and cleandepend so they
	  don't break on missing new mkfiles
2011-07-27 18:23:36 +02:00
Arun Thomas
8a8f27951b sys.mk: Check that MKEMBED is defined 2011-07-26 11:26:26 +02:00
Arun Thomas
1d45d57e40 Add MKEMBED option 2011-07-26 08:54:32 +02:00
Ben Gras
51ffecc181 import elf-only -lcurses
. abandons mixer, gomoku, talk, talkd, top from base system
	. compile top with clang so no ack-compiled program
	  needs -lcurses any more
2011-07-25 11:08:17 +02:00
Arun Thomas
1e56737131 bsd.prog.mk: include minix.gcov.mk 2011-07-22 18:00:09 +02:00
Arun Thomas
06617e7fdf Import NetBSD mkdep 2011-07-22 17:28:27 +02:00
Erik van der Kouwe
e3d0bb59ab Avoid MMX and SSE instructions in the MINIX programs and libraries 2011-07-22 11:59:31 +02:00
Arun Thomas
3eb22ca4d1 MK option to build only system image programs 2011-07-20 09:57:46 +02:00
Antoine LECA
afd132adc9 share/mk: use original stat -qf 2011-07-19 18:38:06 +02:00