cahirwpz / mimiker Goto Github PK
View Code? Open in Web Editor NEWSimple unix-like operating system for education and research purposes
Home Page: https://mimiker.ii.uni.wroc.pl
License: BSD 3-Clause "New" or "Revised" License
Simple unix-like operating system for education and research purposes
Home Page: https://mimiker.ii.uni.wroc.pl
License: BSD 3-Clause "New" or "Revised" License
For reference look at sys/malloc.h and kern/kern_malloc.c. Please also read malloc(9) manual.
Depends on #42
#define MB_MAGIC 0xC0DECAFE
typedef struct mem_block {
uint32_t mb_magic; /* if overwritten report a memory corruption error */
uint32_t mb_size; /* size < 0 => free, size > 0 => alloc'd */
TAILQ_HEAD(, mem_block) mb_list;
uint8_t mb_data[0];
} mem_block_t;
#define MB_UNIT sizeof(mem_block_t)
#define MA_SINGLE 0 /* Single large block allocated within arena. */
#define MA_BLOCKS 1 /* Arena provides space for small blocks. */
/* Allocation larger than MA_THRESHOLD will be handled by allocation of single arena. */
#define MA_THRESHOLD (2 * VM_PAGESIZE)
typedef struct mem_arena {
TAILQ_ENTRY(mem_arena) ma_list;
uint16_t ma_pages; /* Size in pages. */
uint16_t ma_flags;
TAILQ_HEAD(, mem_block) ma_freeblks;
TAILQ_HEAD(, mem_block) ma_usedblks;
} __attribute__((aligned(MB_UNIT))) mem_arena_t;
/* Flags to malloc */
#define M_WAITOK 0x0000 /* ignore for now */
#define M_NOWAIT 0x0001 /* ignore for now */
#define M_ZERO 0x0002 /* clear allocated block */
typedef struct malloc_pool {
SLIST_ENTRY(malloc_pool) mp_next; /* Next in global chain. */
uint32_t mp_magic; /* Detect programmer error. */
const char *mp_desc; /* Printable type name. */
TAILQ_HEAD(, mem_arena) *mp_arena; /* First managed arena. */
} malloc_pool_t;
/* Defines a local pool of memory for use by a subsystem. */
#define MALLOC_DEFINE(pool, desc) \
malloc_pool_t pool[1] = { \
{ NULL, M_MAGIC, desc, NULL } \
};
#define MALLOC_DECLARE(pool) \
extern malloc_pool_t pool[1]
void malloc_add_arena(malloc_pool_t *mp, void *start, void *end);
void *malloc(size_t size, malloc_pool_t *mp, uint16_t flags);
void *realloc(void *addr, size_t size, malloc_pool_t *mp, uint16_t flags);
void free(void *addr, malloc_pool_t *mp);
Develop a mechanism that will have similar responsibility to FreeBSD's sleepqueue. For reference look at: sleepqueue.h and subr_sleepqueue.h.
Sleep queues provide a mechanism for suspending execution of a thread until some condition is met. Each queue is associated with a specific wait channel when it is active, and only one queue may be associated with a wait channel at any given point in time.
Currently, in void tlb_exception_handler(exc_frame_t *frame)
we do:
log("%s at $%08x, caused by reference to $%08lx!", exceptions[code],
frame->pc, vaddr);
tlb_print();
On numerous occasions already this has caused me problems with tlb_print
looping infinitely (recursively?). It seems as if a tlb exception was triggered while printing. Could this be the case? From my experience, the address referenced (vaddr
) in such troublesome case is usually in range 0xc*******
. Isn't this where we keep our page table?
I've ignored this problem until today, when I found out that removing tlb_print
from tlb_exception_handler
apparently fixes the issues I've experienced - the tlb gets refilled, and any further data access appears correct.
So should we strictly abstain from printing tlb status in the exception handler, or is there something I'm missing?
MIPS32® microAptivTM UP Processor Core Family Software User’s Manual, Chapter 5 and 7.
Use Count Register and Compare Register.
Interrupt handler should just atomically increment a global variable.
It's completely unclear what needs to be implemented to have basic support for signals.
We should perform some code analysis and it will probably start from postsig and sendsig routines. There're (at least) two system calls that are necessary to implement for user-space signals: sigaction and sigreturn.
I'm currently migrating to the new custom toolchain, and I can't get it to build our sources, because it does not come with mips/cpu.h
, and - presumably - other headers. Should we import and include such files in with sources, or is there some other way I should get the new toolchain to work?
Because FPU is not used inside kernel, its state is not saved during user_exc_enter
. However it must get saved and restored properly on context switches as user-space program are allowed to perform floating point computations.
In FreeBSD routines that handle FPU context are named MipsSwitchFPState and MipsSaveCurFPState. However it is not clear when exactly FPU context gets switched.
Please import linker sets either from FreeBSD or DragonFly – whatever you'll find simpler and cleaner. If you need to introduce gcc specific variable attributes please wrap them up and put at the beginning of common.h. Perhaps you'll have to modify linker script as well. Please provide a test that will present how to use linker sets.
def find_triplet():
for triplet in ['mips-mti-elf', 'mipsel-unknown-elf']:
if shutil.which(triplet + '-gcc'):
return triplet
Is the order of this list correct? I thought we prefer our custom toolchain, if available.
Implement a buddy systems for storing physical pages. For reference look at vm/vm_page.h, vm/vm_phys.h, vm/vm_phys.c and mips/include/pmap.h.
/* The largest allocation size is 1MB. */
#define VM_NFREEORDER 9
#define VM_RESERVED 1 /* page cannot be freed */
typedef struct vm_page {
TAILQ_ENTRY(vm_page) freeq; /* free pages of the same order */
TAILQ_ENTRY(vm_page) listq; /* all pages in same object (for vm allocator) */
vm_paddr_t phys_addr;
vm_vaddr_t virt_addr;
size_t size;
uint32_t flags;
...
} vm_page_t;
typedef TAILQ_HEAD(vm_freelist, vm_page) vm_freelist_t;
typedef struct vm_phys_seg {
TAILQ_ENTRY(vm_phys_seg) segq; /* list of physical memory segments (in case we have more than one) */
vm_paddr_t start;
vm_paddr_t end;
vm_freelist_t (*free_queues)[VM_NFREEORDER];
vm_page_t page[0];
} vm_phys_seg_t;
void vm_phys_init();
void vm_phys_add_seg(vm_paddr_t start, vm_paddr_t end);
void vm_phys_reserve(vm_paddr_t start, vm_paddr_t end);
vm_page_t *vm_phys_alloc(size_t order);
void vm_phys_free(vm_page_t* page);
The interface proposal is in klog branch. It's modeled after ktr(9) facility from FreeBSD.
The idea is too keep circular array of klog_entry_t
. Appending to kernel log requires almost to synchronization – just incrementing head pointer will do. The messages will not be printed to the console, unless klog_verbose
parameter is passed to the kernel. Thus such mechanism should have little impact on running kernel.
There're many sources of kernel log messages (for instance: KL_SCHED
, KL_VM
, etc.). A user can specify through klog_mask
kernel parameter which events they want to keep in log buffer. One should be able to print the contents of the buffer by invoking gdb extension script.
I believe prepare_program_stack procedure can be made platform independant. Given exc_frame_t
and call_ctor_t
structure (latter doesn't exist yet) one should be able to fix thread.c:32. Proposal follows:
call_ctor_t call;
call_init(&call, td->td_kframe);
for (int i = 0; i < n; i++)
call_add_argument(&call, va_arg(ap, reg_t));
call_finalize(&call);
Prepare simple TCB that stores only an execution context and a pointer to stack. Allocate two TCBs and run two threads that switch between each other. You'll have to implement an assembly routine that switches threads: void switch_context(TCB *from, TCB *to)
. You can look at setjmp / longjmp implementation - they might prove to be useful starting point.
Find similar functionality in other kernels (Minix / *BSD / Linux) and discuss them here. Propose your data structures and functions.
Though stdc/ctype/ctype_.c
file exists, there're no isspace
and family functions available. Please carefully adapt OpenBSD implementation from isctype.c and ctype.h. After that it'll be possible to take strtol.c function from OpenBSD, which is going to be handy if we want to parse kernel arguments.
EDIT: strtol.c from FreeBSD's libkern is preferred as it does not use errno
.
For reference look at proc.h. Thread structure has been already defined in thread.h. Thread handling functions are implemented in kern_thread.c.
thread_t *thread_create(const char *name, void (*fn)());
void thread_exit() __attribute__((noreturn));
void thread_destroy(thread_t *);
int thread_join(thread_t *);
QEMU won't load ELF file created with current linker script. Thus kernel file has to be converted to S-Record format. It'd be nice to identify the source of problem and fix it.
Read CP0 registers to find out:
Depends on #32
A task queue is needed to run bottom halves of interrupt handlers. A top half of interrupt is executed during interrupt request handling. Time consuming and possibly blocking work is deferred to be executed within high priority kernel thread. Interface proposal:
typedef void task_fn_t(void *data);
struct task_queue {
STAILQ_ENTRY(task_queue) next;
work_fn_t *func;
void *data;
};
void task_queue_add(struct task_queue *queue, task_fn_t *task, void *data);
void task_queue_run(struct task_queue *queue);
Find similar functionality in other kernels (Minix / *BSD / Linux) and discuss them here. Propose your data structures and functions.
Certainly we can. But do we really need it? What are the implications?
More to read here: http://wiki.osdev.org/C++
This is a basic structure to maintain a list of threads ready to be executed. On top of it a simple scheduler should be implemented. For reference look at: runq.h and kern_switch.c. The only dependency for this task is to have a kernel thread structure defined.
We need one directory to hold all header files.
This is simple build tree restructuring task.
It's desired to use standard C function within kernel (like strings.h
, stdlib.h
and so on) - there's no real benefit in implementing them by ourselves.
In the future It'd be even better if we could devise kprintf function for debugging purposes. Obviously I expect this function to reuse some code from newlib.
Write tlb_identity_mapping()
routine in C that will recreate default memory management setup (figure 4-4 from PIC32MZ EC documentation). Use functions described in "Codescape GNU Tools for MIPS Programmer's Guide", chapter 13.
After you turn on memory translation, make sure you get TLB exception when invalid access happens. Then try to create a mapping in user segment (addresses up to 2^31-1) and verify it works.
This is a prerequisite for a task where you will actually need to define page table data structure and write TLB refill handler.
Per this comment, we need a simple implementation of a generic bitset type, and a collection of basic operations on bitsets. An example implementation of such template macros is available in freebsd sources.
Read the document listed below and try to merge following functionality with kernel startup routine:
init_caches.S
init_cp0.S
init_gpr.S
init_tlb.S
Boot-MIPS: Example Boot Code for MIPS® Cores, code archive: Boot-Mips.zip
Look at the files in M14Kc
directory for reference. Note that microAptiv is superset of M14Kc architecture. Your implementation may be written in C.
Having heard that @dyniec has problems with QEMU, I spent this day investigating what happens when running mimiker on QEMU, and I've found some pretty interesting stuff. Obviously, as it always turns out, it is by mere luck that the system runs okay-ish on OVPsim.
So the first problem is something we were already aware of: QEMU always jumps to ebase + 0x200
whenever any interrupt arrives. Vectored interrupts are incorrectly supported in QEMU (see: helper.c:674), and we can either fix that ourself (shouldn't be difficult, but I doubt we want to maintain our qemu fork), or prepare two variants of our exception handlers - one for QEMU, using a single interrupt address ebase + 0x200
.
As for now, this is how we process such interrupt:
irq0:
1: j b1
nop
which causes an endless loop whenever QEMU emits an interrupt. So in order to temporarily fix this, I changed it into:
irq0:
1: j irq7
nop
since the clock interrupt is the only interrupt we currently use anyway.
But that led to even more confusing problems. To explain their cause, please have a look at how we currently do kernel_boot
, taking note on when are interrupts enabled:
cpu_init();
pcpu_init();
pci_init();
pm_init();
intr_init(); // <------- This calls enable_interrupts
callout_init();
tlb_init();
pmap_init();
vm_object_init();
vm_map_init();
sched_init();
sleepq_init();
mips_clock_init(); // <------ This enables clock interrupt, also disables and re-enables interrupts
kprintf("[startup] subsystems initialized\n"); // <----- This takes, relatively, a long time
thread_init((void (*)())main, 2, argc, argv);
Thus it is totally possible for a clock interrupt to arrive while we are printing "subsystems initialized", and it happens quite often when running mimiker under QEMU. Suppose that happens. To process an interrupt, we start by calling kern_exc_enter
, which stores some data onto the stack, but also accesses current thread data (LOAD_PCPU
and PCPU_CURTHREAD
). However, _pcpu_data.currthread
was not set yet, because thread_init
wasn't called yet! This, naturally, lead to immediate crash by dereferencing a null pointer.
Thus we need to either:
kernel_boot
may not enable interrupts, and may not enter critical sections. Currently only mips_clock_init()
does that. However, thread_init
uses generic context switching functions to run the first thread, and these use (among others) vm_map_activate
and pmap_activate
, which use a critical section. Therefore we would also need a clever way of starting the first thread that does not re-enable interrupts in the process, and we would enable them in main
(or a pre-main
)._pcpu_data.currthread
much earlier, so that interrupts may be enabled during the startup process. I'm not sure how to do that. thread_init
does not set currthread
directly, it happens when ctx_boot
runs ctx_resume
- which we can't call earlier. Choosing this option would require us to reorganize the thread_init
process, possibly complicating it a bit.I don't know what would be the better choice, or if there are other options (I suppose that removing references to currthead
in kern_exc_enter
is impossible).
A side-problem I noticed (I tested QEMU with the first proposed fix applied in a brute-force way) is that QEMU triggers interrupts much more frequently than we can handle, so a new interrupt arrives before we are done with the previous one. This may be caused by either:
a) QEMU incorrectly triggerring too frequent clock interrupts
b) QEMU triggering interrupts we have not enabled
c) QEMU providing incorrect values for clock interrupts, causing us to spend infinite amount of time when processing clock interrupts.
I'll investigate this side-problem further once we get interrupts to work correctly.
Extend implementation of mtx_lock
/ mtx_unlock
to support:
Please download OVPsim simulator and run OVPsim.20151203.0.Linux32.exe
installer. Make sure you read README.Linux.txt
file and set up your environment correctly by calling setupImperas
from ${IMPERAS_HOME}/bin/setup.sh
.
Create a symbolic link to Malta platform simulator in one of directories listed in ${PATH}
. ${IMPERAS_HOME}/lib/Linux32/ImperasLib/mips.ovpworld.org/platform/MipsMalta/1.0/platform.Linux32.exe
There's a documentation for simulator provided in following files:
${IMPERAS_HOME}/ImperasLib/source/mips.ovpworld.org/platform/MipsMalta/1.0/doc/Imperas_Platform_User_Guide_MipsMalta.pdf
${IMPERAS_HOME}/ImperasLib/source/mips.ovpworld.org/processor/mips32/1.0/doc/OVP_Model_Specific_Information_mips32_4KEc.pdf
Malta Board is well supported by many existing kernels and has comprehensive documentation:
Processor core is documented in:
kmalloc
and kfree
must be made thread-safe,kmalloc
and kfree
should automatically shrink or grow memory pool unless M_NOWAIT
flag is specified,kmalloc_init
should accept minimum and maximum size in pages,I think I encountered a bug in your code. I modified the code to show how to reproduce the bug:
Steps to reproduce:
make clean
.make qemu
smallclib.a
is missing.Using make all
to build the .elf file will, however, succeed, because the all
target depends on both smallclib
and .srec
file, and they happen to be build in the correct order.
Note: This issue was introduced by 870cbfd
Now that we have a generic rule for building any .elf file, I sincerely have no idea how to add the missing dependency into Makefile.common. Requiring smallclib.a
is not a universal property of all .elf files, so it probably should not be added to the generic .elf pattern rule.
I'm in the process of setting up a comfortable development environment, and I have noticed a potential problem with tags generation. Currently the relevant toolchain files are found by:
find $(SYSROOT)/mips-mti-elf/include -type f -iname 'mips*'
find $(SYSROOT)/lib/gcc/mips-mti-elf/*/include -type f -iname '*.h'
which is mostly fine, but it doesn't include numerous files from $(SYSROOT)/mips-mti-elf/include/mips/
, including m32c0.h
or cpu.h
- and we do use tags from these files. Would it be fine if I made a simple PR that fixes this by expanding the search to include the entire [...]/include/mips
directory?
And while I'm at it, would it be okay to generate both vi-style and emacs-style tags files (and keep them in tags
and etags
)?
Simple interface should be devised for file operations. It'll be a subset of fileops functions, namely: open
, read
, stat
, seek
and close
. Each file is assigned vnode structure, which is bound to filesystem's object representation.
After necessary abstractions are introduced, one can implement very simple filesystem. It's going to be a ramdisk in cpio(5) format visible under /initrd
location.
I noticed this issue while implementing a bootstrap memory allocator, but it can certainly haunt us in any context.
Our intention is to place the kernel stack at the end of the RAM (as it grows downwards). This is expressed in pic32mz.ld
with:
_estack = ORIGIN(ram) + LENGTH(ram);
This value apparently makes sense, pic32mz.map
confirms:
0x00000000a0080000 _estack = (ORIGIN (ram) + 0x80000)
which is, indeed, 512kB after RAM's beggining. This address value is correct.
To initialize stack pointer, we use the la
pseudo-instruction: (startup.S
)
_start: la $sp, _estack // Set stack pointer. la $gp, _gp // Prepare global pointer.
I would expect that the linker will set the value in this instruction so that the effect of this instruction will be setting sp
to 0xa0080000
. However, it is not:
(gdb) disas _start Dump of assembler code for function _start: 0xbfc00500 <+0>: lui sp,0xa000 0xbfc00504 <+4>: addiu sp,sp,4236 0xbfc00508 <+8>: lui gp,0xa000 0xbfc0050c <+12>: addiu gp,gp,0 [...]
so in fact (in this case) the stack pointer will be set instead to 0xa000108c
. And it indeed is:
(gdb) info registers [...] t8 t9 k0 k1 gp sp s8 ra R24 00000000 00000000 00000000 00000000 00000000 a000108c 00000000 00000000 [...]
Obviously, this is incorrect. It will cause all sorts of issues, starting with the fact that the stack ends up having just 1kiB before it crashes onto .bss
. I originally found this issue when I tried to allocate more than 1kiB with the bootstrap memory allocator, and cleared the allocated region, which happened to override stack frames.
I have no idea what may be causing this. The only interesting thing I noticed while investigating this is that apparently that incorrect address is always _end + 1kiB
, so increasing .data
or .bss
by N bytes pushes the stack these N bytes further.
Please read uvm(9) from NetBSD.
Note that some functionality from sys/time.h may have to be stolen. For reference look at _callout.h, callout.h and kern_timeout.c.
typedef void (*timeout_t)(void *);
typedef int64_t sbintime_t;
typedef struct callout {
TAILQ_ENTRY(callout) c_link;
sbintime_t c_time; /* ticks to the event */
timeout_t *c_func; /* function to call */
void *c_arg; /* function argument */
uint16_t c_flags;
...
} callout_t;
#define CALLOUT_ACTIVE 0x0001 /* callout is currently active */
#define CALLOUT_PENDING 0x0002 /* callout is waiting for timeout */
#define callout_active(c) B_SET((c)->c_flags, CALLOUT_ACTIVE)
#define callout_deactivate(c) B_CLR((c)->c_flags, CALLOUT_ACTIVE)
void callout_init();
void callout_setup(callout_t *handle, sbintime_t time, timeout_t fn, void *arg);
void callout_stop(callout_t *handle);
/* Process all timeouts, should be called from hardclock or softclock. */
void callout_process(sbintime_t now);
Please read pool(9) from NetBSD.
Bitmapped memory allocator for objects of fixed size.
typedef struct pool_item_header {
LIST_ENTRY(pool_item_header) ph_pagelist; /* pool page list */
uint16_t ph_nused; /* # of chunks in use */
uint16_t ph_start; /* start offset in page */
uint32_t ph_bitmap[0];
};
LIST_HEAD(pool_page_list, pool_item_header);
typedef struct pool {
pool_page_list_t pp_empty_pages;
pool_page list_t pp_full_pages;
pool_page_list_t pp_part_pages; /* Partially-allocated pages */
size_t pp_itemsize; /* Size of item */
size_t pp_align; /* Requested alignment, must be 2^n */
size_t pp_npages; /* # of pages allocated */
size_t pp_nitems; /* number of available items in pool */
} pool_t;
void pool_init(pool_t *pp, size_t itemsize, size_t align);
void pool_destroy(pool_t *pp);
void *pool_get(pool_t *pp);
void pool_put(pool_t *pp, void *item);
int pool_add(pool_t *pp, int nitems);
I've been looking at vm_page_fault
, and I am very confused about the way it calls pmap_map
at the end:
vm_addr_t fault_page = fault_addr & -PAGESIZE;
vm_addr_t offset = fault_page - entry->start;
vm_page_t *frame = vm_object_find_page(entry->object, offset);
if (!frame)
frame = obj->pgr->pgr_fault(obj, fault_page, offset, fault_type);
pmap_map(map->pmap, fault_addr, fault_addr + PAGESIZE, frame->paddr,
entry->prot);
return 0;
Take note of the second and third argument to pmap_map
(start
and end
). I suppose the range we want to map is fault_page, fault_page + PAGESIZE
, not fault_addr, fault_addr + PAGESIZE
. I also see that pmap_map
requires both addresses to be page-alligned. So in current case, if the first address accessed does not happen to be page-alligned (it happens to be in our current test-cases...), pmap_map
will crash.
I'm sorry I'm creating these questions as issues instead of just sending a PR with a fix, but I am genuinely unsure about how this is intended to work, so I need a confirmation that my findings are correct before I prepare a fix.
Make possible for any part of kernel to register an interrupt handler. An interrupt handler returns true when the handler processed the interrupt. For reference look at sys/interrupt.h, kern/kern_intr.c and sys/bus.h.
#define FILTER_STRAY 0x01 /* this device did not trigger the interrupt */
#define FILTER_HANDLED 0x02 /* the interrupt has been fully handled and can be EOId */
/*
* The filter routine is run in primary interrupt context and may not
* block or use regular mutexes. The filter may either completely
* handle the interrupt or it may perform some of the work and
* defer more expensive work to the regular interrupt handler.
*/
typedef int driver_filter_t(void*);
typedef void driver_intr_t(void*);
/*
* Describe a hardware interrupt handler.
* Multiple interrupt handlers for a specific event can be chained together.
*/
typedef struct intr_handler {
TAILQ_ENTRY(intr_handler) ih_next; /* Next handler for this event. */
driver_filter_t *ih_filter; /* Filter handler function. */
driver_intr_t *ih_handler; /* Handler function. */
intr_event_t *ih_event; /* Event we are connected to. */
void *ih_argument; /* Argument to pass to handlers. */
char *ih_name; /* Name of handler. */
prio_t ih_prio; /* Priority of this handler. */
} intr_handler_t;
/*
* Describe an interrupt event.
* An event holds a list of handlers.
*/
typedef struct intr_event {
TAILQ_ENTRY(intr_event) ie_list;
TAILQ_HEAD(, intr_handler) ie_handlers; /* Interrupt handlers. */
char *ie_name; /* Individual event name. */
uint8_t ie_irq; /* Physical irq number. */
...
} intr_event_t;
void intr_init();
void intr_event_init(intr_event_t *ie, uint8_t irq, char *name)
void intr_event_add_handler(intr_event_t *ie, intr_handler_t *ih);
void intr_event_remove_handler(intr_handler_t *ih);
void intr_event_execute_handlers(intr_event_t *ie);
Develop a mechanism that will have similar responsibility to FreeBSD's turnstiles described in the book. For reference look at: sleepqueue.h and subr_turnstile.c.
Implementation of turnstiles is used to hold queue of threads blocked on non-sleepable locks.
Sleepable locks use condition variables to implement their queues. Turnstiles differ from a sleep
queue in that turnstile queue's are assigned to a lock held by an owning thread. Thus, when one
thread is enqueued onto a turnstile, it can lend its priority to the owning thread.
I predict that in long run implicit size of integer types will cause us a headache. I propose to introduce a kernel source wide policy to adopt int*_t
and uint*_t
types among others available in stdint.h.
We need it badly, now. Two major requirements are:
There're a few solutions:
Just choose bare minimum interface and present it as a header file, don't start implementation yet. I'd go for doubly linked circular list variant - it seems to be the most versatile.
Any plans to add support for pci interrupts?
I've tried working on this myself (on another codebase) and I'm having troubles..
(interrupt gets fired once, then never fires again)
thanks!
Move _start
to startup.s
file. Make sure .data
section is copied to RAM before code jumps to main
function. Loop forever when main
exits. Fix Makefile
and linker script respectively.
I have probably found a test case that proves we are not switching vm spaces correctly:
void vm_map_switching_demo(){
vm_map_t *umap1 = get_active_vm_map(PMAP_USER);
vm_map_t *umap2 = get_active_vm_map(PMAP_USER);
vm_addr_t start = 0x1001000;
vm_addr_t end = 0x1001000 + 2 * PAGESIZE;
vm_map_entry_t *data1 =
vm_map_add_entry(umap1, start, end, VM_PROT_READ|VM_PROT_WRITE);
vm_map_entry_t *data2 =
vm_map_add_entry(umap2, start, end, VM_PROT_READ|VM_PROT_WRITE);
data1->object = default_pager->pgr_alloc();
data2->object = default_pager->pgr_alloc();
volatile char* p = (char*)(start + 0x10);
set_active_vm_map(umap1);
*p = 0x10;
set_active_vm_map(umap2);
*p = 0x20;
set_active_vm_map(umap1);
assert(*p == 0x10);
set_active_vm_map(umap2);
assert(*p == 0x20);
log("Test 2 passed.");
}
When I add it to ./vm_map.c
, I get:
[panic] vm_map.c:66 Assertion '*p == 0x10' failed!
which suggests the second switch to umap
did not happen. I suppose the reason for this may be that the page corresponding to *p
is still in TLB. We're not yet flushing TLB on vm map switch, are we? The fix may be not trivial, as simply adding tlb_invalidate_all
to set_active_vm_map
starts a cascade of recursive page faults.
@coodie could you take a look at this? Is my test case even correct?
We are quite far from using user mode, but virtual address space for kernel is going to be useful. For example, we could make use of memory protection to detect stack overflows (this is impossible without VA). This is going to be easier to implement higher level memory allocator using VA, since buddy system we have returns pages of size of power of two, we might try to create contiguous mappings to these of other sizes which aren't powers of two.
We also already know how to use TLB on our board. I propose two-level page table as described in Memory Systems: Cache, DRAM, Disk book in chapter 31.1.3 (Bottom-up traversal page table).
Our board has some support for this task. For example there is Context register which contains base address of page table.
For simplicity, I wouldn't use variable sized pages to be put in TLB (in order to reduce TLB misses by mapping big segments of memory), unless this turns out to be easier than we might expect. Another solution would be to use bigger pages for kernel (like 2MB pages).
Some references are here:
http://nxr.netbsd.org/xref/src-freebsd/sys/i386/include/pmap.h
http://nxr.netbsd.org/xref/src-freebsd/sys/mips/mips/pmap.c
http://nxr.netbsd.org/xref/src-freebsd/sys/vm/vm_map.h
https://www.freebsd.org/cgi/man.cgi?query=vm_map
I also recommend reading 6.4 of The design and Implementation of FreeBSD operating system
This isn't ready yet. I'll update this comment as I gain more information.
For reference look at sched.h and sched_4bsd.c.
/*
* This is an extension to the thread structure
* and is tailored to the requirements of this scheduler.
*/
typedef struct td_sched {
ts_timeslice; /* As assigned based on priority. */
ts_remaining; /* Remaining part of time slice. */
...
} td_sched_t;
/* Set the priority to an absolute value. */
void sched_prio(thread_t *td, prio_t prio);
/* Record the sleep time. */
void sched_sleep(thread_t *td);
/* Schedule a thread to resume execution and record how long it voluntarily slept. */
void sched_wakeup(thread_t *td);
/*
* Switch threads. This function has to handle threads coming in
* while blocked for some reason or running.
*/
void sched_switch(thread_t *oldtd, thread_t *newtd);
/*
* Select the target thread queue and add a thread to it.
* Request preemption if required.
*/
void sched_add(thread_t *td);
/* Choose the highest priority thread to run. */
thread_t *sched_choose();
/* Runs main loop of the scheduler. */
void sched_run();
This issue was submitted purely for design and discussion purposes.
We want to address some of the deficiencies of physmem
, pmap
and vm_map
subsystems including:
modified
and referenced
bits for MIPS architectureCorresponding NetBSD 9 manuals: pmap(9), uvm(9) and FreeBSD 11 manuals: pmap(9), vm_map(9).
I'd like to avoid situation where we mix up physical and virtual without any control, here's first idea how to handle it:
typedef union {
void *ptr;
intptr_t addr;
} paddr_t;
typedef union {
void *ptr;
intptr_t addr;
} vaddr_t;
/* by explicit casting one can convert underlying type to paddr_t */
paddr_t foo1(void *addr) {
return (paddr_t)addr;
}
/* by virtue of union's field access one can get value of underlying type */
void *foo2(paddr_t pa) {
return pa.ptr;
}
/* and this is the way how one can convert between virtual and physical address */
paddr_t foo3(vaddr_t va) {
return (paddr_t)va.ptr;
}
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.