willdurand / arvernos Goto Github PK

STR:

1. in kshell, type cat /proc/meminfo
2. run a program like cal
3. type cat /proc/meminfo

I am not sure where it leaks but I expect the number of frames in (1) and (3) to be the same, but it's not the case:

To Fix

src/kernel/console.c:13:23: warning: initializer element is not constant [-Wpedantic]
   13 |                       (vtansi_parser_t){ VTSTATE_ESC, { { 0, 0 } }, 0 },
      |                       ^
src/kernel/console.c:13:23: note: (near initialization for 'vtc.ansiparser')
In file included from src/kernel/console.c:4:
libs/vtconsole/vtconsole.h:16:3: warning: initializer element is not constant [-Wpedantic]
   16 |   (vtattr_t) { false, VTC_DEFAULT_FOREGROUND, VTC_DEFAULT_BACKGROUND }
      |   ^
src/kernel/console.c:14:29: note: in expansion of macro 'VTC_DEFAULT_ATTR'
   14 |                     .attr = VTC_DEFAULT_ATTR,
      |                             ^~~~~~~~~~~~~~~~
libs/vtconsole/vtconsole.h:16:3: note: (near initialization for 'vtc.attr')
   16 |   (vtattr_t) { false, VTC_DEFAULT_FOREGROUND, VTC_DEFAULT_BACKGROUND }
      |   ^
src/kernel/console.c:14:29: note: in expansion of macro 'VTC_DEFAULT_ATTR'
   14 |                     .attr = VTC_DEFAULT_ATTR,
      |                             ^~~~~~~~~~~~~~~~
src/kernel/console.c:16:31: warning: initializer element is not constant [-Wpedantic]
   16 |                     .cursor = (vtcursor_t){ 0, 0 },
      |                               ^
src/kernel/console.c:16:31: note: (near initialization for 'vtc.cursor')

Fixed

src/net/dns.c: In function 'dns_request':
src/net/dns.c:73:29: warning: missing braces around initializer [-Wmissing-braces]
   73 |   struct sockaddr_in addr = {
      |                             ^
......
   76 |     .sin_addr = inet_addr2(interface->dns_ip),
      |                 {
   77 |   };
      |   }

src/fs/debug.c: In function 'debug_write':
src/fs/debug.c:48:25: warning: pointer of type 'void *' used in arithmetic [-Wpointer-arith]
   48 |     printf("%s", &buffer[offset]);
      |                         ^
src/fs/debug.c:48:25: warning: dereferencing 'void *' pointer

Similar to #132, we could add a proxy header to ease the process of porting existing code that uses some of the usual unix/posix stuff.

I am thinking that taking a typical C udp client and compiling it as a "userland" program would make a nice demo. That should involve 3 new syscalls (socket, recfrom, sendto) and interesting challenges around file descriptors, buffers for received packets, etc.

/Git/willOS/./.clang-format: Invalid argument
YAML:23:38: error: invalid boolean
AllowShortIfStatementsOnASingleLine: Never

Repeats for each usage of the Never keyword.

"userland" programs run in kernel mode currently (ring 0) but they should run in ring 3 (user mode) instead.

According to this: http://os.phil-opp.com/remap-the-kernel.html, which is a very helpful resource, I have to remap the kernel to do something better. I have to investigate because I am not sure to understand everything right now.

This should bring a (mostly) safe kernel stack and a working page table module, which is the path to add a virtual memory allocator \o/

This could also be due to a gcc version > 5.4.0 (which is what I have for some reasons locally...)

Reading this thread: https://forum.osdev.org/viewtopic.php?f=15&t=27100 as well as my observations when running willOS, I feel like we could simplify the loader by only considering program headers like it was in an early version.

Liballoc is GPL-licensed which makes it incompatible with your own license, since you can't have non-GPL licenses with GPL code. Do you have any plans to replace or remove liballoc?

We'll need this eventually. We could move the HTON*/NTOH* macros/functions there.

I don't get it.

The fcntl.h header is often used in existing programs/libs. The kernel does not provide it but given that we provide unix-like open, read, write, close functions, we might want to have it. We could only define the constants we want to support for now (O_*).

In the debug logs, we can see attempts to map already mapped pages. I didn't pay too much attention to that before but it is probably because we have contiguous sections and we calculate the same "start" and "end" pages.

In order to implement userland later, I need syscalls. While 32bits OS use interrupts, it seems 64bits use a sysenter instruction.

Steps to reproduce

1. run make debug
2. in willOS, type cat /bo and press enter

What happened?

The content of the file booo is displayed.

Expected?

It should indicate "file not found" because bo is not a file.

This happens because of an invalid strncmp() call, but fixing it breaks other test cases 😞

We should create the following type to differentiate frame numbers vs physical addresses. I suppose creating an optional-like type would make sense.

typedef struct frame {
  bool has_value;
  uint64_t addr;
} frame_t;

The frame numbers will continue to use the uint64_t type.

This will allow us to reimplement the meminfo command as a userland program.

Even today the floppies are still being used, for example - as virtual floppies inside the coreboot open source BIOS. Just imagine: your wonderful OS could be a part of someone's BIOS build! (for coreboot supported motherboard, maybe you have or could get one - see https://www.coreboot.org/Supported_Motherboards )

@willdurand , If you already have a coreboot-supported motherboard, or a real chance to get one, - wouldn't it be cool to be able to launch your own OS straight from the BIOS chip? ;) With one simple command its possible to add any floppy to coreboot BIOS build - and then you see it as a boot entry! Multiple floppies could be added this way (as long as you have enough space left inside the BIOS flash chip, luckily LZMA compression could be used for the stored floppies to reduce their occupied size)

Move ls from kshell to init.

There are 2 TODO about this function. It seems worth implementing now.

Similar to #84, let's have a page_t type:

typedef struct page {
  bool has_value;
  uint64_t addr;
} page_t;

When I run an unmodified version of this repo, the init program will crash:

Do you have any idea what the problem might be or how it could be solved?

The current screen driver is a vga text mode driver. It should probably be renamed as such.

When doing this, we could reuse the existing screen "interface" to have an abstract api for putting stuff on a screen. That would allow us to write a new driver (VESA) in the future.

STR:

1. enable the guard page
2. compile, run
3. type a command like help
4. use the arrow up key to recall the previous command

In most cases, it will produce a page fault.

It'd be handy to have INFO logs for things that we should log all the time, even in release mode.

In addition to that, let's rename core/debug.h to logger.h.

It would be great to have one logger per module too and makefile variables for each one so that we can enable debug logs for modules we want.

Instead of maintaining some socket specific information in a descriptor and a probably a buffer in the net/socket module, we could probably create a sockfs FS that would take care of this. That way, we could revert the changes made on the file descriptors, use inodes for sockets, and move the "net" logic from syscalls to sockfs.

In the paging code, there is no huge page support. It seems to be required, so it should be done at some point. I need to read more about this stuff.

When we pass -fstack-protector-all to a new-ish gcc version (like the one used in the Docker image), the kernel panics in the paging_init() function. This behavior isn't reproducible with x86_64-pc-elf-gcc (GCC) 5.4.0.

We use liballoc to manage the kernel heap memory. In addition to handling a bunch of annoying stuff for us, it exposes the following well-known functions: malloc, free, calloc and realloc. We use them in the kernel code. We cannot use them in "userland" programs because the libc does not offer these functions yet.

We should configure a PREFIX for liballoc (something like k_) and use the k_* functions in the kernel code. That should make it clear that we are using kernel heap memory.

This is needed because this project uses pretty much the same code for the libk and libc.

We should build the kernel and run it using qemu in circle ci. Ideally, we'd pass a multiboot command line to tell the kernel to shutdown after initialization.

We currently use a constant for the available memory size, which allows us to initialize the bitmap and so on at build time.

This is nice because it works but we should probably use the multiboot info to determine the memory size at runtime. This would allow to change the RAM config without having to change a constant and recompile the kernel.

I think we could achieve that by creating a bitmap in the first frame we can allocate (in frame_init()). We should double check but 4kb for a bitmap should be enough in most cases.

Move cat from kshell to init

fd stands for file descriptor but we'll have to handle socket descriptors too. Let's use the more generic term descriptor to represent the different descriptors. The fd code currently lives under src/fs but descriptors should not be filesystem-specific. I guess we can create a proc sub-folder ("proc" for "process").

The init program contains stub functions to implement. The end state will be to replace most of the kshell features with init and later move init to user mode (kshell will stay in kernel mode).

• cal
• clear
• date
• exit
• help
• hostname
• reboot
• uptime
• overflow

Note: I suggest creating multiple PRs (like one per command) instead of a big patch that will be hard to review.

We shouldn't be using random registers when syscall'ing, see: https://wiki.osdev.org/Calling_Conventions.

I think it'd be nice to fix that but I am pretty sure we'll need to extract the syscall handler to a separate "thing", i.e. without using the ISR code. Even if we use an interrupt for syscalls, ISR/IRQ handling is different (it seems).

I gave clang-tidy a try locally and that is interesting. It caught a few minor mistakes and gives good warnings. I see it as a linter/static analysis tool to prevent some mistakes.

We'll adopt clang-format to enforce stylistic conventions. clang-tidy is not about style but errors that can be caught with static analysis so having both would make sense.

That's not high priority, though.

We can send ARP requests and process ARP replies but we cannot respond to other machines' ARP requests.

After having remapped the kernel, the VFS does not work anymore and it's not clear why. Maybe we need to map the multiboot module first? (This seems to help a bit but I am not able to list any files).

See: https://github.com/cjlano/tinyprintf

The benefit would be better placeholders and likely less bugs.

Something we need to look into after my ELF fix PR is merged.
I've tried several times to do an elf_unload method but it always causes a page fault when I unmap any of the pages.
Any idea why that could be happening?

I wrote many things to deal with memory, like frames, pages, etc. The frame allocator seems to work and translating a phy address seems to work too. I can get a page, a frame, etc. I think unmapping works too. I can handle page faults (more ore less).

The problem is I have no clue about what to do next. It seems like I need some sort of allocator to build a heap so that I can write kmalloc/kfree and then I am done, but I don't really know how to reuse what I have so far.

The multiboot info contains memory areas, so I should probably use that as well. I actually pass such info to the mmu but...

In the phil'os tutorial, the idea was to remap the kernel. I don't have that, so maybe I should?

I am thinking about restarting from scratch because obviously I cant remember what I did 2 years ago

Instead of having a big init binary, we could create smaller binaries for each command and implement something in init to execute them.