Code Monkey home page Code Monkey logo

os-tutorial's Introduction

os-tutorial

⚠️ Hey! This is an old, abandoned project, with both technical and design issues listed here. Please have fun with this tutorial but do look for more modern and authoritative sources if you want to learn about OS design. ⚠️

How to create an OS from scratch!

I have always wanted to learn how to make an OS from scratch. In college I was taught how to implement advanced features (pagination, semaphores, memory management, etc) but:

  • I never got to start from my own boot sector
  • College is hard so I don't remember most of it.
  • I'm fed up with people who think that reading an already existing kernel, even if small, is a good idea to learn operating systems.

Inspired by this document and the OSDev wiki, I'll try to make short step-by-step READMEs and code samples for anybody to follow. Honestly, this tutorial is basically the first document but split into smaller pieces and without the theory.

Updated: more sources: the little book about OS development, JamesM's kernel development tutorials

Features

  • This course is a code tutorial aimed at people who are comfortable with low level computing. For example, programmers who have curiosity on how an OS works but don't have the time or willpower to start reading the Linux kernel top to bottom.
  • There is little theory. Yes, this is a feature. Google is your theory lecturer. Once you pass college, excessive theory is worse than no theory because it makes things seem more difficult than they really are.
  • The lessons are tiny and may take 5-15 minutes to complete. Trust me and trust yourself. You can do it!

How to use this tutorial

  1. Start with the first folder and go down in order. They build on previous code, so if you jump right to folder 05 and don't know why there is a mov ah, 0x0e, it's because you missed lecture 02. Really, just go in order. You can always skip stuff you already know.

  2. Open the README and read the first line, which details the concepts you should be familiar with before reading the code. Google concepts you are not familiar with. The second line states the goals for each lesson. Read them, because they explain why we do what we do. The "why" is as important as the "how".

  3. Read the rest of the README. It is very concise.

  4. (Optional) Try to write the code files by yourself after reading the README.

  5. Look at the code examples. They are extremely well commented.

  6. (Optional) Experiment with them and try to break things. The only way to make sure you understood something is trying to break it or replicate it with different commands.

TL;DR: First read the README on each folder, then the code files. If you're brave, try to code them yourself.

Strategy

We will want to do many things with our OS:

  • Boot from scratch, without GRUB - DONE!
  • Enter 32-bit mode - DONE
  • Jump from Assembly to C - DONE!
  • Interrupt handling - DONE!
  • Screen output and keyboard input - DONE!
  • A tiny, basic libc which grows to suit our needs - DONE!
  • Memory management
  • Write a filesystem to store files
  • Create a very simple shell
  • User mode
  • Maybe we will write a simple text editor
  • Multiple processes and scheduling

Probably we will go through them in that order, however it's soon to tell.

If we feel brave enough:

  • A BASIC interpreter, like in the 70s!
  • A GUI
  • Networking

Contributing

This is a personal learning project, and even though it hasn't been updated for a long time, I still have hopes to get into it at some point.

I'm thankful to all those who have pointed out bugs and submitted pull requests. I will need some time to review everything and I cannot guarantee that at this moment.

Please feel free to fork this repo. If many of you are interested in continuing the project, let me know and I'll link the "main fork" from here.

os-tutorial's People

Contributors

alhaad avatar amit794 avatar arisingh8 avatar cfenollosa avatar erginbilgin avatar ghifari160 avatar maxkl avatar peroh215 avatar sapsaldog avatar wellingguzman avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

os-tutorial's Issues

Qemu error

In the first chapter, it tells me to run qemu boot_sect_simple. However, the actual command is qemu-system-x86 (or something similar).

Typo in README

"Probably we will go through them in that order, however it's too soon to tell."

Drivers code doesn't seem to work

I'm following this tutorial on Linux (ubuntu), and it worked until chapter 15. First thing is that when I want to write a character on screen I have to offset the character by 1 and the color by 2. When I copied the code it was flickering so I debugged it and managed to make the clear_screen() work (with the offset).
If anyone has an idea on what's the problem I'll be glad to hear it.
Thanks

Error on section 05

boot_sect_main.asm:error: Can't find valid values for all labels after 1004 passes, giving up.
boot_sect_main.asm:error: Possible causes: recursive EQUs, macro abuse.

All of the other files needed are made and ready, but whenever I nasm boot_sect_main.asm -o boot_sect_main.bin it gives this error message.

getting error while installing binutils

../binutils-2.24/configure --target=$TARGET --enable-interwork --enable-multilib --disable-nls --disable-werror --prefix=$PREFIX 2>&1 | tee configure.log

on executing above command I'm getting these errors. what am I doing wrong ?

checking build system type... i386-apple-darwin17.4.0
checking host system type... i386-apple-darwin17.4.0
checking target system type... i386-pc-elf
checking for a BSD-compatible install... /usr/bin/install -c
checking whether ln works... yes
checking whether ln -s works... yes
checking for a sed that does not truncate output... /usr/bin/sed
checking for gawk... no
checking for mawk... no
checking for nawk... no
checking for awk... awk
checking for gcc... /usr/local/bin/gcc-4.9
checking for C compiler default output file name...
configure: error: in '/tmp/src/binutils-build':
configure: error: C compiler cannot create executables
See config.log' for more details.

How to use BIOS interrupt for VGA/SVGA modules ?

I have completed the whole set of tutorial currently to 21-malloc. But I want to extend it to VGA and SVGA modules. I googled that we have to use int 0x10 for VGA Display, but it's only available for BIOS. Can I simply put them into bootsect before we are ever jumping to Protected Mode ?
I also tried to put int 0x10 before jump to pm, but the display turns black.

Endianness

In 01-bootsector-barebones/README.md there is typo.

(beware of indianness, x86 is little-endian).

should read

(beware of endianness, x86 is little-endian).

01-bootsector-barebones issue - expecting raw image?

qemu-system-x86_64 boot_sect_simple.bin WARNING: Image format was not specified for 'boot_sect_simple.bin' and probing guessed raw. Automatically detecting the format is dangerous for raw images, write operations on block 0 will be restricted. Specify the 'raw' format explicitly to remove the restrictions. warning: TCG doesn't support requested feature: CPUID.01H:ECX.vmx [bit 5] Could not initialize SDL(No available video device) - exiting
I receive this error when attempting to run the basic bootsector. Is there something I have to specify?

17-video-scroll The cursor is gone

When I run the code in lesson 17, everything works fine except that the cursor is gone. This happens only when a scroll occurs. Any idea about this? Thanks.

16-video-driver can't pass string

When I pass a string to kprint or kprint_at it always points to null. I have to first declare the string as an array, and then pass it. Why is this happening? I copied/pasted the code.

char msg[] = "test";
kprint_at(msg, 1, 2);

the above works but the below does not

kprint_at("test", 1, 2);

Can't setup cross-compiler

I follow all the instructions (I even tried the updated ones) and I get this:

checking build system type... i386-apple-darwin16.0.0
checking host system type... i386-apple-darwin16.0.0
checking target system type... i386-pc-elf
checking for a BSD-compatible install... /usr/bin/install -c
checking whether ln works... yes
checking whether ln -s works... yes
checking for a sed that does not truncate output... /usr/bin/sed
checking for gawk... no
checking for mawk... no
checking for nawk... no
checking for awk... awk
checking for gcc... /usr/local/bin/gcc-6.3.0_1
checking for C compiler default output file name... 
configure: error: in `/tmp/src/binutils-build':
configure: error: C compiler cannot create executables
See `config.log' for more details.

How do I fix this?

Any new updates?

The last update was 6 months ago. Is this tutorial discontinued?

Add demo gif to README

Disclaimer: This is a bot

It looks like your repo is trending. The github_trending_videos Instgram account automatically shows the demo gifs of trending repos in Github.

Your README doesn't seem to have any demo gifs. Add one and the next time the parser runs it will pick it up and post it on its Instagram feed. If you don't want to just close this issue we won't bother you again.

Minor: three jump after conditionals

Location:
https://github.com/cfenollosa/os-tutorial/tree/master/05-bootsector-functions-strings

cmp ax, 4      ; if ax = 4
je ax_is_four  ; do something (by jumping to that label)
jmp else       ; else, do another thing
jmp endif      ; finally, resume the normal flow    ######### control never gets here, delete this line pls

ax_is_four:
    .....
    jmp endif

else:
    .....
    jmp endif  ; not actually necessary but printed here for completeness

endif:

You should remove line marked with "#"s.

32Bits Enter

I download the code to check if it was not me that I make a mistake but no the code is not correcte.

In the 32bit-main/gdt/print Idk but its not printing "Loaded 32-bit protected mode".
Can someone help me?

Future thoughts

Unix clone? Implement Unix v6?

Minix? BSDs?

Go for a totally different path? (templeOS, MSDOS, MikeOS, other)

Expression syntax error, chapter 1

When trying to run nasm -f bin boot_sect_simple.asm -o boot_sect_simple.bin I got two exceptions of the same type: expression syntax error.
I searched the Internet and managed to fix the problem, I had to rename the loop variable to loop1 and then it worked!

Weird errors with QEMU

Ok, first of all i only tested this with qemu.
I am compiling with gcc and linking with ld, basically because i wasn't able to install the crosscompiler for windows.
The first kernel barebones thing worked perfectly. It just printed an "X" at the top of the screen as it should. But here comes the problem: i just exacly copypasted the kprint code (it didn't work with my version of it) and it does nothing. Nothing appears at the screen at all. Surprisinhly enough, clear_screen() function works just fine, and i can change the color of it and it still does work just fine.
Please can someone help me?

Removing the infinite loop guard in the bootsector does not prevent qemu from executing it

Context

There is a behaviour (maybe) related to qemu I don't understand.

I just tried the first lesson, and I wanted to see what would happen to the V.M. if the infinite loop "guard" was removed.

Note:

bootsector.asm:

jmp $ ; <--<--<--<--<-- the infinite loop i'm talking about
times 510-($-$$) db 0
dw 0xaa55

Expected

Qemu crashes.

Actual

Nothing changes visually. No error or warning is issued.

Issue

I don't understand what happens behind the scene. Is the presence of an infinite loop at the end of a boot-sector optional? Or is qemu some sort of wizard that knows magically when to stop the execution of the boot-sector?

continue the project

can you continue that project to make full featured kernel at the end , it is very useful repo

flickering screen - lesson 10

Hello,

When i am running 32bit_main in qemu, in lesson 10, is not working properly.
The screen is flickering, like is constantly rebooting. This only happens when i got 32 bit enabled.
When i comment out all 32bit code, and keep only 16bit is working properly.. any fixes for that?
I am sure that the problem is something about 32bit, but dont know exactly what.

Please check the bug of function hex_to_ascii in string.c

I am studying os-dev by recoding this repo.
Then I found here have a bug in function hex_to_ascii.
In string.c:32, it is

32        if (tmp > 0xA) append(str, tmp - 0xA + 'a');
33        else append(str, tmp + '0');

When the input int is 0xAA. Then the string of this will be "0x:A".
It should be modified the condition of tmp > 0xA to tmp >= 0xA.

Finally, thank you for this repository gave me great help!

Is this still active

Is this still active and being updated. The last commit was over 4 months ago.

Content of exception_messages string pointer arrays is empty in 18-interrupts

When learning 18-interrupts, content of exception_messages in cpu/isr.c is empty. All items are 0x0. It seems that ld don't replace the string constants with real address.
I do this tutorial on Gentoo with gcc (Gentoo 7.3.0-r3 p1.4) 7.3.0, GNU ld (Gentoo 2.30 p5) 2.30.0.
CFLAGS is -m32 -fno-pie -g. There would be errors if I don't use -fno-pie.

Building in linux - Target-Libgcc

Hello. I'm following the steps from: 11-kernel-crosscompiler

I've installed successfully binutils following your steps. But I'm having problems installing GCC:

In the step:
make all-target-libgcc

I'm getting the next error:
/tmp/ccP9qmbP.s:15: Error: invalid instruction suffix for push' /tmp/ccP9qmbP.s:45: Error: invalid instruction suffix for pop'

I was reading, and I found that is because of the instructions. I don't know how to force to 32 bits mode.

Details:
make[2]: se sale del directorio '/tmp/src/gcc-build/i386-elf/libgcc'
/tmp/src/gcc-build/./gcc/xgcc -B/tmp/src/gcc-build/./gcc/ -B/usr/local/i386elfgcc/i386-elf/bin/ -B/usr/local/i386elfgcc/i386-elf/lib/ -isystem /usr/local/i386elfgcc/i386-elf/include -isystem /usr/local/i386elfgcc/i386-elf/sys-include -g -O2 -O2 -g -O2 -DIN_GCC -DCROSS_DIRECTORY_STRUCTURE -W -Wall -Wno-narrowing -Wwrite-strings -Wcast-qual -Wstrict-prototypes -Wmissing-prototypes -Wold-style-definition -isystem ./include -fpic -g -DIN_LIBGCC2 -fbuilding-libgcc -fno-stack-protector -Dinhibit_libc -fpic -I. -I. -I../.././gcc -I../../../gcc-8.1.0/libgcc -I../../../gcc-8.1.0/libgcc/. -I../../../gcc-8.1.0/libgcc/../gcc -I../../../gcc-8.1.0/libgcc/../include -DUSE_EMUTLS -o _muldi3.o -MT _muldi3.o -MD -MP -MF _muldi3.dep -DL_muldi3 -c ../../../gcc-8.1.0/libgcc/libgcc2.c -fvisibility=hidden -DHIDE_EXPORTS
/tmp/ccP9qmbP.s: Assembler messages:
/tmp/ccP9qmbP.s:15: Error: invalid instruction suffix for push' /tmp/ccP9qmbP.s:45: Error: invalid instruction suffix for pop'
Makefile:494: recipe for target '_muldi3.o' failed
make[1]: *** [_muldi3.o] Error 1
make[1]: se sale del directorio '/tmp/src/gcc-build/i386-elf/libgcc'
Makefile:11670: recipe for target 'all-target-libgcc' failed
make: *** [all-target-libgcc] Error 2

Stuck at 00 :(

Installed qemu and nasm on ubuntu. The later commands are not working. I'm not exactly clear on how to open the software to write code.

How to boot?

I have a usb with os-image.bin on it and it wont boot from physicall machine

Cross GDB build fails

In14th lesson README, GDB build fails if guile version is above 2.0. So add --with-guile=guile-2.0 to configure call

Disk read error: strategies to debug?

hey @cfenollosa ! I am working on this on my own, and really close to getting a Dockerized emulator up and running, and hitting this issue:

image

I've put and organized everything into a Github repo, and I would be so grateful for your help!

https://github.com/vsoch/os

Specifically, I'm building the docker image, and then running (booting) the image with running it!

It's a disk read error, so maybe the issue is in the disk.asm?

If you can't help directly, could you tell me how to debug this? Thank you!

Hangs after loading kernel into memory

My code hangs after loading kernel into memory.
; Identical to lesson 13's boot sector, but the %included files have new paths
[org 0x7c00]
KERNEL_OFFSET equ 0x1000 ; The same one we used when linking the kernel

mov [BOOT_DRIVE], dl ; Remember that the BIOS sets us the boot drive in 'dl' on boot
mov bp, 0x9000
mov sp, bp

mov bx, MSG_REAL_MODE 
call print
call print_nl

call load_kernel ; read the kernel from disk
call switch_to_pm ; disable interrupts, load GDT,  etc. Finally jumps to 'BEGIN_PM'
jmp $ ; Never executed

%include "print.asm"
%include "hexprint.asm"
%include "disk.asm"
%include "32bit-gdt.asm"
%include "32bit-print.asm"
%include "32bit-switch.asm"

[bits 16]
load_kernel:
mov bx, MSG_LOAD_KERNEL
call print
call print_nl

mov bx, KERNEL_OFFSET ; Read from disk and store in 0x1000
mov dh, 64 ; Our future kernel will be larger, make this big
mov dl, [BOOT_DRIVE]
call disk_load
ret

[bits 32]
BEGIN_PM:
mov ebx, MSG_PROT_MODE
call print_string_pm
call KERNEL_OFFSET ; Give control to the kernel
jmp $ ; Stay here when the kernel returns control to us (if ever)

BOOT_DRIVE db 0 ; It is a good idea to store it in memory because 'dl' may get overwritten
MSG_REAL_MODE db "Started in 16-bit Real Mode", 0
MSG_PROT_MODE db "Landed in 32-bit Protected Mode", 0
MSG_LOAD_KERNEL db "Loading kernel into memory", 0

; padding
times 510 - ($-$$) db 0
dw 0xaa55
thats the boot sector

File API info soon?

Been following this series happily, been having fun experimenting and adding new features to the OS.

However, I've run into the issue of File I/O, which I can't find any easy-to-follow info on, mostly just tech docs and the likes. Will you be releasing a tutorial on making a File I/O API soon?

Great work regardless, thanks for making this. :)

EDIT: After extreme amounts of googling, it sounds like the standard for tutorials is FAT12, but SFS (SimpleFS) might be easier to implement/more stable.

I would love to help more, but I'm really only good at high-level languages with already-made APIs and the likes - this is my first adventure into the low-level lands of Assembly and OS development.

Translate

Hi, I'm a Chinese university student. I got a lot of benefit from your code 'os-tutorial'.It occurred to me that maybe I could translate them to Chinese in recent days.Therefore, I suppose I should ask your permission for the rights of translation.Likewise, I have several understandings on my own, so I'm thinking whether I can add some code lines to your finished codes. I will appreciate it if you allow me to do that. Hoping for your response. Thanks so much! 谢谢!!

Problem in running the kernel - lesson 13

First of all, thanks for this awesome tutorial.

I am having problems in the 13th lesson. The qemu screen just flickers. By placing some jmp $ here and there, I find that everything is fine untill the "call KERNEL_OFFSET" in BEGIN_PM in the bootsect.asm.That is control never reaches the first line of kernel_entry.asm.

There are no disk read errors (using fda) and no trouble in reaching 32 bit protected mode. Any ideas?

How exactly do I verify that the kernel binary is placed at 0x1000? Both ndisasm and xxd do not seem to show this.

lesson 13: undefined reference to "_GLOBAL_OFFSET_TABLE"

when I was using ld to link kernel.o and kernel_entry.o, the error below happens

make \n kernel.o: In function ' dummy_test_entrypoint': \n kernel.c (.text+0x9): undefined reference to '_GLOBAL_OFFSET_TABLE_' \n
Any workarounds?

macOS install gcc cross compiler error

follow configure step is right for me

./configure --prefix=$PREFIX \
--target=$TARGET \
--disable-nls \
--enable-languages=c,c++ --without-headers \
--enable-interwork --enable-multilib \
--with-gmp=/usr/local/Cellar --with-mpc=/usr/local/Cellar --with-mpfr=/usr/local/Cellar

the gmp mpc mpfr must target to /usr/local/Cellar.because they are install with brew

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.