yjp20 / turtle Goto Github PK

# turtle

## goals

First and foremost, turtle is a hobby operating system. As a consequence, it needs to be a pleasurable experience to write and read.

Second, I want to keep things simple. turtle should be an operating system that pays careful attention to the cost of abstractions, not only their benefits.

Third, I want to reexamine old, but time-tested abstractions and see if I can do better.

Beyond these abstract goals, there's a few concrete ideas that I want to incorporate into turtle, which I will detail in the sections below.

### language first

turtle as an operating system is inseparable from the language that was designed for it, straw. All programs, including the kernel should be able to be written in straw. Many existing general-purpose languages are capable of being used in this manner. A far more bold requirement for straw is that it needs to be a reasonable language to use as the interactive shell for turtle.

Because of this requirement, it doesn't really make sense for the computer to be configured into running discrete binaries as in unix. In the unix way, we use C to create binaries, which are then composed together using a shell. Shell utilties are versatile since they act entirely on files, or essentially strings. It was also somewhat necessary, because the input/output abstraction encouraged the creation of programs that used very little memory. This isn't a constraint that is super relevant anymore on modern systems.

However, we do lose the rigor of types which can help guide us in creating correct programs. Some new projects such as nushell attempts to get around this by adding typing to commands. This is a pretty good compromise for the unix world, but if we're creating an operating system from scratch, we can surely do better. In an ideal world, the shell is just as pleasant to use as modern IDEs, with full type checking.

Ultimately, I want to see if it is possible to remove the "shell" as we know today and unify everything under one programming language, where individual programs do not exist, only libraries. Instead of the shell, there will really just be an interpreted version of straw, along with loadable compiled versions.

One of the problems with this, however, is that abusing a language in these different ways strains the language's focus and effectiveness at any one task. This is why DSLs exist. This is an inherent problem that cannot be "solved", but nevertheless I have some ideas that will hopefully make a decent compromise.

### universal terminal

The terminal is also something I would like to experiment with. The current understanding of a terminal is essentially just a scroll of text, with some exceptions (such as being able to render images).

This kind of limitation was borne out of necessity a long time ago, but it's very clear that it's no longer a necessity. Because we insist on emulating old terminals, making terminal applications that are delightful to use require some pretty arcane techniques.

But this is a case of "worse is better". While using graphical displays to emulate terminals to emulate graphical displays is quite arguably illogical, a lot of people (including myself) prefer terminal applications over their graphical counterparts. As it turns out, the fundamental limitations of text generally encourage UI/UX that is preferrable to the user. The content is first and foremost out of necessity, applications are fast, and graphical elements are minimized.

To replace the old scroll-of-text, keeping in mind the good limitations are important. We can remove these limitations whilst still encouraging their outcome.

I have a pretty good image of what I want my version of the terminal to look like, but I have a tough time describing it. In essence, I want to create a scroll of graphical elements, not just text. Then, when we go into an application that necessarily requires control of the entire window like vim, then we temporarily hide the scroll. I think the best analog is to the plan9 acme text editor, which was designed in such a way that it could serve almost all purposes.

### accessible resources

Data and access to data is one of the fundamental concerns of a computing system. As in plan9, stateful resources should be mapped to the filesystem under a namespace, like processes, network, audio, etc. Even something like the data in a git repository should be mapped as a filesystem. One problem with files is that they have an implicitly defined format. File name extensions can help with specification, but are insufficient as they are mostly hints. To "fix" this problem, all files that are specified internally within turtle should be written in straw's serialization format, which is built into the language. This serialization format should have an interchangable human readable mode and optimized binary format.

## motivations

For most people, I suppose that this section won't be interesting, but it's necessary for me to make clear exactly why I'm doing this when I'm undertaking such a huge a project. Otherwise, my focus will waver and my motiviation will dim.

For one, I've always wanted to make some kind of complex system, an institution, or really any work that I could call my own. Perhaps it's just a part of the universal human experience, but I feel an intrinsic desire to build things, and an even greater desire to build something that is great, well thought-out, and will out last me. This desire was always a part of me. I would always daydream as a kid about a family of scientist dogs inhabiting their sophisticated castle (not entirely unlike Tony Stark's mansion) filled to the brim with futuristic technology. I think this feeling is what attracted me to programming in the first place -- you could build your own castle made of digital bricks with essentially no cost but time.

As I became a better programmer, I grew more and more sure that I would build a programming language. I may be biased, but I think a programming language is the pinnacle of craftmanship, and failing that, is at least the pinnacle for a programmer. At this point in my life, I think I have just enough knowledge and skill to get started, and so that's exactly what I'm going to do. If I don't get started, I never will, and will always regret it.

But it isn't enough just to build a programming language. Tolkien wrote thousands of pages of lore, an entire meticulously-crafted and consistent world just to build a home for his language. Building an operating system is just a natural extension for a language like mine.

Another reason is that designing something like a language is just so interesting. As with almost anything else, there's way more complexity and nuance to creating a complete system than I could comprehend, and it's difficult to articulate the little that I do. Even in my preliminary phases of designing a language, I have to weigh each feature in terms of conceptual clarity, code complexity, composability, readability, runtime speed, compiletime speed, and a bajillion other factors I wasn't even aware of before. To say that designing a language is NP-hard is selling it short.

Exploring something to such depths, especially something for which I feel compelled to do, is a delightful experience -- the world is a fractal, with each locality having essentially infinite detail. However, even among the various branches of the world, I think computation is an especially fruitful slice to dive into.

Another thing I enjoy about learning through experience is that I appreciate the world more. The same nuances that I find difficult when designing something like a language are things that are taken as granted when learning.

For example when you look at something like Go, the design is so simple and clear that it seems obvious. Learning to use Go is, all things considered, pretty straighforward. Before trying to design my own language, Go was already my favorite language to use for a variety of reasons, but after giving a go of it myself, I have an even greater appreciation for the language and its creators. The same goes for a whole bunch of things that I can't get into.

Each of these reasons is enough for me to at least try creating a programming language and operating system, but with all of these desires in conjuction, I have never been more sure that this is a project that I will either see through or die trying. I'm pretty sure it will take around 10 years to get anywhere close to the point I want to get to. Most likely, it will be more. That's fine.

# os references

## linux

Linux is a tried and true monolithic kernel with a ton of features. I don't have much to say about linux as a whole, because it's the standard and by definition doesn't have many exceptional, non-standard features.

## bsd/openbsd

Personally, I like OpenBSD and the BSD flavors of unix in general. For Linux, my favorite distribution is Void Linux by a wide margin, which was created by a BSD contributor and shares a lot of the BSD ethos. What I like about them is their commitment to an integrated system and quality of simple and effective code. Ultimately it shows that to upkeep maintainable code over a long period of time, there is a need to actively limit feature creep.

OpenBSD's main focus is specifically on security. As much as I would like to emulate their approach, I think security is one of those things that takes second footing to an actually operating operating system. Essentially, I will worry about it later.

## plan9

plan9's main innovation deals with extending unix's file abstraction to practically everything. A similar feature will probably be implemented into turtle. Besides that, there are a whole lot of things taken from plan9 directly and indirectly, so much so that it will be impractical to detail all of them. Some examples include CSP, the plan9 ABI (used in Go), etc.

## serenity

Andreas Kling, the creator of serenity used to work in the Apple ecosystem The main thing about serenity that I find attractive is it's centralized development which enables extremely fast iteration and its GUI that's consistent throughout the entire system. It's not experimental with the various abstractions by design and tries to implement a sane version of the most common, but not necessarily the best abstractions throughout computing history.

## hubris

Hubris is interesting in the sense that it is statically configured and made in Rust. Not too familiar with it, but being statically configured is an interesting concept for an operating system. It's definitely makes sense for embedded applications for pretty clear reasons, but also might be interesting to use within virtualized contexts since static and ultimately repeatable configurations is what docker tries to achieve.

## redox

Redox extends plan9's everything is a file concept to everything is a url. This solves the problem of file specification, where the data format of various different files are unclear. turtle's approach to this problem is a bit different, to the point where the url concept likely won't apply.

# language references

## c

C is nice because the language is so close to the processor. On many architectures, C can be translated almost directly to the actual instructions.

The problem with C is that everything about the language is essentially just hacks on top of other 

## zig

Zig's most interesting feature is its metaprogramming capabilities. comptime represents a few different "kinds" of metaprogramming that can be achieved, such as conditional compilation, generics, and zero-cost abstractions. The "problem" for me is that grouping these metaprogramming faciltiies under one semantic term makes conceptual sense, but less practical sense. I think this may be a case of over-abstraction, although i've yet to see if I can come up with something better.

Such powerful metaprogramming abilities comes with some drawbacks, such as a reduction in static analysis, increased complexity in implementation as a custom interpreter is needed.

For now, the main motivating example is static configuration for speed, but I'm interested to see what the implications of being able to dynamically configure at runtime for certain tasks.

## rust

## vlang

## go

## haskell

## apl

## rc

## limbo

## spiral language

## red/rebol

Red, like straw, is meant to be a full-stack language. Its original plans were to have a "collaborative execution model" that would interop code in interpreted, aot-compiled, and jit-compiled modes. Red would run most code in the slower interpreted mode, but give you the option of writing critcal sections in Red/System which should be considerably faster and comparable to C.

Red also doesn't settle for C-like syntax. The code is homeoiconic, which is an idea of lispy origins where code is data, and data is code. Essentially, the code is just a nested list of symbols which is given life by the compiler, but can also serve as a data-format by itself.

Being such a deeply abitious language, 

## jai

## lua


# serialization formats

## amazon ion

## protobuf