Code Monkey home page Code Monkey logo

thebeambook's Introduction

PDF Build Pages Build

The BEAM Book

This is an attempt to document the internals of the Erlang runtime system and the Erlang virtual machine known as the BEAM.

You can read or download the book as a PDF from the latest stable release or online as a webpage.

The book is written in AsciiDoc and most of it can be read directly from source on GitHub in your browser. To read the book online just open the file book.asciidoc.

You can also read it as an Github IO page.

Contributing

The plan is to make this book project into a collaboration effort so that we can get a complete documentation of the Erlang runtime system as soon as possible. Please feel free to contribute since this work is far from done.

You can contribute by raising an issue, comment on open issues or create a branch with a fix or an addition.

Note that the book is released under a Creative Commons license (see below) and anything you contribute will also be included under that license.

The chapters in the book can be in one of four states:

  1. Placeholder, basically only the title and perhaps an outline of the chapter is done. If you are interested in writing the chapter or parts of the chapter grab the corresponding issue and start writing.
  2. First draft, most of the text is in place but more editing is needed. Feel free to comment, focusing on missing content, hard to read passages, the order of sections within the chapter, diagrams or pictures needed, and plain errors.
  3. Final draft, spelling and other errors probably still need fixing.
  4. Done (for OTP version X), if things changes in later versions of Erlang OTP the chapter will need an update.

(Not all chapters are yet marked in this way.)

Style guide

There are several ways to use AsciiDoc and some constructs work better in some environments or for some targets.

The priority of the AsciiDoc code in this project is that it renders nicely for the following targets in the following order:

  1. The PDF target
  2. The HTML target
  3. View directly on GitHub

We will try to come up with specific guides for which AsciiDoc constructs to use and add them here as we discover what works and what doesn't work.

Comments in AsciiDoc

Each chapter should begin with a comment about the status of the chapter. This should be one of 'Placeholder', 'First Draft', 'Final Draft', or 'Done (for Erlang X.X)'. There can also be a link to an issue describing what is needed to bring the chapter to the next level.

A comment in the code starts with '//'.

Linking to OTP/Erlang source code

When referring to the source code of Erlang/OTP please add a link to a tagged version (not to master) of the code on GitHub, as in:


 link:https://github.com/erlang/otp/blob/OTP-19.1/erts/emulator/beam/erl_time.h[erl_time.h]

Directory structure and build

Try to keep the root directory clean.

Put each chapter in a separate .asciidoc file in the chapters directory. Use underscores "_" to separate words in chapter names but try to use just one-word file names for the chapters.

Put code used in a chapter in code/CHAPTERNAME_chapter/src, and add an include of the code in ap-code_listings.asciidoc.

Put images in the images directory.

How to tag chapters, sections, figures

The following is not yet done consistently so please feel free to contribute by fixing tags in the current version.

Chapter tags should start with 'CH-'. Words in a tag are separated by underscores '_'.

Part tags should start with 'P-'.

Section tags should start with 'SEC-'.

Figure tags should start with 'FIG-'.

Appendix tags should start with 'AP-'.

Code listing tags (in the appendix) should start with 'LISTING-'.

Process

If you find something you do not understand or which is incorrect please raise an issue in the issue tracker.

If you find spelling or formatting errors feel free to fix them and just make a pull request.

For larger rewrites check the status of the chapter and check the issues to see if someone is likely to be working on that chapter right now. If someone else is working on the chapter try to contact that person before doing a major rewrite. Otherwise either just go ahead and do the rewrite and do a pull request or start by opening an issue declaring what you intend to do.

Building the PDF locally from source

The project contains a makefile which will let you build your own PDF from the source, provided that you have all the needed tools installed.

Docker

You can build the project locally using docker by first building the docker image

make docker-build

And then building the project by

make docker

Linux

WIP, to be updated

make

Mac OSX

  1. Install asciidoc
  2. Install asciidoctor-pdf
  3. Install asciidoctor-diagram
  4. Install ditaa
  5. Install graphviz
  6. Install rouge
  7. Install wget
  8. make

Mac OSX (using brew etc)

  1. brew install asciidoctor
  2. gem install asciidoctor-pdf
  3. gem install asciidoctor-diagram
  4. brew install ditaa
  5. brew install graphviz
  6. gem install rouge
  7. brew install wget
  8. make

License

The Erlang Runtime System by Erik Stenman is licensed under a Creative Commons Attribution 4.0 International License. Based on a work at https://github.com/happi/theBeamBook. A complete copy of the license can be found here.

A short and personal history of the book

I, Erik Stenman (Happi), started writing this book back in 2013. At first I was thinking of self publishing the book on my blog, but since English isn't my native language I felt I needed help by a good editor.

I managed to get a deal with O'Reilly and started converting my outline to their build process. My original plan was for a very long and thorough book, which the editor felt would get few readers. I started cutting my content and tried to write more of a tutorial than a manual. Unfortunately progress was slow and pre-sales was even slower and the publisher cancelled the book in 2015.

I managed to get a new deal with Pragmatic and started converting my content to their build system and rewriting the book according to the more pragmatic style of the new publisher, cutting down the content even further. The series editor also wanted me to fit the book into the Elixir series and I tried to add more Elixir examples. I did not really manage to make it into an Elixir book and also my progress was still slow, which led to another cancellation of the book early 2017.

Now I had three repositories with three different book building systems with three different outlines of the book. In the end I more or less went back to the original longer book outline and the original AsciiDoc build system. I started a new repository in a private GitHub account and started pulling in content from the three different versions.

Then on April 7 2017 I opened the repository to the public to share it with some students. I didn't think anyone else would notice and I was not planning to release the book for real yet since the repo currently just contains bits and pieces from the different versions of the book.

There was more interest than I had expected though and fortunately also several who where willing to contribute. From now on the book is a collaborative effort to document the Erlang runtime system ERTS and the Erlang virtual machine BEAM, and it is released with a Creative Commons license (see above).

Watch this space for further news and to see the whole book take shape.

-- Erik Stenman aka Happi

thebeambook's People

Contributors

alexandrejbr avatar anton-ryabkov avatar citizen428 avatar djnym avatar drphil avatar duskyelf avatar garazdawi avatar gorillainduction avatar happi avatar hirotnk avatar itsgreggreg avatar jfacorro avatar karljoad avatar kencausey avatar kianmeng avatar kimshrier avatar kronicdeth avatar kvakvs avatar lincolnbryant avatar michalwski avatar mvw avatar platinumthinker avatar ramkumarr avatar robertoaloi avatar shalokshalom avatar slomo avatar sthagen avatar stratus3d avatar techgaun avatar whatyouhide avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

thebeambook's Issues

Inspecting message handling example

I am stucked with the Inspecting message handling example:

4> P = spawn(fun() -> receive stop -> ok end end).
<0.63.0>
5> P ! start.
start
6> hipe_bifs:show_pcb(P).

It is supposed to show the message queue of a process like this:

...
408 | msg.first | 0x00007fd40962d880 | |
416 | msg.last | 0x00007fd40962d880 | |
424 | msg.save | 0x00007fd40962d880 | |
432 | msg.len | 0x0000000000000001 | |
696 | msg_inq.first | 0x0000000000000000 | |
704 | msg_inq.last | 0x00007fd40a306238 | |
712 | msg_inq.len | 0x0000000000000000 | |
616 | mbuf | 0x0000000000000000 | |
640 | mbuf_sz | 0x0000000000000000 | |
...

All I see is :

P: 0x00007fd9ef6400b0

Offset| Name | Value | *Value |
72 | htop | 0x00007fd9ec1bb020 | |
96 | hend | 0x00007fd9ec1bb2d8 | |
88 | heap | 0x00007fd9ec1bab90 | |
104 | heap_sz | 0x00000000000000e9 | |
80 | stop | 0x00007fd9ec1bb290 | |
592 | gen_gcs | 0x0000000000000000 | |
594 | max_gen_gcs | 0x000000000000ffff | |
552 | high_water | 0x00007fd9ec1bab90 | |
560 | old_hend | 0x0000000000000000 | |
568 | old_htop | 0x0000000000000000 | |
576 | old_head | 0x0000000000000000 | |
112 | min_heap_.. | 0x00000000000000e9 | |
328 | rcount | 0x0000000000000000 | |
0 | id | 0x0000021000000423 | |
336 | reds | 0x000000000000001a | |
16 | tracer | 0xfffffffffffffffb | |
24 | trace_fla.. | 0x0000000000000000 | |
344 | group_lea.. | 0x0000019800000333 | |
352 | flags | 0x0000000000002004 | |
360 | fvalue | 0x000000000000640b | |
368 | freason | 0x000000000000000a | |
320 | fcalls | 0x00000000000007c9 | |
384 | next | 0x00007fd9ef6444b0 | |
48 | reg | 0x0000000000000000 | |
56 | nlinks | 0x0000000000000000 | |
616 | mbuf | 0x0000000000000000 | |
640 | mbuf_sz | 0x0000000000000000 | |
464 | dictionary | 0x0000000000000000 | |
472 | seq..clock | 0x0000000000000000 | |
480 | seq..astcnt | 0x0000000000000000 | |
488 | seq..token | 0xfffffffffffffffb | |
496 | intial[0] | 0x000000000000320b | |
504 | intial[1] | 0x0000000000000c8b | |
512 | intial[2] | 0x0000000000000002 | |
520 | current | 0x0000000000000000 | 0xffffffffffffffff |
296 | cp | 0x0000000000000000 | 0xffffffffffffffff |
304 | i | 0x00007fd9ed8294c8 | 0x000000000043f0bb |
312 | catches | 0x0000000000000000 | |
224 | arity | 0x0000000000000000 | |
232 | arg_reg | 0x00007fd9ef6401a8 | 0x000000000000320b |
240 | max_arg_reg | 0x0000000000000006 | |
248 | def..reg[0] | 0x000000000000320b | |
256 | def..reg[1] | 0x0000000000000c8b | |
264 | def..reg[2] | 0x00007fd9ec1bab91 | |
272 | def..reg[3] | 0x0000000000000000 | |
280 | def..reg[4] | 0x0000000000000000 | |
288 | def..reg[5] | 0x00000000000007d0 | |
136 | nsp | 0x0000000000000000 | |
144 | nstack | 0x0000000000000000 | |
152 | nstend | 0x0000000000000000 | |
160 | ncallee | 0x0000000000000000 | |
56 | ncsp | 0x0000000000000000 | |
64 | narity | 0x0000000000000000 | |

true

What am I missing?

Incorrect Extended Literal Tag

In the information box in section 6.2.14 if all the tag values are shifted down by one then:

and literal is 2#1010111

should read

and literal is 2#1000111

Please continue being awesome

This book is ridiculously good. I have only read a few bits of it so far and have learned a lot already.

Please continue being awesome! πŸ‘ πŸ‘ πŸ’― πŸ₯‡ πŸ˜€

Timer Wheel

There is a description below. In my understanding timer wheel exists per scheduler after around 18. Is it the case ?

The timing wheel is a global resource and there might be contention for the write lock to the timing wheel if you have many processes inserting timers into the wheel

Possible typo in section "7.2 BEAM code listings"

First of all, thanks a lot for your great work! πŸ™‚ πŸ‘

In the last paragraph of section 7.2 BEAM code listings, right above section 7.3 Calls (pdf page 96) it reads:

Then the instruction {move,{atom,beamexample1},{x,0}} moves the module name atom to the first argument register x1

Doesn't {x,0} mean that it moves it to x0?

Docker image for building the book

Provide a Docker image including all dependencies required to build the book, to avoid users and contributors having to install dependencies (well, apart from Docker).

Strongly statically typed?

On type_system.asciidoc there's the following sentence:

These aspects of the Erlang type system, strongly statically typed
with an order on the types puts some constraints on the implementation
of the language.

Given what is previously written in the chapter saying that Erlang is strongly statically typed confuses me. Is it really correct?

Compilation Failure

MacOS. All listed dependencies installed.

When executing 'make' I get this error.

Processing title:Chapter 1. Introducing the Erlang Runtime System undefined method absolute_path?' for File:Class
Did you mean? absolute_path
Use --trace to show backtrace
make: *** [beam-book.pdf] Error 1`

Chinese Translation Request

I started the translation work in my fork https://github.com/hitdavid/theBeamBook
and suffer some error in Chinese charset display in pdf build using asciidoc.

I'll keep my focus on the content first, and maybe solve asciidoc config problem at last.

Please keep on write the book till complete, It's a GREAT work, thank you contributors.

Observer images are missing

Hi, the images like followings are missing in chapter 3:
fig-observer_system, images/observer_applications.png, images/observer_processes.png, images/observer_code_server.png

(If it's ok to add images of observers on Mac, I can add them).

Travis CI Integration

It would be great to get some basic CI pipeline in place, so that only Pull Requests which would not break compilation could be merged. The pipeline could also be used to ensure that the generated PDF and HTML are stored as artifacts, so that readers would not need to build their own.

Typo on BEAM instructions

call Arity Label
Does a call to the function of arity Arity in the same module at label Label. First count down the reductions and if needed do a context switch. Current code address before the call is saved into CP.

The before the call should be after the call.

Source code ref.

show.erl file is missing

In PCB section, there is a mention to the file show.erl to show the tag type, but I don't see the file. Is it supposed to be added ?

HTML Entity for * (&#42;) is not rendered properly

I haven't checked the whole of the book yet, but at least from the BEAM section I can see:

screen shot 2018-10-29 at 08 24 22

I'd happily replace it all over by the * symbol but I'm not sure what consequences it'll have for the PDF rendering.

Build the PDF artefact and tag it as release version

Hi,

it would be useful to have a pdf available to download that represents the current release version.

Currently we could create version tag 0.1.0 and deliver the pdf. This way we avoid to make everyone to build the project, install the dependencies and "fight" against the troubles that can arise when building.

For example rebar3 project make the binary version available in each release, what is really cool.

Erik, I was waiting to buy your book before it was canceled. Thank you very much for make it available now as open source.

(see you at EUC2017 )
Cheers

Pedro

Add support for Chinese?

Hi guys, I wonder whether this project have plans for other languages? I could translate it to Chinese if it's ok ;)

Missing words / Typo in The Erlang Type System section

In "The Erlang Type System" section, the 2nd paragraph below the "Erlang Type Lattice diagram" (diagram is Figure 18 in the web version on your website), it says:

Both (1 < 1.0) and (1.0 < 1) are false, and (1 =< 1.0 and 1 >= 1.0) and (1 =/= 1.0).

I believe it should instead say:

Both (1 < 1.0) and (1.0 < 1) are false, while (1 =< 1.0), (1 >= 1.0) and (1 =/= 1.0) are true.

Misprint or not?

In the chapter 1.3.2 you example some OTP behaviours, one of them gen_ftp:

The fifth layer, OTP, supplies the Erlang standard libraries. OTP originally stood for "Open Telecom Platform" and was a number of Erlang libraries supplying building blocks (such as supervisor, gen_server and gen_ftp)

May be you means gen_tcp?

The Debugging chapter

I was thinking of making turning the chapter on the debugger into a broader chapter on "Finding and Fixing Bugs"

The top level idea is:

  • Finding and Fixing Bugs
    ** In a long running 24/7 system there will always be bugs,
    or at east unwanted features. In this chapter we will look
    at different ways to find and fix bugs, without disrupting
    the running service.
    ** You want to read this chapter to learn about different ways to test code and
    about the tools and frameworks that are available for testing.
    This chapter will also cover some common sources of bugs and problems
    in the system, such as deadlocks, message overflow, and memory
    problems, and how to identify and fix them.
    ** You will learn about the "let it crash" principle and how to
    best use it in your system. You will also learn how exceptions
    works and how to design your system with supervisor trees.
    You will learn how to use the debugger to debug your program
    and how to use GDB to debug the runtime system.
    You will learn how to read crash dumps, you will learn
    why you often don't get crash dumps and what to do about it.
    You will learn how the Erlang debugger works.
    You will learn how redbug can make debugging easier and
    somewhat safer to run on a live system.
    You will learn a little bit about how to use gdb
    to debug on the ERTS level.
    You will also learn a little about dtrace and systemtap.
    And maybe, perhaps, you will pick up some debugging philosophy.
    ** You will be able to test your system in a systematic way,
    and you will be able to test individual components of your system.
    You will be able to identify some common mistakes and problems.
    You will be able to use the debugger.

Suggested outline:

Testing

Eunit

Common Test

Mocking

The Usual Suspects

Deadlocks

Mailboxes

Binaries

Let it Crash

Exceptions

Supervisors

The Debugger

The Erlang debugger dbg

redbug

starting, stopping

choosing what to debug

setting limits.

Crash Dumps

Debugging the Runtime

gdb

gdb macros

systemtap and dtrace

Remove the Revision History from the book and add a THANKS section, instead

The Revision History section in the book is becoming larger and larger and I see very little point in maintaining a complete history of the book inside the produced PDF, now that the project is open source. I would personally remove and add a THANKS section, instead (ideally automated by running the git log, extracting the contributors and u-sorting them. Opinions?

Atom no longer mandatory

The chunk named Atom is mandatory

This is no longer true, either the Atom or AtU8 chunk need to be present.

Suggestion: Make explicit the version of Erlang in the book

As stated here, it may be helpful to explicitly state somewhere in the book which version it targets.
Then readers can guess about the contents of the book even after the implementation is changed later. Since sometimes there are major changes that affect the contents of this book.

Possible wording issue in scheduling section with regards to reduction checking

In the Scheduling: Non-preemptive, Reduction Counting section I think there is a minor mistake in the wording of the sixth paragraph, but I'm not 100% sure.

It reads:

Each function call is counted as a reduction, and BEAM does a test upon entry to each function if the process has used up all its reductions.

Should this instead be (changes in bold):

Each function call is counted as a reduction, and BEAM does a test upon entry to each function [to check] if the process has used up all its reductions.

It currently reads as if BEAM only performs the test if the process has already used up its reductions, but from surrounding context it seems like BEAM should be performing the test to determine if reductions are used up or not.

I'm not an expert (yet!) so this could just be my lack of understanding of how the BEAM works rather than an issue with the wording.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    πŸ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. πŸ“ŠπŸ“ˆπŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❀️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.