Commands

pwd

Print the current working directory you're in.

cd

Change working directory.

ls

List the files and directories in the current working directory. Note that options can be combined, like ls -alt.

• ls -a: List all files, including . files.
• ls -l: List all files as a long list, with more details about each file. From left to right the information shown is: permissions, number of hard links, owner name, owner group, file size, time of last modification, and file/directory name.
• ls -t: List all files by last modified date.

cp

Copy files from one directory to another, or from one file into another.

• cp frida.txt lincoln.txt: Copy frida.txt into lincoln.txt.
• cp biopic/cleopatra.txt historical/: Copy cleopatra.txt into historical directory.
• cp biopic/ray.txt biopic/notorious.txt historical/ : Copy multiple files into historical directory.
• cp * satire/: Copy everything in the current working directory into satire directory.
• cp m*.txt scifi/: Copy all txt files that start with the lowercase letter m into scifi directory.

mv

Move files.

• mv superman.txt superhero/: Move superman.txt into superhero directory.
• mv blah.txt yes.txt hello/: Move multiple files into one directory.
• mv batman.txt spiderman.txt: Rename batman.txt to spiderman.txt.

rm

Delete files and directories.

• rm blah.txt: Delete file from current working directory.
• rm -r slapstick: Delete slapstick directory and all its subdirectories from current working directory. (-r option = "recursive)

cat file.txt

Outputs the contents of a file to the terminal.

wc file.txt

Outputs the number of lines, words, and characters found in the file.

sort file.txt

Sorts the contents of the file alphabetically.

uniq file.txt

Removes adjacent duplicate items. Duplicate items that are not adjacent remain.

grep

Stands for global regular expression print. Searches files for lines that match a pattern and returns the results. It's also case sensitive.

• grep Mount mountains.txt: Searches for 'Mount' in mountains.txt
• grep -i Mount mountains.txt: Searches for capital OR lowercase strings that match 'Mount' in mountains.txt
• grep -R Arctic /home/ccuser/workspace/geography: Searches all files in the given directory for the string "Arctic" and outputs filenames and lines with matched results (-R = "recursive")
• grep -Rl Arctic /home/ccuser/workspace/geography: Same as above, but only outputs the filenames where matches were found (-R = "recursive", -l = "files with matches")

sed

Stands for stream editor. Accepts stdin and modifies it based on an expression, before displaying it as stdout. Similar to "find and replace".

• sed 's/snow/rain/' forests.txt: Expression s stands for "substitution". snow = the text to find, and rain = the replacement string. Thus, sed searches forests.txt for the string "snow" and replaces it with "rain". Will only replace the FIRST instance of "snow" on a line.
• sed 's/snow/rain/g' forests.txt: Same as above except now, because of the g (global) flag, sed will replace ALL instances of "snow" with "rain".

The below todos are generated from the roadmaps for backend dev and devops. As I think they complement each other, and they both represent spheres of knowledge that I want to be intimately familiar with, I'm just creating one todo list.

As I complete each item I'll check it off. "Completeness" of an item is a bit subjective, but I'll judge something is complete when I have something tangible to show for that todo. Whether that's a program, a blog post, or something else, I will link to such "evidence" here.

Roadmap Todos

• Pick a language: Go
• Learn & practice what you've learned by building some command line apps
  • gophercises
    • quiz
  • ehonnnavi.net web scraper
  • aggregate and send self a daily email of hackernews and lobste.rs
  • aggregate and send self a daily email with google analytics data for all web properties
  • build a pretty command line interface for working with go processes
• Understand different OS concepts
  • Process management (Evidence: Notes, Blog post)
  • Threads and Concurrency
  • Sockets
  • I/O Management
  • Virtualization
  • Memory / Storage
  • File Systems

What are threads?

Previously we learned that a program is just a set of instructions for the computer, and a process is the execution of those instructions.

"A thread is the basic unit to which the operating system allocates processor time."
"A thread is a path of execution within a process. A process can contain multiple threads."
"A thread is also known as lightweight process. The idea is to achieve parallelism by dividing a process into multiple threads. For example, in a browser, multiple tabs can be different threads. MS Word uses multiple threads: one thread to format the text, another thread to process inputs, etc."
"The primary difference is that threads within the same process run in a shared memory space, while processes run in separate memory spaces.
Threads are not independent of one another like processes are, and as a result threads share with other threads their code section, data section, and OS resources (like open files and signals). But, like process, a thread has its own program counter (PC), register set, and stack space."
"A thread is the unit of execution within a process. A process can have anywhere from just one thread to many threads."

"When a process starts, it is assigned memory and resources. Each thread in the process shares that memory and resources. In single-threaded processes, the process contains one thread. The process and the thread are one and the same, and there is only one thing happening.

In multithreaded processes, the process contains more than one thread, and the process is accomplishing a number of things at the same time (technically, sometimes it’s almost at the same time—read more on that in the “What about Parallelism and Concurrency?” section below)."

"In computer science, a thread of execution is the smallest sequence of programmed instructions that can be managed independently by a scheduler, which is typically a part of the operating system.[1] The implementation of threads and processes differs between operating systems, but in most cases a thread is a component of a process. Multiple threads can exist within one process, executing concurrently and sharing resources such as memory, while different processes do not share these resources. In particular, the threads of a process share its executable code and the values of its dynamically allocated variables and non-thread-local global variables at any given time."

Concurrency vs. Parallelism

"A question you might ask is whether processes or threads can run at the same time. The answer is: it depends. On a system with multiple processors or CPU cores (as is common with modern processors), multiple processes or threads can be executed in parallel. On a single processor, though, it is not possible to have processes or threads truly executing at the same time. In this case, the CPU is shared among running processes or threads using a process scheduling algorithm that divides the CPU’s time and yields the illusion of parallel execution. The time given to each task is called a “time slice.” The switching back and forth between tasks happens so fast it is usually not perceptible. The terms parallelism (genuine simultaneous execution) and concurrency (interleaving of processes in time to give the appearance of simultaneous execution), distinguish between the two types of real or approximate simultaneous operation."

Resources

https://www.backblaze.com/blog/whats-the-diff-programs-processes-and-threads/
https://en.wikipedia.org/wiki/Thread_(computing)

What is a process?

Before talking about a process, first we must define what a program is. In short, a program is a passive collection of instructions for the server to run.

A process is the active execution of those instructions. The basic flow is:

write code (program) > computer compiles it to binary > run the binary (process)

A single program can be run multiple times at the same time, thus creating multiple processes.

Processes are represented by process identifiers (PID), a unique numeric value. (These numeric values can be recycled, so two processes could have the same PID, just not at the same time.)

Processes can be short or long-lived.

When a program becomes a process, it can be divided into four sections:

• Stack: contains the temporary data, like method/function parameters, return addresses, local variables
• Heap: dynamically allocated memory to a process during runtime
• Data: global and static variables
• Text: compiled program code

Types of processes

Foreground: A foreground process is initialized and controlled through a terminal session. A user starts these, not the system.

Background: Not connected to a terminal and doesn't expect any user input. A daemon is a type of background process that starts at system startup and continues to run forever as a service.

Processes can also be divided into Parent and Child processes. Child processes are created by a parent process at runtime.

Process states

Processes can have a number of possible states. The names for these states vary by operating system, but the following five are pretty standard.

• New: process is being created
• Ready: process has all resources available that it needs to run, but CPU is not currently working on this process's instructions
• Running: CPU is working on this process's instructions
• Waiting: process can't run at the moment because it's waiting for some resource to become available or an event to occur (keyboard input, disk access request, a child process to finish)
• Terminated: process has completed