unit-code-cnbd / cs198.1x Goto Github PK

1.01. What is Bitcoin?

1.02. Bitcoin vs. Banks

1.03. Bitcoin from the Ground Up

1.04. Identity (Stage 1)

1.05. Transactions (Stage 2)

1.06. UTXO’s

1.07. Record Keeping (Stage 3)

1.08. Consensus (Stage 4)

1.09. Forking

1.10. Bitcoin Review

1.11. Chapter 1 Summary

2.01. Pre-Bitcoin

2.02. Libertarian Dreams

2.03. Bitcoin Precursors

2.04. Bitcoin Invention

2.05. Hacks and Scandals

2.06. Bitcoin Bubble and Altcoins

2.07. Scalability

2.08. Ethereum Timeline

2.09. Intro: Enterprise Blockchain

2.10. Intro: State of the Industry

2.11. Chapter 2 Summary

3.01. Motivations and Definitions

3.02. Cryptographic Hash Functions

3.03. SHA256

3.04. A Tamper-Evident Database

3.05. Merkle Trees

3.06. Proof-of-Work: Mining

3.07. Signatures and Authentication

3.08. Bitcoin Script

3.09. P2PKH

3.10. Chapter 3 Summary

4.01. Key Components

4.02. Wallets

4.03. Wallet Mechanics

4.04. Mining

4.05. Real World Mining

4.06. Bitcoin Governance

4.07. Chapter 4 Summary

5.01. Pool Strategies

5.02. Pool Hopping

5.03. Pool Cannibalization

5.04. Nash Equilibrium and the Tragedy of the Commons

5.05. Double Spending

5.06. Censorship Attacks

5.07. Selfish Mining

5.08. The Bitcoin Network

5.09. Chapter 5 Summary

6.01. Smart Contracts

6.02. Ethereum

6.03. Ethereum Virtual Machine: Overview

6.04. Ethereum Use Cases

6.05. Blockchain vs. Internet

6.06. Blockchain Generalizations

6.07. Supplement: Use Cases

6.08. Ethereum Ecosystem

6.09. Chapter 6 Summary

This course is a comprehensive and in-depth overview of the fundamental concepts of the crypto space, with a particular emphasis on Bitcoin. By building intuition first from the context of cryptocurrencies, the first application of blockchain, we understand the key strengths and distinguishing factors of blockchain versus traditional database systems. We then leverage these core features of blockchain to solve new problems.

The course is divided into 6 modules: Bitcoin High-Level Overview, Blockchain History, Bitcoin Mechanics Technical Overview, Bitcoin in Real Life, Game Theory & Network Attacks, and Ethereum & Smart Contracts. 

1. Bitcoin Protocol & Consensus: A High-Level Overview

We begin with some fundamental concepts such as the basic properties and intent of centralized/decentralized currency. We then build an in-depth understanding of Bitcoin from the ground up, divided into four stages: a. Identity, b. Transactions, c. Record Keeping, and d. Consensus. 

2. Blockchain History: From the Cypherpunk Movement to JP Morgan Chase

This module delves into the origins and historical significance of Bitcoin. We look into the roots of Bitcoin in the Cypherpunk movement and Libertarian ideals, and examine the revolutionary significance of Bitcoin compared to some of its early predecessors. We then move onto exploring the history of the crypto space as a whole.

3. Bitcoin Mechanics & Optimizations: A Technical Overview

We examine the in-depth mechanics behind Bitcoin, such as the Bitcoin network, cryptography and cryptographic hash functions, Bitcoin Script, privacy, and hash commitment schemes.

4. Bitcoin In Real Life: Wallets, Mining, and More

We examine the most frequently used real-world aspects of Bitcoin, such as wallets, wallet mechanics, mining, transactions, and Bitcoin governance. We explain the various ways one can interface with the Bitcoin network, depending on the specific software they run. 

5. Game Theory & Network Attacks: How to Destroy Bitcoin

We look into how to destroy Bitcoin, including various network attacks. Specifically, we look into vulnerabilities such as pool cannibalization, double spending and forking attacks, network attacks, the Goldfinger attack, malicious mining profit strategies, and 51% attacks.

6. Ethereum & Smart Contracts: Enabling a Decentralized Future

This module focuses on the properties behind the second largest blockchain platform, Ethereum. We introduce the Ethereum Virtual Machine and the idea of Turing completeness and examine some of the key protocol differences between Bitcoin and Ethereum, such as the UTXO vs. accounts model and functionality. We then look into some of the use cases of Ethereum, and conclude with an overview of smart contracts and building decentralized applications. While the last modules primarily focus on cryptocurrencies, this module encourages students to think about blockchain use cases outside of cryptocurrency.

• Bitcoin and Cryptocurrency Technologies by Arvind Narayanan, Joseph Bonneau, Edward Felten, Andrew Miller, and Steven Goldfeder

• Mastering Bitcoin by Andreas Antonopoulos

Also, a good unofficial resource is the Blockchain at Berkeley Public Slack, where we discuss various topics related to blockchain. You can request access to our Slack workspace at the bottom of the Blockchain at Berkeley website under "Join Blockchain at Berkeley on Slack."

Rustie Lin, Mengyi (Gloria) Wang, Nadir Akhtar, Jennifer Hu, Janice Ng.

Blockchain Fundamentals started off as the Cryptocurrency DeCal (Democratic Education at Cal, student run) course on UC Berkeley campus in Fall 2016, taking inspiration from existing courses, videos, and textbooks. From the beginning, the vision was to lower the barrier of entry and provide a survey into the blockchain and cryptocurrency space -- to explain concepts from the ground up, as clearly and concisely as possible.

Each semester since then, we have had many student volunteers take time out of their already hectic schedules to help develop and fine-tune content, to make Blockchain Fundamentals what it is today. 

Blockchain Fundamentals is constantly being improved, in terms of new information as well as course design, but its vision has remained constant. Here are some of the major contributors to Blockchain Fundamentals, who have not already been featured on the previous page.

Max Fang, Philip Hayes, Sunny Aggarwal, Aparna Krishnan, Gloria Zhao, Gillian Chu, Brian Ho.

Welcome to the first module in Cryptocurrencies: Bitcoin and the Crypto Space, the first course in the Blockchain Fundamentals program.

Whether you’re an enthusiast, trader, aspiring blockchain developer, or just a curious individual looking to learn where and how blockchain can be used, you’ve come to the right place.

Blockchain technology is poised to affect so much of the world today, -- in the financial, energy, identity, and IoT industries just to name a few.

But blockchain use cases weren’t always this pervasive.

Throughout its infancy, blockchain found its primary use in cryptocurrencies -- and Bitcoin was its original inspiration.

To understand the wider crypto and blockchain space, we must first seek to understand Bitcoin, the oldest and most widely understood blockchain application.

Throughout this module, you’ll first learn what Bitcoin is and what its primary motivations are.

Then, we’ll dive into four-part buildup of Bitcoin consensus and explain why Bitcoin is built the way it is.

What is Bitcoin? 1 min, 37 sec

In this section, you will learn about the origins and motivations of Bitcoin, and the key distinctions between cryptocurrencies and normal currencies.

We’ll clear up some common misconception about Bitcoin and Blockchain, and also explain briefly Bitcoin’s origins in the Cypherpunk movement.

We’ll then draw a comparison between Bitcoin and Banks and analyze each of the features and services they provide.

And this all sets the stage for the next section, in which we’ll build Bitcoin from the ground up, incorporating each of these features and services and seeing how they fit together.

Most people struggle to answer the question, “What is Bitcoin?,” because there are so many different ways to respond.

First, Bitcoin is considered the first and the most widely used cryptocurrency.

A cryptocurrency is a completely digital, decentralized currency that is built using principles of computer science, cryptography, and economics.
The term “Bitcoin” refers to the protocol governing this currency.

Second, bitcoin lowercase refers to the actual units of currency.

A Bitcoin user will say they have a certain amount of bitcoins, similar to how we say we have a certain amount of dollars when referring to the US Dollar.

Third, Bitcoin is the inspiration for the blockchain, which is the underlying data structure of this cryptocurrency.

A data structure is a virtual format for organizing, retrieving, and storing information.

The Bitcoin blockchain in particular stores a permanent history of all transactions to ever occur in the history of Bitcoin.

It is an append-only ledger, meaning that any information added to the ledger cannot be deleted.

But most importantly, Bitcoin is a cultural revolution.

Rooted in ideals from Cypherpunks and libertarians, Bitcoin represents a shift towards privacy and decentralization.

This cryptocurrency is not backed by any central organization, government, or company.

Instead, Bitcoin is built by the users, for the users.

As we said, Bitcoin was inspired by the Cypherpunk Movement of the late 80s.

Cypherpunks advocate for the protection of privacy using cryptography.

They don’t trust governments, corporations, or large organizations to respect privacy.

These points of centralization accumulate a great deal of power over society by collecting unimaginable amounts of information from millions of users.

And the Cypherpunks were some of the first to be concerned about central entities stripping away the freedom of the general public.

One massive point of centralization in modern day society is the financial system, where:

banks govern the economies of entire countries.

Several different companies and researchers attempted to make a decentralized or anonymous currency, but all of them failed.

Bitcoin was the first technology to succeed as a cryptocurrency.

The Bitcoin whitepaper, or research paper, was published in October 2008 by Satoshi Nakamoto.

The whitepaper was a 9-page, concise proposal for the structure and function of a peer-to-peer electronic currency.

Satoshi Nakamoto is a pseudonym, or a false identity, of an individual or a group of individuals.

No one knows their real identity. However, what's important is that this whitepaper envisioned a currency where users do not rely on financial interimagesries or trust anyone in order to make transactions with each other.

In Bitcoin, users do not need to use their real world identities; instead, they are represented by addresses, strings of random letters and numbers.

Bitcoin takes control out of the hands of third parties and gives users the freedom to transact while protecting their privacy.

How does Bitcoin do it? On a high level, the Bitcoin network validates transactions and stores the entire transaction history.

The Bitcoin network is a group of users communicating with each other as part of the Bitcoin protocol.

This network serves as the substitute for the central bank and must have certain properties to function correctly.

Bitcoin is trying to create an open, accessible cryptocurrency not subject to censorship or centralization.

But what are the problems?

Keep in mind the problems of trying to create an open, accessible cryptocurrency not subject to censorship or centralization:

There are no central parties to ask for information about user accounts, and there are no central parties to kick out or censor malicious users.

Decentralized networks generally suffer from these problems, leading to inconsistencies between parties and malicious messages infecting the network.

The most popular attack is known as the double spending attack, an attack where some value is used for more than it i’s worth.

In real life, it’s easy to prevent double spending: since dollar bills can’t be copied and pasted.

However, in digital currencies, there needs to be assurance that the virtual tokens have not been promised to more than one person.

Bitcoin as a technology is trying to solve a very specific problem in the realm of distributed systems: when any “node,” or computer within the network, can come and leave as it pleases and behave however it likes.

There are enormous possibilities for failures given the complete removal of centralization, which is why there were so so many Bitcoin’s predecessors to Bitcoin which failed.

How does bitcoin solve these problems?

Bitcoin solves these problems through two things:

First, the blockchain, and the Proof-of-Work consensus protocol, both of which are Satoshi Nakamoto’s most popular and influential innovations.

Because of these two things, anyone with access to internet and a computer can join the Bitcoin Network.

There are no banks or any equivalent of the Federal Reserve on the Bitcoin Network.

Instead, everyone can verify and audit the transaction history on their own.

And even the creation of money is decided not by a central authority, but through the process of mining, or Proof-of-Work.

Bitcoin aims to get rid of the central entity, the bank.

In order to understand how it does that, we first need to understand what purpose a bank serves and what features it provides to users, then understand the parallels in Bitcoin.

The first thing that banks do for us is manage accounts.

Banks verify that we are the legitimate owner of the bank account and only we can spend the money or the funds. How does a bank do that? Banks will ask us to provide identification before any activity can take place.

Every transaction we conduct can be traced back to our identity.

On top of that, banks transfer and redeem money on your behalf.

We send money to each other through banks.

We rely on banks to honestly record our account balances.

This way, we don’t need to send money through envelopes to relatives -- we let these central institutions move money in safe and, established ways on our behalf.

To keep track of all this information, we rely on banks to keep track of our account balances.

Banks update our account balances whenever we make a new transaction.

They also let us see statements so that you’re aware of your past history of activity.

But most importantly, banks provide trust: banks are run by educated professionals from top-tier universities and under the constant regulation of the U.S. Government.

If you trust the quality of education and standards of the U.S. government, then you can trust the bank.

But if you don’t, then you start looking for alternatives.

And this is where Bitcoin comes in.

Let's take a look at how Bitcoin can fulfill a bank's functions.

In Bitcoin, identity and account management are completely autonomous.

Each user of Bitcoin creates their own identity instead of asking a bank to create one.

Anyone can generate a Bitcoin identity on their own.

This identity is disconnected from their real world identity, providing a high degree of privacy.

On top of that, transactions are also peer-to-peer: instead of talking to a bank which will talk to another bank which will eventually talk to the recipient of some money, we can make transactions directly can be made between with our peers and be confident that they are confirmed by the rest of the network.

Therefore, in Bitcoin, users can send funds to each other directly knowing that their transactions will be validated by the entire network without the presence of a trusted third party.

To store all this information, each Bitcoin user gets to possess their individual copy of the ledger.

This decentralized approach of record keeping ensures the integrity of data despite the presence of faulty nodes who might record the information dishonestly.

The decentralized nature of bitcoin also prevents the risk of single point of failure.

In the event that a particular node is hacked in bitcoin, because everyone is a record keeper, the rest of the network can still ensure the integrity of the transaction record and keeps running.

But finally, there is still a need for trust in Bitcoin: instead of trusting people in suits, we trust math and logic.

We trust that the Bitcoin protocol is correct, allowing us not to trust the users and still have certainty that transactions are being validated correctly.

We trust in the incentive alignment and publicly verifiable, tamper evident ledger instead of our fellow users.

All of these pieces together make Bitcoin the technological revolution that kicked off the Cryptocurrency Movement.

We saw in the section on Bitcoin versus Banks that we need to enable account and identity management, money transfer as a service, record management, and trust.

In this section: titled Bitcoin from the Ground Up, we’ll take each of these four concepts and see how they fit into Bitcoin’s architecture.

By the end, you should have a high level understanding of how Bitcoin works, and why things are the way they are in Bitcoin.

We’ll start off by taking a look at one of the most fundamental ideas in Bitcoin:

Identity

We saw earlier that banks need to keep track of the accounts and identities of their customers.

But how do we do this in Bitcoin, without a bank or central entity to keep track of who’s who and who owns what?

To understand why we need identity in Bitcoin in the first place, let’s understand first why we need identity at all in the context of currencies.

In currencies, we need to ensure that all users can authenticate themselves through some identification method, and that all identification methods have integrity.

Authentication is required to ensure that no one else acts on your behalf.

Claiming, receiving, and spending money on your behalf are things that only you should be able to do.

Like a bank, by associating yourself with your funds through authentication, you are able to receive money from others and spend your own money.

Only you have access to your own money, since others are not authenticated to handle your money.

Without an authentication process, anyone could spend my money, which is obviously undesirable.

Additionally, authentication can also be used to enforce blaming.

If someone tries to do something incorrect within the network, such as spend someone else’s funds, you want to be able to call them out with proof.

For example, if someone tries to withdraw your funds from a bank, you would want a record of this, including the identity of who was trying to take your funds, to get rid of malicious activity in the future by banning or refusing to interact with those people.

Integrity, the other half of identity, means that all our authentication methods cannot be replicated by anyone else.

We can understand integrity by the process of signing a check: once you sign a check or transaction, no one should be able to intercept and/or manipulate it.

The check has been signed by you, and since no one can replicate your signature, it should be tamper evident.

Integrity ensures that no one but you, the signer, can use your signature.

Imagine the following scenario in Bitcoin: If I wanted to send Alice 10 bitcoins, she shouldn’t be able to add another zero and make me send her 100 bitcoins.

She also shouldn’t be able to copy or replicate my signature elsewhere.

Only by preventing all these things can we trust the integrity of Bitcoin identities.

Identity is a simple concept appearing everywhere in daily life.

For example, houses have both addresses and mailbox keys.

When you ask people to send you mail, you give away your address, so they know where to send the package to.

Meanwhile, you alone control the mailbox key.

Similarly, emails have aliases and passwords.

People who want to send you email have access to your email address or alias, while you alone have access to the password to your email account, so only you can read these emails.

Bitcoin, following this pattern, has both public keys and private keys.

The items in orange on the left hand side -- mailbox addresses, email aliases, and public keys -- represent one’s public identity.

They are what you give out to the public so that they know how to communicate with you, so that they know how to recognize and identify you.

Meanwhile, the items on the right hand side in red -- mailbox keys, email passwords, and private keys -- are secret keys that you alone should own.

With these personal keys, you access the orange items on the left.

It gives you control and ownership.

It gives you your identity.

If anyone else gets their hands on your email password, for example, they can pretend to be you, receiving and sending emails on your behalf.

Similarly, you would never want anyone else to have your Bitcoin private key as that would give control of your Bitcoin public key and, by implication, your Bitcoin identity.

In Bitcoin, you can think of your public and private key as a chest and key, respectively.

You use your private key to prove you have ownership of your public key and to access the funds associated with that public key, just as you can use a physical key to unlock a chest that contains your physical money.

To reiterate, public keys are used for receiving, and private keys are used for redeeming.

Received money is associated with the public key, and you can access or spend those funds with your private key.

You are safe letting people know about your chest of bitcoins -- your public key – so long as you don’t give them the key to get inside -- your private key.

A small note: in actuality, other users send transactions to your address, not your public key, as your Bitcoin public key is not identical to your Bitcoin address.

Your address is actually derived from your public key.

We will make the distinction clear when we go more in depth into Bitcoin mechanics, but you can for now think of addresses, pseudonyms, and public keys as synonyms.

The reason why we need this public-private key pair scheme in Bitcoin is because there is no central authority to generate unique identities for users.

This means that users have to generate their own identities.

To generate an identity, a user picks a private key at random generates the public key from the private key through a mathematical function.

You may be concerned about the danger of two users ending up with the same public key without a central registry ensuring that no two people have the same public key.

A bank, for example, has an internal list of users with unique identifiers and can easily assign a new, unique identity to any new user.

How do we simulate a bank’s identification process in Bitcoin?

How do we trust that no two users end up with the same identity?

To illustrate the probability of two users choosing the same identity as someone else in Bitcoin, let’s consider something we’re all familiar with: the Earth.

Bitcoin has 2^160^th different possible addresses.

To get a sense of how many different addresses exist for users to choose out of, let us compare the number of addresses in Bitcoin to the number of grains of sand on Earth.

The Earth has 2^63^rd grains of sand.

Imagine every person in the world choosing a grain of sand at random.

The likelihood of two people picking the same grain of sand is less than 0.0001%.

In comparison, the number of addresses in Bitcoin is many, many magnitudes greater than the number of grains on sand on Earth, making the probability even smaller.

Let’s consider something more drastic: imagine that for every grain of sand on Earth, there exists another Earth, also with 2^63^rd grains of sand.

Now we have 2^63^rd times 2^63^rd grains of sand per earth, meaning that there are a total of 2^126^th grains of sand.

Even the number of grains of sand upon all these Earths is only 0.0000000058% (8 zeroes) of all the possible Bitcoin addresses.

So now that we have a clear understanding of the structure and purpose of identity in Bitcoin, we can start asking the question, “How do we make transactions between one another in Bitcoin?”

After all, that’s the primary goal of the Bitcoin network -- to enable secure transactions between any two users in the network.

As we did before with the concept of identity, we will now be analyzing the idea of a transaction.

Take a moment to ask yourself, “What makes a transaction valid?”

For a transaction to be valid, it must have these three components:

1. a proof of ownership, aka a signature,

2. available and sufficient funds to spend, and

3. a guarantee that no other transaction is using or has used the same funds.

Let’s understand these criteria in the context of checks and banks:

When you want to make a transaction via a check, you have to sign the check to validate it -- that’s a proof of ownership.

The bank then has to verify that you have enough funds in your account -- that you have available and sufficient funds for the transaction.

They must also ensure that you do not spend the same money more than once -- that you cannot send two or more checks each spending 100 dollars when your account only has 100 in total.

To ensure that one uses funds not in their possession, all the same conditions apply to Bitcoin.

To enforce this, Bitcoin uses what’s known as a UTXO, or Unspent Transaction Output, model.

Traditionally, when we think of banks, we think of a single account where all our funds are aggregated into one count.

For example, after buying $10 worth of burgers and fries, your account balance goes from 20 to 10 dollars.

This is not the case with Bitcoin.

Accounts are easy for users to understand but surprisingly difficult for computers when it comes to all the complexity of a decentralized network.

There is no longer a central entity to track all the transactions being made by my account, nor a central clock to track when transactions are being made.

If I make three different transactions of 5 bitcoins from an account of only 10 bitcoins, which of the two transactions should go through?

How do we ensure that we do not accidentally let all three transactions go through?

To make transaction processing much easier and more secure, users do not spend from an account; instead, they spend directly from transactions made to them.

An easy way to wrap our heads around this unintuitive model is to think of UTXOs as piggy banks!

Every time a transaction is made to us, we put all that money into a UTXO, or piggy bank.

When we want to spend money, we break open that piggy bank...spend whatever we like, and then put the rest into another piggy bank.

(It’s hard to put a piggy bank’s shattered pieces back together.)

This way, the complexity of checking for transaction validity goes down.

Instead of asking the more difficult question, “Is this account trying to, at this time, spend more money than it owns across multiple transactions?,” we only need to ask, “Does this single piggy bank have enough funds?”

On the other hand, the complexity of keeping track of one’s own funds goes up, as each of these piggy banks has to be tracked and secured individually.

The amount of bitcoin you own is calculated by summing up the value of each of your piggy banks, or UTXOs.

Here’s an example of how transactions work in Bitcoin:

Let’s say that Gloria has two UTXOs, from past transactions, one worth one hundred (100) bitcoins... and one worth fifty bitcoins.

She, being incredibly wealthy and generous, wants to send one hundred and one bitcoins to me (Rustie).

To start off, she first breaks open the one hundred (100) UTXO and makes that an input for the transaction to me.

However, she recognizes that she doesn’t yet have enough in the input, so she breaks open her next UTXO worth fifty (50)...and makes that a second input for the transaction.

The transaction now currently has two inputs, one worth one hundred (100) and one worth fifty (50), and one output worth one hundred and one (101) to me.

What about the remaining 49 bitcoins that were originally Gloria’s?

Where do those go?

To make sure that Gloria can spend these bitcoins at a later date, she makes a second output to herself known as a change UTXO, or another piggy bank, containing all the remaining unspent bitcoins from the inputs.

This way, she can give one hundred and one (101) of her one hundred and fifty (150) bitcoins to me and keep her remaining forty nine (49) even though she doesn’t have UTXOs which sum up to exactly the amount she wants to spend.

From here, if I'm equally generous and want to give that remaining amount to Derrick, I can just make my new UTXO an input into a different transaction...the output of which will be Derrick’s new UTXO.

This is how Bitcoin effectively keeps track of individual transactions and prevents anyone from spending incorrect amounts of bitcoin.

We have now understood how to assign unique identities to every user through public and private keys and conduct transactions between these entities through the use of the UTXO, or Unspent Transaction Output, model.

Naturally, the next question to ask is: How do we keep track of the history of transactions?

After all a user’s current balance can be described as a number of transactions, summing to the current amount, and if we don’t know what transactions have happened in the past, we can’t determine what’s valid in the future.

Consider the diagram above.

Let this represent our network, which includes 5 entities.

Each of these circles represents an identity: me (Rustie) in green on top, Gloria in red on the top right, Nick in purple on the bottom right, Nadir in blue on the bottom left, and Derrick in orange on the top left.

We are all connected to each other on this network, as is apparent by the straight edges connecting each circle.

For simplicity's sake, I’ve replaced all public keys with names.

I’ve also replaced the UTXO model with a basic table on the top right hand side of the screen.

We need to store the history of transactions for obvious reasons: to know who owns what in the present, and to use this history to confirm the validity of future transactions.

To save this information, we need some form of database.

A database is a store of information, and there are many types and implementations of databases.

To understand which type of database we need to use in Bitcoin, let’s recall again the requirements of the Bitcoin protocol: we want no central entity in control of the information in the network, and we want a way for anyone to be able to read and write to the history of transactions.

Hence, we want to use a distributed database: as its name entails, information stored in a distributed manner, meaning that the information is not stored by one entity or only in one location.

Because Bitcoin aims to be decentralized, we want to use exactly that.

What does this distributed database look like, and where exactly is it stored?

There’s no central entity to hold on to our information, so the closest thing is having a chosen set of entities hold onto this history.

If we assigned several entities in the network the responsibility of maintaining and sharing our ledger of transactions, we are still seeing some parts of centralization sneak their way into our protocol because we would have to trust the maintainers of this distributed database, going against Bitcoin’s aim to be a trustless system.

We must find another way.

Instead of having any selected maintainers, let’s make a simple and straightforward choice: we have everyone keep a copy of the ledger.

We make everyone the bank.

To get as far as possible from centralization, every individual entity in Bitcoin should be equal.

If every person stores the ledger, then every person has just as much right and legitimacy as the next person to vote on the validity of transactions.

Every person has control of their own data, and no person can decide for anyone else.

There’s no one person to bribe, no one person to hack, no one person to cheat in order to alter the database.

This is the maximum state of individual independence possible for maintaining this history of transactions.

We know that we want each person to store the ledger, but what should the database actually look like?

What data structures hold the transaction history?

We might naively decide to store every transaction individually, but for a network that may be dealing with many transactions per second, updating the database for every received transaction will be costly, especially since this update would have to be delivered to everyone in the network.

Everyone maintains their own ledger after all, and once a change is made for one entity, it must propagate throughout the entire network.

How do we store our ledger efficiently?

Every update to the distributed database, the Bitcoin ledger, is a batch of transactions grouped into what are called blocks.

Every block is built off, or chained to, a previous block.

Altogether, this forms a magical data structure known as a “blockchain.”

By grouping data into blocks, we do not have to strain the network as a result of updating every ledger after every transaction.

With a blockchain, only every block, which may contain thousands of transactions, needs to be appended to the blockchain.

In this way, blockchains efficiently keep track of not only the transactions in any given update but also give the database discrete states.

Every block is an update, and a chain of blocks represents a history.

This process is helpful for identifying discrepancies between two different versions of the database, since it is clearer what happened in the ledger at any given time than if every transaction were individually processed.

Every block contains information about the previous block, as every block is built off the previous one.

If any block is mutated, intentionally or not, information within this block and all future blocks will change.

This makes the blockchain tamper-evident, as tampering with a transaction from the past would invalidate any future blocks linking back to it.

To reiterate, this design choice is both to reduce the strain of storing individual transactions and to facilitate consistency between all members of the Bitcoin network.

Of course, the next question is, “How does everyone come to agreement on the next block?”

We will go over this in the next segment on consensus.

Now we know how to send transactions to one another with unique identities using the UTXO model, and how to store these transactions using a blockchain to maintain a global record.

At this point, we’re only missing one final piece to bring all these different components together.

How do we make updates to the blockchain?

How do we decide which transactions, or blocks of transactions, are valid?

With no central entity deciding on the next update, we have to find a way to make this decision in a decentralized way.

In Bitcoin, users on the network must come to consensus, or agreement, on the next valid update.

Because everyone is storing information about the blockchain, we need consensus to make sure that everyone agrees on the history of transactions.

Not only do all users need to agree on the update, but they also need to agree on a valid update to make sure that no corrupted information is accepted by the network.

If there was no way for different parties to come to agreement on this, then there will forever be this schism between them.

Without agreement, we cannot have a functioning distributed database.

We have to all stay on the same page to make sure that we all believe in the same reality of Bitcoin.

Thus, it is imperative that we have a mechanism by which everyone can come to consensus.

To understand the best consensus mechanism for Bitcoin, we will start off by understanding what problems exist with naive consensus strategies and solve them in the next iteration, continuing this process until we finally recreate Bitcoin’s consensus model.

For this consensus buildup, we will consider the same 5 actors from before: me (Rustie), Gloria, Derrick, Nadir, and Nick.

In the most basic consensus mechanism, updates take the following form:

One node proposes a transaction to the network, sending a message about the transaction directly to every other node.

All other nodes save the transaction into their history if it’s valid and disregard it otherwise.

We have an example for you here.

Gloria wants to make a transaction to Nadir, so she sends that message to all four other entities.

This is represented by the arrows pointing from Gloria to every other entity, blue to represent that she is making a transaction to Nadir, the blue circle.

The implication of this system is that nodes don’t engage in conversation with one another, and the only nodes that see a transaction with one hundred (100) percent certainty are the sender and recipient.

As we will see, this does not work because of what’s known as the double spend attack.

With centralized systems, we trust banks to check for the validity of all the transactions.

Recall that one condition of a transaction being valid is that proposed funds for the transaction are not promised elsewhere in a previous transaction.

Because we do not have a bank to check if there is any malicious behavior, we must build mechanisms that handle these situations.

Let’s say that Gloria is purchasing tons of laptops and is willing to pay both Nadir and myself (Rustie) 10 bitcoins for our laptops.

Gloria promises me (Rustie) 10 BTC in one transaction, and promises 10 BTC to Nadir at the same time.

This is represented by the green arrow from herself to me (Rustie), the green circle, and by the separate blue arrow from herself to Nadir, the blue circle.

However, she only has 10 BTC in total.

Her invalid transaction evades detection because, as you may have realized, she only tells one person about each transaction: the person receiving the bitcoins.

I (Rustie) only know about my (Rustie’s) incoming bitcoins, and Nadir only knows about his.

Derrick and Nick, the other entities in the network, know nothing about either transaction.

This is what’s known as a double spend attack.

Gloria is spending only 10 bitcoins to get 20 bitcoins worth of goods.

In this naive version of consensus, this is legal: with her transaction to me, Gloria tells me to update my copy of the ledger, and with her transaction to Nadir, she tells Nadir to update his copy of the ledger.

Both Nadir and I see that the transaction is valid and each believe that we have received the bitcoins.

Of course, both Nadir and I (Rustie) can’t own the same bitcoins.

The moment we try to redeem these tokens with the network, the issue becomes transparent.

It’s going to look to Derrick and Nick that these transactions never happened.

If I try to convince anyone else that I (Rustie) own these ten bitcoins, they’ll all think I’m crazy because they’ve seen no evidence of me (Rustie) ever having received these bitcoins.

The same applies to Nadir.

In this scheme where entities only see the transactions that directly involve them as sender or recipient, it is impossible to come to consensus on a history of transactions because of dishonest actors.

Thus, it is impossible to prevent these double spend attacks with our current model of consensus.

Instead of individuals doing their own validation of transaction, we can set up a voting system.

The problem with our previous version of consensus was that there wasn’t any consensus!

Instead of making siloed decisions as we did before, let’s implement a system of proposers and voters.

One person at a time makes a proposal about an update, and everyone else votes on whether or not to accept the proposal.

The person who wants to make a transaction sends the transaction to everyone in the entire network, not just the recipient of bitcoins.

Everyone on the network then casts votes based on whether the transaction they saw was valid or not.

Only after receiving a certain number of votes, say a majority, does the transaction get saved.

Like before, there are blue lines from Gloria to the rest of the network to indicate that she is making a transaction to Nadir.

Unlike before, there are dashed blue lines from each node to everyone else (excluding Gloria) as an indication of communication about two things: the received transaction and a vote for or against its validity.

Let’s observe what happens when Gloria tries to double spend under these circumstances.

Now, when Gloria attempts to double spend, she will be rejected by observing peers.

She again sends only two messages: One of a transaction to me (Rustie), as indicated with the solid green arrow, and one to Nadir, as indicated with the solid blue arrow.

However, we introduce a new component: communication.

The dashed blue and green lines from Nadir and myself (Rustie) respectively represent the relay of those message to the rest of the network.

By looking to the rest of the network for input, considering everyone else the third party, we are protected against Gloria’s attempts at malicious behavior.

Peers in the network vote “no” on Gloria’s proposal, as they notice multiple transactions trying to spend the same funds.

The transaction doesn’t go through, and is not included in an update to the blockchain.

It looks like we’ve solved all our problems: we have a voting system that ensures that no one can double spend the same funds, and each peer stores the whole history of transactions so that they can verify for themselves that the funds exist.

It looks as if we are victorious!

However, we forgot one fatal truth about Bitcoin: the anonymity.

Recall that Bitcoin is an accessible, anonymous network with no central registry.

Banks keep track of everyone’s identities and accounts, but no such infrastructure is available in Bitcoin to prevent anyone from producing multiple identities.

In Bitcoin, anyone can join, and anyone can participate.

Because of the ease with which Bitcoin addresses can be generated, nothing is stopping Gloria from generating more identities and posing as Derrick and Nick.

It’s inexpensive to create multiple identities, requiring only the generation of a random number, as seen in the identity section.

Because of this low cost, Gloria can easily hold multiple Bitcoin identities to cast more votes than she should be allowed.

Gloria, a real-world minority, could easily propose and vote for her own malicious transactions by creating sufficient identities and occupying a network majority.

In other words, this current version of consensus is susceptible to a Sybil attack, where a user creates multiple identities for some malicious purpose.

With such a little cost to vote, votes become meaningless.

There’s no value in a vote because anyone with spare time can make as many identities as they want.

We see that this is a problem when Gloria attempts the Double Spend attack because, with these extra identities under her control, she succeeds.

She sends the transactions to Nadir and me (Rustie), and we send these transactions to the rest of the network, and they all vote that both transactions are valid.

Because the majority votes on both transactions being valid, we now have a lack of consensus about what the truth is.

Gloria has broken Bitcoin all because it was easy for her to pose as multiple people and cast multiple votes, overwhelming Nadir and myself (Rustie).

Clearly, to solve the problem, we cannot assume that each online identity deserves the same voting power, as some people have multiple identities.

To ensure that every real person only has one vote, we have to make a vote expensive.

We have to make it such that anyone trying to vote has the same amount of voting power as anyone else, regardless of how many Bitcoin identities they have.

But how?

Is it even possible to solve this problem?

This is where the innovation of Satoshi Nakamoto comes into play.

They recognized that in order to ensure every real entity has equal voting power, they cannot vote with identities at all.

Instead, Satoshi realized that, to preserve freedom and anonymity while solving the issue of endless identities, each vote must be cast with resources.

Scarce, valuable, tangible assets.

The particular resource that Satoshi identified was one available to all users of Bitcoin:

computing power!

Online entities can be generated easily, but you can’t copy-paste a computer.

By tethering voting power to scarce computing power, Satoshi ensured that all users have scarce voting power.

In his whitepaper, he envisioned a “1-CPU-1-vote” network, rather than the traditional “1-identity-1-vote” system.

It is this new concept of voting with resources instead of with identities that earned the title “Nakamoto Consensus.”

The particular consensus algorithm that Satoshi Nakamoto came up with is known as “Proof-of-Work.”

Let’s break down the term “Proof-of-Work”.

We know “proof” to mean “evidence,” and “work” to mean “spent resources.”

In other words, “Proof-of-Work” is the method by which users provide evidence of spending resources.

It is the method by which computing power is translated into voting power.

It is the method by which users vote in Bitcoin.

It is this method of voting that made Bitcoin the first successful cryptocurrency and inspired voting mechanisms for practically every other cryptocurrency to follow.

Here’s how Proof-of-Work works: whenever someone wants to make a proposal to the rest of the Bitcoin network, they first have to solve a computationally difficult problem.

In other words, their computer is tasked with a problem that can only be solved by doing a significant amount of work.

This problem is uniquely generated based on the information within the proposed block and cannot be predicted beforehand.

There’s no way to predict the solution to the problem.

Instead, it’s similar to brute forcing a password: all you can do is trial and error.

Your computer will try a bunch of inputs until it finds a solution to the problem, at which point it will submit the successful input along with the proposed block to the rest of the network.

The unpredictability of the form of correct inputs ensures that there’s no way to “game the system,” keeping any user from cheating.

By having every user solve a brute force problem, you have a reasonable expectation that they’ve done a lot of work to solve it.

You might ask, “What if a user gets lucky and finds the solution to this brute force problem on the first try?”

Well, this is possible, but not probable.

For example, it’s possible for me to guess the PIN on your phone in one try -- but it’s because we believe in the unlikeliness of that situation that we aren’t afraid of anyone breaking into our phones.

Although we can’t guarantee that it takes exactly some amount of computation to solve these problems, we have reasonable expectations based on how long on average it takes to solve these problems.

Let’s see Proof-of-Work in action.

Again, Gloria tries to double spend on Nadir and me (Rustie), and these two transactions are sent to the rest of the network.

We all vote on the transactions, but Gloria no longer has a voting advantage even though she still has control of two extra Bitcoin identities.

Her voting power is bound by her computer, just as everyone else’s is.

Even though there are five digital identities, there are only three real-world identities.

Under Proof-of-Work, no matter how many digital identities are formed, there will only ever be three voting entities.

Because Nadir and I (Rustie) recognize that there are two transactions trying to spend the same funds, we can vote against both transactions being included in the blockchain.

Gloria with her single computer is incapable of outvoting us, which is precisely what we want!

By tying our voting power to our real-world identities through the scarce resource of computing power, we have successfully eliminated Sybil attacks from Bitcoin and solved the double spending problem.

This is what made Bitcoin a revolutionary technology.

Author: Rea Savla

The Bitcoin network requires Proof-of-Work to add new blocks to the blockchain. Users in the Bitcoin network vote on new blocks, and come to consensus on whether or not new blocks should be included in the blockchain. Proof-of-Work ties voting power to computational power rather than digital identity, and was designed to prevent the Sybil attack, where a malicious actor creates many identities to skew the vote. However, due to an uneven distribution of computational power, Satoshi Nakamoto’s “1 CPU 1 Vote” vision is not reflected perfectly in reality.

Bitcoin’s correct operation hinges on one key assumption: that there is an honest majority of computational power. An honest majority would be able to mine faster than a malicious minority, and thus have a higher probability of creating the next block. Once the network comes to consensus on these new blocks, generally it is in a miner’s best interest to follow protocol and mine on the longest observed blockchain. The longest chain is seen as the “true” valid transaction history because it has had the most work put into it. Therefore, the majority defines the transaction history.

However, if a malicious entity controls more than 400px; of the mining power (say 51%), it has the majority and is now able to mine an alternative chain (with a different transaction history) and make it the longest chain. Bitcoin users would then accept that chain as the “true” transaction history. This happens for the same reasons that an honest majority might be able to maintain the longest chain. Giving power to the majority is a requirement of the decentralization Bitcoin aims to achieve. If we allow an honest minority to control the transaction history, then we’ve created a centralized entity consisting of the collection of these honest actors and thus defeated the purpose of decentralization. 

With 51% of the mining power, malicious actors can double spend, and use the same bitcoins for two different transactions. A malicious actor may send the same bitcoin to a third party and then to itself, choosing to include and validate the latter transaction and avoiding payment altogether. 

One entity holding majority mining power also makes the network susceptible to the Goldfinger Attack, where a 51% is used to destroy the value of a cryptocurrency. 

We’ll explore all these attacks in further detail in future course modules.

Sometimes, different miners may create different blocks, either intentionally (e.g. double spending) or unintentionally, to add at the same point on the blockchain. This creates multiple chains: multiple different versions of the transaction history. We say that the blocks are competing at the same block height, and that there has been a fork. Following protocol, miners eventually resolve the fork and agree upon one of the chains to be the valid blockchain, and continue to build blocks upon it. 

While some forks occur naturally, and some are the result of double spending attempts, there also exist purposeful cases of forking, used to make changes to the Bitcoin protocol. 

Two categories of these protocol changes are soft and hard forks. Soft forks implement protocol updates that strictly reduces the set of valid transactions, while hard forks, conversely, allow for previously invalid transactions to become valid.

We’ll explain more in the course modules for Chapter 4.

Let's go over what we talked about in lecture.

First, we talked about the concept of identity on the Bitcoin network.

In Bitcoin, each node’s identity is represented by their public key.

However, ultimately the public keys are controlled by the owner of private keys.

Only the private key can be used to spend money.

Another thing to note about identity is that users can generate as many private/public key pairs as they want.

How does it work again?

Remember that there are 2 to the 160 total possible private keys.

It is extremely unlikely that someone might happen to generate the same private key as yours.

Nor is it at all likely that someone can guess your private key to spend money on your behalf.

Bitcoin doesn’t have the account balance model that banks have.

Instead, users spend outputs from previous transactions.

These specific outputs are called “Unspent Transaction Outputs” or UTXO.

The total value of bitcoins you have is the sum of all of UTXOs you own.

These UTXOs are uniquely identifiable and make tracking payments at the protocol level much more straightforward: The UTXO record system makes it easy for nodes to see how funds change hands between users and UTXOs.

The UTXO model might not be the most intuitive model for us to understand, but it works well for bitcoin from an architectural standpoint.

As we mentioned earlier, the blockchain is the key data structure for recording Bitcoin activity.

New transactions are recorded within new blocks added to the existing, established chain.

Once a transaction is recorded, it is close to impossible to undo without changing every single version of this database in the universe.

The way that the network reaches consensus is through Proof-of-Work.

How does it work again?

Proof-of-Work requires that voters expense a considerable amount of computational power in order to validate transactions.

But why do we need Proof-of-Work?

Because, again, there is no central authority to make sure that one person only vote once, and there is no limitation on how many identities one person can generate, Bitcoin uses computational power as a resource constraint to limit the voting power of malicious entities.

Proof-of-Work hence aims to make votes expensive for everyone, so that the voting power one has is based on how much computational power one has, instead of based on the number of identities.

Given all this information, you can now justify many of the common buzzwords associated with Bitcoin which you may have heard.

The most common descriptors of Bitcoin are “pseudonymous,” “decentralized”, “immutable”, and “trustless”.

Pseudonymity is a combination of the words “pseudo,” meaning fake, and “anonymous,” meaning unknown.

Bitcoin attempts to be anonymous through having every user represent themselves with a random number, the public key.

However, because it is not impossible to trace back these virtual identities to real world identities, bitcoin is not complete anonymity --- it is only mimicking anonymity.

In Bitcoin, addresses and pseudonyms are synonyms -- it’s a fake name, but it can still be used to trace back to you associated with you with enough effort.

In addition, decentralization refers to taking an activity that is typically performed by one central entity and repeating the storage of information and computation among more than one party.

Bitcoin achieves decentralization by having every single participant in the Bitcoin network store the full history of transactions, as we’ve seen.

This way, every user possesses a copy of the transaction history and does not have to ask anyone else for that information.

Immutability, referring to the inability to change information, is another property of Bitcoin achieved through decentralization.

Once all users in the Bitcoin network decide on the validity of some transaction, it is extremely difficult for anyone, including themselves, to undo their decision.

This feature helps foster trust among nodes on the network.

If one wanted to alter the history of transactions, they would have to change every single user’s local history simultaneously, which in the present day is close to ten thousand different users.

As a result of these three properties of pseudonymity, decentralization, and immutability, we achieve in Bitcoin a trustless network.

Because every user is by default a stranger to everyone else, one may ask, “How do we trust others in the network?

If we do not trust a majority of Bitcoin users, how do we trust Bitcoin?”

The Bitcoin protocol ensures that one does not need to trust their peers in order to be certain that any transaction they make will be accurately recorded by the rest of the Bitcoin network.

First, the ledger is publicly verifiable.

Anyone can see any and all information about the history of transactions in Bitcoin.

You can go to the blockchain and check if your transaction has gone through.

In addition, the Bitcoin network is secured through the Proof-of-Work consensus protocol designed by Satoshi Nakamoto which changed the way everyone thinks about cryptocurrencies.

These are the four essential unique properties of Bitcoin: 1. pseudonymous, 2. decentralized, 3. immutable, and 4. trustless.

Cryptocurrency is a completely digital, formless currency, tied together using computer science, cryptography and economics. Bitcoin is the first and most widely used cryptocurrency.

Bitcoin is inspired by the Cypherpunk Movement in the 1980s, which advocated for protection of privacy from external entities using cryptography. Satoshi Nakamoto first outlined and created Bitcoin in 2008 and 2009.

Bitcoin aims to be pseudonymous, trustless, decentralized, and immutable. In addition, anyone with a computer and internet connection can join the Bitcoin network. Each computer is a node in the Bitcoin network, and each node may verify and audit the transaction history of their own funds. In Bitcoin, the minting and distribution of bitcoins is determined through mining; since anyone can mine and win bitcoins, this process also aims to be decentralized.

Some of the challenges Bitcoin addresses are:

• The difficulty to ensure every Bitcoin node holds a consistent version of the transaction history

• The difficulty to identify malicious actors

These conditions may normally allow a node to conduct a Double Spend Attack, in which the one spends the same funds more than once by tricking parts of the network to believe different versions of the transaction history. However, Satoshi overcame this problem using blockchain and Proof-of-Work. 

Bitcoin is robust because it serves the same functions as a bank:

• Account management; the Bitcoin protocol gives users a way to create and manage their own identities (account)

• Legitimacy; It ensures we are legitimate owners and accessors of our accounts

• Record-keeping; It honestly records account balances at each transaction.

Unlike a bank, Bitcoin is decentralized and ensures a high degree of privacy and trust. 

Trust is built on the blockchain due to a high level of transparency: blockchain is a publicly verifiable ledger, not owned by any entity, and it prevents any single point of failure.

To maintain this trust, we need identity in Bitcoin for authentication and assigning blame. Bitcoin uses public keys to send funds and private keys to prove ownership of the public key and redeem the sent funds. Each individual is responsible for creating and managing their own private and public keys. Public keys are generated from Private keys and are used to send/receive funds. Private keys are randomly generated and used to prove ownership of the public key. The chances of guessing the same private key are very low.

In addition to proof of ownership, in order to be considered valid, transactions must also have enough available funds to spend from and guarantee that no other transaction uses the same funds. Bitcoin uses the Unspent Transaction Output (UTXO), in which users spend directly from transactions made to them.

Instead of storing transactions individually, Bitcoin batches them into “blocks,” built off of their previous blocks, thus forming the tamper-evident, blockchain data structure.

Users on the blockchain must come to a consensus on which updates and blocks to add to the blockchain. Doing so also prevents Double Spend Attacks. Bitcoin uses a form of peer validation to build a shared transaction history; everyone on the network casts votes on the validity of a transaction. To prevent a Sybil Attack, where users create multiple identities for malicious purposes, Bitcoin employs Proof of Work, where voting power is based on computational power, to make voting expensive.

The shy college student who helped build Bitcoin into a global phenomenon 

The Essence of How Bitcoin Works (Non-Technical) 

(Optional) The Untold Story of Silk Road, Part 1 

(Optional) Bitcoin: A Peer-to-Peer Electronic Cash System  

(Optional) Bitcoin Developer Guide (up to but not including “P2PKH Script Validation”)

(Optional) How Bitcoin Works in 5 Minutes (Technical) 

A question we sometimes get from students in our class here at UC Berkeley is: why use Bitcoin?

After all, we’ve seen that it’s slow, it’s redundant, and inefficient.

Banks are so much more convenient!

While we’ve mainly been focusing on how Bitcoin works, we’ll be taking a step back and understanding why Bitcoin is the way it is.

We will examine the cultural, political, and technological influences that drove the development of cryptocurrencies.

As we’ll see, Bitcoin didn’t just pop up out of nowhere.

Understanding its historical background and numerous failed predecessors will give us a deeper understanding of Bitcoin.

From there, we’ll explore the present-day state of the blockchain space, including various new cryptocurrencies inspired by Bitcoin along with how blockchain technology has diverged from its origins in Bitcoin.

This journey will take us from the Cypherpunks, who hated big banks and championed privacy and power for individuals, to adoption of blockchain technology by JPMorgan Chase, one of the largest banking institutions in the United States.

The roots of Bitcoin start with libertarianism.

Libertarianism is a political ideology which believes that centralized authority should be as minimal as possible.

Individuals should have as much power as possible to decide the course of their own lives and shouldn’t have to sacrifice rights or properties to a government excessively.

You may already see how this ideology relates to Bitcoin.

Amongst increasing centralized state control, especially of services and of individual information, libertarian dreamers grew concerned with privacy.

From online discussions on forums and mailing lists, two groups formed: the Cypherpunks and the Crypto-anarchists, both of which advocated for the use of cryptography to protect one’s privacy.

Before explaining a bit of the background behind all of this, let’s look at this quote first:

“Privacy is necessary for an open society in the electronic age. Privacy is not secrecy. A private matter is something one doesn’t want the whole world to know, but a secret matter is something one doesn’t want anybody to know. Privacy is the power to selectively reveal oneself to the world.”

He’s one of the founders of the Cypherpunk movement, and in the 90s founded and administered the Cypherpunk mailing list.

The Cypherpunks and Crypto-anarchists hated the idea of national agencies being able to spy on them and have access to their information.

They also hated censorship.

But most of all, they hated big banks and governments, both of which were giant centers of power diminishing the autonomy of the average citizen.

This dichotomy of power only grew as the technology of these institutions evolved faster than that of the general public.

In the past, before these technological innovations, cash was pretty anonymous in purely its physical form.

Once you spend it, there’s no easy way to trace it.

But in an increasingly digital world, the convenience of big banks tracking account balances and transfers comes with the cost of privacy.

And so, these libertarians saw early on the need for an anonymous digital “transaction system” or currency -- one that gave users “the power to selectively reveal oneself to the world.”

The Cypherpunks didn’t get together, design a cryptocurrency, and succeed on their first attempt.

Several early attempts at making a true cryptocurrency failed, though they came close.

These early failures inspired Bitcoin to adopt some key features that they implemented and learned from their mistakes.

As the existing financial system was one of the greatest threats to individual privacy, cryptographer David Chaum implemented Digicash using the latest advancements in public and private key cryptography.

Chaum himself had invented “blind signatures” while studying at UC Berkeley.

Blind signatures allowed users to sign off on transactions without revealing their identity.

Digicash promised complete privacy for users conducting online transactions and included a system of cryptographic protocols that prevented banks and governments from tracing personal online payments.

Ironically, however, Digicash failed because of what it feared the most: centralization.

Digicash was hosted by Chaum’s own company, and if his company ever went down, Digicash would also go with it.

And that’s exactly what happened.

Chaum’s company DigiCash Inc bore the overwhelming burden of having to validate each and every digital signature in the DigiCash system, which eventually led to bankruptcy in 1998.

Hashcash was originally invented as a mechanism to limit email spam.

In order to send out an email, one would have to solve a cryptographic hash puzzle and provide a proof of work.

Only after proving that they have expended computational resources -- with a Hashcash stamp added to the email header -- can someone send out a valid email.

Email recipients can then verify emails that they receive are valid, by looking at the email headers for a valid Hashcash stamp.

The idea is that spammers wouldn’t be able to spam emails anymore.

It would be too costly, as spammers goals are to send out huge numbers of emails with little cost per message.

So, by making it computationally expensive to send out email, hashcash disincentivized spammers.

B-money was an early proposal for a cryptocurrency created by Wei Dai.

In 1998, Dai published a paper titled “B-money, an anonymous, distributed electronic cash system,” and laid out some core concepts that would later be used to implement Bitcoin and other cryptocurrencies.

Among B-money’s core concepts were that:

A proof of work function like Hashcash is used as a means of creating money.

Everyone maintains a copy of the database showing who owns what, and work is verified by the community, who all work to update a collective ledger.

And transactions are accomplished by collective bookkeeping and authenticated with cryptographic hashes.

Workers are awarded funds for their efforts in creating money through expending computational resources.

Contracts and transactions are enforced through the broadcast and signing of transactions with digital signatures.

These ideas from B-money would later influence the development and design philosophy of Bitcoin.

In October 2008, a whitepaper was published online, titled, “Bitcoin: A Peer-to-Peer Electronic Cash System.”

This 9 page whitepaper outlined the design and justification for a digital currency, or cryptocurrency, controlled by no single entity.

Traditionally, we trust the singular bank to conduct financial services on our behalf.

With Bitcoin, the intention is to do what no other attempt at a cryptocurrency could do before: create an anonymous, trustless, decentralized currency.

Instead of trusting any individual humans, we put trust into math, cryptography, and logic.

We trust the Bitcoin protocol, even if we don’t trust any individual in the network.

This cryptocurrency relies on computational power to cast votes.

As mentioned earlier, this is known as Proof-of-Work.

And as we saw earlier, the idea of solving a cryptographic hash puzzle was also used in Hashcash.

Satoshi wrote in the whitepaper that this is meant to enforce a “one-CPU-one-vote” system, such that every computer only has the ability to cast one vote in the consensus process.

This is different from the traditional “one-identity-one-vote” process used in government elections because we can’t assume that each real world identity has only one digital identity.

In addition, each person maintains their own identity through public and private keys, authenticating themselves through blind signatures, which was introduced by David Chaum from DigiCash.

This ensures that every user maintains their own privacy until they voluntarily choose to reveal themselves.

In Bitcoin, every full node maintains their own copy of the blockchain, similar to how in B-Money’s protocol, every participant maintains their own individual database, to keep track of how much money belongs to each user.

Bitcoin was designed to be a deflationary currency, meaning that there will only be a limited amount of bitcoins that will ever exist, specifically 21 million.

Every miner gets a block reward for finding a block, but this block reward decreases over time.

It began at 50 bitcoins and is halved every (two hundred and ten thousand) 210,000 blocks.

Eventually, the block reward will become 0, at which point 21 million bitcoins will have been minted.

Interestingly enough, Satoshi chose to include a message within the first block of the Bitcoin blockchain.

This information is a powerful glimpse into the mentality and motivations of Satoshi Nakamoto.

As we’ll discuss further in the next module, the coinbase transaction gives a miner a block reward.

This particular transaction has empty space into which extra information can be included.

Satoshi chose to reference a story in the Times of London mentioning a Chancellor bailing out a bank.

As we can tell from several posts and comments made by Satoshi, he was not fond of the modern banking system, particularly fractional-reserve banking.

Bitcoin’s design aims to eliminate these issues.

During this first year, people sent bitcoins to each other out of interest, playing around with the software.

During this time, however, bitcoins were never exchanged for any tangible good.

And then in May of 2010, this post showed up on the bitcoin talk forum.

Feel free to pause the video and read through the post, or follow the source link.

In the post, we see that...

...in it, a man named Laszlo Hanyecz asked for pizza in exchange of 10,000 bitcoins.

And on May 22, 2010, he received a $25 order of pizza in exchange for his bitcoins.

This was the world’s first ever Bitcoin transaction for a tangible asset.

In this moment, Bitcoin went from worthless internet money to something with real value.

As a fun fact: as of March 15, 2018, the pizzas Laszlo ordered are now worth 81 million dollars.

And here’s Laszlo reporting back on the forum about the pizza’s safe delivery.

Most people ridicule Laszlo for spending bitcoins on such a small purchase.

However, they are not aware that Laszlo wanted to see Bitcoin flourish.

Before this purchase, the idea that bitcoins could be used to purchase real world goods was ridiculous.

For most people, mining Bitcoin was merely a hobby.

This purchase validated the use of Bitcoin for its original purpose as a currency actually used for buying goods.

In a way, by trailblazing the currency’s use, Laszlo is a hero to Bitcoin enthusiasts.

This was the very first time a Bitcoin transaction traded “magic worthless internet money” for a tangible item of value - therefore bitcoins have value.

However, it’s not like bitcoins were worth that much when Laszlo spent them on pizza.

It took years of development and spread before Bitcoin became accepted as a legitimate technology, yet it’s still got a way to go before reaching worldwide legitimacy.

As Bitcoin started to gain popularity, it quickly picked up a bad reputation due to its focus on privacy and decentralized nature.

Regarding privacy, the difficulty to tie digital identities to the real world user posed issues in some situations.

On top of that, because Bitcoin is decentralized, there is no central entity or court to fix any mistakes within the blockchain.

If someone found your private key and stole your funds, no one else can give your stolen funds back to you.

Early Bitcoin was thus often the target of scandals, hacks, and illegal activity.

In the early stages of Bitcoin, there existed no exchange for trading between bitcoins and regular currencies.

In 2010, Jed McCaleb, turned his domain mtgox.com into a Bitcoin exchange website.

What was originally the Magic: The Gathering Online Exchange, where players of the online game Magic the Gathering Online traded cards like stocks, quickly became one of the largest bitcoin exchanges during the early stages of Bitcoin.

Keep in mind an issue with this exchange: it’s a central point of failure, and its popularity drew a lot of negative attention.

On June 19, 2011, it was discovered that a hacker was using a Mt. Gox auditor’s computer to siphon an abundance of bitcoins to themselves.

Mt. Gox was subsequently shut down for seven days to investigate the situation.

It received several lawsuits, some of which are still being disputed.

Despite this breach of security, people kept on using Mt. Gox.

For a while, it seemed like all was well.

By 2014, Mt. Gox was handling up to 70% of all Bitcoin transactions -- way too much for what the infrastructure could handle.

In February 2014, Mt. Gox announced that it had lost 744,408 of its customers bitcoins in an ongoing theft that had gone unnoticed for years.

By the end of the month, Mt. Gox filed for bankruptcy.

Meanwhile, in the depths of the dark web, Bitcoin was also used to purchase drugs.

In February 2011, the website Silk Road opened, quickly earning the reputation of the anonymous “eBay of Drugs.”

It was led by a man named Ross Ulbricht.

Silk Road leveraged Tor, an anonymous networking protocol, and handled transactions using Bitcoin.

This combination made it difficult for users’ online identities to be linked back to their real life identities.

Because of this, drugs and the black market became the primary use case that came to mind when anyone mentioned Bitcoin.

Even to this day, Bitcoin is tainted with this association.

In October 2013, the FBI shut down Silk Road.

Ross Ulbricht was sentenced to life in jail, and the FBI seized upwards of 26,000 bitcoins, worth $3.6 million at the time.

Between November 1, 2013 and November 30, 2013 -- less than a month -- the price of Bitcoin rose from just under $200 to over $1000

In late 2013, the Bitcoin bubble hit its peak at around $1,165 before suddenly bursting and entering a bearish run that lasted until the end of 2015.

There are a couple speculations about why the bubble occurred:

First, Chinese investors bought bitcoins as a speculative financial investment.

Many sold because of warnings issued by the Chinese government

Second, automated trading in Mt. Gox may have artificially driven the price up by continuously buying bitcoins.

Soon after the creation of Bitcoin, other cryptocurrencies began popping up on the internet as well, each tailored to a different use case or audience.

These cryptocurrencies other than Bitcoin -- Litecoin, ZCash, Stellar, Ripple, Ethereum, Dogecoin, DASH, Monero, and others -- are known as altcoins, alt- standing for alternative.

For example, Litecoin aims to be the silver to Bitcoin’s gold.

It is more progressive in its software updates and serves as a testing ground for proposed Bitcoin software updates.

In addition, ZCash uses innovative zero knowledge proofs, which are a way to prove a fact without revealing information about the fact itself.

This furthers cryptocurrency privacy by allowing transactions to be validated without revealing information about the sender, recipient, and value transferred.

Stellar and Ripple pioneered new federated consensus algorithms, eliminating the need to waste electricity solving cryptographic hash puzzles.

These are just a few examples of how coins other than Bitcoin are trying to solve their own individual problems, making for a wider crypto space.

Following years of hacks and bad reputation, Bitcoin finally began to grow in general popularity.

Here are some of the big headlines following Mt. Gox’s theft and subsequent declaration of bankruptcy in February 2014:

In March 2014, people thought they had found Bitcoin inventor Satoshi Nakamoto in California, but that was a false alarm.

In September 2014, venture capitalist Tim Draper announced his predictions of Bitcoin’s price heading up to $10,000.

For context about his perspective, he was interested in cryptocurrencies since before Bitcoin.

In 2003, he met a father in South Korea who bought a virtual sword for his son with fiat money, and was curious ever since.

He also won some bitcoins from the FBI auction of confiscated bitcoins from the Silk Road shutdown.

2014 was also the year when merchants began to accept bitcoin as a form of payment.

In January, Overstock.com became the first major retailer to accept bitcoins.

Then, in September, PayPal partnered with Coinbase, BitPay, and GoCoin.

Fun fact: Blockchain at Berkeley was previously known as the Bitcoin Association of Berkeley, and in 2014, these headlines were happening every week.

At every club meeting, our 7 members would discuss the latest hack, the latest bankruptcy, and the latest Ponzi scheme.

We’ve grown so much since then, and that just comes to show how much the blockchain space has matured over the years.

A bunch of Bitcoin startups began popping up too.

Wallet companies helped other companies or users handle bitcoin without having to personally join the Bitcoin network.

For example, Coinbase is an online exchange that manages wallets and lets users buy and sell bitcoin for fiat currency.

Bitpay allows merchants to accept bitcoin.

Blockchain.info is a block explorer that allows users to see individual blocks and transactions in the Bitcoin blockchain in browser, without having to download the entire blockchain themselves.

Most importantly, during this time, the term “blockchain” started becoming a buzz word.

This is a graph of Bitcoin prices from 2014-2015, where you can see that first prices shot up immensely, then slowly started to fall.

Of course, we all know that the price recovered and went back up after this, but this was a huge shock to the community at the time.

There are a few theories as to why Bitcoin burst at this time:

One was that investors who had speculated and bought a lot of bitcoin had second thoughts, and began to sell.

Especially, Chinese investors had sold because of warnings issued by the Chinese government.

And then of course the market amplified the current trend, so people further dumped because they feared a loss in value.

Historically, Bitcoin has faced a number of challenges as a technology, including scalability and of course public perception.

Even today, being the first successful and most widely used decentralized cryptocurrency, Bitcoin still has several problems to solve before it can be used by the masses.

In this section, we’re going to talk about the next stage of Bitcoin’s history, when disputes about the core Bitcoin technology sparked internal debates, and when a separate platform called Ethereum quickly came to the mainstream.

One issue that has been the topic of much debate is that of scalability.

Scalability refers to the ability of a technology to be used by increasing numbers of people.

In the context of Bitcoin, it refers to the number of transactions that the network can confirm in a certain amount of time.

As of March 2018, Bitcoin blocks are created every 10 minutes and can only hold 1MB of transactions, which is about one to three thousand transactions per block.

Thus, we can estimate that the Bitcoin blockchain can process about three transactions a second.

In 2015, the transaction volume exceeded the network’s capacity so much that blocks actually began to run out of space, meaning that transactions were left unconfirmed, left without being included in a block.

This led to proposals like Bitcoin XT, Bitcoin Classic, Segregated Witness (commonly abbreviated as SegWit), and more recently, SegWit2x in order to improve the network’s transaction processing capacity.

Some of these proposals rely on increasing the block size in order to fit more transactions, while other ones modify the underlying protocol.

Each of these solutions have their pros and their cons, but we won’t discuss them in too much detail right now.

The important thing to understand right now are the questions that the scalability debate raises about decentralized governance.

For example: what do we do when we want to change Bitcoin?

Governance is the mechanism by which a protocol makes changes to itself, but no such mechanism was encoded into the Bitcoin protocol.

Instead, users introduce proposals called BIPs, or Bitcoin Improvement Proposals, outside of the Bitcoin network, on forums and online discussion boards, and then the community votes ad hoc on whether or not they want to go through with that change.

Members can't directly propose and vote on updates to Bitcoin within the actual software, but at least the current mechanism gets the job done.

After Bitcoin, the next most influential blockchain platform is Ethereum.

Bitcoin is a storage of value, that is to say, it’s “coin-centric”.

It was created as a medium of payment transaction and a store of value, an alternative to regular money.

Ethereum, on the other hand, was developed as a platform to execute peer-to-peer “smart contracts” and applications.

It supports Turing-complete languages, meaning that it can perform general computation.

In other words, any type of code that I run on a regular computer can also be run on Ethereum.

Code execution on Ethereum is fueled by Ethereum’s internal token, called ether.

Ethereum was first described in a whitepaper released in late 2013 by then 19-year-old Vitalik Buterin, a programmer from the University of Waterloo.

The platform had a token sale between July and August 2014 and sold 7.4 million ether for 3700 BTC in the first 12 hours of the presale.

At the time, this was equivalent to 2.3 million USD, or in Bugatti terms, about 1.2 Veyrons.

The Ethereum blockchain officially went live on July 30th 2015, and by May 2016, the cumulative value of Ethereum tokens was more than $1 billion.

Around that time, the idea of Decentralized Autonomous Organizations (DAOs for short) became hugely popular.

DAOs are essentially programs on the Ethereum blockchain that create a distributed government.

“TheDAO” was a specific project that would serve as a decentralized Venture Capital, allowing their investors to vote and decide on the distribution of funds between startups.

However, in July 2016, these dreams came crashing down when a hacker exploited a bug in the underlying code, stealing about $120 million worth of Ether from TheDAO smart contract.

Outraged by the enormous theft, several voices in the community proposed to defy the protocol and undo the hack.

The majority of community decided to simultaneously rewind their own chain and ignore all activity starting from the hack, but a small subset chose not to undo that activity with the belief that “code is law.”

The split that rewinded (zurückgespult) history is the split that is currently branded as Ethereum.

The remainder that believed that nothing - including catastrophic events like the DAO Hack – should be reverted stayed on the main chain, now known as Ethereum Classic.

On June 21st 2017, the price of Ethereum on the exchange GDAX crashed briefly to 10 cents USD per ether due to a massive sell order.

This is a testament to the massive volatility of cryptocurrency prices.

Much of the price is dependent on the public perception of the currency.

Here are a few things that affected it.

Speculation about how the SEC would rule on the DAO Hack led to a drop in prices, since more SEC regulation would mean that it would be much harder to trade cryptocurrencies.

On the other hand, the advent of cryptocurrency exchange-traded-funds like the Winklevoss Bitcoin ETF meant that the average person could invest in cryptocurrencies without having to worry about dealing with exchanges and storing the tokens.

Initial coin offerings, or ICOs, have also been a huge factor in the price of Ether.

We’ll talk more about them later on, but know that in the third quarter of 2017 Q3’17, ICOs have raised $1.3B with 150 ICOs while seed/angel investing across all tech sectors has raised $1.4B across 1602 deals.

Venture capital funds have also started investing into Ethereum technology, either into the ICO like Blockchain Capital, or directly into the token like Polychain Capital.

Given the exploding interest surrounding cryptocurrencies, FOMO, or “Fear of Missing Out” plays a role in many people’s investing decisions.

For better or for worse, people don’t want to miss out on the “next bitcoin”, and end up investing in cryptocurrencies like Ether, driving the price in upin a positive feedback loop.

The fact that Bitcoin’s and other cryptocurrencies’ prices are starting to be broadcasted on public radio further contributes to this “FOMO.”

And economic and political circumstances like Brexit, Trump’s election, or India’s war on cash can also contribute to driving up the price of cryptocurrencies, since they tend to undermine people’s trust in centralized systems and cause them to shift towards more decentralized systems like cryptocurrencies.

Due to some economic and political situations, and of course the aforementioned Fear of Missing Out, cryptocurrencies became extremely popular, resulting in a massive hype train.

In December 2017, cryptocurrencies started attracting much more “mainstream” attention, and let to a change in the demographic of the people who were invest, in particular, an increase in the number of millennials getting involved.

A decentralized application built on top of Ethereum called CryptoKitties, an online marketplace for virtual cats, became so popular that at one point, it contributed to 10% of Ethereum’s total network transaction volume.

It’s important to have an educated understanding of the space…...rather than make decisions based on volatility and prices.

Although the price of Bitcoin is many, many times higher than what it was a year ago, it’s started to take a downward turn in the beginning of 2018 after peaking in December 2017.

An increase in the amount of international regulation especially in India...South Korea…US and the UK, combined with a “mob mentality” of people investing without truly believing in the technology led to quickly changing market caps, and eventually the hype train crashed.

While blockchain technology was initially developed to underlie cryptocurrencies, it quickly drew the attention of enterprises, which wanted to harness the same technology to create private or permissioned blockchains.

Thus, gave rise to enterprise blockchains.

We’ve been talking a lot about public blockchains like Bitcoin and Ethereum.

Now we’ll be switching gears to see how blockchain fits in with enterprises, particularly with banks -- ironically enough.

As a technology that was initially created to avoid large banks and powerful centralized institutions, blockchain seems to have come full circle...

As blockchain became more mainstream, banks started to pick up on the technology as well.

They took note of Bitcoin and what it offered as a digital currency, but did not agree with Bitcoin’s design goals of being open, decentralized, and trustless.

After all, banks want their users to trust them, and keep operations private and controllable.

Banks looked for a way to apply blockchain technology WITHOUT having to replace the US dollar or any other national currency with cryptocurrency.

They wanted to find ways to leverage this new distributed ledger technology without inheriting the trustless, distributed, and decentralized overhead from Bitcoin.

This led to a rise of interest in “private blockchains” or “permissioned ledgers,” where the network is not open, not trustless, and does not have a mining scheme with underlying economic incentives (mining rewards).

In a sense, they wanted to separate “blockchain” from “Bitcoin”.

These blockchains take the fundamental cryptographic technology from bitcoin (public key cryptography) and modify it to be more compliant to enterprise use.

There are many different enterprise technologies in the space today, including R3’s Corda, Chain, JP Morgan’s Quorum and Juno, and Digital Asset Holdings.

The Hyperledger project is an open source blockchain run by Digital Asset Holdings and the Linux Foundation.

IBM’s Open Blockchain is a platform that is now part of the Hyperledger project as “Fabric”.

The way that companies, especially financial institutions, have looked at blockchain technology has changed drastically over the past few years.

Let’s take a look specifically at Jamie Dimon, the CEO of JP Morgan Chase...

...As one of the most influential figures in the financial sector, his words can play a huge part in shaping the public opinion on blockchain.

In January 2014, he said about Bitcoin “It’s a terrible store of value. It could be replicated over and over.”, which doesn’t really mean anything, indicating that people really didn’t, and still don’t understand what Bitcoin is.

In October 2014, he said “Bitcoin developers are going to try and eat our lunch. And that’s fine. That’s called competition, and we’ll be competing.”

The statement seems aggressive, but he actually gives legitimacy to Bitcoin by suggesting that it’s a competitor to traditional banks and finance.

In November 2015, “Virtual currency, where it’s called a bitcoin vs. a US Dollar, that’s going to be stopped…No government will ever support a virtual currency that goes around borders and doesn’t have the same control. It’s not going to happen.”

Now, Bitcoin isn’t just competition, it’s a bona fide threat.

It’s something that bankers can’t control, and they hate it.

In October 2017, he said “Bitcoin is a fraud that won’t end well. If you’re stupid enough to buy Bitcoin, you’ll pay for the price of it one day. The blockchain is a technology which is a good technology. We actually use it...God bless the blockchain.”

Note that it looks like Dimon has started to understand the merits of blockchain technology as opposed to just Bitcoin itself.

His stance towards Bitcoin doesn’t seem to have changed.

These comments were actually made at an event hosted by the Institute of International Finance, which was hugely publicized.

Just to drive his point home, here are some more comments he made in September 2017.

“I’d fire a JP Morgan trader in a second who traded [Bitcoin]. It’s against the rules, it’s stupid, it’s dangerous.”

And in another quote he said “One of my daughters bought bitcoin and it went up, she thinks she is a genius.”

In January 2018, he said “The blockchain is real. You can have crypto yen and dollars and stuff like that … the bitcoin to me was always what the governments are gonna feel about bitcoin as it gets really big, and I just have a different opinion than other people.”

Cryptocurrency and blockchain technology started its roller coaster of a journey with a small group of cypherpunks, who believed in cryptography as means to promote privacy…it had a rough beginning as we saw with the failed DigiCash…but now we’ve ended up here at JP Morgan Chase, one of the largest American multinational financial services firms.

The history of Bitcoin and Blockchain is a story of rapid transformation.

Originating as a fledgling technology founded on libertarian ideals, Bitcoin had its beginnings in an industry full of scandals and activity.

As bitcoin began to rise in value and attract attention from a wider and wider audience, there was a shift in focus away from Bitcoin itself and more into the other innovations made possible by the underlying technology, the blockchain.

Where exactly do people discuss bitcoin and blockchain technology?

There are a few different forums that people use.

There’s Reddit’s r/bitcoin, online discussion boards like Bitcointalk.org, Bitcoin meetups and conferences, and Bitcoin specific IRC Channels.

There are also organizations like us (Blockchain at Berkeley), that are dedicated to discussing and pushing forward blockchain technology.

We have a Slack channel with over 2000 members that people use to discuss news and developments in the blockchain space.

We also host many public events, workshops, and meetups, teach multiple classes on the UC Berkeley campus, and also this one online.

And this all done to foster and develop the blockchain community.

But within the community, there’s always some internal political debate going on.

For example, there’s a lot of the internal politics that occurs between miners, who want higher transaction fees, and merchants, who want lower transaction fees.

There’s also politics regarding the scalability debate in Bitcoin, the Ethereum split, and many other issues in the space.

Most of the core Bitcoin community can be very libertarian, as we’ve seen with the roots of Bitcoin and the cypherpunks.

And oftentimes, people’s political views influence how they feel towards technological debates as well.

It’s a common sentiment that getting die hard libertarians to agree is a very hard problem.

Many libertarians think that implementing blockchain into the government is a good way to hold governments accountable.

On the other hand, libertarians also think that people should be able to do what they want.

The flag on this slide is the Gadsen flag, which has strong Libertarian roots.

It’s used as a libertarian symbol because the porcupine is the “quiet”, unassuming warrior of the forest, which doesn’t attack so long as you leave it alone.

The United States government tries to best represent its citizens’ opinions and interests through a representative democracy.

In the blockchain community, there are different ways of coming to consensus about the changes that happen, and the process isn’t always smooth.

For example, in a previous section, we talked about Bitcoin Improvement Proposals.

Some controversial topics in the blockchain space have been surrounding problems like block size (specifically segwit2x), confirmation times, and centralization in third party companies (Intel’s SGX).

In this course, we explain these issues in an objective fashion, and let you form your own opinion on these debates.

We’ve told the story of Bitcoin and blockchain: from the cypherpunk movement all the way to JP Morgan Chase.

We’ve also explained a bit about the Bitcoin and blockchain community: where it exists both online and offline, and a bit about the political landscape.

So where is the space now?

That’s what we’re going to explore in this next section on the state of the industry.

Nowadays, you can’t speak about the state of the blockchain industry without at least mentioning ICOs, or initial coin offerings.

They’re a way for new projects, startups, and companies to sell their underlying crypto tokens in exchange for investors’ money.

Think of ICOs as Initial Public Offerings, but instead of investors purchasing shares of a company, they purchase the coin underlying a new project.

ICOs are very different from equity.

Having a new project’s coin doesn’t give you ownership of the project, but instead enable you to use the project when it becomes available.

Thereby, by buying into an ICO, it shows that you are interested in this new project.

This incentivizes others to do the same, showing that the project will probably be widely used, since without the associated token, you can’t use the project.

For example, here are a couple famous ICOs: Bancor ICO raised $150 million

Tezos (XTZ) ICO raised $200 million Filecoin ICO raised $253 million

ICOs are permissionless and enable ANYONE to invest in a project that they feel will be successful.

We put emphasis on the word ANYONE because of the open and public nature of these cryptocurrencies.

And when you think of it, it’s pretty wild.

From this tweet:

“95% of Americans are not allowed by law to invest in start-ups.

Only ‘accredited investors’ are entitled to do so, but you can buy lottery tickets all you want or go to Las Vegas to gamble”

This shows that ICOs are leveling the playing field for investments.

Normal people now can invest in any blockchain project they want to.

And with so many new projects coming out, some with more potential than others, the community needs some way to support them.

That’s where ICOs come in.

As a comparison:

In Q3’17, ICOs have raised $1.3B with 150 ICOs while seed/angel investing across all tech sectors has raised $1.4B across 1,602 deals.

Perhaps one of the most popular projects of 2017 was Cryptokitties.

The idea behind Cryptokitties was to allow users to purchase, collect, breed, and sell various types of virtual cats in a virtual game run on the Ethereum blockchain.

Each Cryptokitty is unique and ownership is validated through the blockchain.

Initially presented as a hackathon project at EthWaterloo in October 2017, Cryptokitties launched at the end of November 2017 and its popularity quickly skyrocketed -- so much so that in December, Cryptokitties was responsible for an all-time high transaction volume, congesting and significantly slowing down the Ethereum network.

Some have blamed Cryptokitties for blocking out “more serious” transactions on the Ethereum network, and some have compared Cryptokitties to the Beanie Baby craze, saying that both are naturally devoid of value.

On the bright side, however, some people think that cryptokitties contributed to the popularity of ethereum and brought public awareness to the cryptocurrency space, which is helpful for the public adoption of cryptocurrencies.

It’s clear that cryptocurrencies and blockchain are hugely popular now.

Whether or not its mainstream spotlight and growing popularity is a good thing is another question.

With the growing popularity of blockchain and cryptocurrencies, it’s important to have good wallet software.

Parity wallet is a popular multisignature wallet, which requires multiple people to sign off on transactions, and was created by Ethereum co-founder Gavin Wood and his team at Parity Technologies.

Parity marketed itself as the fastest and most secure way of interacting with the Ethereum blockchain, but in November 2017, this happened…

A user by the name of devops199 accidentally locked up 300 million US dollars worth of ether that had been owned by users of Parity multisig wallet.

One note: by the time I went to GitHub to take a screenshot of the issue devops199 opened up, they had already deleted their account -- which is why it says “Ghost,” which is what it says for deleted users.

Parity had relied on an external smart contract to use as a software library for more heavy computation, and this smart contract had not had an extensive security audit before launch on the public Ethereum network.

All it took was for one curious learner to call the “kill” function -- and then 300 million was lost.

Due to Ethereum architecture, and the immutability of public blockchains, code that is published is hard to take down, so it’s important that smart contracts do not have bugs.

This was a huge eye-opening moment for the Ethereum community as a whole, and taught developers -- especially open source developers -- to fully audit the security of mission critical code.

Coincheck was marketed as one of the largest and most popular cryptocurrency exchanges in Asia -- at least up until a hack in January 2018 that after the dust had settled turned out to be the largest cryptocurrency hack in history.

More than half a billion US dollars were stolen in the Coincheck hack of January 2018.

Compare this to the already immense Mt. Gox hack in 2014 that resulted in 400 million US dollars stolen or lost.

Coincheck estimated that upwards of 250,000 (two hundred fifty thousand) users were affected by the hack, and reassured users that it was not an inside job, but a legitimate breach and hack.

Most of the cryptocurrency stolen in the Coincheck hack was lost in a single event.

And compare this with Mt. Gox, which we had mentioned lost a majority of its cryptocurrency through multiple thefts across a span of multiple years.

The landscape for crypto and blockchain is pretty wild in the current day.

There are a lot of exchanges, wallet software, and other blockchain utility applications, and an even greater number of ICOs out there.

Since blockchain is still young, it’s important to do your own due diligence when using blockchain software.

Looking at the broader picture, that’s one of the goals of this course -- to get you to start thinking critically about blockchain fundamentals and the more technical aspects about the technology that can then help inform your own personal decisions.

In the face of increasingly powerful banks and national agencies, the Cypherpunks and Crypto-anarchists of the late 1980s advocated the use of cryptography to preserve privacy, which they defined as the power to selectively reveal oneself. They sought to develop an anonymous digital transaction system. 

In October 2008, Satoshi Nakamoto released the Bitcoin whitepaper, which outlined, for the first time, an anonymous, trustless, decentralized cryptocurrency.Bitcoin was built upon a history of failed cryptocurrencies, including Digicash, Hashcash, and B-money. Bitcoin relies on Proof-of-Work, a peer validation protocol introduced by Hashcash, that expends computational power to solve cryptographic puzzles and to cast votes. As in Digicash, each node in Bitcoin maintains their own identity through public and private keys, authenticating transactions using blind signatures. As in B-money, every Bitcoin full node maintains a copy of the blockchain. Bitcoin is a deflationary currency, with 21 million total bitcoins that will be slowly introduced to the bitcoin supply via block rewards.

The first transaction using Bitcoin to purchase a tangible asset occurred in 2010, when Laszlo Hanyecz exchanged 10,000 bitcoin for $25 worth of pizza. This exchange gave bitcoins real value, a first step towards legitimacy. 

As Bitcoin began increasing in popularity, it also began facing increasing cases of thefts, hacks, and illegal activity. The first was a large hack on Magic: The Gathering Online Exchange (Mt. Gox), one of the first Bitcoin exchange website created in 2010. 

Bitcoin was also used for the purchase of illegal substances on the dark web, especially through the website “Silk Road,” nicknamed the “eBay for Drugs.” As Bitcoin became more accessible and useful to the public, it steadily grew in value, reaching a bubble in 2013.

Altcoins, such as Litecoin, ZCash, Stellar, Ripple, Ethereum, Dogecoin, DASH, and Monero, began popping up soon after Bitcoin’s success, each serving a different functionality.

In 2014, merchants, such as Overstock.com and PayPal (with Coinbase), started accepting Bitcoin. Bitcoin startups and wallet companies, including Coinbase, Bitpay, and Blockchain.info, started appearing around this time as well. Around this time, people started to differentiate between the term blockchain from Bitcoin. 

Bitcoin is far from perfect. One of the biggest technological challenges Bitcoin faces is that of scalability. Such issues raise concern of decentralized governance. Currently, Bitcoin users may propose Bitcoin Improvement Protocols on online forums and gather ad hoc community votes on proposed matters.

Another influential blockchain platform is Ethereum. While Bitcoin is a storage of value, Ethereum is a platform designed to execute arbitrary code called “smart contracts”. Users pay for code execution using Ethereum’s internal token, called ether. With Ethereum’s introduction, Distributed Autonomous Organizations (DAOs), programs on the Ethereum blockchain that create a distributed government, gained popularity. “The DAO” was a decentralized Venture Capital DAO that suffered a $120 M hack in 2016. Disagreement about whether to roll back the history of transactions to before the hack led to two versions of Ethereum. Ethereum(the main chain) rewound the transaction history to before the hack, whereas Ethereum Classic continued the original chain despite the hack. 

While concerns of the SEC’s reaction to The DAO hack initially lowered Ethereum’s price, the growing popularity of Initial Coin Offerings and cryptocurrency exchange-traded-funds increased Ethereum’s price overall. Economic and political changes in early 2017 along with an expansion of the crypto user base to include millennials further contributed to growth in Ethereum value and transaction volume.

Meanwhile, banks started seeking ways to apply blockchain technology leading to an increased interest in “private blockchains.” Enterprise blockchain technologies today include R3’s Corda, Chain, JP Morgan’s Quorum and Juno, Digital Asset Holdings, and IBM’s hyperledger. Blockchain has come a long way -- from online ideation amongst Cypherpunks to adoption by JP Morgan Chase.

The blockchain space has expanded to include not only major financial institutions, but also the general public. Initial Coin Offerings, equity-less fundraising schemes for new crypto startups that allow anyone to participate, and the advent of Cryptokitties, an online marketplace for virtual cats, indicate the spread of blockchain across communities and also to popular culture.

A Cypherpunk’s Manifesto by Eric Hughes 

Coindesk: A Bot Named Willy: Did Mt. Gox's Automated Trading Pump Bitcoin's Price? 

The DAO, The Hack, The Soft Fork and The Hard Fork 

(Optional, to Get Ahead) Princeton Textbook 1.1 Cryptographic Hash Functions (pages 23-31) 

(Optional) All You Need to Know About ICOs

(Optional) Digital Gold by Nathaniel Popper 

We’ve seen how Bitcoin works at a high level.

We know what needs to be implemented.

However, we don’t know how that happens.

This lecture will teach you everything about the low-level specifics of Bitcoin that make it work.

First, let’s discuss cryptographic hash functions: what they are and how they are used in Bitcoin.

Then, we’ll see how we can use cryptographic hash functions to create a tamper-evident database -- the blockchain that we all know and love.

We’ll then discuss digital signatures, the elliptic curve digital signature algorithm, and how we actually generate and use our private key, public key, and address.

Finally, we’ll introduce Bitcoin script and its functionality as well as its role in making Bitcoin more flexible.

Recall our earlier assumptions about Bitcoin: we can’t trust anyone.

Before, we considered trust in the context of consensus, but what about information that moves through the network?

For example, if someone sees a block with my transaction in it, what’s to stop them from modifying or even replacing it?

We previously demonstrated how to have consensus on updates to the blockchain through Proof-of-Work.

Now, we will demonstrate how we can be certain that the update is the same for everyone by designing a tamper evident system.

Tamper evident means that although information can be tampered, it is obvious that there has been some manipulation of the information.

We’ll see that in order to design our tamper evident database, we first need a source of standardized randomness.

For this, we’ll utilize cryptographic hash functions.

A simple way to do this is with some fingerprinting system.

How do fingerprints apply?

Well, think of human fingerprints.

Each one is unique, difficult to forge, and close to impossible to predict.

When entering another country, they often ask for your fingerprint as that is a unique identifier of you.

If you are someone else, then your fingerprint changes.

The same applies in Bitcoin.

Except instead of humans and thumbs, we have meaningful information and random data.

If we can design a way to generate fingerprints of our meaningful data, then we can ensure the integrity of our information.

With this fingerprint system, if the information is changed, then so is the fingerprint.

Here’s the fascinating thing about fingerprints: when you really think about it, fingerprints are just standardized randomness.

You can’t guess what someone's fingerprint will look like, just by looking at them.

In the same way, you shouldn't be able to guess the data that produced a digital fingerprint.

But how do we all agree on a way to generate fingerprints?

We need standardized randomness.

Cryptographic hash functions are one way functions that take some input and produce a pseudorandom output.

We say pseudorandom because, while it appears random to us, it is actually always going to be the same output for some given input.

In other words, if I take the hash of “hello, world!” once, it will be the same if I do it a second time.

This is necessary for standardization between all parties.

A cryptographic hash function always outputs a message of some given size.

One note is that we call the output the image, and the input the pre image.

Cryptographic hash functions differ from regular hash functions in that they are built for security, but for convenience, we will now refer to cryptographic hash functions as hash functions.

What are the properties of hash functions then?

Well, we have three important properties to discuss: pre-image and second-preimage resistance, along with collision resistance.

To clarify what these terms mean, let’s go over a couple definitions.

“Pre-image” resistance is synonymous with “input.”

“Resistance” is synonymous with difficulty.

For example, “pre-image resistance” refers to the difficulty of finding an input given some output.

We’ll dive more into each shortly.

Key Properties of Cryptographic Hash Functions: Pre-Image Resistance

First off, we want to make sure that no one can reverse engineer our fingerprint.

Otherwise, certain issues come up, such as exposing information that we didn’t want to reveal.

To make sure this doesn't happen, we need what’s known as preimage resistance.

The technical definition is as follows: given some hash function H and some output of the function H of x, it is computationally difficult to find x.

In other words, we cannot easily discover the input which created some output.

Key Properties of Cryptographic Hash Functions: Second Preimage Resistance

Next, we want to make sure that no one can steal fingerprints.

If someone else can generate my fingerprint with a different input, then no one can tell who originally made the fingerprint.

To make sure this doesn’t happen, we need what’s known as second preimage resistance.

The technical definition is as follows: given some hash function H and some output H of x, it is computationally difficult to find a different input x’ such that H of x equals H of x’.

Expanding the concept of second preimage resistance, we generally recognize that any two arbitrary inputs mapping to the same output is a bad thing.

To prevent this, we need collision resistance.

The technical definition is as follows: given some hash function H, it is computationally difficult to find two different inputs x and y such that H of x equals H of y.

Consequences of these properties is what’s known as the Avalanche Effect.

This means that any change in the input leads to a pseudorandom change in the output.

This prevents a hot or cold game with inputs, where you attempt to guess the output based on inputs.

On the right side, we've hashed the string "I am Satoshi Nakamoto0", "I am Satoshi Nakamoto1", all the way to 19...

And each result is pseudo random.

There's no relation between the Hashes despite the inputs being so similar.

The particular hash function that Bitcoin chooses to use in many scenarios is called SHA-256.

SHA-256 is a cryptographic hash function that was designed by the NSA.

SHA-256 is a member of the SHA-2 family of cryptographic hash functions, SHA standing for Secure Hash Algorithm.

SHA-256 takes in an input of size less than 2^64 bits and produces a 256 bit fixed size output.

In practice, the 2^64 bit upper bound on the input size is so massive that we usually just say that it takes in an arbitrary amount of information.

Bitcoin, in many cases, uses SHA-256 squared, or SHA256d, which simply means that SHA-256 is used twice in a row.

Once on the original message you want to hash, and another time on the output of this first hash.

Now that we know about cryptographic hash functions and their properties, how do we then apply them to design our tamper evident database for Bitcoin?

In this section, we’ll be taking a look at the structure of a block, looking specifically at some of the metadata contained within the block header that will help us achieve tamper evidence.

What does this tamper evident database actually look like?

Well, if you’re an average user, here’s a real example of a block found on Blockchain.info, a company that keeps track of all information about the Bitcoin Blockchain for public access.

We’ll explain each of these fields by the end of the video.

Here’s the exact same block, but now in JSON format.

This is a much more understandable format for a computer.

But what is the role of each of these different pieces of information?

Here’s what a block looks like conceptually.

It has four main components: the Block Header, Block Size, Transaction Counter, and Transactions.

The “Block Size” field tells how large the block is.

The “Block Header” field represents the metadata necessary for understanding the components of the block.

The “Transaction Counter” field says how many transactions are within the block.

The “Transactions” field is the actual transaction data.

For this lecture, we’re going to focus first on the composition of a block header, the information that enforces the security of the blockchain.

How does this ensure tamper evidence?

Well, let’s go ahead and look at where hash functions are applied within Bitcoin.

The block header refers to all the metadata associated with every block.

There are six fields in the implementation, but we’re going to speak to the three important ones that implement the protocol explained in module 1.

Those are;

1. the Merkle Root,

2. Previous Block Hash, and

3. Nonce fields

The Merkle Root represents a summary of transactions.

The Previous Block Hash represents the chaining.

And the Nonce represents the proof-of-work.

The block header is simply the hash of all these fields concatenated.

Let’s first take a look at the Merkle Root.

What is a Merkle Root, and what do we mean by a summary of transactions?

Does this mean that we have contained all information about our transactions in one piece of information?

Well, it actually means that we have contained all fingerprints of information in one piece of data.

The Merkle Root is the top of a Merkle Tree, which is a cryptographic data structure.

This is definitely complicated, so let’s take it slow and start from the very beginning.

First and foremost, a tree in Computer science is a data structure that has some root node and some children, which may also be the roots of other trees.

A binary tree is a tree in which every node has at most two children.

A Merkle Tree is just a very specific version of a binary tree, where two things are true:

• there are a power of two children on the bottommost level, and

• the lowest level is made of the hashes of the information that you would like to summarize.

Perhaps a better way to explain the Merkle Tree is to describe its construction.

We start off with a set of transactions that we have verified.

We lay them out one by one.

We hash each one to get a level of hashes.

We then hash each pair together, making a new level with half as many hashes.

We continue this process until we finally have only one hash at the top.

This hash at the very top is the Merkle Root.

This way, we can detect any transaction changes after the commitment with the Merkle Root.

We see here that if any transaction is changed, then the Merkle Root is affected.

This is demonstrated in the following diagram: when Transaction 3 is changed, then so is its hash.

Because of that, the hash above it is also effective, propagating all the way up to the topmost level.

You may ask, “Isn’t it expensive to verify that a transaction was included within a Merkle Tree?"

Don’t worry, we don’t need to recollect every transaction to do that.

We can actually prove it with just one piece of information at each depth.

You’ll notice that we only need to hash Transaction 1 with these two pieces of information:

Hash B, and Hash F.

This is actually a great efficiency gain as our tree grows larger.

It’s pretty obvious what the Previous Block Hash does: It merely contains the hash of the previous block.

But notice this: every block does that.

This means that if any block is altered, then the block after will also be altered, and so will the block after that.

Changing any part of this history also changes the entire future after that point.

Here’s an example of what happens when that tampered Merkle Root from before manifests within the blockchain.

It changes the block header, which changes the next block’s Prev Block Hash, which affects the rest of the future block hashes as well.

The rest of the network will reject the history proposed by this blockchain because they will not agree.

This is why the blockchain is considered immutable, because all tampering is easy to see.

Proof-of-Work: Partial Preimage Hash Puzzle

Now that we’ve finished talking about how Prev Block Hashes, let’s talk about the nonce, the physical manifestation of Proof-of-Work.

Keep in mind that we need entities to prove that they’ve done some work before they’re allowed to submit a vote to the network, meaning that we have to design a problem or puzzle for their computers to solve to prove that they’ve done work.

The way we do this in Bitcoin is with a partial preimage hash puzzle.

Keep in mind the definition of preimage from before, which was the input.

Partial preimage then means a partial input.

In Bitcoin, we are given part of an input, and we have to find the other part which produces some particular output.

The condition that needs to be met is that the hash of the block header is less than some target value.

This condition is how Proof-of-Work is implemented in Bitcoin and almost every other popular Proof-of-Work cryptocurrency.

To satisfy our needs for a puzzle, these hash puzzles need to have three characteristics:

1. Computationally difficult,

2. Parameterizable, and

3. Easily verifiable.

Computational difficulty means ensuring that the solution to the puzzle cannot be easily found.

There’s no point if Proof-of-Work takes little work.

Parameterizable means adjustable.

The difficulty of the puzzle should be adjustable to ensure that it never gets too easy or too hard.

And finally, an easily verifiable puzzle ensures that computers don’t have to do too much work to see that an answer is correct.

It should just take one hash for example to prove that some nonce is correct, even if finding the nonce takes millions of tries.

For a better idea of how mining works, let’s bring up an analogy.

Mining is like throwing darts at a target while blindfolded.

This implies two things.

First, we have an equal likelihood of hitting any part of the target.

We have no way of knowing what’s closer to or farther from the center.

Two, the only way we can be more likely to ever hit the center is by throwing more darts.

Again, since we have no way of trying to make any individual dart more likely to hit the center, the best we can do is throw more.

This compares to mining because miners are looking for some hash output that is below some algorithmically decided target.

It’s as if that target is the green line dividing the valid and invalid blocks in the diagram.

How do we adjust the size of this circle?

How do we set and change the target?

This all happens through the difficulty, which is a representation of the number of expected computations required to find a block.

Again, we can’t predict how many computations anyone produces to solve the puzzle, but we can approximate based on how quickly the puzzle is solved on average.

The difficulty is implemented as a requirement of a leading number of zeros on the block header hash.

This is why the example block at the start of this section had many zeros at the start of its block hash.

As the number of zeros increases, so will the difficulty, and vice versa.

This difficulty adjusts with the global hashrate.

We know that the amount of computing power in the network will always be changing, as miners join and leave the network, but we want to maintain a block time of ten minutes.

For this reason, we have to raise and lower the difficulty alongside the hashpower growth and decay of the network.

The way we recalculate this difficulty is with the equation on the slide: difficulty is equal to itself times the ratio of two weeks to the time taken to mine the previous 2016 blocks.

Every two weeks, we check to see how long it took to calculate those 2016 blocks.

If every block took exactly 10 minutes, then it should have taken precisely two weeks to produce those 2016 blocks.

If we took too long, it’s because the puzzle was too hard to solve, and if we didn’t take long enough, then the puzzle was too easy.

We make adjustments on the puzzle difficulty accordingly going forward.

A quick sanity check to see if your understanding is correct.

Let’s say that the current difficulty is 10.

Then what’s the new difficulty if the time to mine 2016 blocks is exactly two weeks?

Yes, it’s still 10!

The puzzle was precisely as hard as we wanted, so the difficulty stays exactly the same.

What about when the time to mine those 2016 blocks is just one week?

Or a staggering 4 weeks?

If time to mine is one week, then the difficulty is 20!

We mined those blocks in half the expected time, meaning that the puzzle was half as hard as necessary, so we make it twice as difficult.

If the time to mine is 4 weeks now, the difficulty is now 5!

We mined those blocks in twice the expected time, meaning that the puzzle was twice as hard as necessary, so we make it half as difficult.

The difficulty is inversely proportional to the time to mine.

You might be asking, where does the block reward to miners even go?

Well, it goes in the coinbase transaction!

Whenever a miner produces a block, they first make a coinbase transaction which is always the first transaction of the Merkle Tree.

This coinbase transaction grants miners a reward of some bitcoins which can be spent at some later date.

This is how new bitcoins are minted, or introduced, into the network.

Also, the Coinbase transaction has a separate nonce field that is used in our hash puzzle as well.

Here's some mining pseudo code.

We loop infinitely until we find a valid block.

In each loop, we try a Coinbase nonce first, and then in an inner loop, exhaust all possible values for header nonces.

Then we increment the Coinbase nonce, and repeat this cycle.

The reason why we want to do this in this order: to exhaust all header nonces for each coinbase nonce, is that changing the Coinbase nonce changes the merkle root.

It would be more expensive to recalculate the merkle root each time on each iteration of the inner loop.

We optimize by trying nonces in such a way as to only have to calculate the merkle root in each iteration of the outermost loop.

Now we’ll go into how to use digital signatures to send messages and transactions both pseudonymously and trustlessly.

In the previous section, we showed how transactions contained in previous blocks in the blockchain are tamper evident.

Using digital signatures, we can also ensure that the current transactions we send to the rest of the network are tamper evident as well.

Recall from early in the course that our motivation for having signatures is to authenticate identity, so that you know a message you receive could only have come from one other user, because it was signed with that user’s specific private key, proving ownership of a public key.

In this next section, we’ll go over what we want from a digital signature scheme at a high level.

Consider two users on the network: call them Alice and Bob.

Recall that they both have private and public keys.

Private and public keys in Bitcoin are generated through an algorithm called ECDSA, or Elliptic

We’ll go over that in the next section.

Alice wants to send over a message to Bob.

How can she do this and make sure no one tampers with her message?

First we point out that Bob has access to Alice’s public key, since that’s public information that’s used to identify Alice.

We’ll see later on that this is important to help Bob verify that a message coming from Alice was actually sent by Alice.

Before sending out her message, Alice signs her message with her private key.

This generates a unique signature that proves that she created the message as it is.

She then sends her signature along with her original message to Bob.

The idea is that the message is the main payload, and the signature can be used to prove that Alice was the one who created that exact message.

Bob is then able to easily verify that the signature is valid, given Alice’s public key and the original message.

And if the signature is valid, then that means the message wasn’t tampered with.

In order for Alice to have created that signature, she must have had the associated private key.

Also, at any point in time if her message was tampered with, the signature could not be valid anymore.

This is how digital signature schemes make messages tamper evident.

A key point is that Bob or anyone else should not be able to guess Alice’s private key, given just her public key.

Otherwise, Bob or anyone else would be able to recreate Alice’s signatures and thus fake her identity.

We’ll see how this is prevented in the next section, where we explain the way we generate public and private key pairs.

In summary, here are some key properties that we should keep in mind.

These are essential properties of digital signature schemes that are enabled by the functionality that we went over just now with Alice sending a message to Bob.

First, given a message and a signature, as well as a copy of the sender’s public key, the recipient should be able to identify the message origin.

Since the message has been signed by the sender’s private key, this shows that the original sender has authorized this message.

There’s also non-repudiation, meaning that the original sender should not be able to backtrack.

Since they have already signed the message with their private key, there should not be away to undo that signature, or to nullify it.

And changing the message would make it such that signature wouldn’t match up with it anymore.

Finally, message recipients should also be able to verify message integrity.

And this is what we saw earlier: that signatures can be used so that messages can't be modified by anyone after signing.

Here’s a diagram showing a high-level overview of what we’re going to be diving into in the next couple slides.

We start off with a private key that we generate randomly.

We then can use Elliptic Curve point scalar multiplication to derive a public key.

And then from there, we can use hash functions to finally arrive at our Bitcoin address.

One important note is that the processes mentioned -- the elliptic curve point scalar multiplication and the hashing -- are all one-way processes.

We can go forward, but not backwards.

A private key wouldn’t be so private if we could guess the private key from the corresponding public key or address.

Alright, so let’s take a look at elliptic curves first.

Bitcoin uses ECDSA, or Elliptic Curve Digital Signature Algorithm, to produce private keys and public keys.

An elliptic curve is just a mathematical curve defined by the general form y^2 = x^3 + ax + b.

We take everything over a finite field because we want to encode every value possible in a constant amount of space.

Here’s a picture of Bitcoin’s elliptic curve, secp256k1, on the right side.

To illustrate, here’s a demo.

We start off with a generator point P that is known to everyone, since that’s built into secp256k1’s specification.

To add P with itself, simply take the tangent at the point P. Where the line intersects the curve let’s call it -2P.

We then reflect this point across the x axis, resulting in our answer 2P.

To get 3P, we can add P to 2P via the same process.

Draw a line through P and 2P, find where they intersect, which is -3P, and then reflect that across the x-axis again to yield our answer 3P.

Its equation is y^2 = x^3 + 7, and was engineered to have some unique properties that we’ll be explaining in the coming slides.

As mentioned earlier, it’s taken over a finite field so as to limit key size.

To illustrate, on the left side is the same curve, but over a small field where we only have integer values -128 to 128.

When we take it over a finite field, it doesn’t look like a smooth curve anymore.

However, there are unique properties of elliptic curves that will help us in creating a one-way function to generate public keys from private keys.

As mentioned earlier, Bitcoin’s elliptic curve is secp256k1.

This curve is specified with a couple parameters, including the actual curve formula itself (by its coefficients), the field, and a generator point.

Note the symmetry of the curve across the x axis.

This symmetry is preserved even when taking it over a finite field.

Also note that any non-vertical line on this curve will intersect the curve in at most 3 points.

On the graph on the right hand side, the line through points P and Q intersect at point R.

On this elliptic curve, we can do point addition using lines and points.

To add P plus Q, we can simply draw a line through them, intersecting at a third point R, and then reflect the point R across the x-axis , yielding our result P + Q.

And this is called the chord tangent process.

This is a trapdoor, or one-way, function, because given a point K that is P + Q, it is difficult to find the individual points P and Q.

The user generates a random number n for their private key.

They use elliptic curve point scalar multiplication by n to generate their public key: they add a known generator point P for secp256k1 with itself n times.

Then, to generate the address, we hash the public key nP with SHA-256 first, and then with another hash function called RIPEMD160.

SHA-256 makes the address quantum resistant, as quantum computers could possibly be able to break elliptic curves and reverse the one way point scalar multiplication we defined, but quantum computers cannot reverse hash functions.

RIPEMD160 then shortens the address size from 256 bits to 160 bits.

Now you might be wondering: given a public key nP, is it possible to get the private key n?

This problem is known as the Elliptic Curve Discrete Logarithm Problem, and is known to be computationally infeasible.

We can safely say that our procedure of generating public keys from private keys is fairly secure.

Public Key to Public Key Hash

Now that we have our public key, how do we get to our address?

We’ll be going over that in this section.

If quantum computers exist, we could use them to solve the elliptic curve discrete logarithm problem and find the private key given a public key.

Quantum computers can’t reverse hash functions, so we choose to run two hash functions on our public key.

First SHA-256, which we mentioned previously as a hash function prevalent throughout the Bitcoin protocol, and second, RIPEMD-160, which shortens the address size down from 256 bits to 160 bits.

Together, the combination of using SHA-256 and RIPEMD-160 is called “double hash” or HASH160.

After all this, we have our public key hash.

And after we have our public key hash, we’re just a little bit away from arriving at our Bitcoin address.

The next step is to make our pub key hash a bit more human friendly.

First, we add a version byte in front of our previous RIPEMD-160 hash from the last section.

In the version byte, we specify which network we’re on: the main Bitcoin network, or a smaller test network.

We’ll save this for how and call it our extended RIPEMD-160 hash.

We then run a Base58Check encode, which first involves converting what we have so far into base 58.

Base 58 is an alphabet that’s like our usual 62 character alphabet.

It has all single digits 0 to 9 and all upper and lower case letters, except for those that are hard to distinguish from one another.

We take out the 0 and capital letter O, and also capital “I” and lower case “l.”

This is to avoid ambiguity when copying down your address.

Then, we calculate a checksum.

We perform SHA-256d on our extended RIPEMD-160 hash, running SHA-256 on it twice.

We take the first 4 bytes of our SHA-256d hash as our checksum, and append it to the end of our extended RIPEMD-160 hash.

After converting the result to base58, we finally have our Bitcoin address.

As a recap, in our address, we have successfully encoded our public key hash, which allows people to pay us, a version byte that says what network we’re on, and also a checksum that accounts for human error.

If someone types or copies down your address wrong, then the address would not be valid given the checksum.

And this is something that decoding software will do for us.

It can hash the beginning of our address and compare it with the checksum to see if it is valid.

Now we’ll dive into Bitcoin script.

Bitcoin script, or just Script, is a language that was designed to be used for Bitcoin to process a variety of transactions -- from payments between two people, to more complex multi-signature transactions -- greatly extending the functionality of Bitcoin.

Before we dive into specifics about how Bitcoin scripts work, it’s important to first review how transactions work.

Remember the UTXO model?

Bitcoin doesn’t have “accounts” with associated bitcoin tallies.

Rather, you can think of each transaction as outputs from previous transactions feeding into new inputs.

Transactions contain the signature of the owner of the unspent funds.

These unspent transactions are UTXOs.

How do you spend your bitcoin, or “your UTXOs”?

This is important: spending bitcoin is the act of redeeming previous outputs with a proof that you are the legitimate redeemer, and then specifying who can redeem the output you are now creating by encoding that person’s information in your transaction.

For example, if I send bitcoin (in the form of UTXOs) to Rustie, then in that transaction it’s encoded that only Rustie can redeem those UTXOs.

That proof that Rustie would use to redeem his bitcoin is constructed with two things: a public key, and a signature.

We’ll go more into detail later.

Contents of a Transaction

Let’s take a deeper look into the contents of my transaction to Rustie.

As we can see, a transaction has three main “sections.”

The metadata section contains some housekeeping data, a unique ID of this transaction, locktime, and size.

The inputs section contains a list of previously created UTXOs as well as a proof that I am eligible of redeeming the money.

This proof allows me to redeem the UTXOs and use them to produce new outputs.

The outputs section contains a list of new UTXOs that will be sent to new addresses.

Each of these values is accompanied with a script that locks the value away from everyone except the intended redeemer who can provide a valid proof.

Taking a deeper look into the transaction, we start with the metadata.

The first piece of metadata is the hash of our transaction, or the unique “ID” of this transaction.

We also get information here in the field vin_sz, which stands for vector input size, or the number of input UTXOs being referenced in this transaction, as well as vout_sz, which stands for vector output size, or the number of new UTXOs being created.

In this case, I created this transaction with 2 UTXOs and send them as one new UTXO to Rustie.

Finally, we have the version and locktime.

Version is a number specifying the version of the Bitcoin software you are using.

We will talk more about locktime later in this lecture.

Now let’s talk about inputs.

Remember that in the metadata section we have a unique ID for this entire transaction object that we’re talking about.

Well these come in handy in the inputs, with each of the hashes you see here being references to the unique IDs of previous transactions containing the relevant UTXOs that are being redeemed now.

We also get a reference to the index of the input in the previous transaction.

“0” means the first input.

“1” means the second, and so on.

The scriptSig is the most important part of these inputs because they’re the required proofs that prove that you can redeem the associated UTXO.

Last but not least, the end product of our transaction is the output(s).

You can see here in this particular transaction we have only one output with a value of about 10 bitcoin.

Satoshi is the smallest unit of bitcoin.

One bitcoin can be converted to 100 million satoshi, which means that it can be divisible to the 8th decimal place.

The accompanying output address is a script that locks the transaction and makes it redeemable only by the specific proof that I specified in this transaction.

In this case, only after Rustie provides his proof can he unlock and spend my UTXOs.

Remember: in this transaction, the output addresses are actually scripts.

For this particular output script, we can read it as “This amount can be redeemed by the public key that hashes to address X, plus a signature from the owner of that public key.”

This brings us back to Script.

Do you have a better idea of why Bitcoin connects inputs and outputs through scripts?

Rather than just connecting public keys together, connecting inputs and outputs through scripts allows for potentially complex transaction types.

The Script language is stack-based, has a native support for cryptocurrency, and is purposely limited in capability for security reasons.

Let’s walk through an example of how Script works under the hood.

The example we are looking at is at the top of the screen.

This script specifies the most common type of transaction in Bitcoin, which is to redeem a previous transaction output.

To redeem this previous transaction, we need to prove our identities with (1) a public key that, when hashed, yields the address to which the previous transaction was sent, and (2) a signature that proves ownership of the private key corresponding to the public key we provided.

This is the most common type of script in Bitcoin Pay to Public Key Hash (shorthanded).

But how do we implement this functionality?

With locking and unlocking scripts.

Unlocking scripts are scripts that you provide in your input when you want to spend from a previous transaction, allowing you to redeem the associated bitcoin.

You provide the signature and your public key, which are needed to prove your identity, and to unlock the transaction output.

Your public key is then hashed and checked against the address that owns the UTXO.

Because you provide a signature, this script is called scriptSig.

Locking scripts are the scripts that are found in previous transaction outputs.

The locking script specifies the requirements for redeeming a UTXO.

This essentially “locks” down the UTXO, so that it can only be spent by whoever can unlock it.

In our Pay to Pub Key Hash example, the locking script requires that the users that want to spend from a previous transaction output MUST prove that they possess a private key that hashes to a specific address.

For this reason, this script is called scriptPubKey.

To make unlocking and locking scripts work in tandem, we simply concatenate them.

We put the unlocking script on top of the locking script, scriptSig on top of scriptPubKey, and then run them together.

And the entire resulting script must execute successfully in order for the transaction to be considered valid.

Now let’s piece everything that we’ve learned so far about Bitcoin script together, and walk through a sample execution of Pay to Public Key Hash.

First notice how the execution order is defined.

On the left hand side, we have the scriptSig on top, and the scriptPubKey on the bottom.

The scriptSig on top is the input script to a new transaction we are creating, and the scriptPubKey on the bottom is the output script of an old transaction we are trying to spend from.

Bitcoin Script is stack-based, which is first in last out, so execution will proceed as follows:

We first put our signature on the stack.

Then we put our public key on the stack.

OP_DUP will duplicate the top item on the stack OP_HASH160 hashes the previous item on the stack, first with SHA-256, and then with RIPEMD-160

Next, <pubKeyHash?> -- with the question mark in the angle brackets -- is the actual public key hash specified by the previous transaction output.

We put this on the stack as well OP_EQUALVERIFY then checks to see if the top two items in the stack are equal.

If they are equal, we continue with execution.

If not, then the transaction is invalid and we stop execution.

In this case, let’s say the transaction is valid, so we keep going with execution.

Lastly, OP_CHECKSIG checks the validity of the signature with the given public key.

If it’s valid, then it returns true.

And then finally, since the final return value is true, we know that the transaction was valid, and so the transaction will go through.

Something cool you can do with Bitcoin Script is write arbitrary data into the Bitcoin blockchain.

This is the idea behind proof of burn.

There’s an opcode in script called OP_RETURN that throws an error if it is reached.

If you specify a script that has OP_RETURN before the output script, then the output can’t be spent, since execution terminates before then.

If this transaction is published on the blockchain, then you’ve successfully proven that you have destroyed some bitcoin.

No one can ever spend that bitcoin again, since you’ve burned it.

Below OP_RETURN, there’s space to put whatever you want.

One thing you can do with this concept of burning bitcoin is that you could bootstrap an altcoin, requiring that you must destroy some Bitcoin, and show a proof of this, in order to get some altcoin.

Some altcoins such as CalCoin do this.

Aside from using OP_RETURN for the sake of deriving an altcoin’s value, you can also write arbitrary data into the blockchain.

Under OP_RETURN, there’s space to write whatever you want, so people have been creative with this.

You could potentially prove the existence of something at a particular point in time.

For example, if you coined a new word, and wanted to prove in the future that you had coined this word in the past, you could simply burn this data into the blockchain.

As we mentioned before, the Bitcoin blockchain is immutable, meaning that it cannot be changed realistically.

Transactions are also timestamped, so people can see that you had actually coined this word in the past.

Aside from words, you could also prove the existence of a document, a piece of music, other creative works, or anything else, so long as you have bitcoin to burn.

In this section, we’ll look at how to create arbitrary scripts in a scheme called P2SH, or Pay to Script Hash.

For a better understanding, we’ll compare pay to script hash to the scheme from the previous section: pay to pub key hash.

To make for a better mental model, we’ll be explaining everything in terms of a vendor-customer scenario, where the customer creates a transaction to pay the vendor.

In other words, the customer is the sender, and the vendor is the recipient.

As a recap, in Bitcoin, senders specify a locking script in the output of the transaction they are sending, and recipients provide an unlocking script in the input whenever they want to redeem their bitcoin.

In the previous section, we gave the example of Pay to Pub Key Hash, or P2PKH.

In this scheme, the vendor, or the recipient of the transaction, says: “Send your coins to the hash of this Public Key.”

And later, the vendor has to provide a public key and a signature, in order to redeem these bitcoins.

This is by far the simplest and most common case.

On the other hand, there is the more flexible Pay to Script Hash.

Rather than paying to just a public key hash, we can now pay to a more complicated set of instructions.

In this scheme, the vendor says: “Send your coins to the hash of this Script.

I’ll provide the script and the data to make the script evaluate to true when I redeem the coins.”

This makes sense because we don’t really want vendors to require customers to create a complicated output script first.

It’s much more customer friendly if the recipient of the transaction, the vendor, specifies the script.

As a refresher, here’s how everything works in Pay to Pub Key Hash.

If Bob wants to receive bitcoin from Alice, Alice has to have access to Bob’s public key hash.

She can then send a transaction paying to Bob’s public key hash.

In order for Bob to redeem the bitcoin that Alice just sent him, he must provide a signature and full public key, proving his identity and allowing him to spend from this transaction output.

Now let’s say that Bob wants to introduce some more functionality into his transaction with Alice.

Say he wants to use multisig or a time-lock -- two types of scripts that we’ll introduce later.

Instead of providing Alice a public key hash, Bob first creates a script, takes the hash of it, and sends it off to Alice.

Alice can then send a transaction paying to Bob’s script hash.

In order for Bob to redeem the bitcoin that Alice just sent him, he must provide a signature and the full script that he had written.

Why Pay to Script Hash instead of just pay to a script?

By providing a hash of the script to the customer, the vendor offloads the complicated task of having to compose a bitcoin script to himself.

This is particularly helpful from a vendor-customer standpoint, where the vendor is the recipient of bitcoin, and the customer is the sender.

Let’s say a vendor wants to receive money and use complicated features such as multisignature, they don’t want to burden their customers by forcing them to write complicated scripts just to make a transaction.

Therefore, all P2SH requires is for sender to use only hash of the script to compose a locking script.

The customer doesn't care what the script actually is -- all they care about is getting their goods.

It's up to the vendor that they write the correct output script so that they and ONLY they are able to redeem that output.

It also makes sense that the customer shouldn't need to know anything about how the vendor holds their funds in order to be able to easily send them money.

Pay to Script Hash was an update to Bitcoin back in 2012, and since then has been one of the most important improvements to Bitcoin since Bitcoin’s inception.

One of the most widely used example of P2SH is multisignature, where a specific number of pre-specified signatures are required to unlock a UTXO.

A typical multisig scheme is an m-of-n, where you need m of n signatures users to sign off on a transaction before it is considered valid.

Some unique benefits of multisig are...First, multisig scheme increases the difficulty of stealing funds.

Instead of using one private key to unlock the fund, the thief now has to have the minimum number of keys that satisfies the multisignature requirements.

Second, using multiple keys to unlock accounts prevents losses.

Before using multisig, losing the one and only bitcoin private key would be an end-all-be-all situation.

Now, with a 2-of-4 address, for example, you can still redeem the funds within the address by only having to provide signatures from 2 keys.

Thirdly, it can also be used to give control of a single address to multiple people.

For instance, the executive board of a company can mandate that 3 out of 5 executives must approve of a budget before a fund can be spent, and using a 3-of-5 signature suits the exact purpose.

Let’s take a look at the diagram on the slide to see how a multisignature script is constructed and executed.

In the first box of the left hand side diagram, you see the unlocking script, which contains a number of signatures that are just enough to satisfy the conditions of the locking script which allows the user to spend the UTXO.

Below the unlocking script is the redeeming script, which contains all possible and eligible signatures to spending the bitcoins.

It sets the rules on how and who can spend money from a particular account.

The letter m on the top of the box represents the minimum number of signatures, and the letter n shows the total number of valid signatures.

This redeeming script is only revealed and used when the redeemer wants to spend the money.

The redeeming script is then hashed, to check against the redeeming script hash, and evaluated.

Lastly, if you look at the bottom of the diagram, the locking script contains the hash against which the hash of the redeem script will later be compared with.

For a multisig transaction to be valid, the full redeem script is hashed and then compared with the script hash.

If they match, the UTXO is unlocked.

Timelocks are a type of functionality in Bitcoin that restrict the spending of funds until a later time or a specific block height.

Time in this case can either be represented as a UNIX timestamp, block height, or the number of blocks built on top of the block containing the transaction.

Timelocks can be absolute or relative.

Absolute timelocks specify an absolute point in the future when a particular transaction can go through.

This is done using a UNIX timestamp or blockheight.

Relative timelocks on the other hand specify a block depth, or the number of blocks that have been built on top of current one.

There is also the distinction between transaction-level and script-level or UTXO-level timelocks.

As their names imply, transaction-level timelocks impose a timelock on the transaction itself, whereas script-level timelocks can impose timelocks on specific UTXOs, and are specified in Bitcoin scripts.

Transaction-level timelocks were the first implementation of timelocks on bitcoin.

This timelock ensures that a transaction cannot not be included in a valid block until a particular time in the future.

However, the caveat is that since the timelock is only placed on the transaction but not the UTXO that is being spent, the sender could simply send another transaction spending from the same UTXO as the timelocked transaction.

This new transaction will go through first so long as the first transaction is still timelocked.

And so long as it is valid, the new transaction in effect invalidates and overrides the timelocked transaction, since the outputs that the timelocked transaction tries to spend from will have already been used, thereby cancelling out the original timelocked transaction.

In order to prevent this caveat, the time lock can also be placed on the UTXO itself.

The UTXO- level timelock specifies that a particular UTXO cannot be spent until the specified time.

This ensures that the sender of the UTXO cannot override the transaction by sending the UTXO to another address before the timelock is unlocked.

One note when using timelocks is that users must be careful near the expiry time of a timelock.

There is no notion of global time on the blockchain since peers’ clocks on the network could potentially be off.

Therefore, be wary when using absolute timelocks with UNIX timestamps.

A transaction that seems ready to spend given that a specific amount of time has passed might be viewed as premature by others.

Also, blocks are not created at guaranteed intervals either, so any attempts to cancel out of a timelocked transaction should be made a few hours before the timelock, specified in blockheight, expires.

In this lecture, we dove into the low-level specifics of Bitcoin that make it work. Bitcoin was innovative because it allowed a decentralized network to reach consensus. It achieved this via tamper-evidence, which means although one can modify the information that passes along the Bitcoin network, it would be obvious that some modification has been made. This tamper evident system allows us to be sure any update on Bitcoin is the same for everyone.

We achieve a tamper evident system using cryptographic hash functions to produce standardized random “fingerprints” of our data. If the data changes, so will the fingerprints. Cryptographic hash functions do the following:

Cryptographic hash functions are pseudorandom: although the output for any given input seems random, the output will remain consistent for that input.

Important Properties of Cryptographic Hash Functions:

1. Pre-image Resistance: Given H(x), it is computationally difficult to determine x

2. Second-image Resistance: Given x, it is computationally difficult to find some value x’ such that H(x) == H(x’)

3. Collision Resistance: It is computationally difficult to find x and y such that H(x) == H(y)

These properties produce the Avalanche effect, where even any small change in the input leads to a significant pseudorandom change in the output.

The particular hash function Bitcoin uses is SHA256, which takes in an input of size less than 2^64 bits and produces a 256 bit fix sized output.

This cryptographic hash function is used to make an entire tamper evident database in Bitcoin. The Block Header of a block on Bitcoin, is a hash of many contents within the block, most notably its Merkle Root, Previous Block Hash, and Nonce fields. The Merkle Root represents a summary of transactions, the Previous Block Hash represents the chaining, and the Nonce represents the Proof-of-Work.

The Merkle Root is the head of the Merkle Tree, a binary tree of hashes of all the previous transactions. The Previous Block Hash contains the hash of the previous block. Both of these hashes change if any of the previous transactions or blocks is modified.

The Nonce is the manifestation of the proof-of-work in Bitcoin; it is a numerical value that must be found to solve the partial preimage hash puzzle. Miners hash the entire block header (the input) and tweak the nonce and coinbase until they find an output that solves the hash puzzle. 

Hash puzzles must be:

1. Computationally Difficult: The solution to the hash puzzle cannot be easily found

2. Parameterizable: The difficulty of the hash puzzle should be adjustable 

3. Easily Verifiable: Computers should have to do little work to ensure the answer is correct

The difficulty of the hash puzzle in Bitcoin is:

difficulty = difficulty * two weeks / time to mine previous 2016 blocks

Once miners solve the puzzle, they receive bitcoin via a coinbase transaction. Whenever miners produce a block, they first create a coinbase transaction, which is the first transaction of the Merkle Tree.

Using cryptographic hash functions, we ensure previous blocks remain tamper evident; we now turn our attention to how digital signatures help us ensure current transactions are tamper evident as well. Public and Private keys in Bitcoin are generated using Elliptic Curve Digital Signature Algorithm (ECDSA). ECDSA has three key properties:

1. Given the encrypted message and the sender’s public key, the recipient should be able to identify the message origin. Since the message has been signed by the sender’s private key, the ability to encode it using the public key demonstrates the original sender has authorized this message.

2. The digital signature scheme must also ensure non-repudiation: once the sender signs the message, they should not be able to undo it. 

3. Finally, the scheme must maintain integrity; since messages are signed with the private key, they cannot be modified after signing.

Identity in Bitcoin is derived from private keys, which are generated randomly. Public keys are the result of Elliptic curve point multiplication of the private key against a known generator point on the curve. Given the public key, it is computationally infeasible to arrive at the private key.

We can apply these concepts of private and public keys to understand how transactions in Bitcoin work. Spending bitcoin is the act of **redeeming **previous transaction outputs with a proof that you are the legitimate redeemer, and then **specifying who can redeem **the output of the transaction you are now creating, by encoding that per's information in your transaction.

A transaction has three main sections:

1. Metadata: Contains housekeeping data, a unique ID of this transaction, locktime, and size

2. Inputs: Contains a list of previously created UTXOs and proof of eligibility to redeem this money

3. Outputs: Contains a list of new UTXOs that will be sent to new addresses. These values are locked by a script only the intended redeemer can unlock.

Bitcoin uses the stack-based, Turing-incomplete language named Script to create transactions. Locking and Unlocking Scripts are contained in transaction input and previous transaction output and are used to redeem the output of a previous transaction and specify requirements for redeeming transactions, respectively. Senders specify a Locking Script, and recipients specify an Unlocking Script. In Pay-to-Pub-Key-Hash (P2PKH), the recipient says “send your coins to the hash of this Public Key.” In Pay-to-Script-Hash (P2SH), the recipient says “Send your coins to the hash of this Script; I will provide the script and the data to make the script evaluate to true when I redeem the coins.” The latter is popular among customer-vendor transactions, where the vendor (recipient) is responsible for writing the script.

• Where is Double hashing performed in Bitcoin? 

Princeton Textbook 5.1-5.4 (pg. 131 - 157)

• Bitcoin Wallets Explained: How to Choose the Best Wallet for You 

• (Optional) Bitcoin Developer Guide (There's a lot; don't try to read it all in one day) 

• (Optional) Tech explained: Hash puzzles and proofs of work 

• (Optional) Secure Hash Standard (SHS) (Insane math: a blessing or curse depending on your preference)

Bitcoin operates under certain assumptions, such as an honest majority of computational power. The assumption we discussed in this last lecture was that cryptographic hashes are unique, given the properties of preimage, second preimage, and collision resistance.

Let’s say that you manage to break computer science as we know it and devise a way to make any input look like your preferred output. Formally, you can always construct an input y such that you can control the value of H(y) to be in your favor. You can create collisions at will. How can you then manipulate the Bitcoin protocol in your favor? There are several different answers, but you only need to provide one example.

Feel free to discuss amongst each other in the discussion board topic below. We'll be grading the responses you paste into the text input.

We’ve looked at Bitcoin mechanics from both a high and low level, explaining the primary motivations of why we do what we do in Bitcoin, and its implementation.

But what about the users?

What does the Bitcoin experience look like from their perspective?

If I want to get involved sending and receiving Bitcoin, mining, or running a full node, what does that look like?

The user experience varies greatly depending on your involvement with Bitcoin, but whether you’re a light user, full node, solo miner, or running a mining farm, you interface with the Bitcoin network in some way.

This module will touch upon the various user experiences in the Bitcoin ecosystem.

We’ll start off with a short discussion on the types of users in Bitcoin, then move on to wallets as a way to manage your identity.

We’ll then formalize the Bitcoin mining procedure and explore real world mining endeavors.

Finally, we’ll explore the various ways through which users can change Bitcoin and its underlying protocol.

While we may primarily only think of miners or casual traders using exchanges or their own wallet software, there are actually a plethora of different types of Bitcoin users, each categorized by the different software that they run, and how they interface with the Bitcoin network.

In our previous discussions about Bitcoin mechanics, we explained everything from the perspective of a full node and miner: a user with networking capability to interface with the Bitcoin network, actively maintains a full up-to-date copy of the Bitcoin blockchain, has a wallet as a means to manage public and private keys, and mines to support the network and also to earn a mining reward.

There’s a total of four functions: network routing, mining, maintaining a full blockchain, and handling wallet services, and our full node miner contains all of them.

But normal clients don’t need all this functionality.

For example, not every client is a miner.

If my computer isn't fast enough, I won't be making a profit mining, so why bother?

Not every client needs to have the entire blockchain.

If I just want to send bitcoin through my phone, why would I want to have the 160 plus gigabyte blockchain downloaded?

My phone doesn’t even have that much storage!

Not every client even needs a wallet, especially if they don’t transact often.

Say you’re just a full node hosting the entire blockchain, but have no interest trading yourself.

Or you have a separate wallet software.

And finally, as we’ll soon see, not every client needs to be connected to the Bitcoin network, or even the internet!

In actuality, the distinction between different types of users isn’t just through these four main functionalities -- mining, routing, having a full blockchain, and key management through wallets -- but it does summarize a lot about the Bitcoin user experience.

The first type of user we’ll be talking about is the most common Bitcoin user.

Doesn’t have the entire blockchain downloaded, and doesn't need fancy functionality.

Just a wallet to help manage keys, and this type of user can send and receive bitcoins with the rest of the network.

As we’ll soon find out, wallets come in many different shapes and sizes – hosted on the web, on your computer, digital, or physical.

As we’ve previously mentioned, Bitcoin is all about granting real-world entities virtual identities.

Through these Bitcoin identities, we can easily authenticate ownership and send bitcoins between users.

Also as mentioned previously, the private key is what unlocks a virtual identity.

It’s what gives access to a Bitcoin address.

But how do we actually protect these private keys?

How do we make sure that we’re not subject to identity theft?

This is where wallets come in.

The primary function of Bitcoin wallets is to keep track of your identity.

Unlike how we use physical wallets to conveniently hold onto and store our physical money, our digital Bitcoin wallets don’t actually let us store bitcoins.

Bitcoin wallets are simply a method of storing and accessing your private key, which then allows you to spend the corresponding bitcoin.

Most wallet software will also store, send, receive, and list transactions for you, since it’s easy for software to do this but not necessarily for humans.

It would be infeasible to manually keep track of all blockchain activity involving you.

To take care of this, wallet software will store all relevant information about the blockchain on your behalf.

Wallets come in many forms.

We generally make the distinction between two main categories of wallets -- hot and cold -- based on their connectivity to the internet.

Hot wallets are connected to the internet, and cold storage is not.

Some examples of hot wallets are smartphone apps such as Mycelium and AirBitz (I personally use AirBitz).

There are also online web wallets such as those hosted on Blockchain.info and coinbase.com.

And all these are connected to the internet.

On the flip side, cold storage never touches the internet.

For example, there’s paper wallets, hardware wallets, and brain wallets.

Paper wallets are literally pieces of paper with your private key printed on it.

You can generate fancy paper wallets on websites such as Bitcoinpaperwallet.com and Bitaddress.org.

For maximum security, these websites will ask you to download the entire website and run them locally, but only after shutting off your entire internet connection for maximum security.

They also warn of smart printers that cache files that it prints, and operating systems that might already be compromised.

In situations like these, users of paper wallets suggest using a dumber old- fashioned printer, and also dedicating an entire new, fresh computer just for the purpose of printing paper wallets.

This is how far some people take security.

There are also hardware wallets such as Ledger, Trezor, Case, and Keep Key, which are little devices that plug into your computer or smartphone via USB or some other method and sign transactions for you.

The idea is that while your computer may be connected to the internet, the hardware wallet handles your private key and signs transactions in a trusted execution environment that runs separately from your computer and never touches the internet.

And then there’s brain wallets.

Imagine a wallet that is so secure that it’ s practically impossible to hack. Online web wallets might get hacked, your smartphone or hardware wallet might get stolen, or your house might burn down and take with it your paper wallet.

Imagine a wallet that’s immune to all of these disasters AND more.

Well look no further than your brain, the most secure wallet of them all.

Simply memorize your private key.

Just kidding.

While it is possible to memorize your raw private key, an easier alternative would to memorize a collection of words or phrases or a mnemonic.

The idea is to just have a more easily memorizable set of characters than your straight up private key.

On the right, you can see that I’ve chosen a list of 12 words.

Multiply, scrap, submit, select, adjust, end, accuse, fuel, nose, hope, chair, afraid.

After I’ve chosen this set, I would then hash them together to get my private key.

This way, I can have the security of a brain wallet, having no physical evidence of my private key and keeping everything in my head, and also get around memorizing my actual private key.

However, there’s a catch.

Humans aren’t as random as we think, so the words someone chooses for their brain wallet might easily be guessed -- such an attack is called a dictionary attack.

Dictionary attacks are especially dangerous to those who take their brain wallet words from a famous quote from a speech or movie or book.

In general, so long as your words are not closely related, it’s still expensive and improbable that someone would be able to guess and successfully dictionary attack your brain wallet.

However, for the security minded, to further lessen the probability of someone being able to guess your brain wallet, you can employ what’s known as key stretching.

There’s nowhere saying what hash function you have to use to get from your brain wallet to your private key, nor how many times you’re allowed to hash -- that’s all up to you as the user.

The idea behind key stretching is to hash your brain wallet a large number of times: say 2^20 or some other huge number.

I take my set of words “multiply, scrap, submit, etc” and I…...Hash and hash and hash it….

Not only will an attacker have to guess your initial set of words, but they would also have to guess how many times you hashed your set of words.

Hacking a brain wallet that has been key stretched is exponentially harder to brute force than a brain wallet that is only hashed once.

There’s a lot of wallets out there, definitely a lot more than what we’ve gone over, and that’s why users have to do their own research to find the wallet that’s best for them.

For example, oftentimes you have to trade off between convenience and security.

Coinbase.com is very easy to use, but they actually store your private keys on the cloud, and you never get to see it.

Some people don’t like this because you’re putting trust in a powerful, trusted third party -- Coinbase in this case.

However, others are willing to trade this for the convenience of having their bitcoin at their fingertips through an easy to use website.

On the other hand, wallets like Mycelium and Electrum do not hold anyone’s private keys on the cloud.

Users hold their own private keys and are responsible for their own funds.

There’s no recourse from these services in case someone loses their private key.

The benefit of course, is security, since no one else but you controls your private keys.

However, it’s a huge responsibility and burden, especially for casual Bitcoin users.

At the end of the day, your choice of wallet is really up to you as an individual.

Whether you’re more partial to convenience features such as payment via smartphones and QR codes, or fancy multisignature wallets, or go the extra mile to use a hardware wallet, print out a paper wallet, or memorize a brain wallet… it all boils down to your personal needs.

You might be thinking: Alright Rustie, we’ve been talking a lot about Bitcoin wallets and how to manage your private keys, but how do I get bitcoins in the first place?

Bitcoin’s not some magic internet money after all.

Well lucky for our students in Berkeley, we actually have Bitcoin ATMs a couple blocks from campus.

And for all you non-UC Berkeley folks, don’t worry, these are also available elsewhere!

Bitcoin ATMs are pretty cool.

You first scan your QR code from a smartphone wallet app, or any other device (or piece of paper) that can display a QR code.

The important thing is that this QR code contains your Bitcoin address.

You then insert cash, press send, and then bitcoin is sent over!

Just like that!

Some Bitcoin ATMs work a bit differently, and instead of having you scan your QR code, they simply print out a paper wallet for you after you’ve paid your equivalent amount through cash, debit, or credit card.

You can also get Bitcoin through an exchange, where you can trade between different types of traditional currency and cryptocurrencies.

For example, you can trade from bitcoin to USD, and from USD to bitcoin, naturally.

Exchanges are important because they set the price of bitcoin, since they define the market value.

There are a lot of exchanges, some more reputable than others, so it’s important to do your own due diligence in choosing which exchange to use.

For example, there’s the distinction between centralized and decentralized exchanges.

Centralized exchanges are easy to use and easy to access, but could pose a risk for your funds.

There have been some infamous hacking incidents over the years.

Think of the Mt. Gox hack and bankruptcy, and also the more recent BitFinex hack in which approximately 120,000 bitcoins were stolen.

As a general rule of thumb, it’s never safe to keep your money in exchanges in the long term.

A lot of people do this, and it’s unfortunate when they lose all their funds when a hack does occur.

So naturally, following the trend of decentralizing Everything for security reasons, we now have decentralized exchanges

They don’t rely on a third party service to hold the customer’s funds, or do any of the mission critical back end work.

Instead, trades only occur directly between users (think P2P).

Generally this is done by creating proxy tokens, or assets, that represent certain amount of fiat or crypto currency.

Decentralized exchanges can also be trustless.

You don’t have to trust the security or honesty of the exchange since the funds are held by you in your personal wallet, and not by a third party.

Some examples of decentralized exchanges are Bitsquare, Bitshares, Openledger, NXT, CounterParty, etc.

Decentralized exchanges might not necessarily be better than centralized exchanges, especially since large centralized exchanges are generally backed by powerful companies that can invest in high security, but of course, do your own due diligence.

We now know that wallets help manage your identity: your public and private keys.

In the case of brain wallets and paper wallets, there’s not much else to say, since the idea behind them is simple: just write or print out a copy of your private key, or have some mnemonic or way to derive your private key.

But when it comes to wallet software, it’s a bit more complex.

In this section, we’ll be going into some wallet mechanics and how they enable the functionality in your favorite Bitcoin wallet app.

First off, let’s address the elephant in the room.

How come when I use my wallet software, I don’t have to download the entire Bitcoin blockchain?

Well, we have SPV to thank for that.

SPV stands for Simple Payment Verification, and it’s a method for verifying if particular transactions (specifically your own transactions) are included in a block by only having to download the blockheaders of each block, instead of the entire block, which includes all the transactions.

This is particularly useful for clients that are constrained by storage capacity, as they can now operate without storing the full blockchain.

Imagine your typical Bitcoin smartphone wallet app.

You don’t have the entire blockchain stored.

Instead, you have a wallet, to keep track of your keys, and a network routing component that allows you to connect to the Bitcoin p2p protocol.

This is all done without a full blockchain, and for this reason, clients that run SPV are called lightweight or thin clients.

In SPV, by having the block headers, you can simply run a merkle proof of inclusion, which we explained in the previous module in the tamper evident database section.

One major assumption you have to make when using SPV is that incoming block headers are not from a false chain.

If the block headers you receive are fake or invalid, then a transaction you see as valid may not actually be so, and someone could potentially double spend you!

Thankfully, it’s generally ok to assume that incoming block headers are not from a false chain.

As long as you connect to many different nodes, then the probability of each one being controlled by one malicious entity is significantly lowered.

And as long as you’re connected to a variety of nodes, in the long term, the chain you see will be honest: a core assumption that we make in Bitcoin to begin with.

And realistically, we can’t afford to put the entire blockchain on our phones or other lightweight devices, so having a thin client running SPV is a pretty decent tradeoff.

We mentioned multisignature in the previous module in regards to pay to script hash, but just to explain again and elaborate on its potential uses, we generalize, saying that multisig enables what are called M-of-N transactions.

In simple terms, multisig is a way of cryptographically sharing a key between participants, and serves to distribute points of failure.

Imagine if you want to share a wallet with your family, or even with yourself.

For example, consider a 2-of-3 multisig.

Imagine I have a 2 of 3 multisig with my friends Derrick and Gloria.

Only two of us need to sign off on a transaction in order to send it.

But no one can just run off with our bitcoin since you need at least two of three keys.

And in this example, I have set up a two of three multisig with myself.

Instead of losing my private key and then subsequently losing all of my funds, I have more than one point of failure.

After losing one private key, i still have two of the three total keys, so I still have sufficient keys to access my funds in a 2-of-3 multisig scheme.

As another example, some exchanges and software wallet companies provide 2-of-3 or 3-of-5 multisig services.

In the 2-of-3 multisig case, you keep two of your keys, and a trusted third party such as the company holds the third.

The exchange can’t use your funds since they only have one key, but if you ever lose one of your keys, the company can step in and offer their key to help you recover your funds.

As you can imagine, this works similarly for 3-of-5 or any other M-of-N multisig.

It’s generally considered best practice to never reuse pseudonyms.

This requires generating a new public and private key pair for every transaction you make.

You want to do this so that it’s more difficult to determine how much bitcoin you own, and what you’ve been spending that bitcoin on.

Also, imagine if you didn’t generate a new public and private key pair for every transaction, and you had all your funds associated with one identity.

As soon as you lose the corresponding private key, all your funds are gone.

It’s best practice to spread out your funds across multiple identities.

After all your private keys are just random numbers -- there’s no relation between them -- so compromising one key is independent of the others.

And public and private keys are computationally easy to generate anyways, so why wouldn’t you generate new public and private keys for every transaction?

It’s a little effort that could potentially go along way in saving your Bitcoin user experience.

Wallet software will generally handle this anyways, making it so much easier to comply with best practices.

Now if you’re complying with best practices and generating a new public and private key for every transaction you make, the natural question is: how do I manage an increasing number of keys?

This is especially the case if you’re making frequent transactions.

The number of keys you control can quickly become overwhelming.

The traditional way to keep track of keys was through what are known as JBOK wallets, where JBOK is an acronym standing for “Just a Bunch of Keys.”

The idea behind JBOK wallets was that after making every transaction, you would make a new backup of the new key pair you generate.

Alternatively, you could also start off with a bunch of keys generated beforehand.

However, after you use up all of these keys, you would still have the same issue of having to generate more keys.

JBOK wallets aren’t too convenient because you have to store each and every key pair.

Consider if you’re an exchange, and you have thousands of users trading constantly.

You’d have to deal with so many keys, and even more back ups.

That’d be pretty unmanageable.

Instead of having just a bunch of keys, let’s try something a bit more clever.

What if we could instead come up with a way of deriving keys from a original seed value?

That’s the idea behind HD Wallets, or Hierarchical Deterministic wallets.

We start off with a randomly generated seed value.

Instead of having to back up every single key from now on, we could just deterministically generate a new key pair, so long as we know our original seed.

We can think of this seed as a master key.

We first generate our master key, and for each subsequent key we want to generate, we can just take the hash of the master key with some counter or index number, and then we’ll arrive at our derived child key.

For example, if I want to generate my third child key, I would append 3 to my master key and hash the whole thing.

We can think of the master key as being a parent key, and any keys that are generated from it are its child keys.

And to make things more interesting, we can also use child keys as parent keys to generate even more child keys.

By generating keys in a known way instead of randomly, we greatly lessen the load on wallets having to keep track of all your keys -- especially if you’re dealing with thousands of keys like exchanges do.

Of course, exchanges can use HD wallets to greatly reduce their load, but HD wallets are also useful elsewhere.

Imagine an organization with a hierarchical structure.

Imagine that we have the president control our master key.

From the master key, we can generate child keys for each of the department heads.

And from the keys of the department heads, we can generate child keys again for each of the department employees.

Since keys are derived in a hierarchical fashion, someone controlling a parent key can derive a child key, and can thus spend from it.

In our hierarchical organization, if everyone in the org were to have an HD wallet as previously defined, then the department heads could spend from the wallets of employees in their department, since the employees keys were generated from the department head’s keys.

And the president would have the most power.

Everyone’s key originates in some way from the president’s master key, so the president would be able to spend from everyone’s wallet.

We’ve talked about consensus and Proof-of-Work, but we haven’t really talked about the implementation of mining in Bitcoin.

In this module, we’ll be diving into the nitty gritty of how mining manifests in the real world, and how even the brilliant Satoshi Nakamoto may not have envisioned the future of mining.

We will go through what a miner actually does to validate blocks.

Let’s first take a look at the generic steps for mining as a full node.

Step 0, the preliminary step for mining, is downloading the entire Bitcoin blockchain.

This allows us to know the history so that we can verify future transactions.

Note: this step is optional if you mine in a mining pool; we’ll talk more about this later.

Step 1, the next step, is to verify transactions, which we fill up our block with valid transactions.

Step 2, create the block using the given transactions, and all necessary metadata, such as time, version, and target.

Step 3, find the proof-of-work, aka a valid nonce that solves the partial preimage hash puzzle.

Step 4, broadcast your block if you have not seen any competitor blocks yet.

Step 5, if your block gets included in the longest chain: profit!

Let’s go into detail on how each of these steps plays out.

Step 0, download the history of transactions.

This is Step 0 for two reasons: It only needs to be done once.

Once you have the full Bitcoin blockchain history, you don’t need to redownload that info; you only need to keep track of incoming blocks from other miners.

If you’re a miner with an SPV node, you only need to store the block headers and request info from full nodes for verifying transactions as needed.

Meaning that this step is optional.

We’ll describe further different types of miners.

If you want to be a full, independent miner though, you have to do this step.

Now that we have the full history, let’s go ahead and get started with mining!

Bitcoin users are sending transactions to the Bitcoin network every second, and it’s the miners’ jobs to verify the validity of those transactions.

As transactions come to the miner, they will store those in a “mempool,” where “mem” means “memory” and “pool” means supply (referring to supply of transactions).

It’s where all the pending transactions live before they make their way into a block.

Miners will then choose transactions to verify based on their transaction fees, which we’ll get into shortly.

They verify the validity of each by running the unlocking script as mentioned in Lecture 3.

If that piece of code is able to successfully unlock the previous bitcoins, then the transaction goes through.

After validating all these transactions, it’s time to generate a block.

As previously mentioned, there are several pieces of information necessary to construct a block, For example, the Merkle Root, which we generate from our list of transactions.

After constructing the block data, we are finally able to start working on the most expensive part of the mining process, the step that earned this whole process of Proof-of-Work the “mining” title: finding a valid nonce.

As mentioned before, every miner needs to find a nonce which makes the hash of the block header less than some target value.

This is how we implement Proof-of-Work in Bitcoin and every other Proof-of-Work cryptocurrency.

By finding the nonce, we have translated the energy burned in computation directly into voting power in the Proof-of-Work consensus protocol.

You may have noticed in this pseudocode that there are two different nonces, the header nonce and the coinbase nonce.

Keep in mind that the nonce in the header is a reasonably small number: only a 32 bit number.

This means that a powerful device can run through all the nonce possibilities within a second.

To make it such that we don’t reach a dead end with our mining puzzle, we need to change our puzzle if the header nonce has no viable options.

To do this, we change the coinbase nonce, a field within the coinbase transaction.

By changing this nonce, we change the Merkle Root, and therefore, we're able to explore more options and hopefully find the answer to the mining puzzle.

Our mining puzzle is completely different.

We go through these loops until finally we find a valid nonce.

After finding the nonce, the typical miner broadcasts it as soon as possible so that other miners are aware that a block was found.

Those miners will validate the block for themselves before accepting it into their own chain, and then broadcast the block once more.

By broadcasting the block first, other miners will abandon their previous blocks and start to mine on this new longest chain.

At least, that’s the hope: there’s the possibility that someone else broadcasted a block before you that you haven’t yet seen!

Only if your block makes its way into the longest chain will you reap the rewards.

So how do we ensure that happens?

Well, we can’t!

All we can do is hope that we got lucky.

That’s all Proof-of-Work really is at its base, a random lottery.

Assuming that our block made its way into the longest chain, we profit!

We receive both the block reward, represented by the coinbase transactions, and transaction fees.

All the transactions within that block are added to the canon transaction history.

A situation in which our block is not in the longest chain is where our block is competing with another block that was submitted to the network at the same time.

In which case, miners will likely choose randomly between the two, meaning that it’s once again luck.

If your block is orphaned, or in a fork that’s not the longest chain, then you get no profit.

We won’t actually ever be 100% certain that our block is in the longest chain.

We just assume that the probability of a fork happening gets lower over time.

We’ll talk about that more in the game theory lecture.

Text: Recipe for Mining:

Author: Rea Savla

• Download the entire Bitcoin blockchain. This only has to be done once. 

  • This allows us to know the history so we can validate future transactions. 

  • Note: This step is optional if you mine in a mining pool or are doing lightweight mining. 

• Verify transactions.

  • You store newly received, unprocessed transactions in a “mempool,” where all pending transactions live before making their way into a block.

  • You then choose the transactions with the highest fee per byte or size ratio to verify. 

  • You verify the validity of each transaction by running the unlocking script. 

  • If that script runs successfully, then the transaction is included within our block.

• Create the block with the given transactions and necessary metadata, such as time, version, and target.

  • Construct the block data from our list of valid transactions.

  • Construct the Merkle Root by hashing the hashes of each pair of transactions 

  • Construct the Previous Block Hash by hashing the previous block’s header

• Find the proof-of-work that solves the partial preimage hash puzzle. 

  • A valid nonce makes the hash of the block header less than some algorithmically generated value known as the “target.”

  • By finding the nonce, we have translated the energy burned in computation into voting power, as designed by the Proof-of-Work consensus protocol.

  • Note: There are two different nonces, the header nonce and the coinbase nonce. In the event that no permutation of the header nonce solves the hash puzzle, alter the coinbase nonce. This changes the Merkle root, yielding an entirely different hash puzzle.

• Broadcast your block if you have not yet seen any competitor blocks.

  • After finding the valid proof-of-work, broadcast the block as soon as possible.

  • Other miners will validate the block for themselves before accepting it into their chain and propagating it further through the network. 

• If your block makes it into the longest chain, you profit!

  • You receive both the block reward and the transaction fees, 

  • All the transactions within that block are added to the transaction history.

  • Note: When two valid blocks are submitted to the network at roughly the same time, resulting in a fork, honest miners choose to mine on whichever block they see first. You will not receive block reward if the other fork grows longer. 

Why do we do things?

Why do we work at a job?

Why do we trade cryptocurrencies?

Why do you choose to watch these lectures?

Well, it’s a pretty simple answer:

Profit!

Let me make that a little bigger for you.

Yes, profit!

As you know intuitively, everything that anyone does is with the expectation of gaining some profit.

Each person’s definition of profit is different, but it’s pretty straightforward in Bitcoin: more money means more profit.

It’s such a simple word, yet this concept of profit will come up again and again throughout the rest of this course.

In fact, any type of incentive mechanisms within blockchain protocols focus on aligning personal profit with overall profit.

Pay good attention to this slide.

Yes, it’s very basic information, but this equation is fundamental to all incentive structures.

It is the fact that the profit equals revenue minus cost.

This means that, only if the revenue exceeds the cost, will you get profit.

Every single person in the Bitcoin network is presumably maximizing the value of their own profit to the best of their ability.

Let’s go ahead and break down the various components of profit within Bitcoin.

First, we’ll start by discussing the block reward, the most significant source of profit currently for miners.

The block reward is a reward that goes to a miner whose block is included in the longest chain.

As of May 2018, the block reward is around 12.5 bitcoins.

As mentioned before, the miner includes a special transaction to themselves: the coinbase transaction.

That transaction is what both allows for the minting of bitcoins and, more importantly, incentivizes honest actors to validate blocks.

To understand better the rationale for the block reward, remember that profit is the primary motivator for any activity, and that a higher incentive for honest behavior leads to a more secure network.

However, we can’t directly punish dishonest behavior: everyone’s anonymous, and there’s no effective way to enforce punishment.

The conclusion is that, if we can’t punish dishonest behavior, we might as well reward honest behavior!

Because miners get rewarded for producing blocks, they’re incentivized to produce more blocks.

However, they only get rewarded in bitcoins.

This means that, if a miner sees no value in Bitcoin, they are less likely to spend money on electricity and hardware to get bitcoins.

Because of this, we expect honest mining power to be drawn more heavily to the Bitcoin network.

As you notice on the graph, the Bitcoin supply cap tapers off at a certain point, calculated at 21 million.

This is because the block reward halves every 210,000 (two hundred and ten thousand) blocks, which is approximately 4 years.

It started off at 50 bitcoins per block when Bitcoin first started, then it dropped to 25 after the first two years, and has been halving ever since.

By using a geometric series, the supply cap is calculated to be 21 million bitcoins.

One caveat about this: Not all 21 million bitcoins will be liquid.

For example, approximately 1 million belongs to Satoshi Nakamoto, and it is possible that those bitcoins will never move.

In addition, some private keys have been lost, and some bitcoins have been burned, all decreasing the usable supply cap.

Now that we’ve talked about block rewards, let’s go ahead and talk about transaction fees.

Transaction fees are the second form of profit for miners.

The transaction fee is a price set by the sender of a transaction.

You can consider the transaction fee the cost of service for using the power of the bitcoin network.

Providing transaction fees are not required for a transaction to go through, but they incentivize miners to consider choosing your transaction over other ones due to limited block space.

Because a miner can only approve so many transactions at once given the one megabyte size limit, they will want to choose the transactions that give them the most profit within that block.

In fact, the way the miner calculates the transactions they will collect into a block is through maximizing the ratio of transaction fees to unit of storage.

What this means is that the overall transaction fees for their block will be maximized as they’re taking all the most profitable transactions.

Notice one thing as the block reward diminishes, transaction fees will go up if miners seek the same amount of profit in bitcoins.

As block rewards approach zero, transaction fees will become the primary source of revenue for miners.

Now that we’ve discussed all the different sources of revenue for miners, we can move on to the costs.

Let’s start off by discussing the fixed costs, and then get into the variable costs after.

As you all know, hardware is what allows users to mine, and the cost of hardware is fixed.

It’s a one-time purchase.

There are a few different types of hardware, ranging from generic computation devices to specialized mining hardware.

This logarithmically scaled graph here shows a quick overview of the growth of mining power produced by miners, which as you can tell has grown incredibly, several orders of magnitude since the start of Bitcoin.

The first type of mining hardware used was a CPU, or central processing unit.

As of late September 2017, mining with a CPU would take you about 7.6 million years to find a block.

This is because there is much more powerful hardware nowadays, which makes it much harder for a CPU to compete with the rest of the network.

The pseudocode above is what was run originally by CPU miners.

They were the original type of computing power available to the general public.

This is because CPUs are meant to do generic computation, but they are not great at specifically calculating hashes.

Some members of the gaming community quickly recognized that they had some more powerful computation devices available at their hands: GPUs.

GPUs, or graphic processing units, are devices typically used for processing images, like video games.

These devices are also generic computational devices in that they can handle many different types of computation, including hashing.

They are an order of magnitude faster than CPUs when it comes to computing, meaning that they can hash 10 times as fast.

If you look in the chart, you’ll see that in order to mine a block, GPUs only take about a mere 762 thousands years!

GPU miners were most common about 6 years ago, back in 2012, when still not many people had heard of Bitcoin.

They were repurposed from gaming to mining, providing GPU miners a significant advantage above CPU miners.

Some of the disadvantages of using GPUs, however, stems from their components irrelevant to mining.

For example, mining does not require any floating point numbers, so the excess components are not helpful but are still included in the cost of GPU hardware.

In addition, they are not meant to be run in farms side by side, so it is difficult to operate a large scale GPU mining setup.

This is where FPGAs, or Field Programmable Gate Arrays, come into play.

These were the first forms of specialized mining hardware in Bitcoin.

These devices were optimized for mining Bitcoin, but they did not lose all general computation power.

The reason for this tradeoff was because there was still some uncertainty around Bitcoin’s future success, and Bitcoin-specific hardware would become useless if Bitcoin disappeared.

However, if Bitcoin thrives, then there is a desire to beat out other mining competition.

However, FPGAs were only used briefly, because much more powerful hardware quickly came into the market around 2014.

ASICs, or Application Specific Integrated Circuits, are mentioned throughout the Bitcoin ecosystem.

These are fully specialized devices that can do only one thing.

Bitcoin ASICs are capable of one type of computation alone: mining.

These devices do nothing but solve the Bitcoin hash puzzle, but they do it better than anything else.

There are large amounts of customizability when it comes to ASICs.

These range from choosing a lower base cost in exchange for higher electricity use, or choosing a smaller device and losing out on hashrate.

Depending on your specific needs, there probably exists a relevant ASIC.

One issue with ASICs, however, is their initial cost.

Not only do ASICs take a lot of upfront capital to produce, but they also need to be bought in tremendous batches to make a reasonable profit.

A fun stat: one of the most popular and powerful ASICs, the Antminer S9, can perform 14 trillion hashes per second but would still take about 10.9 years to find a block.

Now that we’ve discussed fixed costs, or hardware costs, let’s finish up the equations with an understanding of the variable costs in mining.

As we know, there’s several different costs of energy consumed in the process of mining, all of which are considerations for the cost of Bitcoin on the environment.

These energy costs come in multiple forms, but primarily the following three.

Embodied energy is consumed when producing hardware, electricity powers hardware, and cooling maintains hardware as it heats up.

All these types of energy are considerations, but electricity and cooling are the variable energy costs that are the bigger considerations when looking at long-term profit projections.

On top of that, there is infrastructure and overhead to maintain as well, such as space and employees.

Depending on the scale of your operation, you may need to go so far as to purchase entire warehouses and a few maintainers to ensure that nothing goes wrong with the hardware, or to take care of things when hardware goes down.

Finally, you understand everything there is to know about how to profit from mining.

If you keep your revenue above your costs, you’ll make a profit from mining!

Keep in mind that most mining operations, don’t turn a profit for quite a while, so do much more research before spinning up your own ASIC farm.

Speaking of ASIC farms, we’ll go ahead and examine what some mining operations look like in the real world in the next section.

We’ve seen and understood mining from the theoretical and technical side, but what does it look like in practice?

In this section, we’ll see what mining operations look like in the real world.

Theory and practice often differ in a lot of scenarios, and this is definitely the case in Bitcoin.

We’ll see that while the initial goal of Bitcoin was to be decentralized, honoring the “one CPU one vote” mantra, this could not be farther from the truth.

Here, we see a few pictures of what an ASIC mining farm in China looks like.

As you can see, the look is very industrial.

This is because the farm focuses on reducing unnecessary costs, which increases profit for the miners.

The racks are for systematic allocation of hardware, and the giant fans are for cooling.

Notice the water cooling used in the bottom right image.

Anyone familiar with hardware maintenance will recognize that water cooling is much more expensive than air cooling, meaning that whoever is cooling their devices with water cooling is willing to take a bigger cost for better maintenance of their hardware.

Perhaps in the long run, better maintenance means that there’s less need to replace hardware.

Either way, the goal of each of these mining farms is to maximize their profit.

This is why places that are cold -- hence providing natural cooling -- or that provide cheap electricity are more advantageous to set up a long term mining operation.

Here is an example of an ASIC.

[asic goes here]

The machine on the left is an Antminer S9, with 14 trillion hashes per second.

It’s one of the most powerful ASICs on the market currently.

The company that produces these, BITMAIN, claims that it has provided approximately 70% of the world’s mining power in hardware.

On the right we can see racks and racks of Antminer S9’s in a datacenter.

This particular photo was taken from an online listing that rented out Antminer S9’s by the month.

But of course, even with the most impressive technology, it still takes far too long to find a block.

Is there any way that we can reduce the amount of time it takes for us to get profit from our hardware?

Well, that’s where mining pools come in.

Mining pools allow individual miners to join their hash power together.

This way, they will collectively find blocks and get rewards more often, even if they only get a smaller amount with each block the pool finds.

Mining pools are run by a pool manager, or pool operator, that takes care of running a Bitcoin full node and distributing jobs to other miners.

They also take a cut of the rewards as compensation.

This way, miners don’t even have to store the blockchain themselves: similar to cloud computing, they give pools access to their hardware in exchange for a share of profits from the pool.

You may be asking, “Bitcoin is decentralized! How can I prove to a pool that I’m spending computation power to help them out?”

Remember something really neat: this problem is a smaller version of the problem Bitcoin itself solved!

We can have miners prove that they’re doing work for the pool through shares.

Shares are defined as near-valid or valid blocks.

Here’s how that works:

Let’s say I’m in Berkeley Pool, the hypothetical UC Berkeley mining pool.

My miner just found a block hash with half as many leading zeros as a valid block.

This is an unlikely event that resulted from expending a significant amount of computational resources.

Mining pools can use this information to gauge just how much mining power you’re contributing, and they can reward you for valid shares.

That way, I can get paid by BerkeleyPool even if I never find a valid block, because these near-valid blocks are indicators that I’m at least trying to find a valid block.

This is how we can create mining pools through Bitcoin.

On top of that, we know that no miner in the mining pool can just take the rewards for themselves.

Keep in mind that rewards are given out through the coinbase transaction.

In mining pools, the puzzle you are trying to solve contains a coinbase transaction that pays the pool manager the block reward.

And then the pool manager would hopefully redistribute the block reward.

Miners are solving a problem based on the Merkle Root, since they’re hashing everything in the block header.

And the Merkle Root of the block is based on the coinbase transaction.

If a miner wants to get rewards for themselves, they’d have to change the coinbase transaction to redirect the block rewards to themselves.

Thus, they wouldn’t get any shares from the mining pool, since the mining pool manager would see that the miner’s coinbase transaction does not pay to the manager’s address.

There are various schemes for payout that mining pool use.

The two most fundamental categories are Pay-per-Share and Proportional.

You can probably gather the basic ideas from the names.

Pay-per-share schemes will pay you a fixed amount of money for each share.

This payment is guaranteed and constant per share, regardless of how much reward the pool makes.

Most pools use this scheme, as it’s easy to implement and to understand.

As you can tell, this is more advantageous for the miner, as variance of their payout is reduced.

This implies that pools take on more risk because they will incur costs if blocks aren’t found by the pool.

On top of that, there’s no incentive for a miner to actually submit valid blocks to the pool.

We’ll see why this is an issue in the Game Theory lecture.

Proportional schemes are the other category which pay out only when a block is found.

The reward of each miner is proportional to the number of shares submitted before the block was found.

This is more advantageous for the pool, as they’ll never pay out more money than they have, but it leads to more variance and risk for miners.

Because of the difficulty of buy-in, proportional schemes aren’t seen often, if ever, in the mining world.

This is a demonstration of several different payout schemes that have been thought of.

The point is that there are many ways to go about paying miners, and each one has a different set of incentive tradeoffs.

It all depends on the size and assumptions of the mining pool.

So now that we’re familiar with mining pool schemes, we can talk about the pros and cons of mining pools.

The pros of mining pools are that they give individual and smaller miners the opportunity to make profit without waiting decades to get payment.

On top of that, software changes are easy to make.

Only one person is running a full node for the mining pool, and that person can upgrade on behalf of the pool.

Cons include trust in the pool manager.

You have to rely on them not to misuse your mining power and not to withhold rewards.

This is a consequence of centralization.

On top of that, a multitude of attacks are enabled by mining pools, which we will discuss further in the Game Theory lecture.

The community typically dislikes large mining pools.

For example, GHash.io once approached 400px; of the network mining power.

Miners within GHash voluntarily pulled out of the mining pool because they were aware of the dangers of approaching 51% of the network.

In addition, another concern is that a single entity might be hiding their total amount of mining power.

We see here that BTC.com has about 29% of the mining power in their pool, but what’s to stop them from submitting hashpower to other pools?

This is known as laundering hashes, by hiding the origin of mining power.

This allows you to leverage great amounts of mining power without revealing your prowess to the community.

By doing so, you experience no backlash while still receiving major profits.

Because of this, the graph is only a high-level estimate.

We don’t know the true concentration of control over mining hardware, and we may never know.

This will be a rundown to show the contrast between solo mining and pool mining.

We’ll show an interesting paradox which you may have noticed: more mining power leads to centralization.

So today’s network hashrate according to blockchain.info is 33,971,919 TH/s (about 34 million).

Mining reward is currently 12.5 BTC per block, and since blocks are mined every 10 minutes on average, we can calculate that the total mining reward yearly is 657,000 BTC/yr. (six hundred fifty seven thousand)

Also, for the sake of this calculation, we assume that the price of bitcoin is constant, staying at 10,000 USD

Suppose you want to start solo mining today.

First you need to buy your mining hardware, and let’s you’re buying a Antminer S9, which costs 3000 dollars with a hash rate of 14 TH/s.

Your hashrate in relation to the entire network’s hashrate is very small.

If we do the math, you end up having 0.0000004% of the total network hash power.

You expected annual reward in this case would be 2707 dollar per year, and this is just an estimate of your average income.

However, if you are unlucky, you might end up with not finding a block in a very long time.

Taking into account the variance of having such small proportion of the network hash rate, we can see that with our current numbers, we can expect to mine 1 block every 2,426,566 (about 2.4 million) blocks, which translates to a reward of approximately $125,000 once every 46.2 years

On the other hand, if you were to mine with a pool, you could expect to have much more regular payouts.

Assuming a pool you choose to join has one sixth of the network hashrate, this means that the pool finds every 6th block, or every hour.

This means that you can expect to early 31 cents every hour.

The question is, would you rather have a huge payout every couple decades as a solo miner, or a small but regular payout mining in a pool.

Obviously, pool mining sounds more attractive due to its low variance.

And anyways, your ASICs will probably be really out of date in a couple years, so you can’t realistically afford to have high variance in your mining payout.

This leads to the paradox that: the more secure Bitcoin gets, with more and more hash power, the greater the appeal is for joining a mining pool, since your individual hash power decreases when more hash power joins the network.

Of course, remember that there are various types of miners.

Some of these are known as “Reference Clients”--these control a wallet to store private keys, a routing node to communicate with others, a blockchain to validate transactions, and mining software to submit hashing power to the network.

This is the code released by Bitcoin Core to miners.

Solo miners, on the other hand, have no wallet functionality, probably have some other separate wallet software, and solely mine.

Aside from the wallet, they are identical to the Reference Client.

Mining nodes are general nodes which have some mining hardware and a protocol by which they connect to a mining pool.

There is the Stratum mining protocol, an unofficial standard for submitting hashpower to a pool, but it’s not the only possibility.

These nodes, you’ll notice, have no burden of storing any information.

They only need to store code and control hardware to interact with the mining pools.

Supplement: Lightweight Mining

Author: David Luo

Simple Payment Verification (SPV) is a method for verifying if particular transactions are included in a block by only having to download the block headers of each block, instead of the entire block, which includes all of the transactions.

These SPV/lightweight clients are especially useful for clients who have limited memory space and simply want bitcoin wallets without downloading the entire blockchain.

There is also lightweight mining software, which differs from lightweight wallet software. Instead of using SPV, users can mine using other methods without downloading any data from the blockchain.

One way users with these lightweight clients are able to mine is by joining a mining pool.

These miners work outside the network and only need to mine the transactions the pool operator gives them, trusting that the operators have the entire blockchain and the transactions given to them are valid.

It is up to the pool operator to listen for new blocks and validate the transactions.

To do this, the operator sends the miner a template of the block to be working on.

If a block is found, it is bound to this template, which includes the coinbase transaction that gives the reward to the pool operator.

This way, a user cannot steal the reward for themselves if they find a block.

getwork is a remote procedure call method that a client can call to get a block header from a mining pool operator.

However, a caller of this method does not receive any information about the block or the transactions within it.

A malicious server could easily send this user a block header hash that contains invalid transactions, and the user would mine on this invalid block.

getwork has been replaced by getblocktemplate, which allows miners to create their own blocks instead of having to trust somebody else (e.g. a pool operator) to provide them a valid block to mine on.

Only a block template is provided with specific configuration parameters, with block data that the client can modify if needed.

This way, the control is given back to the miner, because they can now choose the transactions to be included in the block they are mining on.

However, miners are incentivized to include pool specific data, such as a coinbase transaction to the pool operator in order to make valid shares and get rewards from the mining pool.

Although getblocktemplate does provide good functionality, it requires a lot of bandwidth due to the transferring of the entire block.

Because of this scalability issue, some pools now use Stratum, which is another protocol for mining.

Similar to getblocktemplate, the server sends the client a block template.

However, in Stratum only the block header and first transaction are included, instead of the entire block.

In order to mine, a client needs to have the block header hash of the previous block, which comes from the block header of the previous block, which in turn is generated from previous block.

One piece of information our new block needs to have is the block header hash, as we must reference the previous block in the chain.

They can simply receive this value from an external source and mine on that block.

This comes with one caveat, however, as the miner cannot validate the block header hash and must place trust in whatever source such as a pool operator they are getting the block header hash from.

In mining pools, once a block is found, the block header hash is sent imimagestely to the miners so they can all start mining on the new block as soon as possible. Competing miners that are outside of this mining pool can take advantage of this by connecting to the mining pool, listen for the block header hash, and mine on it for themselves.

We’ve discussed many of Bitcoin’s underlying mechanisms and protocols, but we haven’t talked about how those changes came to be.

People want to change Bitcoin for a plethora of reasons.

For example, some may feel that it needs to be tied in more to the original cypherpunk vision of decentralization, while others want higher levels of security.

In this section, we’ll be specifically talking about concerns of mining centralization, proposed changes to Bitcoin’s mining puzzle, and making consensus updates in Bitcoin.

Right now, one of the big features of Bitcoin, as well as other cryptocurrencies is the fact they can theoretically be mined in a decentralized manner.

There’s a reason we say “theoretically”, and it’s because mining is tending towards becoming more centralized with the development of ASICs, mining pools, and mining farms.

The fact that mining pools and mining farms is pretty straight forward, as they’re usually run by a single entity.

ASICs on the other hand, are also centralized, but in the sense that ASICs can cost a lot of money.

This means that only people with the enough capital can acquire them, centralizing the mining process.

We’re going to attempt to address the problem of mining centralization by considering the design of the underlying hash puzzle, and see if there’s any way we can redesign it.

Here’s a quick reminder on what properties a cryptographic puzzle should have.

A puzzle should be hard to solve but easy to verify.

In Bitcoin, finding a nonce is incredibly difficult, but verifying that a particular nonce is correct is much much easier.

The difficulty should be adjustable to account for changes in mining technology (i.e. ASICs).

In Bitcoin, the difficulty is adjusted every two weeks.

You need a solving rate proportional to computational power, meaning that the difficulty reflects the amount of hashpower in the network.

A puzzle should be “progress free”, meaning that finding a hash does not make it easier to find the next hash.

Every solution is independent of the previous ones.

Lastly, you need a pseudo randomly generated puzzle.

Bitcoin’s puzzle is described as a partial hash preimage puzzle.

You don’t have to find the exact preimage of the hash, as long as you have the prerequisite number of leading zeros.

ASICs work because it’s easy to create hardware to compute just a certain puzzle.

They’re computation bound.

A way to get around the problem of ASIC domination is to make puzzles that rely not only on computation speed, but also on memory, giving rise to the notions of memory-hard and memory-bound problems.

A memory hard problem is one that needs a large amount of requisite memory to solve,

and a memory bound problem is a problem that scales based on the amount of memory that you have.

These problems deter ASICs since optimizing computation is useless if memory is the limit agent.

Dogecoin and Litecoin implement a memory-bound hash function called Scrypt (pronounced ess-crypt).

They still use a partial hash-preimage puzzle like in Bitcoin, just with a different hash function.

This means that ASICs that are built for Bitcoin don’t work for Dogecoin or Litecoin.

Scrypt was originally designed to secure passwords and make them hard to brute force, so a memory-bound type of problem makes sense.

Scrypt has two main steps: first fill a buffer with interdependent data, and then access that data in a pseudorandom way.

To see why this is memory-bound, let’s take a look at what happens when you fill a buffer with non-interdependent data.

Looking something up is easy, since you just find the index of the datum you want.

However, if the data is interdependent on other pieces of data, you have to look up those too.

If those aren’t stored in the buffer, then you have to spend time computing them on the fly.

Depending on how much data there is, this can quickly become infeasible, meaning that storing as much interdependent data in memory is the only efficient option.

That’s what makes Scrypt memory-bound.

However, Scrypt also has its drawbacks.

It requires an equal amount of memory to verify, since verifiers have to fill and access an equivalent buffer, with the same parameters.

Remember that puzzles should be easy to verify, so this is not ideal.

In addition, even though Scrpyt was developed to be ASIC resistant, an ASIC has actually been developed for it, so it’s no longer considered ASIC resistant.

Another idea to achieve ASIC resistance that people tried was to chain together a bunch of hash functions.

The idea was that it’s much harder to create an ASIC that can deal with so many hash functions, not just once.

And that was the idea behind x11 and x13, which chain together 11 and 13 different hash functions together respectfully.

The cryptocurrency Dash uses the x11 hashing algorithm, using a chain of SHA3 variants.

Dash’s x11 was designed to be hard to make an ASIC for, but not impossible, so it was never really intended to be ASIC resistant.

Developers wanted to get a good distribution of coin early in Dash’s lifecycle,.

Later on, if someone happened to develop an ASIC for x11, coins would have already been distributed in a fair way.

And that’s exactly what happened.

Another idea that was tossed around was to design coins that periodically switched mining puzzles, making it difficult to optimize.

For example, it could switch around from SHA-1 to SHA-3 to Scrypt for 6 months at a time each.

However, the overhead required to create such an algorithm has historically deterred people from implementing it.

As a closing note, whenever there’s money that can be made, someone’s going to be working to make the most profit.

In cryptocurrencies, that can take the form of working to create ASICs for new hashing algorithms.

Mike Hearn, a Bitcoin Core developer said (quote) “There’s really no such thing as an ASIC-resistant algorithm.”

As it turns out, ASIC resistance might actually not be that great of an idea.

It’s still an ongoing debate.

In a non-ASIC-resistant system, ASICs tend to dominate the network, suppressing regular people.

One common mantra in the early days of Bitcoin was “1 CPU 1 Vote”: the idea was that mining was truly decentralized, with all nodes on the network having an equal say in validating blocks.

The rise of ASICs has since diminished this, effectively giving some people more votes than others.

ASIC-resistance brings back this idea of democracy, and decreases mining centralization in the network.

A common counter-argument on the other hand, is that since ASICs are designed solely to solve a cryptographic puzzle, they’re incapable of anything else.

Miners who own ASICs are thus committed to the network.

Their hardware only has value in the network their ASICs were designed for.

If a system with a large amount of miners relying in ASICs switches to an ASIC-resistant algorithm and then the exchange rate of the cryptocurrency crashes, many miners will suddenly be left with useless electricity-gobbling hardware since they have no incentive to mine anymore.

ASIC miners have already committed to the network by buying their ASICs, so they do not want to do anything that would disrupt the network.

Mining pools have historically disbanded once they’ve gotten too large, since the news of a mining pool with a large amount of hashpower would decrease trust in the network.

This idea of useless hardware and wasted hashpower leads us to our next slide.

Since so much computational power is being expended to solve cryptographic puzzles, which at their core are just pumping out random numbers, why not make those puzzles something that can be useful to the rest of the world?

For example, miners can search for large primes, look for aliens and planets, simulate proteins at the atomic level, or generate predictive climate models.

If we can use the whole computational power on the Bitcoin network to work on these problems, we could solve them so much faster!

Wouldn’t being able to predict the weather be great?

It seems like a great idea, but there are issues.

These distributed computing problems are usually unsuitable for proof-of-work and don’t work as cryptographic puzzles.

There’s a fixed amount of data in the system, meaning that there is a finite number of solutions to the problem.

As soon as somebody has found the solution to a particular problem, what happens then?

Or what if there’s not enough data to find a solution?

The problem can’t be used for proof-of-work anymore.

In Bitcoin, since the puzzle is based on the previous block, the problem space is inexhaustible.

You can always generate more problems with more solutions.

In contrast, we used SETI@Home data for our puzzle, we could run out of raw radio telescope data, leaving no problem to solve.

Next, potential solutions are not equally probable.

For example, if you’re looking for planets in the sky, one part of the sky might be more populated than others.

It’s not equally likely that you’ll find a planet in all sections of the sky.

In the hash functions of Bitcoin, it’s on average equally difficult to find a solution.

Having a uniform solution space means that the only way to find a solution is through pure brute force computation.

You can’t find any tricks or shortcuts that will let you avoid that computation.

Lastly, these problems are delegated by central entities, as opposed to to in Bitcoin, whose problems are created by its protocol.

It opens up the potential for collusion and exploitation.

When we rely on a central authority or central entity to create the problem.

In summary, proof of useful work sounds like a great idea, but it’s incredibly difficult to implement.

The idea behind a consensus update is that since Bitcoin is decentralized, no one person can say “this is the change we’re going to make in the protocol.”

Everybody has to agree, and come to consensus for an update to go through.

There is a centralized team working on these protocol changes, the Bitcoin Core team, along with a few other people that work on specific parts of the software.

This is the software that’s used in full nodes.

This brings us to the two ways that blockchains can be updated: hard forks and soft forks.

Hard forks and soft forks refer not to forks as previously mentioned, where two blocks are produced at the same time, but to protocol forks.

Keep in mind that all miners operate independently, and it is not mandatory for them to update their protocols.

When a large group adopts protocol changes, then the blockchain will react in a couple different ways depending on the new protocol’s rules.

A hard fork results from a new protocol that does things not allowed in the old protocol.

An example is Bitcoin Cash, which increased the block size from 1MB to 8MB.

The nodes on the old, legacy Bitcoin protocol are going to refuse the new 8MB blocks, leading to a permanent fork as seen in the diagram.

These two forks would then theoretically continue forever as long as both chains have support, miners who continuously run that software.

Essentially, the hard fork is not backwards compatible.

In a soft fork, the rules of the new protocol are only restrained.

For example, if a new Bitcoin fork called Bitcoin Dollars were to reduce the block size to half a megabyte, then everything valid in the new protocol is still valid in the old protocol.

However, the old protocol’s blocks are no longer valid in the new protocol, meaning that a soft fork is backwards compatible but not forwards compatible.

As seen in the bottom picture, the new and old nodes will accept blocks from the new protocols, but the old nodes will never have their blocks accepted into the longest chain mined by the new nodes.

The new nodes then pressure the old nodes to adapt the new protocol, so that they’re not left behind.

Before every fork, hard or soft, the entire community launches into debates about the future of Bitcoin.

In Bitcoin, changes to the protocol come in the form of BIPs, or Bitcoin Improvement Proposals.

These can be changes in the network protocol, block or transaction validation, or anything affecting interoperability.

This is the way the community votes on changes that they want to support.

There are three types of BIPS: standard, informational, and process.

A standard BIP is an actual update or change to the protocol.

This is how the ecosystem changes, and what miners vote on.

Informational BIPs are more so guidelines on how people should do things in the future, but it doesn’t change the protocol.

An example of an informational BIP would be a BIP talking about how to run a mining pool.

The last BIP, process BIPs, are specific things that should be done as well as best practices.

The first BIP was by Amit Taaki in 2011, and was actually a BIP to make BIPs.

Miners vote on BIPs by including a reference to that BIP in the block that they mine.

Author: Rea Savla

Bitcoin makes the distinction of 4 key functionalities (Mastering Bitcoin Opens in new window, p.152) that every node in the network is a combination of:

• Wallets: management of keys and addresses
• Mining: the voting process which expends computational power
• Full Blockchain: a copy of the full Bitcoin blockchain
• Routing: software that allows you to talk to other Bitcoin nodes

Image Source: Mastering Bitcoin Opens in new window.

The primary function of Bitcoin wallets is to keep track of your identity. In Bitcoin, a private key gives you a claim to your virtual identity; it unlocks your virtual identity and gives you access to your Bitcoin address. Wallets primarily store your private keys, but most wallet softwares also generate public/private keys with each new transaction and track your transactions. Unlike the physical wallets we use for money, Bitcoin wallets do not store physical bitcoins. They’re simply impossible to store. Hence, control over private keys is essentially control over identity and bitcoins.

There are two types of Bitcoin wallets: hot wallets (Mycelium, AirBitz, Webapp, Mobile App, etc) are connected at some point to the internet, while cold wallets (i.e. paper and hardware) are not. 

You can find them at Bitcoin ATMs, where you can trade cash for bitcoins. You can also get bitcoins through an exchange, where you can trade between different types of fiat currencies and cryptocurrencies. Centralized exchanges are easy to use and access, but are single points of failure which may be vulnerable to hacks. Decentralized exchanges (i.e. Bitshares and Bitsquare) conduct trades directly between users by creating proxy tokens and are mostly trustless but may suffer from liquidity problems. 

For wallets to successfully keep track of your identity and bitcoin, they must implement mechanisms to verify transactions and keep track of many keys. 

When using a wallet software, you do not have to download the full Bitcoin blockchain thanks to Simple Payment Verification (SPV). SPV is a method for verifying if particular transactions (specifically your own transactions) are included in a block without having to download entire blocks. Instead, you can simply ask your neighbors for relevant info about the blocks, though you must trust they will give you legitimate data. 

To maintain anonymity, it is best practice never to reuse pseudonyms. Bitcoin addresses are cheap to generate, and using a new pseudonym for each transaction makes tracking your activity and discovering your identity much more difficult. 

The traditional way to keep track of a large number of keys is through a “Just a Bunch of Keys” (JBOK) wallet: a new key pair is created and backed up for every transaction. However, JBOKs are hard to scale. You can see how exchanges have to deal with thousands of transactions a day for thousands of users would need something more efficient than JBOK wallets. 

Hierarchically Deterministic (HD) Wallets overcome this problem by deriving all new keys from an original random seed value. In HD Wallets, anyone controlling the parent key can also generate and control child keys.

Mining in Bitcoin requires miners to verify incoming transactions, create a block, find a valid nonce, broadcast the block, and earn a profit in bitcoins.

Miners are primarily motivated to maintain the network, or execute the Proof-of-Work consensus mechanism and bring new bitcoin into circulation, by earning profit. Miners only earn a profit from mining when the revenues from the block reward and the transaction fees are greater than the fixed and variable costs of mining.

A block reward is a reward that goes to miners whose blocks are included in the longest chain. Bitcoin incentivizes honest mining by rewarding miners in bitcoin. The block reward halves every 210,000 blocks, approximately 2 years, and will fall to 0 once 21 million bitcoins have been mined.

Regardless of the number of bitcoins in circulation, miners will always be rewarded via transaction fees.

The main fixed cost in mining is the hardware that allows users to mine. CPUs, GPUs, FPGAs, and ASICs are all types of hardware that have been used for mining, and each is more computationally powerful than the last. ASICs, which stands for Application-Specific Integrated Circuits, are currently capable of computing 14 trillion hashes per second. They are fully specialized devices that can only do one activity: mine for Bitcoin.

The main variable costs associated with mining are energy and infrastructure costs (i.e. warehouse and personnel). Energy costs are further divided into three types: Embodied Energy, to produce hardware; Electricity, to power hardware; and Cooling, to maintain hardware.

To get a better sense of how mining looks in real life, here is what a real-world ASIC mining farm in China looks like:

Image Source: The Register Opens in new window

Even with the most impressive technology, it still takes a great amount of time and power to find a block. Mining pools allow miners to join their hash power together so that they will collectively find blocks and collect rewards more often, reducing variance. As a member of a mining pool, your reward is calculated by the number of “shares” you contribute. A share is defined as a near valid block, used to estimate the mining power you contributed. More shares equals more attempts at finding a valid nonce equals a larger slice of the pie.

The two main payment mechanisms in mining pools are:

Pay-per-Share: where miners are paid a fixed amount of money for each share, regardless of how much reward the pool makes

Proportional Schemes, where miners receive a reward proportional to the number of shares submitted before the block was found

Pay-per-Share pools are advantageous to the miner because of guaranteed payouts, but risky for the pool because miners have no incentive to submit valid blocks to the pool. Proportional Schemes are advantageous for the pool because the pool operator only pays out after a block is found, but more risky for the miner due to greater variance of reward.

One of the current challenges of Bitcoin is that mining is tending towards centralization with the development of ASICs, mining pools, and mining farms. ASICs cost a lot of money, which means only those with access to high volumes of capital can afford them, thereby concentrating the mining power into the hands of a few. To address this issue, we must develop a tweak to the protocol that reduces centralization but still adheres by the puzzle requirements (quick to verify, adjustable difficulty, and computationally difficult).

We can get around the ASIC domination by creating a memory-hard or memory-bound puzzle, periodically switching mining puzzles, or replacing Proof of Work by Proof of Useful Work.

While ASIC resistance may decentralize mining and make it a more democratic process, achieving ASIC resistance may also cause disruptions to the mining community that would undermine the network. 

Real changes are implemented in Bitcoin via hard or soft forks. Hard forks result from a new protocol that does things not allowed in the old protocol. In a soft fork, the rules of the protocol are only restrained. A soft fork is backwards compatible but not forwards compatible.

Readings

Mining Bitcoin with pencil and paper: 0.67 hashes per day Opens in new window

ViaBTC Rises: How A Mysterious Miner Could Decide Bitcoin's Future Opens in new window

Antbleed: Bitcoin's Newest New Controversy Explained Opens in new window

(Optional) Time stamped chatlog: Why Jihan and Jiang want to block segwit at all cost Opens in new window

(Optional) Bitcoin Hash Rate vs Difficulty Opens in new window (play around with different variables!)

(Optional) BlockShell Opens in new window (set up and play around with the CLI)

With all our knowledge so far, we have a pretty good understanding of Bitcoin high level motivations and design, as well as low level mechanics.

Now with our knowledge of the ins and outs of Bitcoin… let’s destroy it :)

Looking from a game theoretical perspective, we’ll formulate attacks and other malicious behavior that can potentially destroy Bitcoin.

One note is that although we have been focusing primarily on Bitcoin so far, now we’re starting to build an understanding of not just Bitcoin, but cryptocurrencies and blockchain in general.

This module is called “How to Destroy Bitcoin: Game Theory and Attacks” but you can understand that the game theoretical approach and analysis of attack vectors applies to all proof-of-work blockchains -- not just Bitcoin.

As we explained in the previous module, everything is driven by profit -- and Bitcoin is certainly no exception.

For example, it might not be profitable to be a solo miner, especially if you don’t have a lot of specialized hardware.

You might want to join a mining pool, where users pool together their computing resources so that collectively they all have an increased likelihood of finding the next block, and thus receive the block reward.

As we’ll see in this section, there are some strategies you can employ to increase your profits even further.

And when it comes to making the most profit, honest strategies are NOT usually the best way to go.

Remember from last lecture that pay-per-share schemes will pay you a fixed amount of money for each share, or near-valid block.

This payment is guaranteed and constant per share, regardless of how much reward the pool makes.

One issue is that there’s no incentive for a miner to actually submit valid blocks to the pool because the amount of revenue collected from that share is equal to any other share.

Miners will stop doing work on their current block as soon as it satisfies the near-valid block requirements.

Proportional schemes are the other category which pay out only when a block is found.

The reward of each miner is proportional to the number of shares submitted before the block was found.

However, this scheme is disadvantageous for the miner because the miner now has to bear the risk of not finding a block, and therefore the variance of rewards.

They only get paid when a valid block is found by the pool.

For the same reason, proportional pay-out schemes are beneficial to the pool, since they only have to pay miners when a valid block is found.

Given the incentive structures in both pay per share and proportional pool pay-out schemes, is it possible that pools are vulnerable to some incentive misalignment?

Remember that we as miners want to maximize their payout.

Well, is there a way that we can take advantage of proportional payout schemes?

This graph is a general representation of the difference between pay-per-share payouts and proportional payouts.

We know there must be some intersection between these two because the proportional payout starts off high but decreases while the pay-per-share scheme stays constant.

This graph might reveal something to you right away.

If not, don’t worry -- you’re about to see how we can increase our profit margins though a little bit of cleverness.

Here, you see the area under the curve, which represents the total reward per share times the number of shares.

If we mine under just the proportional scheme, the reward gained per share decreases as more and more shares get submitted towards this block, meaning that our marginal rewards start to approach zero.

In this example, we see the same type of graph but for pay-per-share.

It’s a giant rectangle, extended a small amount by every share that we mine.

Unlike the previous scheme, this one doesn’t approach zero.

Instead, the value of each share stays constant.

A clever miner will look for ways to take advantage of both payout schemes to increase that total area under the curve, to increase their total profit from shares.

And here’s how they can do that.

Let’s say a miner starts off mining under the proportional payout scheme.

This means that each share is very valuable in the beginning.

If we start mining here and find a block quickly, then we make a large amount of profit per share.

However, if we don’t find a block for quite a while in this pool, our shares start to become less profitable.

To get maximum profit, we can reconsider where we put our mining power.

Instead of continuing to mine in the proportional scheme, we switch to the pay-per-share scheme.

Right at the intersection point.

Why?

Because the reward per additional share there is higher, and will yield more profit, than the proportional payout.

To fully understand this attack, you need to see the point at which the two curves intersect.

And this is when the value per share of the proportional scheme decreases beneath the value per share of the pay-per-share scheme.

This is why the attack is known as pool hopping: the miner hops to the most valuable pool as they see fit.

A generalization is to say that a miner will rationally mine in the pool providing the most reward per share.

Because of this, honest and loyal miners will be cheated out of profit by the rational miners, who choose personal profit over the group.

Because of this, proportional pools are not feasible in practice.

Thus far, designing a mining pool reward scheme that both aligns incentives fully and is not vulnerable to pool hopping remains an open problem.

A pool with fully aligned incentives would both have miners aim to submit nonces that produce valid block headers to the pool and stay loyal to the pool.

The proportional scheme incentivizes miners to submit such nonces, but it doesn’t incentivize miner loyalty.

Pay-per-share pools are the inverse, as previously mentioned.

We’ll see how this misalignment of incentives in pay-per-share models can lead to even further attacks.

Remember, in a pay-per-share model, a miner is not incentivized to give their mining pool any valid blocks.

There’s no extra reward to the individual for finding a share that gives the pool a block.

Is there a way that we can exploit this incentive misalignment?

Turns out there is!

You might have started to predict the attack by now.

If we get paid out for finding near-valid shares, why not just get paid for work without submitting valid blocks? And this is known as pool cannibalization.

You may recognize the person in this image on the right side of the slide.

This is Hannibal Lecter, fictitious cannibal, from the TV Show “Hannibal.”

Just like Hannibal is a person that consumes other people, pools can “consume” parts of other pools through pool cannibalization.

The way we “consume” other pools is that if we have a decent amount of mining power, we can distribute some of our power into other pay-per-share schemes, but for these pools, we don’t submit valid blocks.

This way, we can be paid by those other pools without increasing their revenue.

With pool cannibalization, we increase own personal profit at the detriment of other pools.

The advantage of this attack is that it’s very hard to detect in small amounts.

Let’s go ahead and show why this is the case through some calculations.

Let’s first establish some assumptions.

Let’s say we have 30 hashes per second of power in your pool, and the total network has 100 hashes per second.

Easily, we see that we have 30 percent of the network hashrate.

On top of that, there’s a current block reward of 1 bitcoin per block.

These numbers are for simplicity’s sake to set up the stage for the pool cannibalization analysis and not at all representative of real world conditions.

With basic statistics, we can calculate an expectation of profit per block.

This is the average profit made per block, which is not to be confused with the actual profit per block.

If we have thirty percent of the network hashrate, this means we are also expected to get thirty percent of the blocks in the longest chain.

By implication, we are also expecting to earn thirty percent of the total mining rewards on average, which is 0.3 bitcoins per block.

Now that we’ve established this scenario, let’s consider something: we buy a bit more hardware, currently worth 1 percent of the total network hashrate.

Let’s consider what we can do with this extra mining power.

The standard, simple thing to do with this extra mining power is add it to your own resources.

This means we have 31 hashes per second, and the total network has 101 hashes per second.

We now have 30.69 percent of the total hashrate.

This means that our increase in profit is 0.0069 bitcoins from our extra hardware.

But how can we leverage our previous knowledge about pay-per-share pools?

Well, by doing something mean but profitable: distributing our 1 unit of mining power among all the other pools -- the other 70% of the network hash power.

However, we make sure to that we do not add to their revenue.

We withhold any valid blocks while submitting all other shares.

The result is that the pools pay out to us -- because we’re still submitting shares -- but does not gain any benefit from our hash power -- since we don’t submit valid blocks.

What that looks like in this particular scenario is as this: the rest of the pools are 70 parts honest, 1 part dishonest.

By honest, we mean mining and submitting valid blocks.

Dishonest is disobeying this intent and withholding valid blocks, and that’s what we’re doing.

What this means is that the effective hashrate of the other pools has not changed: it’s still 70%.

However, the mining pools are still paying out to us.

This means that our expected value of mining with this one percent of hashpower ends up being 0.0098 bitcoins.

You’ll notice a scary conclusion.

Dishonesty is more profitable than honesty!

We’ve just shown that it’s more profitable to cannibalize pools than to mine directly through our own pool.

This is a scary deduction that we’ve made, but it doesn’t stop here.

If it’s more profitable for any pool to do this, then what does that look like across the network?

Will pools all start cannibalizing each other?

Is everyone going to start being dishonest for profit?

Are pools going to wage war on each other through this attack?

Like all other competitions for resources, we can model this problem as a game and analyze it with economic game theory.

We have several players all attempting to maximize their individual profit, and they each have choices about whether to cannibalize.

We can, for the sake of analysis, view it as an iterative game.

In other words, there are distinct rounds, and a choice needs to be made each round.

There are several different mining pools in the Bitcoin network, but let’s focus on two theoretical pools: Pool 1, and Pool 2.

Each round, or iteration, is a case of the Prisoner’s Dilemma.

The Prisoner’s Dilemma refers to a problem of cooperation.

Pool 1 and Pool 2 both want to attack each other to increase their own personal profit at the expense of the other.

However, if they both attack each other, they’ll be worse off than if they both chose to cooperate and not attack at all.

Let’s say Pool 1 chooses to attack Pool 2.

Pool 1 increases its own personal profit, as demonstrated in the last section, but Pool 2 suffers.

What’s to stop Pool 2 from retaliating to increase revenue?

Well, nothing!

Pool 2 has no reason not to attack Pool 1 back.

Because of this, choosing to attack is tempting for both pools.

In fact, we can formally say that both pools attacking each other is a Nash Equilibrium.

A Nash Equilibrium, put simply, refers to the choices that actors see as most profitable, as long as the others' decisions remain unchanged.

Both Pool 1 and Pool 2 would not stop attacking, even if the other stopped attacking, because that would imply less profit for themselves.

However, if both Pool 1 and Pool 2 attack each other, they both lose mining power relative to the rest of the network.

This means that their profit from attacking is actually less than if they both chose not to attack each other.

Why don’t they just not attack each other and make more profit than in this Nash Equilibrium?

Well, the no-pool-attacks scenario is not a Nash Equilibrium.

In other words, if both pools choose not to attack each other, they’re not making as much profit as they could be.

There’s a more profitable option out there for them.

By attacking, they can increase their profit, and that’s what we’ve established all actors want to do.

Because of this, you fall into this issue where rational actors will eventually choose the Nash Equilibrium if they’re pursuing profit.

They’ll continually attack each other in hopes of making profit.

Sure, if pools adhere to a culture where they agree not to attack each other through cannibalization, then everyone benefits in the long run.

However, it’s not a guarantee that all pools will adhere to this: anonymously or not, nothing is stopping them at this point other than social constructs.

Except for the fear of getting attacked, there’s a large amount of incentive to attack other pools for profit.

Pools might be able to detect these attacks in the future, and detection of these attacks would allow for a long term solution, but as of now it’s still very difficult to detect.

We can’t always tell the origin of mining power, meaning that we either trust that we’re not being attacked or accept that there’s no solution to this problem.

Because of this, you can see the problem as a Tragedy of the Commons, in which all pools are individually incentivized to act in ways that do not as a whole improve the wellbeing of the entire group.

It all comes back to this issue of aligning personal incentives with group incentives.

Author: Rea Savla

We can model the behavior of different actors in the Bitcoin network for a given situation using a game theoretical analysis. Our first assumption is that actors will act in a rational way, where rationality is defined as taking the actions that maximize their utility. In our scenario, utility is defined by the monetary gains resulting from an action. While not everyone in Bitcoin is purely motivated by monetary gain, this is still a powerful generalization.

A Pure Strategy Nash Equilibrium is the set of actions that maximize each actor's utility given the responses of all the other actors. Note, the utility each player receives depends on the player's own decisions and the decisions of all the other actors, in the same way that the rewards a miner gets in the mining pool depend on the miner's own decision to attack or cooperate and the rest of the pool's decision to attack or cooperate. Players in the Bitcoin network or in a mining pool will converge to acting according to the Pure Strategy Nash Equilibrium for a given scenario, assuming they all behave rationally.

Let’s take a look at a simple scenario. Suppose there are only 2 mining pools in the Bitcoin network, Pool A and Pool B. Their utilities are shown below. The numerical value of utility is arbitrary in economics; it does not have any associated units, and it is calculated using a utility function of other parameters, defined by the economist. We are only interested in the comparative value of an action’s utility, whether it is higher or lower than another; we are not interested in its absolute value. In this example, the respective utilities are derived from the monetary gains each player would receive from the given scenarios of attacking or cooperating with the other.

The specific numerical values for the utilities in this example were chosen arbitrarily to reflect a scenario where players are incentivized to act dishonestly, but fare worse when both players are dishonest than when both players are honest. Let’s take the perspective of Player A. Given Player B acts honestly, player A gains the most utility from acting dishonestly, since a utility of 3 is greater than the utility of 2 that A would receive by acting honestly.

Given Player B acts dishonestly, Player A would prefer also acting dishonestly since a utility of 1 >utility of 0.

Player B makes the same conclusions, and makes preferences as follows:

We can see that the scenario in which both players choose to act dishonestly is the Pure Strategy Nash Equilibrium, since at this position, both players are maximizing their payoffs given the other player’s actions. Thus, despite receiving higher returns from both acting honestly, Players A and B will both act dishonestly.

If you are interested in learning more about introductory game theory, we recommend the following reading: A Brief Introduction to Basics of Game Theory by Matthew O. Jackson, Stanford University.

Now that we’ve discussed some pool game theory, let’s go ahead and revisit the classic attack of the cryptocurrency world: the double spend attack.

We previously discussed this at a high level in the first lecture, mentioning that we use computational resources to prevent double-spend attacks.

But how does this actually look within the blockchain?

On top of that, is there still a way to double spend?

Can I double spend with less than a majority of the voting power?

We’ll answer all those questions for you in this upcoming module.

Let’s revisit the definition of a Double Spend attack.

It means being able to successfully spend the same value more than once.

Keep this in mind as we go through the attacks in the following slides.

Imagine this scenario: in the year 2099, I’m an old lady, and I want to buy the new iPhone 92XCS from Rustie on the black market.

I’m offering a whopping 100 bitcoins, but I doesn’t actually want to give up my money.

How can I double spend on Rustie?

Well, it all depends on how early Rustie’s willing to give me the iPhone.

We’ll explain what that means right away.

Suppose Rustie is very naive and trusting.

The moment he sees a valid transaction from me, he sends me the iPhone.

This is before the transaction even enters a block -- he makes the assumption that if the transaction is floating around the network, it will eventually make its way into a block and he’ll get paid.

I can take advantage of that.

I can trick him by sending him a valid transaction and sending the rest of the network a conflicting transaction.

While Rustie has the impression that I am sending the UTXO to his own address, I tell the rest of the network that I want to send that same UTXO to a different address, which is actually another address that I control.

I can incentivize miners to choose the second transaction by making the transaction fee for that higher as well.

It’s possible, then, that Rustie sends me the iPhone but I get to keep my money.

I just successfully double spent on Rustie through a race attack.

It’s called a race attack because the timing of transactions affects the outcome.

The transaction to Rustie doesn’t go through because it’s beat by the second transaction that spends from the same UTXO.

The second transaction making its way into the blockchain prevents poor Rustie from ever getting his bitcoins.

Defense: Confirmations

How might Rustie protect himself from this?

Well, we know that Rustie only gets to keep his bitcoins if a transaction to him makes its way into the longest chain.

And, as we know, the longest chain could be forked early on.

To provide some confidence in the immutability of the transaction, he can look for what we call confirmations.

Instead of just accepting a transaction as valid when he sees it floating around the network or when it first makes its way into a block, Rustie waits for the transaction to get a certain number of confirmations.

A confirmation is defined as the number of blocks built off of some particular block.

In this diagram, there’s the block holding my transaction to Rustie, and there’s two blocks built on top of that.

This means that there’s two confirmations on this block.

The more confirmations a transaction has, the harder it is to double spend, since a malicious miner such as myself would have to fork the chain starting from before the block that contains that transaction, and mine fast enough to surpass with the honest chain.

This is a reasonably simple concept, but we can see some interesting things that involve confirmations.

The question you’re probably asking: how many confirmations does Rustie need before he feels confident about the finality of his transaction?

Well, let’s go back to the example with me and Rustie.

Rustie’s gotten wiser from the last time I double spent on him, so he’s now going to wait for some confirmations before sending the iPhone.

Let’s say he waits for “k” confirmations.

This means he waits for k blocks to be built on top of his transaction before deeming the transaction finalized.

From my perspective, I now needs to find a way for Rustie’s transaction to get k confirmations,

but I then need to find a way to produce a longer chain containing the transaction spending from the exact same UXTO back to myself.

This longer chain will invalidate Rustie’s transaction.

The way I do this is by starting a private chain containing my own malicious transaction, mine k blocks, and then broadcast the chain after Rustie has sent the iPhone to me.

By doing so, I will have received the goods and have kept my own bitcoin.

A quick note: in this demonstration, I will mine k + 1 blocks on top of the block that contains my own transaction to myself.

If I mined just k blocks, I would have caught up with the rest of the network.

But if I want to invalidate the honest chain with the transaction from me to Rustie, my chain has to be longer.

Hence, the plus one.

Let’s go ahead and see a demonstration.

Let’s say that k in this case is equal to 2 confirmations.

In other words, Rustie will assume that the transaction is finalized after 2 confirmations.

I publish my transaction to Rustie, and it has been included within a block.

One confirmation….

Two confirmations.

Rustie notices the two confirmations and feels confident about the transaction’s finality.

He now sends me the iPhone so as not to keep me waiting.

But that was exactly what I was expecting.

This whole time, I was mining on a private chain which I did not reveal to the rest of the network.

This chain contains a transaction from the same UTXO, to a different address which I also control.

This transaction from me to myself is to make sure that the transaction from me to Rustie is forever invalid and can never be republished.

This is because once the network accepts the transaction from me to myself, the corresponding

UTXO is used up, and since both this transaction as well as the transaction from me to Rustie spend from the same UTXO, this then invalidates the transaction from me to Rustie.

When I broadcast my chain, the rest of the network accepts my chain over the previous chain because it’s longer.

Because of this, the entire chain of blocks, containing the transaction from me to Rustie, and onward is invalidated.

The rest of the network will continue mining on my chain, meaning that I will have received Rustie’s iPhone while still keeping my own money.

Hooray!

I have successfully double spent!

However, notice that my double spending of Rustie relied on my ability to produce a longer chain than the honest network.

You might be wondering: what is the probability of me being able to produce a chain longer than the rest of the network?

Well, we’ll see right away how that’s possible.

These graphs show the probability of a successful fork.

The top shows the probability of an attacker catching up to some chain given two parameters:

the amount of mining power possessed by the attacker, and the amount of blocks by which the honest chain is ahead.

The bottom graph is a mirrored image, as it shows the probability that a transaction is safe after some k number of confirmations.

Let’s look at the top graph first.

The blue line represents the probability of an attacker with 10 percent of the network hashrate successfully producing a chain of equal length to the honest network.

Even just catching up to one block is only a 20 percent chance, and it tapers towards zero quickly.

The curves generally look like an inverse exponential curve, in that the probability starts off high but then quickly drops towards zero.

The exception is mainly with the 50 percent model.

As you can see, the 50 percent model has a probability of 1 no matter what.

In other words, an attacker with 50 percent of the network hashrate has a 100 percent chance of creating a chain of equal length to the honest chain.

There’s a lot of fancy statistics to get to these exact values, but we’ll save that for readings.

To quickly comment on the second graph, you’ll notice that it’s a flipped over version of the top graph.

This is because there’s only two possibilities that we’re considering with any transaction: its chain is the longest chain, or it’s invalidated by a longer chain.

Because of this, the probability of one event is 1 minus the probability of the other.

In addition, this leads us to the following conclusion: if the attacker has 400px; of the network hashrate, then we have a 0 percent probability that our transaction is ever safe.

At any time, the 400px; attacker can produce a chain that will catch up with the rest of the honest network.

With more than 400px;, say 51%, I will always be able to double spend.

I will always be in full control of which blocks are included within the history of transactions.

But here’s a question to ask: just because I can double spend doesn’t mean I should.

There’s a lot of real world consequences of double spending that we don’t consider when just discussing theory and math.

If the rest of the world ever realizes that a double spend attack was conducted on Bitcoin, the value would plummet instantaneously because there’s no longer any belief in the finality of transactions and the security of the network.

My own bitcoin would drop in value as well, and that means I’ll lose a lot of money.

In addition, all the specialized hardware I own would be useless as well, since the hardware can only mine Bitcoin.

If Bitcoin is worthless, then so is the hardware.

What are some other options?

Well, I can bribe miners.

If I don’t own the hardware or don’t want to spend that much capital on hardware that’s about to become useless with a successful attack, might as well bribe other miners to mine on my own withheld chain.

But what if I’m a hostile government, adversarial altcoin, large institution, or other enemy to the network with a large amount of capital?

Well, then there’s a way for me to make up my money even after buying all that hardware.

What I can do is called a “Goldfinger” attack, named after the James Bond movie of the same name.

The point of this attack is to profit off the destruction of some assets.

In the James Bond movie, the villain tried to profit off the destruction of gold in Ft. Knox.

With Bitcoin, I can short the currency, meaning to place a bet on its devaluation.

I can first place a bet that Bitcoin is going to go down in value, then I can launch the double spend attack to make my prediction go through.

At this point, double spending on someone is -- interestingly enough -- just a means to an end.

Thus far, we’ve looked at examples of how users with enough hashpower can take advantage of the properties of the blockchain in order to maliciously double spend their coins.

Now, we’ll look at another sort of attack: the censorship of transactions.

With censorship, we can choose to ignore the transactions from an individual or group of individuals, isolating them from the network and effectively rendering their bitcoins useless.

Let’s say that Gloria is in charge of the Glorian nation, and has jurisdiction over all of the mining pools within its boundaries, which sums up to be over 51% of the Bitcoin network's hash power . Gloria doesn’t like me, so her goal is to censor all of my Bitcoin addresses and prevent me from spending any of my Bitcoin.

The blocks in Gloria’s ideal blockchain will include none of my transactions.

The most naive strategy is for Gloria to instruct all of her mining pools not to include my transactions, also known as blacklisting.

However, unless Gloria’s mining pools have 100% of the entire networks hashpower, other miners outside of her jurisdiction will eventually include my transactions in a block.

This doesn’t fully blacklist me, and only ends up causing delays and inconveniences, although those delays and inconveniences can become significant depending on how much hashpower

Gloria has.

Let’s look at another strategy for Gloria.

Remember that Gloria’s mining pools have over 51% of the network hashrate.

She can say that all Glorian pools will refuse to work on a chain containing transactions spending from my address and announce this to the world.

If miners include one of my transactions in a block, Gloria will fork and create a longer proof-of-work chain, which is possible only because she has the majority of the hashrate.

The blocks containing my transactions are now invalid since they aren’t on the longest chain and will never be published.

This also means that the miners on that chain will receive no mining rewards for their efforts.

Non-Glorian miners will eventually stop including my transactions in their blocks since they know that their blocks won’t be part of the longest chain anway.

They'll be invalidated by Glorian miners.

This strategy is known as punitive forking, and can be very powerful in the hands of someone with a 51% majority hashrate.

That entity can prevent anybody from accessing their funds.

However, punitive forking doesn’t work unless Gloria has 51% of the hashpower, which is extremely difficult to achieve in reality. Is there another way? Unfortunately for me,

Yeah...

There is a strategy called feather forking, in which Gloria announces that she will attempt to fork if she sees a block with one of my transactions. But she will give up after a while.

This is in contrast to punitive forking, since that only works if a longer chain is guaranteed by the fork. In short, Gloria gives up after a block with my transactions has k confirmations, where k is any number. In the diagram, Gloria sees a block containing one of my transactions and attempts to fork. However, I get 3 confirmations, so she gives up trying to continue her fork.

Let’s look at a little bit of math. Let q be the proportion of mining power that Gloria has. Let k equal one, so that Gloria gives up after one confirmation on my block. This means that the chance of successfully orphaning my block is q squared, since Gloria has to compete with each confirmation. If q is 20%, then gloria has a 4% chance to orphan my block, which isn’t that great.

However, other miners are now aware that their block has a q^2, or 4% chance of being orphaned.

They now have to decide whether they should include my transaction in their block. The expected value of including my transaction is the chance that my block is included times the block reward, plus the transaction fee that I pay. The expected value of not including my transaction is simply just the block reward, since all miners would be working on that chain.

Therefore, looking at the math, unless I pay at least q^2 times the block reward in transaction fees, other miners will mine on the malicious chain. As of April 17th 2018, that would be a $4000 minimum tx fee, which is incredibly limiting. It’s very likely that I won’t be able to afford those transaction costs, so I won’t be even be able to send my transactions through. Thus, Gloria has succeeded in censoring my transactions with only 20% of the total hashrate.

It's important to consider how rational and malicious actors will behave to increase the amount of money that they make.

Think back to the previous module, specifically the motivations for mining.

If there’s a way for miners to maximize profit, assume that they will do so, even if it’s not totally fair to everyone else.

One of the best examples of this sort of behavior is selfish mining.

Let’s say that you’re a miner, and you just found a block.

You want to give yourself an advantage over the rest of the network, so instead of publishing the block through the network and receiving the block reward, you keep your block a secret.

Since you haven’t released your block, you’re effectively the only person working on the most recent block, and you effectively have 100% of the hashpower on your hidden chain.

Nobody else on the network can even attempt to solve the hash puzzle.

This is called selfish mining, or block withholding.

Note that Block withholding is sometimes also used in the context of mining pools, where you submit shares but withhold blocks.

If you find two blocks on your secret chain before the network finds the next one, you suddenly have the longest chain.

The network believes it’s mining on the longest chain, but little does it know that you actually have the longest chain.

You essentially fooled the network.

You’ve guaranteed yourself at least two block rewards because you can just continue to mine on your own chain, assuming that you always stay ahead of the honest network’s chain.

The moment the honest network is about to catch up to you, you can just publish your hidden chain, and then suddenly the honest chain is invalidated, since your chain is the longest, and as we already know, honest nodes in the network will always choose the longest chain.

So, you can continue mining this way and continue cheating the system until the honest network catches up to you.

You’ve guaranteed yourself more block reward than the honest network, and most importantly, you deny the honest network the chance to do any meaningful work because you have your own secret chain, and you can imimagestely invalidate the honest chain as soon as you publish your secret chain.

Assuming you’re two blocks ahead of the honest network, if the honest network then finds a block, you broadcast your two secret blocks to claim your two block rewards and to keep the honest network from catching up.

This makes the honest network’s block invalid, since the honest chain is now the shorter one.

Because the honest network was mining on chain that was then invalidated, you effectively got time to mine by yourself, for free.

Since you had no competition, you had a higher proportion of the effective hashrate, and therefore, a higher expected return.

The key thing to note here is that you assume that you as a malicious miner can find two blocks faster than the rest of the network can find one block.

If you can’t, then your computation is the one that goes to waste.

What if the network finds a block before you find a second one?

Then it becomes a race to propagate the new blocks.

As it turns out, if you have the capability of telling 400px; of the network about your block before the other party, as well as greater than 25% of the hashpower, then a malicious strategy is more profitable.

If you have greater than 33% of the hashpower, you can lose the propagation race every time and the malicious strategy will still be more profitable!

The math behind these results is omitted here due to complexity, but you’re more than encouraged to look into it!

As malicious miners are incentivized by higher potential profits to attack or try to subvert the network, honest individuals are incentivized to prevent such behavior.

There’s a constant struggle to try to patch up the Bitcoin ecosystem to prevent anything that would completely destroy Bitcoin.

In this next section we’ll be taking a look at some of the defenses we have to counter selfish mining.

We’ll first take a look at some of the more naive defenses, and see what we can improve from there.

One of the earlier proposed solutions to solving selfish mining was to validate blocks using dummy blocks that hold metadata.

This technique was proposed by Schultz in 2015, and later by Solat and Potop-Butucaru in 2016.

The idea is that in addition to having regular blocks, we also create additional (dummy) blocks that contain signatures for that block.

For any given block, it is easy to look at the corresponding dummy block to verify how many users have seen and signed off on that block.

Blocks that are selfishly mined would only have been seen by the one withholding the blocks, and would not have sufficient signatures.

This proves that an honest block is witnessed by the network, and also proves that a competing block is absent before miners are able to work on it.

When a malicious actor that has been withholding blocks finally publishes their chain, all but the first block will be automatically rejected, since the first block would not have sufficient signatures.

With this method, selfish miners can’t make more than their expected reward, which is just for one block.

This invalidates a lot of the work they do to build the chain off of the first block they find.

Therefore, block validation with this dummy block signature scheme disincentivizes selfish mining.

One imimageste flaw that we can see is that this technique is vulnerable to Sybil attacks.

There’s no way to guarantee that the signatures on a block all come from different users.

And so the selfish miner could just generate as many fake signatures as needed to make it seem like the withheld blocks are valid.

They can do this because as we know from before, it’s easy to generate new identities in Bitcoin.

And furthermore, there’s no real way of determining how many signatures is enough to show that a block is valid.

For example…

What order of magnitude is the threshold number of signatures?

And would it have to change depending on the changing number of users within the network?

These questions are all hard to answer but are important to this schemes implementation

Also, no matter how many signatures are needed, a malicious user could still easily Sybil attack the system.

So in the end the fact that there’s no good way of determining a signature volume threshold and the fact that it’s susceptible to a Sybil attack makes the entire defense useless.

Furthermore, another downside to having dummy signature blocks is that their implementation would require fundamental changes to the already established block validity rules.

Such a change would require a hard fork, which is undesirable.

Another idea for a defense against selfish mining was fork-punishment, which was proposed by Lear Bahack in 2013.

The fork-punishment rule is as it sounds: it punishes anyone who forks the blockchain.

Any competing blocks receive no block reward, regardless of whether they were mined selfishly or honestly, and the first miner who proves that there was indeed a fork gets half of the forfeited rewards from the previous block height.

This disincentivizes malicious users to attempt to fork the blockchain by working on their own secret chain, and also incentivizes honest miners to report any forks that they have seen.

In the diagram below, we see that there’s a fork, the top being the honest chain and the bottom being the dishonest chain.

Since two blocks are broadcast at the same block height, they are competing blocks and thus neither of them have an associated block reward.

The first miner who proves that there was a fork, and includes this in the next block, gets the block reward for the block they just mined, as well as half of the block reward from the previous block height.

One of the imimageste drawbacks is that because we punish all competing blocks, honest miners suffer collateral damage from fork-punishment.

And the fact that we’re punishing honest miners constitutes a different kind of attack Fork-punishment also suffers from the fact that we would have to fundamentally change Bitcoin’s reward distribution rules.

We’d have to have some way of changing the coinbase transactions of blocks where there was a fork.

To do this and also to forward half of the block reward to the next block, we’d have to leverage a concept called transaction malleability, which is out of scope for this course.

And in the end, fork punishment would require a hard fork to implement, and as before, that’s pretty undesirable.

Instead of punishing everyone whenever there’s a fork, like in fork punishment, is there another way?

Perhaps we can instead prevent the block propagation race, and consider each submitted block at each block height.

For this, we can use uniform tie-breaking, which was an idea proposed by Eyal and Sirer in 2014.

When we have a race to propagate, the miner who has a network advantage is more likely to get their block accepted into the longest chain.

This is because they are better connected in the Bitcoin network and are able to get their block out to more people.

We can use uniform tie-breaking to protect against selfish miners who have a network advantage.

After each block is published, instead of taking the first seen block as valid, since this might be a block from a well-connected selfish miner, each miner first waits to hear about all competing blocks.

In the case of a tie, where a miner sees multiple blocks at the same block height, the miner randomly chooses which chain to mine on.

By randomly choosing which chain to mine on, miners mitigate an attacker’s network-level dominance.

Eyal and Sirer found that the default Bitcoin protocol has a profit threshold of 0% if there exist selfish miners that are really well connected in the network and can thus influence the way data flows through the network.

This would then mean that for Bitcoin to be entirely secure, 100% of the network has to be honest.

In their paper, with their proposed uniform tie-breaking, they claimed that it raised the profit threshold for selfish mining to 25%.

However, later in 2015, a separate paper published by Sapirshtein proposed a more optimal selfish mining strategy that reduced the profit threshold down to 23.2%.

A 25% profit threshold using uniform tie-breaking is pretty bad, considering that selfish mining is always profitable when you have 33% of the mining power.

Even more shocking is when we compare this to our original design for Bitcoin, which was supposed to be tolerant of up to 400px; malicious miners.

How do we improve 25%?

In 2014, Ethan Heilman proposed a defense of selfish mining using timestamps.

The idea is that timestamps would be issued by a trusted party and incorporated into every block by miners.

Timestamps would be global across the entire Bitcoin network, and would be issued regularly, say every 60 seconds.

They also would be publically accessible, but unforgeable because we’d use cryptographic signatures.

Blocks are considered in competition if they are received roughly within the same timeframe, say 120 seconds of each other, and when the situations arises when there is competition between blocks, then miners simply will pick the block whose timestamp is fresher, or the most recent.

In the paper, it says that the profit threshold for selfish miners using this method of unforeable timestamps goes up to 32%, a solid improvement from the 25% we saw with uniform tie-breaking.

As it turns out, selfish miners can easily break this defense.

Tie-breaking rules only apply when there’s a block propagation race, NOT when the selfish mining chain is longer than the public chain.

And as it turns out, it has been shown that if an attacker has a large amount of computational power, say greater than 40%, then these tie-breaking defenses against selfish mining are essentially worthless.

As you can see in the diagram on the bottom, a selfish miner can still invalidate the honest chain if they have the longest chain.

Miners will pick the longest chain, since it has the most work done on it, and will weight this more than the fact that a block in the honest chain is “fresher”, or more recent.

And another thing you might have realized is that this defense requires a trusted third party to distribute timestamps to everyone.

The drawback here is that Bitcoin aims to be as decentralized as possible, and while this might work with other systems, centralization certainly doesn’t mix with Bitcoin’s philosophy.

We’ll skip this for now.

Now we get into one of the more recent developments in selfish mining defenses.

This one is called Publish or Perish, and was published by Zhang and Prenell in 2017.

Previous defenses we’ve seen had many flaws, such as requiring a hard fork – for example in fork punishment – or not disincentivizing selfish mining when the selfish miner has a longer chain – for example in unforgeable timestamps.

Publish or perish claims to be the best – yet defense of selfish mining, and addresses all these previous issues.

It doesn’t require a hard fork, which means that it’s backwards compatible with the old chain, and it also disincentivized selfish mining even when the selfish miner has a longer chain.

The idea of selfish mining boils down to maliciously building a secret block, and the publishing that off into a chain.

Publish or perish’s main goal was to make sure that secret blocks do not help selfish miners at all.

To do this, Publish or perish’s approach focuses on amending Bitcoin’s Fork Resolving Policy, or FRP.

By default, we know that miners choose the longest chain to be valid, so for Bitcoin, we say that it has a length FRP. Publish or perish’s insight was to switch from length to weight as a heuristic for the fork resolving policy.

Instead of just counting the length of chains, we also consider weight, which is roughly making it disadvantageous to withhold blocks.

There’s a lot of terminology and formal definitions, so we’ll be going into that first.

We define Tau as assumed upper bound on the time it takes to propagate blocks across the Bitcoin network.

And from a miner’s perspective, the term in time means that for a valid receiving block, either (1) it height value is greater than that of the local head, or (2) its height value is the same as that of the local head, but was propagated within tau time.

The bottom diagrams illustrate the two possible requirements for a block to be in time.

On the left side, a block is received withing tau time, so that block is in competition with the current head.

On the right side, the block received has a height value that’s one greater than the local head, so we just append it to the blockchain.

The term uncle refers to an in-time block that competes with a block parent – the parent being the previous block to the current.

To formalize, the uncle of a block B is one less the height of B. Makes sense because the uncle of B has to be at the same height as B’s parent.

And for the uncle of B to be in time, it must be within tau time of B’s parent.

The diagram on the right side nicely summarized the definition of an uncle block.

We have our child block B, and previous to that, we have its parent A. At the same block height to A, we have blocks C and D. Block C is an uncle to B because it is within tau time of B’s parent, A. Block D on the other hand is not an uncle to B because while it has the correct height – one less the height of B – it was not seen in tau time of B’s parent, A. So it’s not an uncle.

The weight of a chain – from a miner’s perspective – is the number of its in time blocks plus the number of in time uncle blocks. And to make sure we have valid in time blocks and in time uncles, we make miners embed Hashes of the blocks that count toward blocks weight.

That way, others can easily verify the blocks weight.

And again, we say everything from a miner’s perspective because whether a block in time is evaluated from the miner’s local perspective.

And we calculate the weight of a chain starting from the block after its root, since competing chains always have a shared root.

For example, in the diagram on the right hand side, all three of these chains are rooted block R.

Having defined weight, you can kinda see how miners are disincentivized from withholding blocks.

If you withhold blocks, then your blocks won’t be in time, and so they’ll have less weight.

With all these definitions, we can now specify the weighted fork resolving policy.

It has three main rules.

If one chain is longer height-wise than the other(s) by k or greater blocks, then miners will mine on this chain.

If a chain is longer than the others by k blocks, we can probably trust it to be a valid chain.

This kind of inherits the notion of an honest majority from the default Bitcoin length FRP. If a chain is longer than other chains by k blocks and its malicious, then we have a bigger problem on our hands; everything’s broken so we should just stop using Bitcoin since someone probably owns a majority of a network hash power.

If all chains are within k blocks height of each other, then the miner will choose to mine on the chain with the largest weight.

If the largest weight is achieved by multiple chains simultaneously, the the miner chooses one among them randomly.

One note is that in the first rule k is a failsafe parameter that gauges the allowed amount of network partition.

If we set k to infinite, then the rule never applies.

So now that we’ve spent all this time defining all these terms and procedures, let’s look at how publish or perish actually disincentivizes selfish mining.

Here’s the general scenario.

Say that a selfish miner has already secretly mined one block. Before they can mine a second block, the honest network publishes a competing block. There’s now a block propagation race. At his point in time, the selfish miner has two options: option 1, to publish, or option 2, not to publish.

If the selfish miner publishes their block, the next honest block would gain a higher weight because it could embed a proof of having seen its uncle.

On the other hand, publishing a block would ensure that it would contribute to the weight of the selfish chain, since it’s in time. So, the selfish miner’s block gets a weight of 1, but also contributes to the weight of the honest chain.

On the other hand, if the selfish miner keeps their block secret and misses the time window for being in time, then the secret block would not contribute to the weight of its own chain. So, the selfish miner’s block gets a weight of 0, but since the honest network wouldn’t have seen this secret block either, it wouldn’t contribute to the weight of the honest chain.

The key takeway is that no matter which option the selfish miner chooses in the end, neither option gives weight to the selfish chain that is not also given to the honest chain.

For a bit more clarity, let’s take a more rigorous approach to our analysis of the previous scenario.

In the diagram on the bottom, the dark circles represent blocks that are found by the selfish miner, and the dark triangles represent blocks are found by honest miners.

The circle and triangle outlines represent a count in the weight of the selfish and honest chains, respectively, and as always, blocks are mined from left to right.

Imagine at this point in time, as before, the selfish miner has one block, let’s call it S, and the honest network just published a competitor to S.

In option 1, the selfish miner chooses to publish S. Since S is published within tau time of the honest competitor, it counts into the weight of the selfish chain.

However, since S then becomes an uncle to the next honest block, it also counts into the weight of the honest chain.

S counts into the weight of both the honest and selfish chains.

Fast forward to the point in time where we the have diagram below, both the honest and selfish chains have a weight of 3 despite the fact that there are only two honest blocks but three selfishly mined blocks

In option 2, the selfish miner doesn’t publish S, and instead chooses to wait, and publish later on as part of the selfish chain.

Since S is published late -- as in not within tau time -- it would not contribute to the weight of the selfish chain.

And also it wouldn't an uncle of the next honest block, so S doesn’t contribute to the weight of the honest chain either.

In option 2, S contributes to neither the weight of the honest nor the selfish chain.

If we have scenario depicted in the diagram below, both the honest and selfish chains have a weight of 2, despite there being three selfishly minedblocks and only two honestly mined blocks.

The result is that regardless of which option the selfish miner chooses, S will NOT contribute to only the weight of the selfish chain.

It will only contribute to both or neither.

This completely nullifies the advantage of having S in the first place.

Here are a couple graphs on the expected revenue of selfish miners when publish or perish is in effect.

On the left side, we have a graph of selfish miners’ profit, given that certain defenses are in place.

Particularly notable is the performance of publish or perish in comparison to optimal tie breaking.

Optimal tie breaking is the theoretical optimal defense that always rejects selfish blocks whenever there’s a tie, with 100% accuracy.

Notice that publish or perish even outperforms this theoretical optimal tie breaking defense.

On the right side, we have revenue given different values of k, which again is the threshold number of blocks ahead a chain has to be in order for miners to just automatically choose to mine on that one.

As it turns out, the higher the value of k, the less revenue selfish miners will make.

Revenue is the lowest when we set k to infinity, at which point miners will always look to mine on the chain with the most weight.

One important note is that there’s still a big gap between the ideal expected profits, which is linear in the amount of mining power one has, and even the lowest revenue publish or perish scheme at k = infinity.

So as good as publish or perish says it is, it still doesn’t reach Bitcoin’s intended goal of having hash power being directly proportional to mining reward.

As it stands, selfish mining is still an effective way to gain more profit, given that you have a significant amount of hash power, but with publish or perish it’s more difficult to pull off.

Publish or perish actually has a lot of limitations too.

First off, it assumes synchrony, and Bitcoin is asynchronous.

It’s not useful to define an upper bound on block propagation time because you can never be sure if all the blocks at a certain block height have been delivered or not.

This is something inherent in large distributed systems like Bitcoin, and we’ll be exploring this in detail in our second course.

For example, for any fail-safe parameter k > 1, an attacker could broadcast their blocks right before they are late.

This would cause inconsistent views among honest miners, since some would see the blocks as in time, and some would see them as late, depending on how long it took these blocks to propagate across the network.

It’s not just publish or perish though.

For all defenses or upgrades or proposals that assume Bitcoin is synchronous, everything just falls apart, because you can’t realistically make these assumptions about the network, especially since Bitcoin is publish and open to anyone.

Also, since publish or perish would be rolled out as a software update to bitcoin miners, not all miners would update their software at the same time.

During the transition period from Bitcoin’s default length FRP to the weighted FRP, attackers could potentially leverage the situation and launch double-spend attacks on confused users.

Another limitation of publish or perish is that it neglects some real world factors.

Due to network latency, there are often naturally occurring forks, but there are not permitted in this model.

It doesn’t consider transaction fees as an additional incentive on top of block rewards.

And it also doesn’t consider when there are multiple selfish miners, and how they would interact with each other and with the rest of the network.

In the end, publish or perish doesn’t achieve true incentive compatibility.

However, analyzing publish or perish and all the other works we’ve looked through is a very good start in designing better defenses for Bitcoin.

Maybe porting over publish or perish into an asynchronous model is a good direction for how we can approach defending against selfish miners.

Taking a step back from analyzing individual defenses, it’s important to understand the actual Bitcoin network itself.

After all, as we saw with publish or perish, which sounded really awesome in theory, it ultimately fails to work in practice due to assumptions about certain properties about the Bitcoin network -- particularly that of synchrony.

First off, the Bitcoin network is peer to peer, and there’s no central entity sending and receiving messages for us.

The way that messages get sent around the network is through a gossip protocol, which also called flooding.

If i wanted to send out a transaction I have to tell all the nodes I’m connected to, which we call my neighbors.

All my neighbors tell their neighbors, and their neighbors tell their neighbors, and so on.

For example, if the network actually looked like this…

...and I wanted to send a message to Nick…...I would tell my neighbors Derrick and Gloria.

Derrick and Gloria would then tell their neighbors, and thankfully Derrick is connected to Nick, so Nick gets my message.

Also, one note is that as we discussed in the Bitcoin mechanics module, my message would be digitally signed.

The way you get connected and join the Bitcoin network in the first place is by a recursive procedure.

Hardcoded into the Bitcoin core software is a list of seed peers that you connect to initially.

You ask these seed nodes for their neighbors, pick some, ask these nodes for their neighbors, and repeat this until you think you have a fairly random set of connections.

Bitcoin allows each node to have a maximum of 125 connections, and generally you have 8 outbound connections and 117 inbound connections.

To further understand how information propagates through the Bitcoin network, we have to understand the network topology and latency.

Network topology means how the Bitcoin network looks like if you were to graph it out like we’ve been doing with circles -- for individual nodes -- and lines -- for connections between nodes.

In Bitcoin, we want an even topology.

We want each node to have roughly the same weight, mining power in our case, have random connections, and look fairly uniform.

But this isn’t really the case.

As we know, there’s a huge difference in mining power node to node.

This means that some nodes are more influential than others.

Is the Bitcoin network truly as decentralized and distributed as we thought?

Nope.

Let’s say Derrick has an incredibly high hash rate…

It could be that he’s running a mining farm or pool.

We can’t really tell, since mining farms and pools sometimes want to remain secret so people don’t attack them.

Derrick is arguably more influential than other nodes in the network.

Compare his hash power with someone who’s just an SPV node hanging out in the network, or a node that’s just a solo miner.

Derrick’s pool a lot more hash power than any of these.

And there’s no easy way of determining the network topology.

If Derrick is running a mining pool, his miners don’t have to be directly connected on the Bitcoin network at all.

They could just be connecting to Derrick’s node directly through some lightweight protocol.

So all of Derrick’s miners form a secret subgraph essentially.

And it turns out that due to this hidden graph topology, and the fact that Derrick could be multiple nodes on the network, possibly running multiple pools, Derrick could potentially hide the fact that he has more than 400px; of the network hash power.

And scarier yet, Derrick doesn’t even need 400px; of the network hash power.

As we saw with the attacks and defenses previously, Bitcoin suffers from some incentive alignment problems, especially in the case of selfish mining.

As it turns out, the Bitcoin network could be in jeopardy if Derrick’s pool -- or pools -- had 33% of the network hash rate.

Or 25%... or 23.2%....or 32%....

Perhaps there’s an attack or selfish mining scheme that hasn’t been discovered yet that lowers the profit threshold even lower.

In addition to network topology, we also have to worry about the network latency.

As we saw in publish or perish, a selfish miner could publish his secret blocks right before it’s too late.

Some nodes would see it as in time, and others would see it as late.

And that’s a pretty good example for how network latency is an issue here.

Also, consider the case where two miners find a valid block at the same time.

There’s a race to propagate like before, and the miner who is better connected in the network and has a faster propagation time would ultimately win.

This leads to disproportionate profits in miners.

Miners who are better connected in the network see things faster and are able to send out messages and blocks faster too.

Meanwhile, it’s possible that a poorly connected miner could spend all this time finding a block, and when it finally finds one, its propagation time could be so slow due to network latency that within that time, another miner would have already found and submitted a competing block to a majority of the network.

And now for something even bigger.

Think back to the first module, where we discussed Sybil attacks and how it’s not beneficial to have more identities because in Proof-of-Work, we make computational power the limiting factor.

Well, if we’re clever enough, we can launch a Sybil attack in another kind of way, leveraging what we just went over: network topology and latency.

Let’s say Derrick has the ability to flood the network with nodes.

These could either be his miners, or if he’s clever enough, he could even write malware that make victims’ computers act like 0-power nodes -- just sitting there listening to the Bitcoin network, and relaying information back to Derrick.

They’d act like sensors.

Derrick could sybil attack honest miners by leveraging his network level dominance.

Let’s say Derrick is also selfish mining, and already has some secret blocks, which are distributed to all of his 0-power nodes.

When Derricks nodes hear about the next honest block X, Derrick could ignore X, and publish his secret block P.

If block P reaches a miner before block X, then the miner by default mines on block P.

In this attack, Derrick leverages both the network latency and topology.

Derrick is well connected and has less latency, and is able to propagate his secret block P faster than if he only had control of one node.

He also has unique ownership of the network topology, since he owns a lot of nodes around the network, and also since he has a large proportion of the network mining power.

As a closing note, know that outside of what we presented in this module, there are a lot more attacks and defenses, such as eclipse attacks and stubborn mining, which take into account network latency, topology, and more.

Author: Rea Savla 

Miners can maximize profits via pool hopping. This occurs when miners switch between Pay-per-Share and Proportional scheme mining pools to whichever payment protocol produces a higher rewards per additional share. In this scenario, honest and loyal pool miners will be cheated out of their profits by miners who pool hop and cause inconsistency in the pool’s hash rate.

In this attack, pool miners distribute a small percent of mining power equally among other pools, without ever submitting valid blocks. In doing so, we increase personal profits to the detriment of our mining pools. Pool Cannibalization is very difficult to detect, unless statistically significant. Since it is more profitable to be dishonest than honest, miners are incentivized to spend mining power cannibalizing each other rather than increasing the useful hashrate for finding the next valid block.

If we model the choice between attacking and not attacking the Bitcoin network using game theory, we see that the dominant strategy is for each miner to attack the network, leading to the ultimate detriment of Bitcoin.

If you, the attacker, have any more than 50 percent of network hashrate, then you will always be able to double spend because you will always be able to create the longest chain, and since you would be in full control of which blocks are included within that chain. Since individuals on the network would just leave the network when they see a 51 percent attack, and subsequently devalue bitcoin, conducting such an attack is too risky.

The Goldfinger Attack allows attackers to still profit off of the destruction of Bitcoin. Attackers can “short,” or place a bet on the devaluation of bitcoin.

In addition to attacking Bitcoin by leveraging hashpower, you can also attack the network via censorship. With censorship, we can choose to ignore the transactions from an individual or group of individuals, isolating them from the network and effectively rendering their bitcoins useless.

A mining pool with more than 51 percent of the network hashrate can decide not to work on a chain containing transactions spending from a particular address. Other miners will see that if they include a transaction with that address, their block will not be included in the chain, since the mining pool with the majority hashrate can fork and create a longer proof-of-work chain. Therefore, they will be incentivized to exclude the transactions with that address from their chain. This is called punitive forking.  

Even without 51 percent of the network hashrate, you can censor another address’ transactions using feather forking. In feather forking, the attacker announces they will attempt to fork if they see a block with a certain address’ transactions, but will give up after kconfirmations. Even though the attacker has a low chance of orphaning, or excluding, that block, miners will ignore blocks with the given address. This happens because miners are incentivized primarily by the profit they expect to earn from mining a particular block. Even if there is a small chance of the attacker forking a block with the given address, the math works out such that the expected profit of ignoring that transaction will exceed that of including it. Thus the victim has to pay a much higher transaction fee to incentivize miners to include their transaction.

Upon being the first to find a valid block, you can withhold that block from the network and continue to work on it privately. If you find two blocks on your secret chain before the network finds the next one, then you suddenly have the longest chain. This means you get at least two block rewards and you have fooled the network into working on an honest chain that you will render useless. You can keep doing this and submit your longer, secret chain to the network right before you think the network is going to catch up to you. This is selfish mining.

 We discussed several theorized defenses to selfish mining including

Block Validation Using Time Signatures Opens in new window, proposed by Schultz (2015) and Solat and Potop-Butucaru (2016)

Fork-punishment rule Opens in new window, proposed by Lear Bahack (2013)

Uniform Tie-Breaking Rule Opens in new window, proposed Eyal and Sirer (2014)

Publish or Perish Opens in new window, proposed by Zhang and Preneel (2017).

However, according to Vitalik Buterin, "in practice, most Bitcoin miners act altruistically to support the network, both out of ideological considerations and because they do not want to destabilize the source of their own revenue. Such higher-level economic concerns are beyond the scope of Eyal and Sirer’s paper, but they seriously reduce the chance that this economic attack will work in practice" (Bitcoin Magazine Opens in new window, 2013).

In addition to individual attacks, the architecture of the Bitcoin network itself contains vulnerabilities. The Bitcoin protocol is P2P, in which messages get sent via a gossip protocol, where each node passes a message to its connected nodes. Though we would like an even topology, this is not the case. Nodes with higher hashrate, perhaps from running mining pools, may have miners connected to them directly on a lightweight protocol, or a secret subgraph. Due to this hidden graph topology, a user could potentially hide that it has more than 400px; of the network hash power.

In addition to network topology, network latency also poses as an issue. Nodes that are better connected to the network see things faster and can send out messages and blocks to the network more quickly. This leads to disproportionate profits in miners.

Selfish Mining: A 25% Attack Against the Bitcoin Network Opens in new window

"Ethereum Whitepaper up to "Miscellanea And Concerns" 

This module will be the culmination of the last 5 modules.

From here, you will have fully understood all the tenets of how we build a blockchain system, from the conceptual level to the implementation.

Our first module examined the basics of Bitcoin, building it from the ground up with four key components: identity, transactions, record-keeping, and consensus.

WIth this high level understanding of Bitcoin, you learned the motivations for Bitcoin’s design, how Bitcoin provides a previously centralized service in a decentralized manner, and were then able to explain Bitcoin in simple terms.

In our second module, we took a step back to look at Bitcoin and blockchain’s historical context, to get a sense for libertarian ideals and the Cypherpunk movement, why blockchains are designed as they are, and the path that current efforts are taking us now.

We then dove into the technical implementation of Bitcoin in our third module, which used tools such as cryptographic hash functions, tamper evident data structures, the elliptic curve digital signature algorithm, and Bitcoin Script.

Through this lecture, you saw features of Bitcoin, such as Merkle trees and nonces, implementing every concept from Module 1.

In the fourth module, we stepped out of the world of hypotheticals and theory and looked at how all these concepts played out in real life.

We examined all the various types of users of Bitcoin and discussed tools to interact with the network, such as wallets and various categories of nodes.

The lecture brought Bitcoin to life, allowing you to decide for yourself how well Satoshi Nakamoto met their mission with Bitcoin.

And then finally, last module, we looked at the vulnerabilities of Bitcoin from a game theoretical standpoint.

Recognizing the costs and implications of both malicious Proof-of-Work attacks, and that of network latency and topology, we saw for ourselves Bitcoin’s imperfections.

By completing these 5 lectures, you have now completed your understanding of Bitcoin fundamentals.

This is only the beginning of your blockchain knowledge.

With these accumulated sets of knowledge, we are now going to take you from Bitcoin-centric material into a new frame of blockchain thinking, leveraging all the blockchain knowledge you accumulated while learning about Bitcoin.

In this module, we’ll be looking at Ethereum and blockchain use cases.

In the past, we talked about how Bitcoin enabled users to conduct transactions that were decentralized and trustless.

A more general case of this is the execution of arbitrary computations on the blockchain, and that’s exactly what smart contracts enable us to do in Ethereum.

We’ve mentioned this before very briefly, saying that Ethereum allows the creation of decentralized apps, otherwise known as dapps.

In this section, we’ll take a look at what smart contracts are at a high level, and why we might want them.

Then, we’ll look at what we need in a framework that allows us to run smart contracts.

As we did with Bitcoin, we’ll first look at the high level features that we want to see in Ethereum -- mainly the support of smart contracts -- and then see how they’re implemented at a lower level.

First, we have to consider what makes Bitcoin so special.

Understanding what Bitcoin has to offer allows us better to understand Ethereum’s own value proposition as a smart contract platform.

Remember in module 1, we built Bitcoin from the ground up, and emphasized the importance of four key components: identity, transactions, record keeping, and consensus.

And it’s the careful design and combination of these components that really differentiated Bitcoin from anything else at the time.

It turns out that these ideas are much more fundamental than Bitcoin itself, and will help us understand the key components of Ethereum.

Let’s take some time to review some of what makes Bitcoin so special.

First off, when we built Bitcoin from the ground up, we first emphasized the importance of identity: especially in order to enable authentication and integrity.

We want everyone to control a unique identity, and we also wanted Bitcoin to be pseudonymous.

We used ECDSA, the elliptic curve digital signature signature algorithm.

Bonus points to anyone who remembers the specific name of the curve.

Users generate their private key at random, or through some hard to guess manner such as a hashed and key stretched brain wallet mnemonic.

And then they use one way functions to generate a public-facing address.

Also remember that Bitcoin is secure since all the numbers we’re dealing with are massive.

Think back to the grains of sand on earth example in module 1.

Of course, the main idea of Bitcoin was to be able to send transactions between users, so naturally, we have a way to do this.

However, the way we did this with Bitcoin was with UTXOs, which might’ve been not that intuitive at first.

UTXOs made it easier to enable higher degrees of privacy, for example, if you sent all your change UTXOs to different addresses under your control.

On the other hand, to calculate your entire balance in bitcoin, you would have to sum up all your UTXOs.

You’d also potentially have to reference more than one UTXO when making a transaction, and in some cases this isn’t that desirable.

Still, implementing UTXOs was the innovation that enabled transactions in Bitcoin, and it works well despite the tradeoffs it makes.

Next, of course, we have the blockchain: the famous data structure that enables record keeping in the network.

Every full node in the network has a copy of the blockchain.

We did this to avoid centralization of power: to avoid having a central bank, we made everyone the bank.

And the blockchain is constructed in a tamper evident manner too.

Each block in the blockchain refers to the hash of the previous block’s header.

And also, within each block, we construct a merkle root, which is a tamper evident “summary” of all the transactions in that block.

Any time a previous transaction is altered in the blockchain that changes the transaction’s hash, which bubbles up to change the merkle root, which then changes the block header hash.

This then invalidates the block, since all blocks refer to the previous block hash

With the blockchain, we have an efficient way to not only store data, but also to make sure that the data is tamper evident: and this is crucial especially since we’re deploying this for the public, where we can’t expect everyone to be honest.

And finally, we have Proof-of-Work consensus.

Nodes on the Bitcoin network needed a way to come to consensus on updates to the blockchain, so we implemented a voting system.

However, it wasn’t as easy as assigning each user a single vote, since it’s so easy to generate new identities.

We saw the need to assign weights to our votes.

The innovation here was to have everyone solve a cryptographic hash puzzle, incentivized by block reward.

And whoever solved the cryptographic hash puzzle first, and provided proof that they solved it correctly, would propose the new update to the blockchain.

And the more compute power you had, the more likely you’d solve the hash puzzle first and get to propose the new block.

Voting power is limited by compute power -- a physical limitation -- rather than by the number of identities a user holds.

And by carefully designing Bitcoin like that, we have all these nice benefits:

Bitcoin is pseudonymous.

Cryptographic identities allow for integrity and authentication, meaning also that we have accountability -- for example if someone tries to double spend or do something malicious, we can catch that.

Bitcoin is democratic, in that decisions made through Proof-of-Work consensus don’t require you to trust anybody else on the network.

You just have to trust the math behind consensus, and that there’s an honest majority on the network, so the network proceeds in a healthy manner.

Also, we can see that the blockchain is an immutable ledger of truth.

Everyone sees the same version of the truth since we have consensus.

And we also have tamper evident data structures in place so that no one can just go in and change history.

Bitcoin was also designed to be uncensorable.

To censor transactions, you would have to control a large proportion of the network, and for any one party, this would be considerably difficult.

Finally, Bitcoin is distributed.

There’s no central point of failure.

Instead, the execution of transactions depends on a network of miners located around the world.

All these properties seem pretty nice, and it would be awesome if we could include them in our smart contract platform.

We’ll keep Bitcoin’s architecture in mind.

Now that we have these design considerations, we can take a look at smart contracts, and how they fall into this whole space.

Taking a step back, we can look at the definition of the word “contract”.

A contract is a written or spoken agreement that is intended to be enforced by law.

Notice specifically that a contract must be agreed upon, and that it is enforceable by law.

Just by these words alone, it sounds like we need some sort of consensus going on in our smart contract system.

And through consensus, we should be able to agree on both the contents of the contract, and also the execution of the contract.

Alright, so now that we know what a contract is, now the question is...what makes a smart contract so “smart?”

A smart contract is a piece of code that facilitates, verifies, or enforces the negotiation or execution of a digital contract.

For us to reach consensus, a trusted entity must run this code.

After all, we need to trust that a digital contract is enforced correctly.

Like a traditional contract, it carries a set of conditions that must be fulfilled, or terms that must be executed on.

The difference is that the execution and enforcement is done through carefully designed algorithms, not through law.

Looking first at what makes Bitcoin special, and the underlying architecture that makes it the way it is, we saw how we could keep bitcoins design philosophy in mind as we sought ways to design a smart contract platform.

In the next sections, we’ll take a look at Ethereum.

We’ve mentioned it throughout the course thus far.

From what we know so far, Ethereum is a technology upon which people have built countless applications, ranging from decentralized autonomous organizations to online markets for digital kittens.

But now we can finally ask, what exactly is Ethereum and how does it work?

As it turns out, leveraging our understanding of Bitcoin’s inner workings makes understanding Ethereum a whole lot more accessible.

Just like Bitcoin, Ethereum is a platform that’s constantly changing.

It’s under active development by not only the official Ethereum Foundation team, but also by its large open source community.

Over the years, it’s been shaped by current events and its community.

And the software improvements and proposals over the years are, like in Bitcoin, a testament to the large user base and also the democratic nature of the software.

As more and more people begin to use Ethereum to build applications, Ethereum needs to adapt more and more.

On their website, it says Ethereum is a decentralized platform designed to run smart contracts.

In short, it’s a distributed computer spread amongst a multitude of nodes across the world that executes code that people feed to it.

And because of this architecture, it can run unstoppable applications.

Applications are run exactly as programmed, without any possibility of downtime, censorship, fraud, or third-party intervention. It uses a blockchain, and its blockchain is account-based, instead of UTXO-based like Bitcoin.

We’ll into more detail about this in a bit.

Formally, Ethereum is implemented as a distributed state machine, and transactions on the network change the global state of the system.

Nodes in Ethereum keep track of and come to consensus on the global system state – which includes data about who owns what.

Execution of transactions bring us from a previous state to a new state, so they’re the state transition functions.

Ethereum’s native asset is ether, which is the basis of value in the Ethereum ecosystem, and is also crucial in aligning incentives.

As a recap from the previous section, what makes Bitcoin so special are the following properties: it’s trustless, immutable, uncensorable, pseudonymous, has no central point of failure, and aims for a one-cpu-one-vote policy.

Ethereum has all of these, but has a different use case.

Before the creation of Ethereum, Vitalik Buterin, the creator of Ethereum had argued that Bitcoin needed a more general purpose scripting language, but he didn’t get much support for his proposal.

So, he made Ethereum.

And one of the most notable features is that it supports a turing complete scripting language.

Ethereum and Bitcoin are the two most popular blockchain platforms in the world today, but have two different goals.

Bitcoin is the “gold standard” of blockchains.

It’s been around for the longest, and its protocol has successfully supported an enormous number of transactions over the past years.

It’s intended purpose is solely to allow the transaction of bitcoins, its native asset.

On the other hand, Ethereum is a smart contract blockchain platform, a distributed world computer.

It’s native asset ether exists to fund computation and to align incentives.

Its primary purpose is not to act as a medium of exchanging value.

Bitcoin is simple and robust.

It’s a global payment system.

That’s all it really sets out to do.

Ethereum has a much larger vision, and therefore supports many more features, for example, a much more powerful scripting language.

Bitcoin has a stack based primitive scripting language which is not turing complete, meaning that the applications we can make on Bitcoin are pretty limited.

Ethereum on the other hand has a Turing-complete scripting language -- one of the motivating factors for Ethereum’s creation in the first place.

So, a wider variety of applications can be made on Ethereum than on Bitcoin, making it more developer friendly.

Another key difference is that Bitcoin is UTXO-based, whereas Ethereum is account based.

In Bitcoin, private keys prove ownership of UTXOs.

When you spend bitcoin, you spend from previous transactions.

And to calculate your current balance, you have to sum up all the UTXOs that you own.

The reason Bitcoin uses UTXOs is that they make it easy to make transactions and prevent double spending.

Think back to the piggy bank analogy from module 1.

In Ethereum, private keys prove ownership of an account, which tracks a current balance.

Accounts are more space efficient than UTXOs, since to calculate your balance, you only have to reference your account, rather than summing across all your UTXOs.

Also, since our ultimate goal is to support smart contracts, it’s much cheaper and easier to look up an account balance and also transfer between accounts when we have an acc