woofpool / cardano-private-testnet-setup Goto Github PK

Setup guide to create private Cardano testnet and run sample transactions including a Plutus contract example. This environment can be used for local development and learning.

License: MIT License

Shell 87.56% Haskell 12.44%

cardano testnet db-sync

cardano-private-testnet-setup's Introduction

Cardano Private-Testnet / Devnet Setup

This project provides instructions and shell scripts to bootstrap a private Cardano testnet and connect a cardano-db-sync process to it. If you don't want to bother with setting up cardano-db-sync, you can easily skip over the sections of this project that are not relevant.

Note: If you want to run a private testnet in the babbage era, you can check out the babbage branch and follow these instructions instead. The babbage instructions do not include instructions for running smart contract transactions yet, but the network will support the new Plutus V2.

The scripts used by this project to create the private Cardano testnet are taken from the IOHK cardano-node project and have been modified as needed. You may find the original script files in the IOHK git repository: cardano-node scripts. In particular, running cardano-db-sync to sync to the private testnet required a few changes to the original scripts provided by IOHK.

Hopefully, this documentation provides a lot of value for others. I welcome your feedback both good and bad!

Minimum system requirements

Operating System : documentation is based on Ubuntu (Debian) linux, but can be adapted to other flavors of linux, as well as Darwin MacOS
- The IOG scripts used to start node processes should be compatible with any linux variant and Darwin MacOS
Processor : In the mkfiles script, each cardano-node is configured to run 4 CPU cores in parallel. This configuration may be changed as necessary to fit your system constraints. Search for cardano-node run in mkfiles.sh to modify.
RAM : Each node is using about 1/3 GB of RAM, which gives 1 GB of RAM for 3 nodes total.
Disk space : The amount of disk space used to run the network will grow the longer you are running. For the author, after running for a full day, each node used 1.8 GB of disk space, which gives 5.4 GB for 3 nodes total. Depending on your system capacity, one could kill the node processes, delete the persisted state, and initialize a fresh network as necessary.
Recommended : Overall, a 4 core CPU with 2+ GB of available RAM and 10 GB of disk space is sufficient and provides some excess capacity.

Why is this useful?

A private Cardano testnet provides a controlled environment to execute transactions
With cardano-db-sync connected to your private testnet, you can use SQL queries to view blockchain activity data.
You can use this controlled environment for local Cardano development.
Lastly, it is a great learning experience to set up your own private testnet and learn about the data stored on the blockchain.

Key Details

The private testnet consists of three block-producing node processes.
The cardano-db-sync process syncs blockchain data to a highly normalized database schema. This enables blockchain data to be queried with SQL.
The example transactions include setting up keys and wallet address for new user, transfering ADA from one user wallet address to another and executing a Plutus script for a vesting contract.

Medium article

For an additional overview of this project, please check out this medium article

Usage Instructions

Install Cardano executables
- Install the following executables: cardano-node, cardano-cli, and optionally cardano-db-sync
- Please refer to the Install executables guide for instructions.
Optional: Install PostgreSQL packages and create Postgres user
- The cardano-db-sync process uses a connection to a PostgreSQL database.
- Please refer to the Install posgreSQL for instructions to set up.
Run scripts to set up & run private Cardano network
- Run scripts to bootstrap the Cardano private network
- Please refer to the Run network scripts guide for instructions.
Optional: Attach DB Sync process the network
- Attach the cardano-db-sync process to the network, which syncs blockchain data to a PostgreSQL database.
- Please refer to the Attach db-sync guide for instructions.
Run simple transaction and optionally query the db-sync database to see results
- Set up a new wallet for user2 and make a payment from user1 to user2. Query the database to confirm the transaction.
- Please refer to the Run transaction guide for instructions.
Run Plutus script transactions
- Build the project code for a simple vesting script, which sets User2 as the beneficiary with a vesting deadline
- Set up transaction to give ADA from User1 to the script address and a transaction to grab ADA from the script by User2
- Please refer to the Run Plutus Script transactions guide for instructions.

Contributors

This project is provided free of charge to the Cardano community. The author of this project is a fan of Cardano, as well as a Cardano stake pool operator. I am not affiliated with IOHK in any official capacity.

If you want to support the continued development of this project, you can delegate or recommend my staking pool:

WOOF Cardano Staking Pool

Contributing

If you'd like to help maintain this project, please feel free to submit a pull request. Please ensure that any script changes have been tested and verified.

License

This project is licensed under the terms of the MIT License.

cardano-private-testnet-setup's People

Contributors

Stargazers

Watchers

cardano-private-testnet-setup's Issues

node config errors

Up to step number 4 I got the following error:

hud@hud:~/src/cardano-private-testnet-setup$ ./scripts/db-sync-start.sh
[db-sync-node:Info:6] [2022-02-02 00:27:17.80 UTC] Schema migration files validated
[db-sync-node:Info:6] [2022-02-02 00:27:17.80 UTC] Running database migrations
[db-sync-node:Info:6] [2022-02-02 00:27:17.80 UTC] Enviroment variable DbSyncAbortOnPanic: False
ExitFailure 2

Errors in file: /tmp/migrate-2022-02-02T002717.log

hud@hud:~/src/cardano-private-testnet-setup$ cat /tmp/migrate-2022-02-02T002717.log
Running : migration-1-0000-20190730.sql
WARNING: password file "postgres-conn/pgpass-privatenet" has group or world access; permissions should be u=rw (0600) or less
psql: FATAL:  database "privatenet" does not exist
ExitFailure 2

After running the possibly missing instructions sudo chmod 700 -R postgres-conn/pgpass-privatenet and createdb privatenet I get the following errors:

hud@hud-PS63:~/src/cardano-private-testnet-setup$ ./scripts/db-sync-start.sh
[db-sync-node:Info:6] [2022-02-02 13:49:06.98 UTC] Schema migration files validated
[db-sync-node:Info:6] [2022-02-02 13:49:06.98 UTC] Running database migrations
[db-sync-node:Info:6] [2022-02-02 13:49:06.98 UTC] Enviroment variable DbSyncAbortOnPanic: False
[db-sync-node:Info:6] [2022-02-02 13:49:09.36 UTC] Using byron genesis file from: "private-testnet/./byron/genesis.json"
[db-sync-node:Info:6] [2022-02-02 13:49:09.36 UTC] Using shelley genesis file from: "private-testnet/./shelley/genesis.json"
[db-sync-node:Info:6] [2022-02-02 13:49:09.36 UTC] Using alonzo genesis file from: "private-testnet/./shelley/genesis.alonzo.json"
[db-sync-node:Info:6] [2022-02-02 13:49:09.36 UTC] NetworkMagic: 42
[db-sync-node.db-sync:Debug:6] [2022-02-02 13:49:09.36 UTC] SELECT "block"."id"
FROM "block"
WHERE "block"."hash" = ?
; [PersistByteString "]\254\ENQ\139\133Q\SUB\254^\150\&6\250\DEL\162\149\163!\196\158\229W@\152\t\SI\242\133|\245\230\203c"]
[db-sync-node:Info:6] [2022-02-02 13:49:09.36 UTC] Inserting Byron Genesis distribution
[db-sync-node.db-sync:Debug:6] [2022-02-02 13:49:09.36 UTC] SELECT COUNT(*)
FROM "block"
; []
[db-sync-node.db-sync:Debug:6] [2022-02-02 13:49:09.36 UTC] INSERT INTO meta ("start_time", "network_name", "version") VALUES (?, ?, ?) ON CONFLICT ON CONSTRAINT unique_meta DO UPDATE SET "start_time" = EXCLUDED."start_time" RETURNING id ;; [PersistUTCTime 2022-02-01 23:43:36 UTC,PersistText "privatenet",PersistText "Version {versionBranch = [12,0,0], versionTags = []}"]
[db-sync-node.db-sync:Debug:6] [2022-02-02 13:49:09.36 UTC] INSERT INTO slot_leader ("hash", "pool_hash_id", "description") VALUES (?, ?, ?) ON CONFLICT ON CONSTRAINT unique_slot_leader DO UPDATE SET "hash" = EXCLUDED."hash" RETURNING id ;; [PersistByteString "]\254\ENQ\139\133Q\SUB\254^\150\&6\250\DEL\162\149\163!\196\158\229W@\152\t\SI\242\133|",PersistNull,PersistText "Genesis slot leader"]
[db-sync-node.db-sync:Debug:6] [2022-02-02 13:49:09.36 UTC] INSERT INTO "block"("hash","epoch_no","slot_no","epoch_slot_no","block_no","previous_id","slot_leader_id","size","time","tx_count","proto_major","proto_minor","vrf_key","op_cert","op_cert_counter") VALUES(?,?,?,?,?,?,?,?,?,?,?,?,?,?,?) RETURNING "id"; [PersistByteString "]\254\ENQ\139\133Q\SUB\254^\150\&6\250\DEL\162\149\163!\196\158\229W@\152\t\SI\242\133|\245\230\203c",PersistNull,PersistNull,PersistNull,PersistNull,PersistNull,PersistInt64 1,PersistInt64 0,PersistUTCTime 2022-02-01 23:43:36 UTC,PersistInt64 2,PersistInt64 0,PersistInt64 0,PersistNull,PersistNull,PersistNull]
[db-sync-node.db-sync:Debug:6] [2022-02-02 13:49:09.36 UTC] INSERT INTO "tx"("hash","block_id","block_index","out_sum","fee","deposit","size","invalid_before","invalid_hereafter","valid_contract","script_size") VALUES(?,?,?,?,?,?,?,?,?,?,?) RETURNING "id"; [PersistByteString "\200\170\198\230\171~Zk\DC2\226\180\158\247n\178:\US\155\234\181V\155\163\198\FS\229\162\185\248\DC1/\151",PersistInt64 1,PersistInt64 0,PersistText "503000000",PersistText "0",PersistInt64 0,PersistInt64 0,PersistNull,PersistNull,PersistBool True,PersistInt64 0]
[db-sync-node.db-sync:Debug:6] [2022-02-02 13:49:09.36 UTC] INSERT INTO "tx_out"("tx_id","index","address","address_raw","address_has_script","payment_cred","stake_address_id","value","data_hash") VALUES(?,?,?,?,?,?,?,?,?) RETURNING "id"; [PersistInt64 1,PersistInt64 0,PersistText "2657WMsDfac5e1VB4yBWoCXxBVvED7Yd1Yy2MbqbFecptEsiS1huUb6kg9jNU5izo",PersistByteString "\130\216\CANX%\131X\FS>n\176\248\236\236\184\rL\b\193\&4\rr\227\152&Ih\179\213\142\233\175Z\194\137\233\161\STXB\CAN*\NUL\SUB\201\&8\n\204",PersistBool False,PersistNull,PersistNull,PersistText "503000000",PersistNull]
[db-sync-node.db-sync:Debug:6] [2022-02-02 13:49:09.36 UTC] INSERT INTO "tx"("hash","block_id","block_index","out_sum","fee","deposit","size","invalid_before","invalid_hereafter","valid_contract","script_size") VALUES(?,?,?,?,?,?,?,?,?,?,?) RETURNING "id"; [PersistByteString "\185\171\131[t\210\201k\236>2\t\219\204\ETX\170\218bi\191w\219o\143\182\DC3S,M\172\145Q",PersistInt64 1,PersistInt64 0,PersistText "503000000",PersistText "0",PersistInt64 0,PersistInt64 0,PersistNull,PersistNull,PersistBool True,PersistInt64 0]
[db-sync-node.db-sync:Debug:6] [2022-02-02 13:49:09.36 UTC] INSERT INTO "tx_out"("tx_id","index","address","address_raw","address_has_script","payment_cred","stake_address_id","value","data_hash") VALUES(?,?,?,?,?,?,?,?,?) RETURNING "id"; [PersistInt64 2,PersistInt64 0,PersistText "2657WMsDfac72SVer6mTMq5GCoFHAKPSvVE89ZoV18HgfiiDRkpJa74YgcMrJbnW5",PersistByteString "\130\216\CANX%\131X\FS\185\158O\168\194P~\200^\136\230\&5\\sU\SO\244/Tf\f\EMO\vh4\187\233\161\STXB\CAN*\NUL\SUBV\183\182j",PersistBool False,PersistNull,PersistNull,PersistText "503000000",PersistNull]
[db-sync-node:Info:6] [2022-02-02 13:49:09.36 UTC] Initial genesis distribution populated. Hash 5dfe058b85511afe5e9636fa7fa295a321c49ee5574098090ff2857cf5e6cb63
[db-sync-node.db-sync:Debug:6] [2022-02-02 13:49:09.36 UTC] SELECT SUM("tx_out"."value")
FROM "tx_out"
WHERE NOT EXISTS (SELECT ?
FROM "tx_in"
WHERE ("tx_out"."tx_id" = "tx_in"."tx_out_id") AND ("tx_out"."index" = "tx_in"."tx_out_index")
)
; [PersistBool True]
[db-sync-node:Info:6] [2022-02-02 13:49:09.36 UTC] Total genesis supply of Ada: 1006.000000
Node configs that don't[db-sync-node:Error:6] [2022-02-02 13:49:09.37 UTC] Node configs that don't fork to Shelley directly and initiate funds or stakes in Shelley Genesis are not supported.
 fork to Shelley directly and initiate funds or stakes in Shelley Genesis are not supported.

I've also tried getting the latest cardano configs but its a mismatch:

Failed reading Byron genesis file "private-testnet/./byron/genesis.json": GenesisHashMismatch (GenesisHash {unGenesisHash = 96fceff972c2c06bd3bb5243c39215333be6d56aaf4823073dca31afe5038471}) 5dfe058b85511afe5e9636fa7fa295a321c49ee5574098090ff2857cf5e6cb63

Please advise.

Change Major Protocol Version to 6

Currently, the network is started with major protocol version 5, which was the version used by Alonzo with time handling broken. Time handling was fixed in late 2021, bumping the protocol version to 6, which is still used now (May 2022) on both the public testnet and the Cardano mainnet.

CI | integration-test the automate.sh script

Ops | Wrap the devnet with podman container

It will be docker compatible, but from a discussion I had with a long term docker/podman ops guy, podman tends to be a bit nicer experience ootb. Especially compared to how rogue for firewall rules a default docker installation can be.
Will investigate as we go.

Main features it has to provide:

export node1's socket via volume mapping
install stable devnet
come with some extras packed. ie. additional addresses (could be funded from user1) etc. TBD

Possible to use Ogmios and create a custom database instead of using Postgres?

Does Ogmios chain sync work with the private test net?

query private testnet

I am wondering if there's an alternative way to query the private testnet e.g. with blockfrost?

Solution to garbage collection/memory issues

After following the tutorial and being able to run everything successfully (thanks for this by the way), I tried closing the "automate.sh" shell to see if I would be able to get it running again later.

When I tried to execute ./run/all.sh, though, I started seeing a lot of errors and I noticed I was running into the same issue mentioned here, more specifically in this section:

Troubleshooting: If you run cardano-cli query tip and the blocks are not advancing or the syncProgress percent is decreasing, it may mean the processes running the nodes are running into garbage collection/memory issues. The author is still researching the cause of this issue. In any event, the best remedy is killing the run node processes, deleting the private-testnet folder and starting over. This garbage collection issue normally happens early in the update process.

After researching a little bit more, I started to think the problem could have something to do with the fact I was suddenly running all the nodes at the same time. So I tried, instead of ./run/all.sh, first ./run/node-bft1.sh and then ./run/node-bft2.sh, etc. To my surprise, it actually worked!

So I'm just submitting this issue, because it might be a good idea to update the "troubleshooting" section, warning users to run a node at a time, if there are no nodes running.

Instructions to stop the nodes

It would be nice to have a 7th point in the instructions list, about how to stop the nodes and to explain what happens in this case.
How to restart them ? Could them be restarted without losing the blockchain content or not ? etc.

Update scripts for Babbage

Hi all,

First, thanks for making these scripts, they have been invaluable for my development via Canonical LLC.

I would love to see them upgraded to support Babbage/Vasil.

Ops | PoC db-sync tests

no idea yet, trying to yolo something that may work 🤷🏾‍♂️

cardano-db-sync workaround

Following up on the cardano-db-sync workaround

Running cabal build all results in:

create a better error message. Original error message:                                                                                                                                                      
Could not resolve dependencies:                                                                                                                                                                             
[__0] trying: Win32-network-0.1.0.0 (user goal)                                                                                                                                                             
[__1] trying: base-4.14.3.0/installed-4.14.3.0 (dependency of Win32-network)                                                                                                                                
[__2] trying: byron-spec-chain-0.1.0.0 (user goal)                                                                                                                                                          
[__3] trying: microlens-th-0.4.3.10 (dependency of byron-spec-chain)                                                                                                                                        
[__4] trying: template-haskell-2.16.0.0/installed-2.16.0.0 (dependency of                                                                                                                                   
microlens-th)                                                                                                                                                                                               
[__5] trying: pretty-1.1.3.6/installed-1.1.3.6 (dependency of                                                                                                                                               
template-haskell)                                                                                                                                                                                           
[__6] trying: hedgehog-1.0.5 (dependency of byron-spec-chain)                                                                                                                                               
[__7] trying: time-1.9.3/installed-1.9.3 (dependency of hedgehog)                                                                                                                                           
[__8] trying: text-1.2.4.1/installed-1.2.4.1 (dependency of hedgehog)                                                                                                                                       
[__9] trying: cardano-api-1.31.0 (user goal)
[_10] trying: scientific-0.3.7.0 (dependency of cardano-api)
[_11] trying: aeson-1.5.6.0 (dependency of cardano-api)
[_12] trying: cardano-db-12.0.2 (user goal)
[_13] trying: postgresql-simple-0.6.4 (dependency of cardano-db)
[_14] trying: postgresql-libpq-0.9.4.3 (dependency of postgresql-simple)
[_15] rejecting: postgresql-libpq:-use-pkg-config (constraint from config
file, command line flag, or user target requires opposite flag selection)
[_15] rejecting: postgresql-libpq:+use-pkg-config (conflict: pkg-config
package libpq>=9.3, not found in the pkg-config database)
[_15] fail (backjumping, conflict set: postgresql-libpq,
postgresql-libpq:use-pkg-config)
After searching the rest of the dependency tree exhaustively, these were the
goals I've had most trouble fulfilling: postgresql-simple, base,
postgresql-libpq, aeson, text, time, postgresql-libpq:use-pkg-config,
scientific, cardano-db, cardano-api, hedgehog, pretty, template-haskell,
byron-spec-chain, microlens-th, Win32-network

Did you use cabal? Or something else like nix-build?

woofpool / cardano-private-testnet-setup Goto Github PK

cardano-private-testnet-setup's Introduction

Cardano Private-Testnet / Devnet Setup

Minimum system requirements

Why is this useful?

Key Details

Medium article

Usage Instructions

Contributors

Contributing

License

cardano-private-testnet-setup's People

Contributors

Stargazers

Watchers

Forkers

cardano-private-testnet-setup's Issues

Recommend Projects

Recommend Topics

Recommend Org