NineChronicles DataProvider
- NineChronicles.DataProvider is an off-chain service that stores NineChronicles game action data to a database mainly for game analysis.
- Currently, this service only supports
MySQLdatabase.
Table of Contents
- Pre-requisite
- Run
- Development Guide
- Current Table Descriptions
- Migrating Past Chain Data to MySQL Database
Pre-requisite
- MySQL and Entity Framework Core tools should be installed in the local machine.
Run
- Before running the program, please refer to the option values in the latest official 9c-launcher-config.json and fill out the variables in appsettings.json.
- In appsettings.json,
AppProtocolVersionToken,StorePath,PeerStrings,MySqlConnectionStringproperties MUST be filled to run the program. - To setup the NineChronicles mainnet blockchain store to use in the
StorePath, download and extract the latest mainnet snapshot to a desired location.
$ dotnet run --project ./NineChronicles.DataProvider.Executable/ --
Development Guide
- This section lays out the steps in how to log a new action in the database.
- The TransferAsset action is used as an example in this guide.
1. Setup Database
- To setup the database, navigate to NineChronicles.DataProvider/NineChronicles.DataProvider.Executable directory on your terminal and run the following migration command.
dotnet ef database update -- [Connection String]
- Connection String example: "server=localhost;database=data_provider;port=3306;uid=root;pwd=root;"
2. Create Model
- In NineChronicles.DataProvider/NineChronicles.DataProvider/Store/Models directory, create a model file called
TransferAssetModel.cs. - In general,
TxId,BlockIndex,Date, andTimestampare useful to add as default properties in a model because these values will help with query speed when table size increases.
namespace NineChronicles.DataProvider.Store.Models
{
using System;
using System.ComponentModel.DataAnnotations;
public class TransferAssetModel
{
[Key]
public string? TxId { get; set; }
public long BlockIndex { get; set; }
public string? Sender { get; set; }
public string? Recipient { get; set; }
public decimal Amount { get; set; }
public DateOnly Date { get; set; }
public DateTimeOffset TimeStamp { get; set; }
}
}
- In NineChronicles.DataProvider/NineChronicles.DataProvider/Store/NineChroniclesContext.cs, add a DbSet called
TransferAssetsand its description for reference.
// Table for storing TransferAsset actions
public DbSet<TransferAssetModel> TransferAssets => Set<TransferAssetModel>();
2. Create Store Method
- In NineChronicles.DataProvider/NineChronicles.DataProvider/Store/MySqlStore.cs, add a following method that stores the
TransferAssetdata into MySQL.
public void StoreTransferAsset(TransferAssetModel model)
{
using NineChroniclesContext ctx = _dbContextFactory.CreateDbContext();
TransferAssetModel? prevModel =
ctx.TransferAssets.FirstOrDefault(r => r.TxId == model.TxId);
if (prevModel is null)
{
ctx.TransferAssets.Add(model);
}
else
{
prevModel.BlockIndex = model.BlockIndex;
prevModel.Sender = model.Sender;
prevModel.Recipient = model.Recipient;
prevModel.Amount = model..Amount;
prevModel.Date = model.Date;
prevModel.TimeStamp = model.TimeStamp;
ctx.TransferAssets.Update(prevModel);
}
ctx.SaveChanges();
}
3. (Optional) Add Data Getter
In some cases, you need to handle state to get data to make model.
To do this easily, you can make your own data getter inside NineChronicles.DataProvider/DataRendering/.
4. Render & Store Action Data
- In NineChronicles.DataProvider/NineChronicles.DataProvider/RenderSubscriber.cs, add a following render code
_actionRenderer.EveryRender<TransferAsset>()
.Subscribe(ev =>
{
try
{
if (ev.Exception is null && ev.Action is { } transferAsset)
{
var model = new TransferAssetModel()
{
TxId = transferAsset.TxId,
BlockIndex = transferAsset.BlockIndex,
Sender = transferAsset.Sender,
Recipient = transferAsset.Recipient,
Amount = Convert.ToDecimal(transferAsset.Amount.GetQuantityString()),
Date = DateOnly.FromDateTime(_blockTimeOffset.DateTime),
TimeStamp = _blockTimeOffset,
};
MySqlStore.StoreTransferAsset(model);
}
}
catch (Exception e)
{
Console.WriteLine(e);
}
5. Add Database Migration
- Navigate to NineChronicles.DataProvider/NineChronicles.DataProvider.Executable directory on your terminal and run the following migration command
dotnet ef migrations add AddTransferAsset -- [Connection String]
- Connection String example: "server=localhost;database=data_provider;port=3306;uid=root;pwd=root;"
Current Table Descriptions
- Tables that
NineChronicles.DataProviderstores data into are listed in NineChroniclesContext.cs. - Please refer to each
DbSet's comment in NineChroniclesContext.cs for table descriptions.
Migrating Past Chain Data to MySQL Database
Migrate All Action Data
- This command migrates all action data based on DataRendering to the designated MySQL database.
- Options such as
offsetandlimitare provided to specify which block data to migrate. - IMPORTANT) This migration tool requires you to have the necessary
statedata in the blocks you want to migrate (If your chain store lacks thestatedata, this tool will not work).
Usage: NineChronicles.DataProvider.Executable mysql-migration [--store-path <String>] [--mysql-server <String>] [--mysql-port <UInt32>] [--mysql-username <String>] [--mysql-password <String>] [--mysql-database <String>] [--offset <I
nt32>] [--limit <Int32>] [--help]
Migrate action data in rocksdb store to mysql db.
Options:
-o, --store-path <String> Rocksdb path to migrate. (Required)
--mysql-server <String> A hostname of MySQL server. (Required)
--mysql-port <UInt32> A port of MySQL server. (Required)
--mysql-username <String> The name of MySQL user. (Required)
--mysql-password <String> The password of MySQL user. (Required)
--mysql-database <String> The name of MySQL database to use. (Required)
--offset <Int32> offset of block index (no entry will migrate from the genesis block).
--limit <Int32> limit of block count (no entry will migrate to the chain tip).
-h, --help Show help message
Migrate Battle Arena Ranking Data
- This command calculates the
battle arenaranking of all participants at a specific block index (migration-block-index) and inserts the data to a designated mysql database.
Usage: NineChronicles.DataProvider.Executable battle-arena-ranking-migration [--store-path <String>] [--mysql-server <String>] [--mysql-port <UInt32>] [--mysql-username <String>] [--mysql-password <String>] [--mysql-database <Stri
ng>] [--migration-block-index <Int64>] [--help]
Migrate battle arena ranking data at a specific block index to a mysql database.
Options:
-o, --store-path <String> RocksDB store path to migrate. (Required)
--mysql-server <String> Hostname of MySQL server. (Required)
--mysql-port <UInt32> Port of MySQL server. (Required)
--mysql-username <String> Name of MySQL user. (Required)
--mysql-password <String> Password of MySQL user. (Required)
--mysql-database <String> Name of MySQL database. (Required)
--migration-block-index <Int64> Block index to migrate.
-h, --help Show help message
Migrate User Staking Data
- This command calculates the
NCG staking amountof all participants at a specific block index (migration-block-index) and inserts the data to a designated mysql database. - Currently, the
slack-tokenandslack-channeloptions for sending the data incsvformat are required, however, these will be changed to optional in the near future.
Usage: NineChronicles.DataProvider.Executable user-staking-migration [--store-path <String>] [--mysql-server <String>] [--mysql-port <UInt32>] [--mysql-username <String>] [--mysql-password <String>] [--mysql-database <String>] [--
slack-token <String>] [--slack-channel <String>] [--migration-block-index <Int64>] [--help]
Migrate staking amounts of users at a specific block index to a mysql database.
Options:
-o, --store-path <String> Rocksdb store path to migrate. (Required)
--mysql-server <String> Hostname of MySQL server. (Required)
--mysql-port <UInt32> Port of MySQL server. (Required)
--mysql-username <String> Name of MySQL user. (Required)
--mysql-password <String> Password of MySQL user. (Required)
--mysql-database <String> Name of MySQL database to use. (Required)
--slack-token <String> slack token to send the migration data. (Required)
--slack-channel <String> slack channel that receives the migration data. (Required)
--migration-block-index <Int64> Block index to migrate.
-h, --help Show help message
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent