Code Monkey home page Code Monkey logo

price-timestamping's Introduction

Committing to a historic record for price data with git

We want to publicly commit to a historical record of price data.

We start by keeping track of the relevant price data in a git repository, and making git commits every so often, eg every hour (or every minute etc).

We regurlarly publish the new commit hash we get from git, eg 6241eeff1c779d53172dcef676aaae5245039fdd, on relevant blockchains.

That hash commits us not only to the record state of the data, but also the entire history.

Here's how we reveal some information we committed to. Let's assume we kept our data in a directory called price_data:

$ ls --all price_data/
.  ..  .git  data
$ tree price_data/
price_data/
`-- data
        `-- USD-GBP

2 directories, 3 files

(This step is just to show that price_data is a normal git repository.)

Cargo/Rust installation

curl https://sh.rustup.rs -sSf | sh

additional dependencies for ubuntu

sudo apt install libssl-dev
sudo apt install pkg-config

We want to reveal the exchange rate for USD-GPB as of 24 hours ago.

So let's produce the proof with the following syntax:

$ cd root-path-containing-repository/

$ cargo run  --manifest-path <path-to-manifest/Cargo.toml> <path-to-repo/> <proof_output/> <some-commit-hash> <integer-number-commits> <path-to-pricefile/USD-GBP>

For our exemplary repo:

$ cd root-path-containing-repository/

$ cargo run  --manifest-path qredo-analytics/git-merkle-ing/Cargo.toml qredo-analytics/ proof_output/ fab308fce5f648b52be0f93a3a3848f79cc4db14 1 git-merkle-ing/price_data/data/USD-GBP

The proof is now in proof_output. It happens to be a git repository, too. A repository that only has exactly what we need to reveal the relevant information, and nothing more.

Here is how you can consume and verify the proof with off-the-shelf standard git:

$ cd proof_output/
$ git show 6241eeff1c779d53172dcef676aaae5245039fdd~24:data/binance/USD-GBP
exchange rate for USD-GBP at 2022-12-14 08:15:09.989070+00:00 is 2.8584

The same hash works for multiple timestamps and currency pairs. Eg for 11 hours in the past of USD-SGD:

$ git-merkle-ing price_data/ proof_output_2/ 6241eeff1c779d53172dcef676aaae5245039fdd 11 data/binance/USD-SGD
content: exchange rate for USD-SGD at 2022-12-14 21:15:09.989070+00:00 is 0.35782

And verification:

$ cd proof_output_2/
$ git show 6241eeff1c779d53172dcef676aaae5245039fdd~11:data/binance/USD-SGD
exchange rate for USD-GBP at 2022-12-14 21:15:09.989070+00:00 is 1.45

For illustration of having just the relevant data only: the proof we just produced only has data to verify the price 11 hours ago. If we try to use it to verify another price, we get an error:

$ cd proof_output_2/
$ git show 6241eeff1c779d53172dcef676aaae5245039fdd~24:data/binance/USD-GBP
error: Could not read 42a0ec5eeb6433f0bfd35cacf80e1610cdc81c60
error: Could not read 42a0ec5eeb6433f0bfd35cacf80e1610cdc81c60
fatal: invalid object name '6241eeff1c779d53172dcef676aaae5245039fdd~24'.

Example from the kernel

Here's an example using the Linux kernel, to show that we can also handle huge repositories:

$ git-merkle-ing linux-kernel/ /tmp/proof 041fae9c105ae342a4245cf1e0dc56a23fbb9d3c 23 Documentation/gpu/backlight.rst
content: =================
Backlight support
=================

.. kernel-doc:: drivers/video/backlight/backlight.c
   :doc: overview

.. kernel-doc:: include/linux/backlight.h
   :internal:

.. kernel-doc:: drivers/video/backlight/backlight.c
   :export:
$ gtar c /tmp/proof/ | xz | wc -c
57232

And verification:

$ cd /tmp/proof/
$ git show 041fae9c105ae342a4245cf1e0dc56a23fbb9d3c~23:Documentation/gpu/backlight.rst
=================
Backlight support
=================

.. kernel-doc:: drivers/video/backlight/backlight.c
   :doc: overview

.. kernel-doc:: include/linux/backlight.h
   :internal:

.. kernel-doc:: drivers/video/backlight/backlight.c
   :export:

Running the services in this repo

This is a work in progress.

Amongst other things, you need:

  • .env and priceapi/fastapi/app/.cfg config files.
  • you need to start running1.bash (and leave it running) and then start running2.bash

THere might be more steps necessarily. We will document them as we clean things up.

price-timestamping's People

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.