Code Monkey home page Code Monkey logo

kucoin_arbitrage's Introduction

Event-Driven KuCoin Triangular Arbitrage Framework in Async Rust

crates docs license ci issues discussions discord

This is an async Rust project to implement zero-risk crypto trinagular arbitrage, explore technical feasiblity of generating passsive income (i.e. sleep to earn!).

How cyclic arbitrage works

Say we hold USDT, it checks all the listed crypto (e.g. ETH) that can trade against BTC and USDT, and compare the profit by either:

  • Buy-Sell-Sell: buy ETH (sell USDT), sell ETH (buy BTC), sell BTC (buy USDT)
  • Buy-Buy-Sell: buy BTC (sell USDT), buy ETH (sell BTC), sell ETH (buy USDT)

The above is triangular arbitrage, which is the simplest form of cyclic arbitrage. We can make more complex say finding the most profitable routes, or placing order at maker price for reduced fees.

Previous mistakes in Python

2 years ago I have made a python script that runs the triangular arbitrage in KuCoin, but it had several technical issues and ended up not following up.
https://github.com/kanekoshoyu/Kucoin-Triangular-Arbitrage

  • Implementation with Python and REST polling was way too slow to obtain the valid arbitratge chances for execution.
  • Generally the Python REST API caused quite high rates of communication error, which took extra time for resorting.
  • It didn't count the actual size of the arbitrage order, which meant that the script kept buying some shitcoin and could not sell properly.
  • It simply took the mean price instead of best bid/ask, which means it took maker positions for each of three actions in one arbitrage, and did not execute arbitrage promptly.

How to run example executables

  1. Rename config_sample.toml as config.toml and set the API key with your own KuCoin API credentials. Configure the event monitor interval and the USD budget per each iteration of cyclic arbitrage.
[KuCoin Credentials]
api_key="YOUR_API_KEY"
secret_key="YOUR_SECRET_KEY"
passphrase="YOUR_PASSPHRASE"

[Behaviour]
# Performance monitor interval in seconds
monitor_interval_sec=120
# max amount of USD to use in a single cyclic arbitrage
usd_cyclic_arbitrage=100
  1. At the root directory of the project(kucoin_arbiitrage), run the command below:
cargo run --bin event_triangular  

event_triangular is one of the example executables that implements [XXX-BTC, XXX-USDT, BTC-USDT] triangular arbitrage. There are other executables in the bin directory.

Overview

Code Structure

The project is split into these components:

Example Executables
  • bin contains example executable codes. Some of them are for network testing purpose.
Internal Structure (Independent of Exchange APIs)
  • model has internal generic data structures used for abstracted representations of markets. This should be independent of exchange APIs so that the the arbitrage strategy algorithm can be conducted across different exchanges.
  • event has the events used to pass states and data passed across different components. It uses the internal model for the same reason.
  • strategy has the implementations of arbitrage strategy algorithm. The algorithms are built upon internal model and event.
  • monitor has the counter used to monitor MPS (message per seconds) for each broadcast channels, and a timers mapped globally by string for easy debug access.
Link to Exchange APIs (e.g. KuCoin)
  • translator has the conversion of exchange API objects into internal models and vice versa. It uses traits and the traits are implemented per API models.
  • broker has the tasks that runs API calls, and converts into internal data structure.

Event Pub/Sub with Tokio Broadcast

Event broadcasts empowers the modularity of tasks. Each async task communicates with eachother using events, pub/sub via tokio's broadcast. Here is the exmaple for event_triangular.rs

Channel Publisher Subscriber
orderbook task_pub_orderbook_event task_monitor_channel_mps, task_sync
orderbook_best task_sync task_monitor_channel_mps, task_pub_chance_all_taker_btc_usd
chance task_pub_chance_all_taker_btc_usd task_monitor_channel_mps, task_gatekeep_chances
order task_gatekeep_chances task_monitor_channel_mps, task_place_order
orderchange task_pub_orderchange_event task_monitor_channel_mps, task_gatekeep_chances

Task Pools with Tokio JoinSet

Tasks are grouped and spawned using JoinSets. We can either await for all the tasks to end with join! or await until a single task ends with select! or join_next. This provides full control over how we want to control these tasks. Here is the exmaple for task pools declared in core function of event_triangular.rs

TaskPool Task
taskpool_infrastructure task_sync_orderbook, task_pub_chance_all_taker_btc_usd, task_gatekeep_chances, task_place_order
taskpool_subscription task_pub_orderbook_event, task_pub_orderchange_event
taskpool_monitor task_monitor_channel_mps, task_log_mps

When a task in taskpool returns, its result is received by join_next, which are received by core's select!. When an external signal is received, or core returns error, it gets detected by select! at the main and terminates the program.

Major Structural Improvements

  • Use compiled Rust code for neat, efficient and proper code
  • WebSocket subscription of real-time order books to get all the latest maker/taker ask/bid
  • Only take a taker position based on the latest best-bid/ask price/size
  • Implement both data bandwidth monitor and arbitrage performance monitor as tasks
  • Abstraction of orderbook sync and arbitrage strategies using internal model and event
  • Concurrency of syncing and strategy tasks

Feature Progress List

Feature Status
Whitelist all coins that can trade against two other quote coins (e.g. ETH, for ETH-BTC, ETH-USDT) Available
Look for arbitrage chance based on best ask/bid price and calculate the profit in percentage Available
Copy and sync local orderbook in real-time Available
Structurally allow multiple strategies to run in pararrel Available
Order placement upon triangular arbitrage chance Available
Resort against limit order that could not get filled Pending
Full triangular arbitrage with the middle coin other than BTC (e.g. ETH-USD, ALT-ETH, ALT-USD) Pending

Deployment

Please refer to my another repo implementing service-level wrappers: chaiwala

Community

kucoin_arbitrage's People

Contributors

0xmimir avatar dependabot[bot] avatar kanekoshoyu avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

kucoin_arbitrage's Issues

orderbook merge error

[2023-10-30 17:34:33][INFO][kucoin_arbitrage::monitor::task] orderbook   : 29601 messages (  986mps)
[2023-10-30 17:34:33][INFO][kucoin_arbitrage::monitor::task] best_price  :  3772 messages (  125mps)
[2023-10-30 17:34:33][INFO][kucoin_arbitrage::monitor::task] chance      :     0 messages (    0mps)
[2023-10-30 17:34:33][INFO][kucoin_arbitrage::monitor::task] order       :     0 messages (    0mps)
[2023-10-30 17:34:33][INFO][kucoin_arbitrage::monitor::task] trade       :     0 messages (    0mps)
thread 'tokio-runtime-worker' panicked at 'called `Option::unwrap()` on a `None` value', src/model/orderbook.rs:59:51
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
core ended first Err(ErrorMessage { msg: "unexpected error [Infrastructure task pool error [Some(Err(JoinError::Panic(Id(17), ...)))]]" })

chance/order counter not being updated

[2023-10-23 17:53:13][INFO][kucoin_arbitrage::broker::gatekeeper::kucoin] time_sent: SystemTime { tv_sec: 1698076393, tv_nsec: 843016101 }
[2023-10-23 17:53:13][ERROR][kucoin_arbitrage::broker::order::kucoin] ErrorMessage { msg: "global timer [order_placement_broadcast] is not found" }
[2023-10-23 17:53:13][INFO][kucoin_arbitrage::broker::order::kucoin] order placement
LimitOrder { id: "MhESL0T33wphLG8OvQS4evTEWFa-oZrAbDPCi7H0", order_type: Limit, side: Buy, symbol: "BCH-USDT", amount: "0.40740000000000004", price: "245.44" }
[2023-10-23 17:53:17][INFO][chaiwala::report::counter] Broadcast channel data rate
[2023-10-23 17:53:17][INFO][chaiwala::report::counter] orderbook : 25482 points ( 5096mps)
[2023-10-23 17:53:17][INFO][chaiwala::report::counter] best_price: 2589 points ( 517mps)
[2023-10-23 17:53:17][INFO][chaiwala::report::counter] chance : 0 points ( 0mps)
[2023-10-23 17:53:17][INFO][chaiwala::report::counter] order : 0 points ( 0mps)
[2023-10-23 17:53:17][INFO][chaiwala::report::counter] orderchange: 0 points ( 0mps)

TradeMatched TradeFilled gets detected faster than LimitOrderPlacement reply

[2023-10-27 21:57:43][INFO][kucoin_arbitrage::broker::trade::kucoin] TradeMatch [335561405913109708508436045567613047709]
[2023-10-27 21:57:43][INFO][kucoin_arbitrage::broker::order::kucoin] Limit order placement successful [335561405913109708508436045567613047709]
[2023-10-27 21:57:43][INFO][kucoin_arbitrage::broker::trade::kucoin] TradeFilled [335561405913109708508436045567613047709]
[2023-10-27 21:57:48][INFO][kucoin_arbitrage::monitor::task] Broadcast channel MPS
[2023-10-27 21:57:48][INFO][kucoin_arbitrage::monitor::task] order       :     1 messages (    0mps)
[2023-10-27 21:57:48][INFO][kucoin_arbitrage::monitor::task] trade       :     2 messages (    0mps)
[2023-10-27 21:57:58][INFO][kucoin_arbitrage::monitor::task] Broadcast channel MPS
[2023-10-27 21:57:58][INFO][kucoin_arbitrage::monitor::task] order       :     1 messages (    0mps)
[2023-10-27 21:57:58][INFO][kucoin_arbitrage::monitor::task] trade       :     0 messages (    0mps)
[2023-10-27 21:57:58][INFO][kucoin_arbitrage::broker::trade::kucoin] TradeMatch [80848103607579503676599271329919883293]
[2023-10-27 21:57:58][INFO][kucoin_arbitrage::broker::trade::kucoin] TradeFilled [80848103607579503676599271329919883293]
[2023-10-27 21:57:58][INFO][kucoin_arbitrage::broker::order::kucoin] Limit order placement successful [80848103607579503676599271329919883293]
[2023-10-27 21:58:08][INFO][kucoin_arbitrage::monitor::task] Broadcast channel MPS
[2023-10-27 21:58:08][INFO][kucoin_arbitrage::monitor::task] order       :     0 messages (    0mps)
[2023-10-27 21:58:08][INFO][kucoin_arbitrage::monitor::task] trade       :     2 messages (    0mps)
[2023-10-27 21:58:13][INFO][kucoin_arbitrage::broker::trade::kucoin] TradeMatch [295437004754534137248699779684710492288]
[2023-10-27 21:58:13][INFO][kucoin_arbitrage::broker::trade::kucoin] TradeFilled [295437004754534137248699779684710492288]
[2023-10-27 21:58:13][INFO][kucoin_arbitrage::broker::order::kucoin] Limit order placement successful [295437004754534137248699779684710492288]

Imcomplete gatekeeper, only trades one out of three pairs

[2023-10-23 18:22:55][INFO][kucoin_arbitrage::broker::gatekeeper::kucoin] All Taker Chance found!
[2023-10-23 18:22:55][INFO][kucoin_arbitrage::broker::gatekeeper::kucoin] TriangularArbitrageChance { profit: OrderedFloat(0.016376796710005692), actions: [ActionInfo { Buy BTC-USDT at 30900.1 for 0.00323623}, ActionInfo { Buy HBAR-BTC at 0.000001593 for 2029.5001000000002}, ActionInfo { Sell HBAR-USDT at 0.04938 for 2027.4705000000001}] }
[2023-10-23 18:22:55][INFO][kucoin_arbitrage::broker::gatekeeper::kucoin] time_sent: SystemTime { tv_sec: 1698078175, tv_nsec: 779915767 }
[2023-10-23 18:22:55][ERROR][kucoin_arbitrage::broker::order::kucoin] ErrorMessage { msg: "global timer [order_placement_broadcast] is not found" }
[2023-10-23 18:22:55][INFO][kucoin_arbitrage::broker::order::kucoin] order placement
LimitOrder { id: "390KyrWm18SG3nP5ioEk71ZZUB8QvPgy2-mTWcol", order_type: Limit, side: Buy, symbol: "BTC-USDT", amount: "0.00323623", price: "30900.1" }
[2023-10-23 18:23:18][INFO][chaiwala::report::counter] Broadcast channel data rate
[2023-10-23 18:23:18][INFO][chaiwala::report::counter] orderbook : 194488 points ( 3241mps)
[2023-10-23 18:23:18][INFO][chaiwala::report::counter] best_price: 23920 points (  398mps)
[2023-10-23 18:23:18][INFO][chaiwala::report::counter] chance    :   356 points (    5mps)
[2023-10-23 18:23:18][INFO][chaiwala::report::counter] order     :     1 points (    0mps)
[2023-10-23 18:23:18][INFO][chaiwala::report::counter] orderchange:     0 points (    0mps)

how to run

hello
Please explain how to Run this project

digit arithmetics

[2023-10-28 15:01:18][INFO][kucoin_arbitrage::broker::gatekeeper::kucoin] All Taker Chance found!
[2023-10-28 15:01:18][INFO][kucoin_arbitrage::broker::gatekeeper::kucoin] TriangularArbitrageChance { profit: OrderedFloat(0.0004159016320031128), actions: [ActionInfo { Buy LINK-USDT at 11.1751 for 4.474200000000001}, ActionInfo { Sell LINK-BTC at 0.00032844 for 4.4697000000000005}, ActionInfo { Sell BTC-USDT at 34127.8 for 0.00146656}] }
[2023-10-28 15:01:18][ERROR][kucoin_arbitrage::broker::order::kucoin] Invalid order size increment LimitOrder { id: "78570f60-1db6-4f81-ad08-3b56633c7b2a", order_type: Limit, side: Buy, symbol: "LINK-USDT", amount: "4.474200000000001", price: "11.1751" }

orderchange receiver not handling the event properly

[2023-10-25 14:40:33][INFO][kucoin_arbitrage::broker::order::kucoin] order_placement_broadcast: 158.196µs
[2023-10-25 14:40:33][INFO][kucoin_arbitrage::broker::order::kucoin] order placement
LimitOrder { id: "13ps68ZRGTOhrqC1irUO3C_qYBlSUwiBX2-EE-ob", order_type: Limit, side: Buy, symbol: "BTC-USDT", amount: "0.001", price: "40000" }
[2023-10-25 14:40:34][INFO][kucoin_arbitrage::broker::orderchange::kucoin] message: WelcomeMsg(DefaultMsg { id: "lthOjqTzii", type: "welcome" })
[2023-10-25 14:40:34][INFO][kucoin_arbitrage::broker::orderchange::kucoin] message: WelcomeMsg(DefaultMsg { id: "1698241234562", type: "ack" })
[2023-10-25 14:40:43][INFO][kucoin_arbitrage::broker::order::kucoin] order_placement_broadcast: 10.001355413s
[2023-10-25 14:40:43][INFO][kucoin_arbitrage::broker::order::kucoin] order placement
LimitOrder { id: "13ps68ZRGTOhrqC1irUO3C_qYBlSUwiBX2-EE-ob", order_type: Limit, side: Buy, symbol: "BTC-USDT", amount: "0.001", price: "40000" }
[2023-10-25 14:40:43][INFO][kucoin_arbitrage::broker::orderchange::kucoin] message: TradeMatchMsg(WSResp { type: "message", topic: "/spotMarket/tradeOrders", subject: "orderChange", data: TradeMatch { symbol: "BTC-USDT", order_type: "limit", side: "buy", liquidity: "taker", type: "match", order_id: "65391adb7877490007663031", order_time: 1698241243615, size: "0.001", filled_size: "0.001", price: "40000", match_price: "34404.8", match_size: "0.001", trade_id: "5665652239777793", client_oid: "13ps68ZRGTOhrqC1irUO3C_qYBlSUwiBX2-EE-ob", remain_size: "0", status: "match", ts: 1698241243630000000 } })
[2023-10-25 14:40:43][INFO][kucoin_arbitrage::broker::orderchange::kucoin] TradeMatchMsg: "/spotMarket/tradeOrders"
TradeMatch {
    symbol: "BTC-USDT",
    order_type: "limit",
    side: "buy",
    liquidity: "taker",
    type: "match",
    order_id: "65391adb7877490007663031",
    order_time: 1698241243615,
    size: "0.001",
    filled_size: "0.001",
    price: "40000",
    match_price: "34404.8",
    match_size: "0.001",
    trade_id: "5665652239777793",
    client_oid: "13ps68ZRGTOhrqC1irUO3C_qYBlSUwiBX2-EE-ob",
    remain_size: "0",
    status: "match",
    ts: 1698241243630000000,
}
[2023-10-25 14:40:43][INFO][kucoin_arbitrage::broker::orderchange::kucoin] message: TradeFilledMsg(WSResp { type: "message", topic: "/spotMarket/tradeOrders", subject: "orderChange", data: TradeFilled { symbol: "BTC-USDT", order_type: "limit", side: "buy", type: "filled", order_id: "65391adb7877490007663031", order_time: 1698241243615, size: "0.001", filled_size: "0.001", price: "40000", client_oid: "13ps68ZRGTOhrqC1irUO3C_qYBlSUwiBX2-EE-ob", remain_size: "0", status: "done", ts: 1698241243630000000 } })
[2023-10-25 14:40:43][INFO][kucoin_arbitrage::broker::orderchange::kucoin] TradeFilledMsg: "/spotMarket/tradeOrders"
TradeFilled {
    symbol: "BTC-USDT",
    order_type: "limit",
    side: "buy",
    type: "filled",
    order_id: "65391adb7877490007663031",
    order_time: 1698241243615,
    size: "0.001",
    filled_size: "0.001",
    price: "40000",
    client_oid: "13ps68ZRGTOhrqC1irUO3C_qYBlSUwiBX2-EE-ob",
    remain_size: "0",
    status: "done",
    ts: 1698241243630000000,
}
thread 'tokio-runtime-worker' panicked at 'Publishing OrderFilled failed: SendError(OrderFilled((0, "TradeFilled { symbol: \"BTC-USDT\", order_type: \"limit\", side: \"buy\", type: \"filled\", order_id: \"65391adb7877490007663031\", order_time: 1698241243615, size: \"0.001\", filled_size: \"0.001\", price: \"40000\", client_oid: \"13ps68ZRGTOhrqC1irUO3C_qYBlSUwiBX2-EE-ob\", remain_size: \"0\", status: \"done\", ts: 1698241243630000000 }")))', /home/sho/Documents/personal/kucoin_arbitrage/src/broker/orderchange/kucoin.rs:46:32
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Orderbook publish error

TriangularArbitrageChance { profit: OrderedFloat(0.025960391571992147), actions: [ActionInfo { Buy EWT-USDT at 1.76 for 28.409000000000002}, ActionInfo { Sell EWT-BTC at 0.00005216 for 28.3805}, ActionInfo { Sell BTC-USDT at 33861.7 for 0.00147884}] }
[2023-10-27 23:37:16][ERROR][kucoin_arbitrage::broker::orderbook::kucoin] Orderbook event publish error, check receiver
core ended first Err(ErrorMessage { msg: "unexpected error [Infrastructure task pool error [Some(Ok(Err(ErrorMessage { msg: \"unrecognised code [\\\"400100\\\"]\" })))]]" })
Good bye!
[2023-10-27 23:37:16][ERROR][kucoin_arbitrage::broker::orderbook::kucoin] Orderbook event publish error, check receiver

some tasks still not terminating gracefully

[2023-10-23 17:39:46][INFO][chaiwala_service] Received [SIGINT] signal
[2023-10-23 17:39:46][INFO][chaiwala_service] received external signal, Good bye!
[2023-10-23 17:39:46][ERROR][kucoin_arbitrage::broker::orderbook::kucoin] Orderbook event publish error, check receiver
[2023-10-23 17:39:46][ERROR][kucoin_arbitrage::broker::orderbook::kucoin] Orderbook event publish error, check receiver

Outdated dependacies

After running cargo build I get following output:

➜  kucoin_arbitrage git:(master) cargo build
   Compiling proc-macro2 v1.0.59
   Compiling parking_lot v0.12.1
   Compiling ahash v0.7.6
   Compiling core-foundation v0.9.3
   Compiling security-framework-sys v2.9.0
   Compiling tempfile v3.5.0
   Compiling tracing-core v0.1.31
   Compiling percent-encoding v2.2.0
error[E0635]: unknown feature `proc_macro_span_shrink`
  --> /home/mimir/.cargo/registry/src/index.crates.io-6f17d22bba15001f/proc-macro2-1.0.59/src/lib.rs:92:30
   |
92 |     feature(proc_macro_span, proc_macro_span_shrink)
   |                              ^^^^^^^^^^^^^^^^^^^^^^

For more information about this error, try `rustc --explain E0635`.
error: could not compile `proc-macro2` (lib) due to previous 

Invalid type

[2023-11-23 19:01:55][WARN][kucoin_arbitrage::broker::orderbook::kucoin] orderbook[UQC-USDT] did not respond (101 tries) [HTTP(reqwest::Error { kind: Decode, source: Error("invalid type: null, expected a string", line: 1, column: 49) })]

HTTP request for UQC fails all the time.

Reception of messages in wrong order

[2023-10-30 17:44:55][INFO][kucoin_arbitrage::broker::gatekeeper::kucoin] All Taker Chance found!
[2023-10-30 17:44:55][INFO][kucoin_arbitrage::broker::gatekeeper::kucoin] TriangularArbitrageChance { profit: OrderedFloat(0.024062122741007386), actions: [ActionInfo { Buy FTM-USDT at 0.24025 for 208.1165}, ActionInfo { Sell FTM-BTC at 0.00000703 for 207.9083}, ActionInfo { Sell BTC-USDT at 34294.3 for 0.00146013}] }
[2023-10-30 17:44:55][INFO][kucoin_arbitrage::broker::gatekeeper::kucoin] waiting for TradeInfo from KuCoin server
[2023-10-30 17:44:55][INFO][kucoin_arbitrage::broker::trade::kucoin] TradeOpen[37999890520258331277283304113892230095]
[2023-10-30 17:44:55][INFO][kucoin_arbitrage::broker::gatekeeper::kucoin] waiting for TradeInfo from KuCoin server
[2023-10-30 17:44:55][INFO][kucoin_arbitrage::broker::order::kucoin] Limit order placement successful [37999890520258331277283304113892230095]

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.