fabianboesiger / ftx Goto Github PK

When doing:

api.request(PlaceOrder {
    market: o.pair.to_string(),
    side: if o.islong {
        ftx::rest::Side::Buy
    } else {
        ftx::rest::Side::Sell
    },
    price: None,
    r#type: ftx::rest::OrderType::Market,
    size: o.real_quantity,
    client_id: None,
    ioc: false,
    post_only: false,
    reduce_only: false,
    reject_on_price_band: false,
})
.await?

It gives the error: Api error: Missing parameter price. I am confused because it says in the API to pass null for market orders. Am I missing something? This worked a few weeks ago but maybe I changed something.

got this error

thread 'tokio-runtime-worker' panicked at 'called Result::unwrap() on an Err value: Tungstenite(Protocol(ResetWithoutClosingHandshake))', src\main.rs:26:70
note: run with RUST_BACKTRACE=1 environment variable to display a backtrace

on the code
websocket.next().await.expect("No data").unwrap()

Any clue what it means? Subscribed to the Trades channel if that helps

e.g. this is the payload: {"channel":"ticker","market":"SRN-PERP","type":"update","data":{"bid":null,"ask":null,"bidSize":null,"askSize":null,"last":0.00385,"time":1656893927.97223}}
Change Ticker to

pub struct Ticker {
    pub bid: Option<Decimal>,
    pub ask: Option<Decimal>,
    pub bid_size: Option<Decimal>,
    pub ask_size: Option<Decimal>,
    pub last: Decimal,
    #[serde_as(as = "TimestampSecondsWithFrac<f64>")]
    pub time: DateTime<Utc>,
}

in src/ws/model.rs should work.

• Ignore tests that place orders
• Ignore tests that require API key
• Make sure that the remaining tests pass
• Enable testing in CI

does the websocket auto reconnect if it disconnects? Or do we need to implement it ourselves?

It would be useful to discuss topics without the need to open issues.

I am struggling to implement orderbook checksums. As messages are expected to be dropped it is quite important that this function works, so the user knows when to query FTX for a fresh order book snapshot. My current progress is in #22

Resources

• FTX documentation on calculating the checksums
• FTX-provided Python example client code
• ws::tests::order_book_checksum
• ws::tests::order_book

What I've confirmed:

• The data returned from the API should be interpreted as an unsigned integer u32 despite their documentation calling the checksum a "signed 32-bit integer". u32 is also the output of hasher.finalize()
• Number of bids and asks added to the crc32 input is correct
• The crc32 input is formatted in the correct order, with the correct number of : separators, with no : at the beginning or the end. The crc32 input looks like

37741:2.0959:37748:0.0500:37740:2.7650:37749:0.1303:37738:3.4262:37750:0.0873:37737:7.4173:37751:0.2960:37735:1.3957:37752:1.8759:37734:0.5218:37754:1.4942:37733:0.9042:37755:0.0325:37732:1.4596:37756:1.6405:37731:0.3473:37757:0.4998:37730:1.1107:37758:1.0184:37729:1.0956:37759:1.3816:37728:0.6690:37760:2.5531:37727:2.0849:37761:0.0108:37726:2.4043:37762:1.6603:37725:0.7519:37763:2.4172:37724:0.9825:37764:1.1406:37723:1.6561:37765:0.2937:37722:1.2277:37766:1.3033 ... 37613:0.1000:37862:0.0073:37612:0.1329:37864:0.5927

What is uncertain:

• The exact format of the input to the crc32 value returned by the API, including the number of decimal places to include in order prices and quantities.

Testing their code using Python

Consider the simple order book

bids = [[4, 5], [3, 10], [2, 15]]
asks = [[5, 20], [6, 30], [7, 40]]
orderbook = { 'asks': asks, 'bids': bids }

According to their docs, the crc32 input should be:
4:5:5:20:3:10:6:30:2:15:7:40, the crc32 of which is 640057369

However, when you run their example code:

checksum_data = [':'.join([f'{float(order[0])}:{float(order[1])}' for order in (bid, offer) if order])
    for (bid, offer) in zip_longest(orderbook['bids'][:100], orderbook['asks'][:100])]
computed_result = int(zlib.crc32(':'.join(checksum_data).encode()))

Their crc input to zlib.crc32 is 4.0:5.0:5.0:20.0:3.0:10.0:6.0:30.0:2.0:15.0:7.0:40.0, and the crc32 of that is 3640375499. Even the example in their documentation is 5005.5:10:5001.0:6:4995.0:5, which has no 0s appended to the quantities.

Computing checksums for the simple order book succeeds

Based on this simplified order book data, I can compute a correct crc32 value within verify_checksum for either 4:5:5:20:3:10:6:30:2:15:7:40 or 4.0:5.0:5.0:20.0:3.0:10.0:6.0:30.0:2.0:15.0:7.0:40.0 by changing the string formatting to:

• input.push(format!("{}:{}", b.0, b.1)); to yield 640057369
• input.push(format!("{:.1}:{:.1}", b.0, b.1)); to yield 3640375499 by padding .0 into the prices and quantities.

These changes allow the ws::tests::order_book_checksum test to pass, and rule out bugs in the separation and ordering of the bids and asks.

Computing checksums for real order book data fails

However, when I try to verify the checksum on real data in the ws::tests::order_book test, the test fails. For example, BTC-PERP has integer prices (0 decimal places), and quantities to four decimal places. None of the following worked:

• input.push(format!("{}:{}", b.0, b.1));
• input.push(format!("{}:{:.4}", b.0, b.1)); (equivalent to the previous one)
• input.push(format!("{:.1}:{:.4}", b.0, b.1));
• input.push(format!("{:.1}:{:.1}", b.0, b.1));
• input.push(format!("{:.4}:{:.4}", b.0, b.1));

At this point, I don't know what to try next. Any help or suggestions would be greatly appreciated.

I believe that this comment from ftx documentation hasn't been taken into account while checksum validation has been implementing

Rules for float formatting for the checksum:

    Values smaller than 1e-04 (0.0001) should be formatted using scientific notation, and should contain zeroes before the exponent. For example, a message containing 7.5e-5 or 0.000075 should be formatted like so for computing the checksum: '7.5e-05'.

Hi,

This is not an issue, it's just a question.
I'm reading through the code and found this comment. I'm wondering if you could provide more information on it? Is this something mentioned from FTX document?

Thanks!

I have implemented the ftx module inside of my rust application and the command api.get_market(...) works fine.
However, when i use api.get_markets() in the same location, it gives me the following error:

Json(Error("invalid type: null, expected a Decimal type representing a fixed-point number", line: 1, column: 17472))

Code for reference (inside a module, inside of a switch function):

        "price"|"p" => {     
            let query = api.get_market(&pair.as_str()).await?;
            println!("  {}: {}", pair, query.price);

            let query2 = api.get_markets().await?;
            println!("{:#?}", query2);
        },

In the above, the api.get_market runs fine however the api.get_markets gives me the error as said above. Am I missing something here?

How would one use the websocket to listen to multiple orderbooks? Looking at the example seems like only one orderbook can be listened to at a time

Heres my code-

    let solbtc = String::from("SOL/BTC");
    let solusd = String::from("SOL/USD");
    let solusdt = String::from("SOL/USDT");
    let fttbtc = String::from("FTT/BTC");
    let fttusd = String::from("FTT/USD");
    let fttusdt = String::from("FTT/USDT");
    let btcusd = String::from("BTC/USD");
    let usdtusd = String::from("USDT/USD");

    let mut solbtcorderbook = Orderbook::new(solbtc.to_owned());
    let mut solusdorderbook = Orderbook::new(solusd.to_owned());
    let mut solusdtorderbook = Orderbook::new(solusdt.to_owned());
    let mut fttbtcorderbook = Orderbook::new(fttbtc.to_owned());
    let mut fttusdorderbook = Orderbook::new(fttusd.to_owned());
    let mut fttusdtorderbook = Orderbook::new(fttusdt.to_owned());
    let mut btcusdorderbook = Orderbook::new(btcusd.to_owned());
    let mut usdtusdorderbook = Orderbook::new(usdtusd.to_owned());

    websocket.subscribe(vec![
        Channel::Orderbook(solbtc.to_owned()),
        Channel::Orderbook(solusd.to_owned()),
        Channel::Orderbook(solusdt.to_owned()),
        Channel::Orderbook(fttbtc.to_owned()),
        Channel::Orderbook(fttusd.to_owned()),
        Channel::Orderbook(fttusdt.to_owned()),
        Channel::Orderbook(btcusd.to_owned()),
        Channel::Orderbook(usdtusd.to_owned()),
    ]).await?;

    tokio::spawn(async move {
        loop {
            let data = websocket.next().await.expect("No data received").unwrap();

            match data {
                (_, Data::OrderbookData(orderbook_data)) => {

                }

                _ => panic!("Unexpected data received: {:?}", data),
            }
        }
    });

Now im not so sure what to put inside the match statement, as all the orderbook data is just grouped into one

In the example file -

(_, Data::OrderbookData(orderbook_data)) => {
                orderbook.update(&orderbook_data);
                print!("."); // To signify orderbook update
                io::stdout().flush().unwrap(); // Emits the output immediately
            }

the data is not categorised by symbol so im puzzled

Hi Admin,

This is just a newbie question, not an issue actually..
I noticed that you have already provided the convenience methods like best_bid and best_ask from Orderbook. Why should we still need a "ticker" channel?

Also related to request / response @dovahcrow

Testing with a version of my codebase that has updated to f045b28 (and changed nothing else), I get this error in the update commit (which still uses the deprecated functions):

[03:07:28.947761 DEBUG] starting new connection: https://ftx.com/
[03:07:28.962191 DEBUG] response '400 Bad Request' for https://ftx.com/api/orders
[03:07:28.962744 WARN] Error during graceful reset: API Error (Add 1INCH): Missing parameter price

which is strange as I am sending a market order at that point. Currently digging into the issue.

Also suggest do not use the enum here: https://github.com/fabianboesiger/ftx/blob/main/src/rest/model.rs#L12.

Crypto exchange protocols change rapidly. Using enum will only produce an unhelpful data did not match any variant of untagged enum Response error.

A better solution would be to deserialize the payload into a string first.
After that, try to deserialize the string into Result { success: bool, result: T }, if succeeded, return it to the user. If not, write down the error into a variable.
Then, deserialize the string into Error { success: bool, error: String }, if succeeded, return it to the user. If not, return the error written down in the first step.

The overhead to string is fine because when using enum, serde will buffer the content as well. https://github.com/serde-rs/serde/blob/master/serde/src/private/de.rs#L219

Originally posted by @dovahcrow in #44 (comment)

Currently we have rest::OrderSide and rest::Side which are functionally identical (except that OrderSide has a Serialize attribute and Side doesn't)

Thoughts on unifying them into a single rest::Side?

• I don't see any reason why they need to be distinct, and in my own implementing project I've had to manually convert between the two enums several times.
• There aren't too many users of this library yet, so we still have some freedom to make API-breaking changes.

I am currently in the process of tracking down a "Not logged in" error appearing in my log messages after updating my fork of this library.

The tests do not reproduce the error, but my implementing codebase (~7k lines) does. Here are some example log messages:

[00:10:16.484965 DEBUG] response '200 OK' for https://ftx.com/api/positions
[00:10:16.955672 DEBUG] response '401 Unauthorized' for https://ftx.com/api/orders?market=ETH-PERP
[00:10:16.956057 ERROR] Error during reset: Could not cancel excess limit orders: Could not fetch open orders: Api error: Not logged in

It appears that some endpoints (e.g. /positions) work fine, but there is an error with the /orders endpoint.

I've bisected my own codebase and tracked down the offending commit to the one where I updated my ftx lib. I'm currently in the process of tracking down which commit it's in and so far have narrowed it down to one of the ones in the request / response PR #47. Opening this issue here because I was hoping you might have some ideas @dovahcrow as to what is going on.

I think it would be nicer if we used pub mod model instead of of pub use model::* so that the rest and ws modules aren't so polluted with model structs.

Library users would then do

use ftx::ws::{Ws, model::{Channel, Data, Orderbook}};

instead of

use ftx::ws::{Channel, Data, Orderbook, Ws};

Any opinions?

Sometimes I found ftx orderbook data handled by Ws object has big time lag, more than 500 seconds. Is it because websocket disconnected? How to reconnect or resubscribe?

GetSubAccount is not consistent with GetSubaccountBalances

When updating to newer version, out of the blue it gave me this:

> $ cargo build                                                                                                                                                                                       [±release ●●]
    Updating git repository `https://github.com/fabianboesiger/ftx`
   Compiling ftx v0.4.0 (https://github.com/fabianboesiger/ftx?branch=main#1bf760f0)
error: unexpected token: `deprecate_msg`
   --> /home/admin/.cargo/git/checkouts/ftx-9fce05f641d55c68/1bf760f/src/rest/mod.rs:174:18
    |
35  | impl Rest {
    |           - while parsing this item list starting here
...
174 |     #[deprecated=deprecate_msg!()]
    |                  ^^^^^^^^^^^^^
...
534 | }
    | - the item list ends here

error: aborting due to previous error

error: could not compile `ftx`

I am using:
Ubuntu 20.04 based os
rust edition = "2018"
and to get module I am using:

ftx = { git="https://github.com/fabianboesiger/ftx", branch = "main" }

FTX added a new parameter "protocol" in Request withdrawal which is not supported by this api, some solana SPL tokens withdrawal cannot be done without this parameter.

I have a piece of code where self.client = Rest::new(options).

    async fn request_balances_snapshot(&self) {
        debug!("requesting balances");

        let balances = self.client.request(GetWalletBalances {}).await;
        debug!("Got balances {:?}", &balances);

I run it from tokio runtime. And it stops executing after the request. Debug message with balances is not printing

Just a discussion: do you want to use the trait-based request/response instead of the current method-based one? e.g. I implement the trait-based request/response idea in Deribit rust client https://github.com/dovahcrow/deribit-rs/blob/master/src/models/trading.rs#L32.

The benefit of trait-based request/response is that:

1. No long parameter list of the functions.
2. No need to remember the function names, everything is request(SomeRequestType) -> SomeRequestType::ResponseType.
3. API endpoint, signature requirement, request-response type linkage can all be bundled into the trait.
4. No need to specify unneeded parameters by leveraging the Default trait, e.g.

   let req = SomeRequest {
                   field_a: val_a,
                    ...Default::default()
   }
   

   instead of

   client.some_method(
                 val_a, // field_a
                 None, // unneeded_optional_field_b
                 None, // unneeded_optional_field_c
   )
   

5. Possible to add sugar functions to the request, e.g. for creating a market order, you just need let req = Order::market(instrument, qty), no need to specify other parameters like post_only, reduce_only etc.

I can help if you'd like to do the trait-based request/response.

When I compile watch_market.rs, there's an error:

let data = websocket.next().await.expect("No data received")?;
^^^^ method not found in Ws

websocket.next() method not found. I use the latest version. How to get orderbook data in the latest sdk version?

The error types in this crate are not compatible with std::error::Error, any plan for this?

This happens when I use the current setup:

q_main_order = api.place_order(
                        pair,
                        if calculation.islong {ftx::rest::Side::Buy} else {ftx::rest::Side::Sell},
                        None,
                        ftx::rest::OrderType::Market, //change this later so you dont get terrible fees
                        calculation.quantity/q_market.price,
                        None,
                        None,
                        None, //when implimenting close limit order change to true
                        None
                    ).await?;

Also it does actually execute the order on FTX but it errors I think when trying to get a result. I have no idea why it thinks ordertype is missing.

Gives this error:

While using place_order: missing field orderType at line 1 column 336

This happens when trying to initiate an order with TakeProfit order type.

Error Message:

Function Exited: unknown variant `take_profit`, expected one of `market`, `limit`, `stop`, `trailingStop`, `takeProfit` at line 1 column 114

Currently, if I subscribe two different markets, there's no way for me to distinguish the message: https://github.com/fabianboesiger/ftx/blob/main/src/ws/model.rs#L101. Maybe just expose the Response type and let the user deal with the data structure?

Hi, I may be showing my Rust naivety here, but am I correct in thinking the the REST implementation is not able to a call to another request until the current one finishes?

For example using roughly Rust ish pseudo code i would like to do the following without blocking.

`
loop {

let order = api.request(PlaceOrder::new(/*fill this in to place a market but of 1 btc */));

//check if order has completed yet if not continue to loop and send orders 

// if order has compelted let's process then continue to market buy 1 btc in this loop

}

`

I apoligise if this is not the correct forum to ask this question, and you can happily delete if this is the case.

thanks!

See #118 for details.

Maybe FTX added a new WS message type. I got this error yesterday: unknown variant info, expected one of subscribed, unsubscribed, update, error, partial, pong at line 1 column 15