Unwrapping on account validation logic will give opaque errors that will hurt the developer experience https://github.com/Bonfida/dex-v3/blob/main/program/src/processor/new_order.rs#L355-L359. Instead, one can provide specific/custom errors.

When trying to get a wallet's orders for a market, the function always returns an empty array, even though the wallet has open orders.

The issue seems to be that in the filterForOpenOrdersFromSlab function, where
openOrders?.address.equals(new PublicKey(o.callbackInfo.slice(32))) //market.ts, line 711

is called.
The public key created from o.callbackInfo.slice(32) always results in ''11111111111111111111111111111111" - this being the toString value of the created public key.

market.tickSize() returns Math.pow(10, -quoteDecimals) currently, I believe this is incorrect.

I am posting this issue to serve more as a discussion platform after the recent situation involving the presumed FTX hack and its effects on Serum DEX.

From my understanding, a summary of what (people think) happened sound like this:

FTX backed the original Project Serum. Also, FTX was holding the private key of the Update Authority of Serum DEX program. With FTX being hacked, Serum's update private key is thought to be compromised, which in turn made a lot of exchanges, wallets, and others to stop using Serum. Volume decreased by 80-90% day by day since the beginning of the weekend. In order to remove any fears of Serum being linked to the FTX Group, developers started working on a fork.

Anyway, correct me if I'm wrong, please.

How will this affect Bonfida's Serum DEX v4 that will be on mainnet at some point?

Hopefully, this thread will clarify fears and questions.

Apparently client order IDs are really important to market makers and this new version of the serum dex doesn't appear to implement them

Suggest to change minOrderSizeand tickSize to return UI amounts to stay backwards compatible with v3, and then create new functions for getting BN quantities e.g. minOrderSizeBN and tickSizeBN

Is there anything stopping an attacker from passing in an arbitrary account here https://github.com/Bonfida/dex-v3/blob/main/program/src/processor/create_market.rs#L103-L104 and overwriting the data?

The cranker code is only getting maker callback whereas taker is also required in order to make it work.

But why it's working without error?
Because in the consume events code instead of getting taker from maker. It might be an intended behavior but getting taker from maker seems a bit strange.

base_vault, quote_vault and ideally the orderbook account would all be PDAs associated with the dex state.

We should use dynamic sysvars instead of passing in the clock here https://github.com/Bonfida/dex-v3/blob/main/program/src/processor/create_market.rs#L85 and rent here https://github.com/Bonfida/dex-v3/blob/main/program/src/processor/initialize_account.rs#L48 (and everywhere else) so that we can minimize tx size.

I am trying to run the dex-v4 project on my local macbook machine but unable to get started.

I am using
solana-install 1.13.5 (src:91384943; feat:1365939126)

Please let me know which right tools need to compile this project.

In the existing dex one cannot close an account in the same transaction as opening + trading. Even IOC take orders mutate some open orders state that needs to be cleared on the consume events crank. Specifically, the free_slot_bits are not cleared and so closing an account hits this line https://github.com/project-serum/serum-dex/blob/master/dex/src/state.rs#L2292.

This is particularly problematic for swaps because it makes fees appear larger than they are, since one has to pay for rent exemption sol for the open orders account.

Could make this as a pull request but probably simpler to post the solution here and let you guys add the fix when convenient. It doesn't look like a loss of funds issue but it definitely could eventually brick someone's account if their locked token accounts hit u64::max.

Consume events function only increases token_free values, it never reduces the token_locked values. Essentially you're missing quote_token_locked.checked_sub and base_token_locked.checked_sub.

Compare with how these values are updated in the cancel orders

Hello,

The filterForOpenOrdersFromSlab from market.ts tries to convert the price and size from BN to JS number (toNumber()) that is capable of only storing 53 bits.

In some cases, this will not work. For example, if you have a quoteCurrencyMint with 6 decimals and a quoteCurrencyMultiplier of 10000 (10^4) and you place an order of UI value of 1000000 (+ with the adjustments of 10^6 because of the 6 decimals and the 10^4 quoteCurrencyMultiplier to get it to raw unscaled form) will fail to be converted toNumber() because it will exceed the 53 bits JS limit.

I can go ahead and create a PR with the simple change of removing the toNumber(), but I don't want to create breaking changes that could affect any other devs using this library.

Maybe we should add an optional parameter that will tell whether we want the price and size to stay BN or to be converted toNumber() and set the default value to be the conversion toNumber() in order to not break anything for the lib consumers?

Also, this will be in the spirit of consistency with the _parseSlab function that returns the price and the size as BN.

We should consider adding a circuit breaker mechanism to automatically pause a market in extraordinary circumstances, e.g., a crank hasn't been run in some period of time or price changes are too extreme.

The AccountInfo owner (not the data owner) field is not validated here https://github.com/Bonfida/dex-v3/blob/main/program/src/processor/create_market.rs#L109, allowing one to hijack a given market address for PDA markets that don't do this validation them self, since subsequent instructions might simply not work if accounts not owned by the spl token program are used as vaults.

Unable to build cranker with the following command:

cargo build

Here is screenshot with errors:

Hello,

I am trying to produce a .so file that can be deployed on the devnet in order for me to experiment with the code.

Reading the documentation, I understood that I should use the cargo build-bpf to produce the .so file by giving as parameters the path to the .toml file (the manifest path) and an output directory for the result.

After some tries, I am getting this:

Can you help me figure out why I get this?

Also, if you can walk me through the steps to produce a .so deployable from the program in this repo I would greatly appreciate it!

Thanks a lot!

L.E.: After a cargo update in program I get:

In Serum V3 many users / market makers submit consume events instructions along with other instructions in order to ensure that any cancelled or matched orders have been processed and the freed assets can be used. However, it's not possible to know ahead of time whether or not the event queue has already been cleared. Therefore it's a bad idea to explicitly error if no events are consumed. It would be better for the function to silently fail or return.

This is more of a general design question. What's the logic / reasoning for not autosettling the proceeds of cancelled orders? I.e. why not just send the SPL tokens from the cancelled order straight back to the users wallet from the vault. Why require an extra settling step / instruction?

I think it would be beneficial to log out the market address with the order summary in the tx. When using the logs to index trade data it is important to know where the order is coming from. Here is an example order summary log from testnet.

https://solscan.io/tx/a7szfpDtYRQ7e1Hnqf7TAV4LBZcMHd2VLMvyF19qL23zqDn32JBr8YMdmkGH6NGPUAh1U1aT4bCv8CFEj5Q45vK?cluster=devnet

The following Cargo.toml settings have been shown to reduce compute units

[profile.release]
lto = "fat"
codegen-units = 1

[profile.release.build-override]
opt-level = 3
incremental = false
codegen-units = 1

Example here project-serum/serum-dex#159

Hello,

I am trying to make sense of the base and quote currency multipliers. I don't think I got why do we use them or how do we use them.

Aren't the decimals numbers of the mints enough?

Let's say I have a market with the following pair:

• base mint: 0 decimals
• quote mint: 6 decimals

1. What should be the base and quote currency multipliers and why?
2. What's the difference between them being 1 or, say, 10^6, 10^5, etc.?
3. Has the tick size anything to do with these?

Sorry if these are silly question, I'm just trying to figure how everything ties together.

Problem:

Markets accounts are susceptible to account hijacking in the event a user closes a market and an attacker immediately refunds the lamports in a subsequent instruction (in the same transaction).

This is problematic because it means the market can be re-initialized with different parameters, while users, UIs, bots, etc may still be expecting the old parameters by virtue of the account's existence. This is particularly problematic in the event a PDA--say, derived from a new program that's created in the future--is used to create a market, since it means one could have a PDA initialized by something other than deriving/base program.

Solution:

On closing, add a "tombstone" flag to the market account and zero out the rest of the data. Whenever the market account is deserialized or initialized, one can check for the tombstone and ensure it doesn't exist. If it does, you know the account is in limbo and so one should abort--and subsequently try to re close the account.

When calling the Market.load function I am always greeted with the followng error:
Unexpected 128 bytes after deserialized data at deserialize (index.js?8074:405:1) at MarketState.retrieve (state.js?92d6:51:1) at async Market.load (market.js?8329:55:1)
I am building the public key, market address and program id using the solana/web3 classes.
This is happening when trying to connect to the devnet.

I'm trying to get the orders of a wallet that I'm connected to with a wallet adapter, but when calling the loadOrdersForOwner function, it gets stuck on the following error:

Error while getting orders: Error: Invalid public key input
at new PublicKey (index.browser.esm.js?156f:151:1)
at eval (market.js?8329:403:1)
at Array.filter ()
at Market.filterForOpenOrdersFromSlab (market.js?8329:403:1)
at Market.filterForOpenOrders (market.js?8329:422:1)
at Market.loadOrdersForOwner

I investigated where that happens, and it seems that in the functions called inside loadOrdersForOwner get to the calling of filterForOpenOrdersFromSlab where it breaks on the following line:
return [...slab]
.filter((o) => openOrders?.address.equals(new PublicKey(o.callbackInfo))) //(market.ts, line 710)

So basically what I suspect is happening is that the callbackInfo from slab objects is not a good input for generating a public key. I tested it with just getting the bids and trying to map their callback info to public keys, and it does indeed break.

Did I do something wrong when creating the market that causes this issue, or is it something else?

Market address:
wUYvayJxdPwXRFuf7E8y9LVSHuVkXhbZWNEbcE2DgTJ

wallet in question's public key:
DrkEjikTWi6VzeLykTaduG9hdXerjiZBMSrhEgb7mX4c

Everything is on devnet.

There's a bunch of weird artifacts left over in how the code and accounts are structured due to the previous use of AOB as an external program.

The obvious ones are

• The orderbook account for the AOB, there's no reason it's a separate account from the dex market account. It's taking up an additional account space.
• Additional compute overhead of calling processor function which were designed to run in a separate program. For example, the consume events function is pretty much a copy paste of the AOB consume events function but then it also calls AOB consume events. The only unique code is in popping events off the event queue but this is something which could very easily be added into the serum v4 code, without needing to call the AOB consume events function.
• Reading order summaries off of the event queue register instead of simply returning them from the AOB new order function

The open orders account should be a user associated PDA to eliminate calls to getProgramAccounts and provide better UX.

I am trying to settle funds for an account on a market I created. The settle funds requires the base currency account and the quote currency account of the wallet. The issue is that since this wallet only had the quote currency of the market, and thus no base currency account. I have tried to find a way to create a base currency account if needed, but this far I have failed at it.

I am getting the currency accounts in the following way:

  const walletAccountsForQuote = await connection.getTokenAccountsByOwner(
          wallet.publicKey,
          {
            programId: new PublicKey(TOKEN_PROGRAM_ID),
            mint: market.quoteMintAddress,
          }
        );
        const quoteCurrencyAccountPubkey =
          walletAccountsForQuote.value[0].pubkey;
          
 ////
 
  const walletAccountsForBase = await connection.getTokenAccountsByOwner(
          wallet.publicKey,
          {
            programId: new PublicKey(TOKEN_PROGRAM_ID),
            mint: new PublicKey(marketInfo.mintAddress),
          }
        );

        const baseCurrencyAccountPubkey = walletAccountsForBase.value[0].pubkey;
        

   market address is: wUYvayJxdPwXRFuf7E8y9LVSHuVkXhbZWNEbcE2DgTJ
   wallet public key is: DrkEjikTWi6VzeLykTaduG9hdXerjiZBMSrhEgb7mX4c

An arbitrary account from an arbitrary program can be used as the orderbook when creating a market https://github.com/Bonfida/dex-v3/blob/main/program/src/processor/create_market.rs#L125. One should check the owner of the orderbook is the expected agnostic orderbook program ID. Without this, one can hijack a PDA market account.

This would be very useful for swap UIs and composing programs that don't want to deal with fees and the added complication of creating/removing an open orders account

Hello,

I have created a market using the following:

• cluster: devnet
• program ID: HEHEVQNZMTmjUt3CUKdddqHWa4dtXqA9QUsi7QptCCHj
• market public key: 7nuVpoFKARP8BYwFXGVujS3CfHWtrbtjbaCzEbbtJtdD
• base decimals = 0
• quote decimals = 6
• base multiplier = 1
• quote multiplier = 10000
• tick size = 0.01 * 2^32
• royalties bps = 1000 (10%)

Then I have set up 2 wallets: a seller (also the one and only creator with 100% share of the royalties fees) and a buyer. Then I have done the following:

1. Seller sells 1 base token for 1 quote token (UI price)
2. Buyer buys 1 base token for 1 quote token. He was prompted to actually send 1.089 quote tokens
3. Buyer settles and gets 1 base token
4. Seller consumes events, settles and gets the 0.99 base tokens
5. Seller invokes sweep fees --> this transaction fails because it tries to send 990 quote tokens to the creator. It should have sent him around 0.099 - 0.1 quote tokens. It seems that it tries to send him 10000 times more then it actually should. Seems like this 10000 it's the same number as the quote multiplier, in case that might have something to do with it.

The transaction can be found at: https://solscan.io/tx/FUtC7fDkJuLTBkPuBmNXKutCNL3XYZJkTmZr176gPdcCv6z6UQQ3MhquUwRacAxQRQDefEgGEuPzUa9L3tske5x?cluster=devnet

I looked a bit through the code, but I don't find a place where a raw amount (with decimals)/UI amount (no decimals) conversion or scale/unscale conversion is missed.

Appreciate it a lot if you could help me with this!

I am trying to connect to a market that I created, but the Market.load function crashesh with the following message:
Error: Number can only safely store up to 53 bits

The stack trace points to the computeUiPrice function in the utils file. The tick size I put for the market was new BN(100) and the minBaseOrderSize new BN(1).

The connection url is: https://api.devnet.solana.com/
The market address: 2v3gEUKpBWRircVW6PXG1Q8bd8C3g5WrYbRNfpoxHRT1
The program id: HEHEVQNZMTmjUt3CUKdddqHWa4dtXqA9QUsi7QptCCHj
The mint address: AF5DV1Ngx5BZ8pSPE1XCUvpoRCeMjHRsH2AFqu3zyCLe

• In createMarket, metadataAccount should be optional
• In makePlaceOrderTransaction, clientOrderId should be optional
• Change tickSize parameter to be a number rather than BN or change minBaseOrderSize to be a BN rather than a number, for consistency
• Document what the unit scaling is for tickSize and minBaseOrderSize (e.g. should you pass in uiAmount, uiAmount * 2 ** 32, uiAmount * 10 ** quoteDecimals, or something else?)

Hello,

I want to ask some questions and touch base a bit about what the cranking process is and what are the best practices by which it should be done.

What I know from examining the code:

1. Cranking is done through the consume_events instruction
2. Each invocation of the consume_events instruction needs to know how many items in the queue it should read and process (through the max_iterations parameters)
3. Each invocation of the consume_events instruction needs to provide the DEX program with the open_orders_accounts of all the users that are referenced by the events which are about to be processed. This can be done by loading the event_queue and extracting their open_orders_account from the Event by the following rule: in case it's an EventFill then it's taken from the event_fill.makerCallbackInfo.slice(0, 32); in case it's an EventOut then it's taken from the event_out.callbackInfo.slice(0, 32)
4. A taker doesn't need to have his events consumed because the new_order instruction already takes care of updating the open_orders_account in case there was at least a partial fill for his order
5. A maker has to have his events consumed by a crank in order to get his open_orders_account updated after orders of his have been matched
6. accumulated_royalties and accumulated_fees belonging to the market are updated by consuming events

What I am not sure about and would like to figure out:

1. I thought about ways of embedding the consume_events instruction in different transactions. For example:

• Transaction(new_order, consume_events) so that after a taker will get his order matched, he will also consume the events that would update the open_orders_account of the maker, making the latter able to settle his funds.
• Transaction(consume_events, settle) so that before a maker tries to settle his funds, he makes sure he consumed the events that will update his open_orders_account in order to actually have something to settle

However, I don't think these solutions are good. For example:

• For the first bullet above,: it seems that the events pushed to the queue in the case of a match are not "seen" in the consume_events instruction that belongs to the same transaction. But if I create a second transaction with just the consume_events instruction and execute it after the new_order this will work
• For the second bullet above: this is too sketchy because there could be multiple events waiting in the queue that don't belong to the maker trying to get his open_orders_account updated. Given the fact that he needs to specify a max_iterations, he could consume the next X events that belong to other users and were not consumed yet, so he would end up cranking the market for others and not consume enough events to reach his own events.

All in all, I think trying to embed the consume_events instruction in users' trading transaction is a dead-end. This leads to the point 2) below.

2. There is a cranker folder in the repo. The code looks like something that needs to be run in a background job by the market admin at a particular frequency. The frequency can be set according to the liquidity in the market I guess.

Here are the questions:

• Is the background job the way to go regarding cranking/consuming events of a market?
• If it is, any suggestions about the number of iterations it should consume at each run? Is there a limit for how many events can be consumed?

Hey, just wondering if the NPM package(js folder) documentation will be coming soon?