aminhp / gym-mtsim Goto Github PK

hi Mohammad ,
is it possible to use this code to live trading via MT5?
if yes how.
thanks

ofer

class LocalMtSimulator(MtSimulator):
    def download_data(self, symbol: str, df, symbol_info: SymbolInfo) -> None:
        """load dataframe from disk"""
        # symbol_df = pd.read_excel(path)
        self.symbols_info[symbol] = symbol_info
        self.symbols_data[symbol] = df

dataset = sqlite2df("data\\my\\kb8w.db", table_name='temp')
dataset['datetime'] = dataset['datetime'].apply(lambda x: bt.num2date(x))
dataset = dataset.rename(
        columns={"datetime": "Time", "open": "Open", "close": "Close", "high": "High", "low": "Low",
                 "volume": "Volume"})

sim = LocalMtSimulator(
        unit='CNY',
        balance=10000.,
        leverage=5,
        # stop_out_level=0.2,
        # hedge=True,

    )
sim.download_data('rb', dataset, symbol_info=SymbolInfo(
    info={
        "name": "rb",
        "path": "CTP\\rb",

        "currency_margin": 0,
        "currency_profit": 0,
        "trade_contract_size": 0,
        "margin_rate": 0.2,
        "volume_min": 0,
        "volume_max": 0,
        "volume_step": 0
    }

the above is code structure of how i load my own df, and when i fill symbolinfo class, i found i dont understand meaning of following parameters :
"currency_margin": 0,
"currency_profit": 0,
"trade_contract_size": 0,
"margin_rate": 0.2,
"volume_min": 0,
"volume_max": 0,
"volume_step": 0
can u explain these? Thank you.

Hi Amin,

What workflow do you recommend to debug changes if overriding methods and testing. This is for making sure everything is correct before training, but could also apply for training cycle itself.

Many thanks.

Code:

sim = MtSimulator(
    unit='USD',
    balance=10000.,
    leverage=100.,
    stop_out_level=0.2,
    hedge=False,
)


sim.download_data(
    symbols=['EURUSD', 'GBPCAD', 'GBPUSD', 'USDCAD', 'USDCHF', 'GBPJPY', 'XAUUSD'],
    time_range=(
        datetime(2021, 5, 5, tzinfo=pytz.UTC),
        datetime(2021, 12, 5, tzinfo=pytz.UTC)
    ),
    timeframe=Timeframe.M5
)
sim.save_symbols("C:\\symbol.pkl")

Error:

  File "test.py", line 15, in <module>
    sim.download_data(
  File "D:\Porjects\rlmt\gym-mtsim-main\gym_mtsim\simulator\mt_simulator.py", line 59, in download_data
    si, df = retrieve_data(symbol, from_dt, to_dt, timeframe)
  File "D:\Porjects\rlmt\gym-mtsim-main\gym_mtsim\metatrader\api.py", line 20, in retrieve_data
    symbol_info = _get_symbol_info(symbol)
  File "D:\Porjects\rlmt\gym-mtsim-main\gym_mtsim\metatrader\api.py", line 53, in _get_symbol_info
    symbol_info = SymbolInfo(info)
  File "D:\Porjects\rlmt\gym-mtsim-main\gym_mtsim\metatrader\symbol.py", line 9, in __init__
    self.name: str = info.name
AttributeError: 'NoneType' object has no attribute 'name'

Just like the parameter 'df' of Env in anytrading, how can I load my own dataframe in MtSim . I see in the example that only the .pkl file is loaded

Hi
how can I download new tick data for Symbols from MT5?
is there any way to get live tick data and test the model on it?

Thanks

---------------------------------------------------------------------------
AssertionError                            Traceback (most recent call last)
[<ipython-input-6-4ca9ec63cf36>](https://localhost:8080/#) in <module>()
     11 # env = gym.make('crypto-hedge-v0')
     12 
---> 13 model = A2C('MultiInputPolicy', env, verbose=0)
     14 model.learn(total_timesteps=1000)
     15 

2 frames
[/usr/local/lib/python3.7/dist-packages/stable_baselines3/common/base_class.py](https://localhost:8080/#) in __init__(self, policy, env, learning_rate, policy_kwargs, tensorboard_log, verbose, device, support_multi_env, create_eval_env, monitor_wrapper, seed, use_sde, sde_sample_freq, supported_action_spaces)
    189                 assert np.all(
    190                     np.isfinite(np.array([self.action_space.low, self.action_space.high]))
--> 191                 ), "Continuous action space must have a finite lower and upper bound"
    192 
    193     @staticmethod

AssertionError: Continuous action space must have a finite lower and upper bound

Where can set the lower and upper bound? Thank you

Is it possible to run a created model on an actual MetaTrader5 Instance?

Hi @AminHP I would appreciate if you could provide example on how to implement stop loss and take profit just like in metatrader5.

Thanks,

Hello,

I recently discovered this project and was eager to try it out. However, after installing the project using pip, attempting to import it immediately resulted in an error. The error message is as follows:

Traceback (most recent call last):
File "/mnt/c/Users/franc/Documents/GitHub/GCollab_Lstm_Renforcement/test.py", line 1, in
from gym_mtsim import OrderType
File "/home/osdann/.local/lib/python3.10/site-packages/gym_mtsim/init.py", line 13, in
'original_simulator': MtSimulator(symbols_filename=FOREX_DATA_PATH, hedge=True),
File "/home/osdann/.local/lib/python3.10/site-packages/gym_mtsim/simulator/mt_simulator.py", line 42, in init
if not self.load_symbols(symbols_filename):
File "/home/osdann/.local/lib/python3.10/site-packages/gym_mtsim/simulator/mt_simulator.py", line 73, in load_symbols
self.symbols_info, self.symbols_data = pickle.load(file)
File "/home/osdann/.local/lib/python3.10/site-packages/pandas/core/internals/blocks.py", line 2728, in new_block
return klass(values, ndim=ndim, placement=placement, refs=refs)
TypeError: Argument 'placement' has incorrect type (expected pandas._libs.internals.BlockPlacement, got slice)".

I am using WSL 2 and have set up a new Conda environment specifically for this.
So far, I have only installed the latest version of mtsim and bs3.

Could there be an issue on my end ? Any help would be greatly appreciated.

Thank you.

how do i set the maximum lots?

Hi, this repository is currently listed in the gymnasium third party environments but we are cleaning the list up to only include maintained gymnasium-compatible repositories.

Would it be possible for it to be upgraded from gym to gymnasium? Gymnasium is the maintained version of openai gym and is compatible with current RL training libraries (rllib and tianshou have already migrated, and stable-baselines3 will soon).

For information about upgrading and compatibility, see migration guide and gym compatibility. The main difference is the API has switched to returning truncated and terminated, rather than done, in order to give more information and mitigate edge case issues.

hello @AminHP ,
There is a bug in the check volume function inside the simulator, you write :

 def _check_volume(self, symbol: str, volume: float) -> None:
        symbol_info = self.symbols_info[symbol]
        if not (symbol_info.volume_min <= volume <= symbol_info.volume_max):
            raise ValueError(
                f"'volume' must be in range [{symbol_info.volume_min}, {symbol_info.volume_max}]"
            )
        if not round(volume / symbol_info.volume_step, 6).is_integer():
            raise ValueError(f"'volume' must be a multiple of {symbol_info.volume_step}")

you are rounding the volume to 6 decimals and expecting an integer as a result, this can only be true if the passed volume was an integer, your error message says mutiple of 0.01 (volume step), this also contradicts with your voume check function inside env._get_modified_volume where you expect volume to be mutiple of 0.01 as well.

i try to add yolo model make detection in the env and the prediction the RL model take it based on this prediction the RL model can take an action in the env

please help me to add technical indicator like any trading

Hi, thank you for sharing your code. Can you provide more examples of models? Because the performance of A2C is not very good, I am trying to use my own data (including newly added signal_features) and experimenting with different models, but they are not performing well, especially the DQN model. This may be a silly question, please forgive my unfamiliarity with reinforcement learning. Additionally, I encountered errors when using the DDPG model. I have included my modifications and the error below. I would be grateful for your help.
` def _process_data(self, keys: List[str]=['Stochastic_K_1', 'Stochastic_D_1', 'Stochastic_K_2', 'Stochastic_D_2', 'MACD_DIF',
'MACD_DEA', 'MACD_Histogram', 'Moving_Average']) -> Dict[str, np.ndarray]:
signal_features = {}

    for symbol in self.trading_symbols:
        get_signal_at = lambda time: \
            self.original_simulator.price_at(symbol, time)[keys]

        if self.multiprocessing_pool is None:
            p = list(map(get_signal_at, self.time_points))
        else:
            p = self.multiprocessing_pool.map(get_signal_at, self.time_points)

        signal_features[symbol] = np.array(p)
    # data = self.prices
    signal_features = np.column_stack(list(signal_features.values()))
    return signal_features`

Model seems working fine, when I try model.learn, I get the error.

If I adjust upper and lower bound in the action space (low=-1e10, high=1e10) the model does not place any order.

Hi HP,

Thanks for ur great project, just a thought on line 85 of mt_simulator.py

    for order in self.orders:
        order.exit_time = self.current_time
        order.exit_price = self.price_at(order.symbol, order.exit_time)['Close'] # Should be Open price
        self._update_order_profit(order)
        self.equity += order.profit

I think you meant open price? because the candlestick is timed at the open time of the bar.

Unsuccessfully trying to get a list of all balance values within each timestep inside of the reward method. So, at each call to _calculate_rewards(), I am trying to extract a list of all historical balances (for that timestep) - i.e. each call should add 1 historical balance.

I've tried calculating it within MtSimulator but this gets too complicated.

Poking around self.history but unable to successfully unwrap it for this.

Thanks

this example does not open any trades

balance: 10000.0, equity: 10000.0, margin: 0.0
free_margin: 10000.0, margin_level: inf

just different from expected results you wrote

One of the core dependency, MetaTrader5 is only available to Windows but not Linux. This causes the gym-mtsim to be unable to run on Linux machines.

Any ideas why the code below (it is basically the same as the code in the README.md file) would return different results each time it is run after the model has been trained? The trained model is no longer being updated and the input (trades) do not change between runs.

`
model = A2C.load(modelName, env)

observation = env.reset()
while True:
    action, _states = model.predict(observation)
    observation, reward, done, info = env.step(action)
    
    state = env.render()
    z = calcsomevalue(state)
    if done:
       x  = getsomevalue(z)
       y = calsomevalue(z)
        print(<results>)
        break

`

hi Mohamad,
I prepared an example of using pyfolio to analyze results, but it require small changes to env and sim.
if you want I can send you the changes to push.
best,
ofer

Hi
Could i please be helped with this issue:

Traceback (most recent call last):
File "c:\Users\seanm\OneDrive\Desktop\Test\test.py", line 2, in
import gym_mtsim
File "C:\Users\seanm\OneDrive\Desktop\Test\sean\lib\site-packages\gym_mtsim-1.3.0-py3.10.egg\gym_mtsim_init_.py", line 13, in
'original_simulator': MtSimulator(symbols_filename=FOREX_DATA_PATH, hedge=True),
File "C:\Users\seanm\OneDrive\Desktop\Test\sean\lib\site-packages\gym_mtsim-1.3.0-py3.10.egg\gym_mtsim\simulator\mt_simulator.py", line 37, in init
if not self.load_symbols(symbols_filename):
File "C:\Users\seanm\OneDrive\Desktop\Test\sean\lib\site-packages\gym_mtsim-1.3.0-py3.10.egg\gym_mtsim\simulator\mt_simulator.py", line 73, in load_symbols
self.symbols_info, self.symbols_data = pickle.load(file)
File "C:\Users\seanm\OneDrive\Desktop\Test\sean\lib\site-packages\pandas-2.1.1-py3.10-win-amd64.egg\pandas\core\internals\blocks.py", line 2400, in new_block
return klass(values, ndim=ndim, placement=placement, refs=refs)
TypeError: Argument 'placement' has incorrect type (expected pandas._libs.internals.BlockPlacement, got slice)

i suspected pandas , tried to uninstall the one i had and used the setup.py but still same problem

please help

1 The _process_data() method takes the return value of _get_prices() as a feature, but the return value of _get_prices() is two columns of bar data which are close and open, and this are not features. Why?
2 _get_observation() returns a dict, but not a np.ndarray, for a typicle gym env， Shouldn't the observation returned by the step method be a np.ndarray?
3 I see many data structures in this framework use Dict, which should be to support multiple symbols? I only load data of a single symbol, so should I change the underlying data structure,? or just put a single key (the single symbol i focus on) in the dict.

Hello @AminHP ,
What data do you store inside the pickle files in the data folder?

Hello,
Thank you so much for the simple and wonderful library.
can you please explain what is this fucntion doing?
why do we need to get the unit ratio to calculate order profit?

    def _get_unit_ratio(self, symbol: str, time: datetime) -> float:
        symbol_info = self.symbols_info[symbol]
        if self.unit == symbol_info.currency_profit:
            return 1.

        if self.unit == symbol_info.currency_margin:
            return 1 / self.price_at(symbol, time)['Close']

        currency = symbol_info.currency_profit
        unit_symbol_info = self._get_unit_symbol_info(currency)
        if unit_symbol_info is None:
            raise SymbolNotFound(f"unit symbol for '{currency}' not found")

        unit_price = self.price_at(unit_symbol_info.name, time)['Close']
        if unit_symbol_info.currency_margin == self.unit:
            unit_price = 1. / unit_price

        return unit_price

Hello @AminHP ,
I noticed that you are using a constant leverage of 100 during the construction of the MtSimulator object, as you know leverage could be variant on the order level, means every order could have a different leverage, did you think of making leverage as part of the action space?
I was thinking that allowing the agent to decide on the leverage (1x,5x,10x...etc) could allow it to make better decisions?

Hello @AminHP ,
why is the low bound of balance, equity and features is -np.inf? see code:

        self.observation_space = spaces.Dict({
            'balance': spaces.Box(low=-np.inf, high=np.inf, shape=(1,)),
            'equity': spaces.Box(low=-np.inf, high=np.inf, shape=(1,)),
            'margin': spaces.Box(low=-np.inf, high=np.inf, shape=(1,)),
            'features': spaces.Box(low=-np.inf, high=np.inf, shape=self.features_shape),
            'orders': spaces.Box(
                low=-np.inf, high=np.inf,
                shape=(len(self.trading_symbols), self.symbol_max_orders, 3)
            )  # symbol, order_i -> [entry_price, volume, profit]
        })

shouldnt low bound start from zero? meaning its impossible for balance be below 0

Specifically, how do I set 'volume', 'volume_step', 'volume_min' and 'volume_max' without creating a child of the Order class? I need to be able to set these values potentially for each order.

May I ask, How can I fix trading volume or lot size?

thanks in advance for your answer

Hi,

just one question is it possible to make simulator to trade XAUUSD ? bcs as i see by default it uses only currency pairs or may be its possible to feed my data to simulator as pandas dataframe or .cvs file?

Thank you in advance!

Hello @AminHP ,

Do you know why is the volume value always small? i noticed its usually in small numbers such as in the rough range [-5,+5]
I changed the volume to "Amount" but i didnt touch the action space (which should be [-inf,+inf]), i am wondering if the action space is close to inf, why am i getting very small numbers when the action is generated by gym?
I know i can mutiply or square the value, but was just wondering if i am missing something ? albeit i didnt change anything in the code.

Hi!

Any way to check if market is closed at a given date and time? Like holydays and weekends.

Thanks!

It appears you have not considered the fee per lot charged on some asset classes by some brokers. Eg it is common in my experience for there to be zero commission on indices but a commission per lot per round turn of $6-7...

In my opinion, balance mean to cash. So when open a new order, the balance should minus cash used in orders. But in _create_hedged_order function, not update balance. It's a bug or I have misunderstand？

Hello @AminHP ,
when running one of your examples (Create an environment with custom parameters), i get the follwoing error:
File "", line 1, in
File "C:\Users\Ali.Khankan\AppData\Roaming\Python\Python39\site-packages\multiprocess\spawn.py", line 116, in spawn_main
exitcode = _main(fd, parent_sentinel)
File "C:\Users\Ali.Khankan\AppData\Roaming\Python\Python39\site-packages\multiprocess\spawn.py", line 125, in _main
prepare(preparation_data)
File "C:\Users\Ali.Khankan\AppData\Roaming\Python\Python39\site-packages\multiprocess\spawn.py", line 236, in prepare
_fixup_main_from_path(data['init_main_from_path'])
File "C:\Users\Ali.Khankan\AppData\Roaming\Python\Python39\site-packages\multiprocess\spawn.py", line 287, in _fixup_main_from_path
main_content = runpy.run_path(main_path,
File "C:\Program Files\Python39\lib\runpy.py", line 268, in run_path
return _run_module_code(code, init_globals, run_name,
File "C:\Program Files\Python39\lib\runpy.py", line 97, in _run_module_code
_run_code(code, mod_globals, init_globals,
File "C:\Program Files\Python39\lib\runpy.py", line 87, in _run_code
exec(code, run_globals)
File "c:\Users\Ali.Khankan\Desktop\gym-mtsim\ali_test.py", line 16, in
env = MtEnv(
File "c:\Users\Ali.Khankan\Desktop\gym-mtsim\gym_mtsim\envs\mt_env.py", line 63, in init
self.multiprocessing_pool = Pool(multiprocessing_processes) if multiprocessing_processes else None
File "C:\Users\Ali.Khankan\AppData\Roaming\Python\Python39\site-packages\pathos\multiprocessing.py", line 111, in init
self._serve()
File "C:\Users\Ali.Khankan\AppData\Roaming\Python\Python39\site-packages\pathos\multiprocessing.py", line 123, in _serve
_pool = Pool(nodes)
File "C:\Users\Ali.Khankan\AppData\Roaming\Python\Python39\site-packages\multiprocess\pool.py", line 212, in init
self._repopulate_pool()
File "C:\Users\Ali.Khankan\AppData\Roaming\Python\Python39\site-packages\multiprocess\pool.py", line 303, in _repopulate_pool
return self._repopulate_pool_static(self._ctx, self.Process,
File "C:\Users\Ali.Khankan\AppData\Roaming\Python\Python39\site-packages\multiprocess\pool.py", line 326, in _repopulate_pool_static
w.start()
File "C:\Users\Ali.Khankan\AppData\Roaming\Python\Python39\site-packages\multiprocess\process.py", line 121, in start
self._popen = self._Popen(self)
File "C:\Users\Ali.Khankan\AppData\Roaming\Python\Python39\site-packages\multiprocess\context.py", line 327, in _Popen
return Popen(process_obj)
File "C:\Users\Ali.Khankan\AppData\Roaming\Python\Python39\site-packages\multiprocess\popen_spawn_win32.py", line 45, in init
prep_data = spawn.get_preparation_data(process_obj._name)
File "C:\Users\Ali.Khankan\AppData\Roaming\Python\Python39\site-packages\multiprocess\spawn.py", line 154, in get_preparation_data
_check_not_importing_main()
File "C:\Users\Ali.Khankan\AppData\Roaming\Python\Python39\site-packages\multiprocess\spawn.py", line 134, in _check_not_importing_main
raise RuntimeError('''
RuntimeError:
An attempt has been made to start a new process before the
current process has finished its bootstrapping phase.

    This probably means that you are not using fork to start your
    child processes and you have forgotten to use the proper idiom
    in the main module:

        if __name__ == '__main__':
            freeze_support()
            ...

    The "freeze_support()" line can be omitted if the program
    is not going to be frozen to produce an executable.

setting multiprocessing_processes = 0 solved the problem

I have tried many time and method for this. I have writen the code below, could you please check for me whether I have done something wrong. In this case I left only this 2 informations for the experiment

model_balance = Sequential()
model_balance.add(Flatten(input_shape=(1,1,1), name='balance'))
model_balance_input = Input(shape=(1,1), name='balance')
model_balance_encoded = model_balance(model_balance_input)

model_equity = Sequential()
model_equity.add(Flatten(input_shape=(1,1,1), name='equity'))
model_equity_input = Input(shape=(1,1), name='equity')
model_equity_encoded = model_Equity(model_equity_input)

con = concatenate([model_Balance_encoded, model_Equity_encoded])

dense = Dense(1024, activation='relu')(con)
dense = Dense(1024, activation='relu')(dense)
output = Dense(actions, activation='softmax')(dense)

model = Model(inputs=[model_Balance_input, model_Equity_input], outputs=output)

memory = SequentialMemory(limit=2000000, window_length=1)
policy = LinearAnnealedPolicy(EpsGreedyQPolicy())
dqn = DQNAgent(model=model, policy=policy, nb_actions=actions, memory=memory)

dqn.processor = MultiInputProcessor(2)

dqn.compile( optimizer = Adam(learning_rate=1e-4), metrics = ['mse'] #'accuracy', 'mae')

dqn.fit(env, nb_steps=1000, verbose=1, visualize=False,)

====================================================================
Then, this problem occured, and I could not find the error.

please tell me the solution.
thanks in advance

.

Hello @AminHP ,
I am struggeling to understand your action space, you say quote: "
The action space is a 1D vector of size count(trading_symbols) * (symbol_max_orders + 2). For each symbol, two types of actions can be performed, closing previous orders and placing a new order. The latter is controlled by the first symbol_max_orders elements and the former is controlled by the last two elements."

you also write:

self.action_space = spaces.Box(
            low=-np.inf, high=np.inf,
            shape=(len(self.trading_symbols) * (self.symbol_max_orders + 2),)
        )  #

why do you +2 to the symbole_max_orders? what are the last 2 elements you are referring to?

for example , my action space has 7 discrete actions, and they correspond to 7 different positions as follows:
heavy long postion (80% long position size)
mid long postion (50% long position size)
light long postion (30% long position size)
not holding any symbols, meaning position size is 0
light short postion (30% short position size)
mid short postion (50% short position size)
heavy short postion (80% short position size)

In order to achieve this, what should I do？

The example you provided uses the same dataset for both training and testing? In order to evaluate the performance of a machine learning model, we need to separate the dataset into training and testing sets? Can you show us how to evaluate the trained model in unseen dataset? thanks

Hello, thanks for this great repo, I wonder, why did you use pkl files and not csv's?
and how can I replace the csv with my own data set to be in the form of csv or even metatrader files?

thanks

Hi Amin,

On gym-anytradnig it's possible to override _process_data and add your own custom indicators (from Finta etc).

Is it possible to do the same with mtsim - or how should I be thinking about this? Currently I can't even see how to add and format my own training data.

Thanks to the author for the excellent repository. Helped me open up a new direction to explore.
I am interested in the issue of creating an environment. In practice, many traders use multiple time frames to decide whether to open a position. For example, I want to analyze the last 200 bars of the H4 time frame and the last 100 bars from the M5 time frame, and I also want to separately transfer bid and ask data to the environment. I wrote several indicators in MQL5 that upload all the necessary data into a sqlite database and it is convenient to use this data to create sequences that form the environment. The stable-baselines3 documentation states that it is possible to create a composite environment. Something like this:
self.observation_space = spaces.Dict({ 'SlowTF': spaces.Box(low=0, high=1, shape=cfg.slowTF_shape, dtype=np.float32), 'FastTF': spaces.Box(low=0, high=1, shape=cfg.fastTF_shape, dtype=np.float32), 'Tick': spaces.Box(low=0, high=1, shape=cfg.tick_shape, dtype=np.float32) })
How can this approach be implemented in gym-mtsim or perhaps in gym-anytrading?
The number of parameters in gym-mtsim, in my opinion, looks unnecessary. unit, balance, equity, margin, leverage, etc. only complicate the learning process and can be implemented directly in the metatrader5 terminal itself using MQL5. If the model is trained only to search for entry and exit points of a position, this will already be enough for its use. MQL5 has built-in support for the ONNX format. The trained model can be exported to ONNX format and used directly in the code of an advisor or indicator as an imported function.

I've check that there is no Metatrader 5 on pip, because it's support Windows OS only.
As I understand, it's impossible to use your module outside of Windows, cause it's contains required for Windows only module.
Maybe you could provide limited using of your module for MacOS / Linux without MetaTrader?
Thank you for your great job, @AminHP!