Install issue about oanda-trading-environment HOT 28 CLOSED

I'll have a look ...
... it needs a closer look. There is probably some python2 ->3 things here. It also looks like virtualenv works differently with the PREFIX var

will try to get the clear picture this weekend

from oanda-trading-environment.

... I noticed there is a long pending python3 pulll request (from myself) also: #9

I will look at this.

from oanda-trading-environment.

Hi, @jjphung

It has been a long time since I used this code. If I would write it now I would focus on deployment by docker.

So, I will strip the daemonizing components from it and see to get stripped version in a container.

... done. Stripped and wrapped it in a container and it seems to work fine. I will clean things up and fine tune the container usage regarding config and logging. I'll try to get this on github this weekend

from oanda-trading-environment.

from oanda-trading-environment.

It looks like you did not create the oanda user and oanda group. The setup.py checks for this user and group.
There is an extensive description on usage/config on: http://hootnot.github.io/oanda-trading-environment/articles/dedicated-server/

It seems you mix-up some things: the environment runs on a linux user/group: oanda/oanda. But to access the REST service of OANDA you will need an account (the account_id) and a token.

Since you run Ubuntu this document should apply. Check the section before IPTABLES "First we want to upgrade the OS and install some extra packages needed to use Ansible"

Ansible takes care of everything: packages, permissions, users ...

After installing you will have a user on you system: "oanda". You can access for instance with : sudo su - oanda. Check the http://hootnot.github.io/oanda-trading-environment/articles/dedicated-server/ for details. It is all there

from oanda-trading-environment.

Thanks for the reply. I've installed python pip, python dev, ansinble, cloned the repo and when running:

sudo ansible-playbook -i hosts ote.yml
Installation fails with this error:
failed: [localhost] (item=[u'packages']) => {"failed": true, "item": ["packages"], "msg": "No package matching 'packages' is available"}

Edit from @hootnot : this was an Ansible related version issue and is patched with #3

from oanda-trading-environment.

@BenGosub : I've done an install on a virtual (debian) myself and run into the same issue.

When I use Ansible 1.9.3 (which was the latest at time of writing OTE) the error is gone, but it seems on Debian it wants libssl-dev installed, so I added it: apt-get install libssl-dev (I guess this goes for Ubuntu also)
Re-running Ansible (which is the nice aspect of an Ansible install) completes the install, but setuptools
did not create the directory structure under the oanda account. You will see this if you su to the oanda-user: sudo su - oanda.
Work-around :

• delete the Ansible version you installed: pip uninstall Ansible
• install Ansible 1.9.3 by : *pip install ansible==1.9.3
• delete the Ansible installed oanda-trading-environment package. You need to be root for that in the virtualenv of the oanda account, so:
  • cd /opt/oanda
  • . ./bin/activate
  • pip uninstall oanda-trading-environment

To re-install:

• mkdir tmp
• cd tmp
• git clone https://github.com/hootnot/oanda-trading-environment
• cd oanda-trading-environment
• python setup.py install

If you now do an ls ~oanda/bin you will see OANDAd, config stuff under ~oanda/etc.

Exit the 'root' session and get access using: sudo su - oanda to configure the environment as described in the docs.

If you think you are lost somewhere, you can always perform a rm -rf /opt/oanda and re-run the Ansible deployment.

It looks like:

• Ansible 2.2.x has problems with the playbook
• on debian (Ubuntu) an extra package is required: libssl-dev
• setuptools fails to create the directory structure with current versions

from oanda-trading-environment.

@hootnot downgrading ansible and installing libssl-dev seems to fixed the issue. After doing this ansible-playbook is able to develop from the deployment branch.

Thanks for the help.

from oanda-trading-environment.

@BenGosub question for you then : do you get the etc/OANDA directory ?
On debian I had to do the setup with: python setup.py install on the git repo. Somehow pip does not install the package_data files. If this goes well on Ubuntu this might be a platform dependant pip-bug.

from oanda-trading-environment.

@hootnot etc/OANDA directory is there. Daemon is running, I just can't open the log file... seems to be missing from var/log. I am also running the pub/sub example script, it doesn't seem to print anything. I've added access token and account id in the config.

from oanda-trading-environment.

It looks like the daemon is sleeping whenever I hit status:

(oanda) oanda@ideapad:~$ sudo /opt/oanda/bin/OANDAd status OANDAd -- pid: 12692, status: sleeping, uptime: 21m, %cpu: 0.0, %mem: 0.3

from oanda-trading-environment.

You should see logging under var/log/OANDA:

• OANDAd.log
• streamdata...

ps ax| grep OANDA should give you 2 processes: a parent and a child

OANDAd logs to OANDAd.log, check it for messages . Raise the loglevel in the config if you still have issues

tail -f var/log/OANDA/streamdata should give you live the ticks received.

from oanda-trading-environment.

@hootnot I've missed to delete one underscore when inserting the access token. Logging and streaming works now.

from oanda-trading-environment.

@BenGosub I guess the 0MQ example will work also ...

from oanda-trading-environment.

Ou, I newly discovered the OTE, but looks like it's been around for a while. I thought that oandapy probably is the best way to get away from MT4, and found your marvellous wrapper for it, even for a newbie ! A question please. When I tried to install ote I get this "
etc/OANDA": 0700,
^
SyntaxError: invalid token" error. I'm running this on Ubuntu 16.04 VPS, and venv as per your guide.
I really would like to try this environment, so highly appreciate for any adivse.
Cheers,

from oanda-trading-environment.

There is a detailed description on how to install in http://hootnot.github.io/oanda-trading-environment/getting-started/. The prior messages in this thread are about the automated installation procedure using ansible. Using this you should have no problems.

You write "I get this ..." that is not much to go on. So pls. checkout the install docs.

Make sure you have a v20 token also

Depending on what you want to achieve you might also be helped with the use of oandapyV20 . You can use that without the need of using OTE .

from oanda-trading-environment.

hello @hootnot . I am studying oanda and i have a problem when install v20. i would appreciate if you can help me out. the error shown as
c:\users\admiral\anaconda3\include\pyconfig.h(59): fatal error C1083: Cannot
open include file: 'io.h': No such file or directory
error: command 'C:\Program Files (x86)\Microsoft Visual Studio\2017\Buil
dTools\VC\Tools\MSVC\14.15.26726\bin\HostX86\x64\cl.exe' failed with exi
t status 2

from oanda-trading-environment.

@alphagone ,

• first of all: seen your message you use windows, I never use windows, so regarding that I cannot be of much assistence. You might try stackoverflow
• your question is about v20. If you mean the v20 python library, you should drop the question at OANDA. If it is about oandapyV20, please ask questions there. OTE is an extension to the oanda API.

It seems to me that when you have Python installed, visualstudio is not needed (?). When you have Python installed you should be able to install either v20 or oandapyV20 (or both). I'm the auhor of oandapyV20. OANDA is author of v20. So, when you have a correctly installed Python environment I think your problem is solved. For any further questions: if it concerns v20, please drop the questions at OANDA.

from oanda-trading-environment.

@hootnot thank you for your assistance. i will follow your suggestions

from oanda-trading-environment.

Hello @hootnot,

Upon running OANDAd start, it throws an error:

Traceback (most recent call last):
  File "/Users/jjphung/opt/anaconda3/envs/forex/bin/OANDAd", line 17, in <module>
    from oanda_trading_environment.stream import CandleFactory, StreamRecord
  File "/Users/jjphung/opt/anaconda3/envs/forex/lib/python3.6/site-packages/oanda_trading_environment/stream/__init__.py", line 2, in <module>
    from .candlefactory import CandleFactory, granularity_to_time
  File "/Users/jjphung/opt/anaconda3/envs/forex/lib/python3.6/site-packages/oanda_trading_environment/stream/candlefactory.py", line 5, in <module>
    from streamrecord import HEARTBEAT
ModuleNotFoundError: No module named 'streamrecord'

I followed the instructions for installation here: http://hootnot.github.io/oanda-trading-environment/getting-started/#using-git-

I do have oanda-trading-environmentand oandapyv20 installed. I am on Mac OSX running this project in a conda environment with Python 3.6.12

Any insights on this error? Slightly confused as to why it can't find modules installed with the oanda-trading-environment module. Thank you.

from oanda-trading-environment.

Your intuition is right. I also edited my original comment above to say that I am running Python 3.6.12 in my conda env. I managed to get the daemon up and running after:

1. pip install 2to3
2. 2to3 -w * at oanda-trading-environment folder root. This will fix all module and import errors, while creating *.bak python files for the files that were modified.
3. Up next is fixing Python2 -> Python3 functions like iteritems() in bin/OANDAd line 134.

There are other errors that need fixing defined in the log file, but I just wanted to give an update. Perhaps it is time to officially update this repo to use Python3.

Will update this comment as I continue to work through this.

from oanda-trading-environment.

Thank you. I will look forward to it. Amazing work as always.

from oanda-trading-environment.

Another update: Subscriber not receiving messages. In the log file there is an error: TickProcessor: unknown exception: name 'handle' is not defined. To my understanding, each 'record completed' in the log file should send a message to subscribers, but nothing is being printed out. Subscriber client python3 compatible code:

import zmq

context = zmq.Context()
socket = context.socket(zmq.SUB)
socket.connect("tcp://127.0.0.1:5550")
socket.setsockopt_string(zmq.SUBSCRIBE, "") # or socket.setsockopt(zmq.SUBSCRIBE, b"")


socket.setsockopt(zmq.RCVBUF, 1000)
while True:
    msg = socket.recv()
    print("GOT: ", msg)

Log file:

2021-04-21 19:42:47,160 [INFO] oanda_trading_environment.stream.candlefactory : mode: dancingBearHighLowExtreme change of extremes for US30_USD, M5
2021-04-21 19:42:47,161 [INFO] root : record completed: {"instrument": "US30_USD", "start": "2021-04-22 02:40:00", "end": "2021-04-22 02:45:00", "granularity": "M5", "completed": false, "data": {"open": 34117.8, "high": 34118.0, "low": 34117.8, "last": 34118.0, "volume": 2}}
2021-04-21 19:42:47,162 [INFO] oanda_trading_environment.daemon.plugin : Executing plugin: {'instrument': 'US30_USD', 'start': '2021-04-22 02:40:00', 'end': '2021-04-22 02:45:00', 'granularity': 'M5', 'completed': False, 'data': {'open': 34117.8, 'high': 34118.0, 'low': 34117.8, 'last': 34118.0, 'volume': 2}}
2021-04-21 19:42:47,162 [INFO] oanda_trading_environment.daemon.plugin : plugin: PubSub execute ...
2021-04-21 19:42:47,163 [ERROR] root : TickProcessor: unknown exception: name 'handle' is not defined
2021-04-21 19:42:47,775 [INFO] root : TICK: {'type': 'HEARTBEAT', 'time': '2021-04-22T02:42:47.691073632Z'}
2021-04-21 19:42:47,777 [INFO] oanda_trading_environment.stream.parserV20 : heartbeat: 2021-04-22T02:42:47.691073Z
2021-04-21 19:42:47,779 [INFO] oanda_trading_environment.stream.parserV20 : processed # 7 ticks
2021-04-21 19:42:47,972 [INFO] root : TICK: {'type': 'PRICE', 'time': '2021-04-22T02:42:47.819738753Z', 'bids': [{'price': '64.925', 'liquidity': 5000}, {'price': '64.915', 'liquidity': 5000}, {'price': '64.905', 'liquidity': 10000}], 'asks': [{'price': '64.955', 'liquidity': 5000}, {'price': '64.965', 'liquidity': 5000}, {'price': '64.975', 'liquidity': 10000}], 'closeoutBid': '64.905', 'closeoutAsk': '64.975', 'status': 'tradeable', 'tradeable': True, 'instrument': 'BCO_USD'}
2021-04-21 19:42:47,973 [INFO] oanda_trading_environment.stream.candlefactory : mode: dancingBearHighLowExtreme change of extremes for BCO_USD, M1
2021-04-21 19:42:47,974 [INFO] root : record completed: {"instrument": "BCO_USD", "start": "2021-04-22 02:42:00", "end": "2021-04-22 02:43:00", "granularity": "M1", "completed": false, "data": {"open": 64.95, "high": 64.95, "low": 64.94, "last": 64.94, "volume": 6}}
2021-04-21 19:42:47,974 [INFO] oanda_trading_environment.daemon.plugin : Executing plugin: {'instrument': 'BCO_USD', 'start': '2021-04-22 02:42:00', 'end': '2021-04-22 02:43:00', 'granularity': 'M1', 'completed': False, 'data': {'open': 64.95, 'high': 64.95, 'low': 64.94, 'last': 64.94, 'volume': 6}}
2021-04-21 19:42:47,975 [INFO] oanda_trading_environment.daemon.plugin : plugin: PubSub execute ...
2021-04-21 19:42:47,976 [ERROR] root : TickProcessor: unknown exception: name 'handle' is not defined

I believe I have tracked the error down to the execute functions. I added logging inside each of the 3 execute functions and it is last logged in __call__ in the PluginHandler class, right before the try except block. Right outside the try block the “plugin execute …” logs, but a log statement inside the try except block doesn’t log to file. Yet the “execute failure” error log isn’t seen inside the log file, no errors.

I uncommented out logger line in pubsub.py here:

Lastly, there is an intermittent infinite feedback loop upon OANDAd start OANDAd stop OANDAd start. The daemon fails to come back online sometimes. The solution is a full restart of computer.

2021-04-21 20:13:08,648 [INFO] root : forking slave streamparser
2021-04-21 20:13:08,651 [INFO] root : starting slave streamparser: 4586 4585
2021-04-21 20:13:08,661 [INFO] root : In PluginHandler set()
2021-04-21 20:13:08,663 [INFO] oanda_trading_environment.daemon.plugin : skipping plugin: plainfile, not configured as enabled
2021-04-21 20:13:08,663 [INFO] oanda_trading_environment.daemon.plugin : plugin added to be setup: pubsub
2021-04-21 20:13:08,715 [INFO] pubsub : ZMQ context PUB: 127.0.0.1:5550
2021-04-21 20:13:08,716 [ERROR] root : Address already in use
2021-04-21 20:13:08,717 [ERROR] root : Daemon is stopping 4586
2021-04-21 20:13:08,757 [INFO] root : slave 4586 died: exit: 17
2021-04-21 20:13:08,758 [ERROR] root : slave 4586 died to soon ... respawn with exp. backoff 1 seconds
2021-04-21 20:13:10,759 [INFO] root : forking slave streamparser
2021-04-21 20:13:10,763 [INFO] root : starting slave streamparser: 4600 4585
2021-04-21 20:13:10,771 [INFO] root : In PluginHandler set()
2021-04-21 20:13:10,772 [INFO] oanda_trading_environment.daemon.plugin : skipping plugin: plainfile, not configured as enabled
2021-04-21 20:13:10,773 [INFO] oanda_trading_environment.daemon.plugin : plugin added to be setup: pubsub
2021-04-21 20:13:10,809 [INFO] pubsub : ZMQ context PUB: 127.0.0.1:5550
2021-04-21 20:13:10,810 [ERROR] root : Address already in use
2021-04-21 20:13:10,811 [ERROR] root : Daemon is stopping 4600
2021-04-21 20:13:10,863 [INFO] root : slave 4600 died: exit: 17
2021-04-21 20:13:10,864 [ERROR] root : slave 4600 died to soon ... respawn with exp. backoff 2 seconds
2021-04-21 20:13:14,866 [INFO] root : forking slave streamparser
2021-04-21 20:13:14,869 [INFO] root : starting slave streamparser: 4623 4585
2021-04-21 20:13:14,880 [INFO] root : In PluginHandler set()
2021-04-21 20:13:14,881 [INFO] oanda_trading_environment.daemon.plugin : skipping plugin: plainfile, not configured as enabled
2021-04-21 20:13:14,882 [INFO] oanda_trading_environment.daemon.plugin : plugin added to be setup: pubsub
2021-04-21 20:13:14,942 [INFO] pubsub : ZMQ context PUB: 127.0.0.1:5550
2021-04-21 20:13:14,943 [ERROR] root : Address already in use
2021-04-21 20:13:14,945 [ERROR] root : Daemon is stopping 4623
2021-04-21 20:13:15,024 [INFO] root : slave 4623 died: exit: 18
2021-04-21 20:13:15,025 [ERROR] root : slave 4623 died ... after 1 seconds
2021-04-21 20:13:15,025 [INFO] root : forking slave streamparser
2021-04-21 20:13:15,027 [INFO] root : starting slave streamparser: 4624 4585
2021-04-21 20:13:15,034 [INFO] root : In PluginHandler set()
2021-04-21 20:13:15,035 [INFO] oanda_trading_environment.daemon.plugin : skipping plugin: plainfile, not configured as enabled
2021-04-21 20:13:15,036 [INFO] oanda_trading_environment.daemon.plugin : plugin added to be setup: pubsub
2021-04-21 20:13:15,081 [INFO] pubsub : ZMQ context PUB: 127.0.0.1:5550
2021-04-21 20:13:15,082 [ERROR] root : Address already in use
2021-04-21 20:13:15,083 [ERROR] root : Daemon is stopping 4624
2021-04-21 20:13:15,141 [INFO] root : slave 4624 died: exit: 18
2021-04-21 20:13:15,143 [ERROR] root : slave 4624 died to soon ... respawn with exp. backoff 4 seconds

Hope this helps!

from oanda-trading-environment.

Thank you. 🙏. I'll keep my eyes peeled for it.

from oanda-trading-environment.

@hootnot Were you able to get the config and logging working with the docker container?

from oanda-trading-environment.

from oanda-trading-environment.

That is odd. May you please resend that invite? I didn't receive any notifications/emails and also checked my invitations.

from oanda-trading-environment.

Hi. I would love to try this if at all possible. Would it be possible to access the Docker image?

from oanda-trading-environment.