rod-persky / investoscope-easy-update Goto Github PK

I've noticed when using this utility that the current days closing price and volume (and sometimes other fields) for the current date do not match what is showing on Yahoo.

For example https://finance.yahoo.com/quote/SDRY.L/history shows:

If after running Investoscope-easy-update I look at the historical quote data loaded to Investoscope I see the following:

It can be seen that for current date (in this case 11-Dec-19) the Close, Volume and Low figures in Investoscope do not match what is showing on Yahoo. The High and Open figures are correct. For all other dates the data is correct.

I have checked with several stocks and they all seem to exhibit this problem.

I ran the updater around 19:45 long after the market had closed, however I have run the updater a couple of times during the day so perhaps this is some issue where an earlier update is not being refreshed?

I just setup a new Macbook Pro, running Catalina 10.15.7 and getting Investoscope 3 and Investoscope Update to work was not as straightforward as I had hoped. Leaving some notes here in case it is useful for someone.

1. Investoscope 3 is not available in the app store any more, even in my "Purchased" list. I transferred the .app from a backup, put it into my Applications folder and then when running it had to authenticate with my Apple credentials (of the account I originally bought Investoscope with). Then it runs. I seem unable to change the default location for where data is store (which is annoying) - I get an error message; any help here appreciated. Error is "Invalid data folder / The selected folder is on a disk that is not suitable for user data".
2. It is possible to import holdings from a backup, so you have instruments and holdings. This will not include historical data.
3. Download Investoscope updater and edit line 6 of investoscope.py to include the correct path to the data folder. This was harder to find than expected. I managed to find mine at /Users/[username]/Library/Containers/com.investoscope.Investoscope3/Data/Library/Application Support/Investoscope 3
4. Install requirements for Investoscope Updater, this is at least sudo pip3 install requests (I did this over a little while and can't remember if there was anything else).
5. Update line 3 of script to usr/bin/python3 which reflects the location of Python3 on Catalina.
6. Run Investoscope Updater - it should now run and update instruments :-)

I am getting a couple of errors and my coding skills are not yet up to fixing them:

​/private/var/folders/37/vzx7pkfs76s5447sgvn0j9c00000gn/T/AppTranslocation/91E0BB27-CB3A-4D4E-A5DD-4B3F419B796E/d/Investoscope Updater.app/Contents/Resources/investoscope.py:57: SyntaxWarning: "is not" with a literal. Did you mean "!="?
if process.returncode is not 0:

and

/private/var/folders/37/vzx7pkfs76s5447sgvn0j9c00000gn/T/AppTranslocation/91E0BB27-CB3A-4C4E-A5DD-4B3F417B796E/d/Investoscope Updater.app/Contents/Resources/qapi.py:59: SyntaxWarning: "is not" with a literal. Did you mean "!="?
if len(code_market) is not 2:

I also see some of these but this may be linked to instruments

An exception of type AttributeError occurred. Arguments:
("'NoneType' object has no attribute 'group'",)

I am testing out some alternatives to Investoscope but nothing is as good yet :-(

Mentioning this here - not as an issue with the script but to draw users attention to an issue with some of Yahoo's data.

Some data from LSE has mixed GBP and GBp quotes - typically GBp for the current quote (so after running script the latest price is upto date) but GBP for the historical data. So this means that any reports done in Investoscope including those instruments are out by a factor of 100 for those instruments!

An wrong share is here: https://finance.yahoo.com/quote/WWH.L/history?p=WWH.L
A correct share is here: https://finance.yahoo.com/quote/VOD.L/history?p=VOD.L

This appears to be a known issue, with comments from November indicating Yahoo would look into it but I am still having the issues.

The script uses some modules and my machines doesnt have them. When I run the script, I see:
ModuleNotFoundError: No module named 'requests'

The README file should contain instructions on how to install dependencies / modules used by the script.

Would @Rod-Persky, @thhermansen, or anyone else with Investoscope 3 installed please share their "/Applications/Investoscope.app" file since it is no longer available on the Mac App Store?

Legal note: the author did make the software freeware on Dec 4, 2016 (source)

I followed your instructions step by step , but unfortunately get the following error message : ​

Traceback (most recent call last):
File "update.py", line 7, in
import investoscope
File "/private/var/folders/8p/khw__s2j6jjf__lyxn99jmmm0000gn/T/AppTranslocation/848923EC-13BD-49DF-A15D-D5AA60AA305D/d/Investoscope Updater.app/Contents/Resources/investoscope.py", line 23, in
CSV_DOWNLOAD_PATH.mkdir(exist_ok=True)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/pathlib.py", line 1230, in mkdir
self._accessor.mkdir(self, mode)
FileNotFoundError: [Errno 2] No such file or directory: '/Users/xxxxx/.investoscope/csv_data'

I'm looking to try and add intraday updates to share prices. I have looked into it and have created a simple Python script using a feed from Alpha Vantage which provides free real-time stock updates.

I have completed the first half of a proof of concept that fetches the required data from Alpha Vantage and I'm not looking into updating the database accordingly.

For two reasons I'm thinking that it would be better to do this with AppleScript using the 'tell application' method but I don't know what apple scripts are available within Investoscope or which I should use. The first reason is that I'm assuming if the backend database is updated then this won't automatically trigger an update to the GUI and secondly I cannot figure out the timestamp format for the 'ztime' field in the 'zquote' table.

I therefore have three questions:

Does anyone know what AppleScript commands are available to use in Investoscope and are there any details on how to use?

Does anyone know how to decode the timestamp field that is used in the SQLite database for Investoscope? For example I have a value of '557427600' which equates to the 31st August 2018 18:00 (UK timezone) but I don't know how to convert to this value as it doesn't seem to be the standard unix timestamp

Is there a better way of achieving what I want to do? Any pointers or guidance would be much appreciated.

I'm happy to share my code, if it can be incorporated into this project great, if not then I can distribute it separately, go easy on me as I'm not the most proficient programmer :)

Turns out if you follow these steps

1. Add an entry into /etc/hosts
2. Add a self signed certificate as trusted
3. Start a https server as root (with sudo)

That you can build a simple proxy for Investoscope which emulates yahoo and avoid many problems.

Getting the following error messages (for mutual fund files). It seems to update ETFs (XIC.TO) No clue how to rectify. Is there an easy way to get Google finance data instead?

Unable to get information for TD Canadian Bond Index - e (MUTF_CA:TDB909)
Unable to get information for Mawer New Canada (MUTF_CA:MAW107)
Unable to get information for Chou RRSP (MUTF_CA:CHO102)
Unable to get information for Chou Asia (MUTF_CA:CHO300)
Unable to get information for CIBC Canadian Bond Index (MUTF_CA:CIB503)
Traceback (most recent call last):
File "investoscope_update.py", line 203, in
main()
File "investoscope_update.py", line 192, in main
csv_text = get_quote(item)
File "investoscope_update.py", line 140, in get_quote
return get_yahoo_quote(item)
File "investoscope_update.py", line 134, in get_yahoo_quote
return download_yahoo_csv(item, yahoo_crumb, yahoo_cookie)
File "investoscope_update.py", line 117, in download_yahoo_csv
return tidy_yahoo_csv(csv_data.decode('utf-8'))
File "investoscope_update.py", line 93, in tidy_yahoo_csv
row.pop(5)
IndexError: pop index out of range

I'm having an issue whereby the Investoscope Updater App does not update all the symbols that it used to. It appears to get to one symbol and then just quits (after a split second), and the app does not stay open or give any error message for troubleshooting.

Is there anywhere I can check (manually is fine) a log of what has happened so I can try and fix it? As the Investoscope Updater just quits when it gets to this symbol, it doesn't update a good chunk of my portfolios.

With further investigation this is due to a symbol that has been delisted (GKN.L) so it would be good to handle this gracefully rather than stopping the update and having to go troubleshoot manually. Other symbols have also changed, and I am now at the stage I'm not even getting a split-second to view (or screengrab via video) the offending symbol.

Could you add logging and then it would be possible to troubleshoot. Many thanks in advance and thanks again for creating this.

Steps undertaken

1. Add an index to Investoscope (i.e. ^AXJO)
2. Run Investoscope-easy-update
3. Confirmed that index is not up to date

Expected behaviour
Indicies in Investoscope are updated to reflect EOD price index

Steps to reproduce

1. Add an index to Investoscope (i.e. ^AXJO)
2. Run Investoscope-easy-update
3. Review data and confirm that index information has not been updated

I noticed that yahoo finance has the PE ratio, EPS, market cap and other metadata about the stock. Would it be possible to update the script to grab this information from yahoo and update this information in Investoscope?

The app just runs the python script that is bundled as a resource. This has the investoscope path hard-coded to user directory. My directory is inside iCloud Drive. App should let the user set this directory.

Hi @Rod-Persky is the following by design? I have looked at the code but am not a coder so don't fully understand.

I thought it grabbed the data from Yahoo Finance - but I am confused as to the data being a day delayed compared to the web interface or the manually downloaded files from Yahoo.

See below image. Main browser window shows latest close price for VOD.L of 175.02 on Aug 15 and this is the same in the bottom Excel window, showing the CSV which was manually downloaded. However, Investoscope Easy Update (latest version) has just run and updated, with no error for Vodafone. It shows a close price of 178.78 for Aug 14. This is correct, but it is not showing the latest price for 15 Aug which seems to be available?

Is this intended behaviour / all the script can get? Or is there maybe a line offset glitch in the script? Once again, the data in Investoscope is correct, just not as up to date as it appears it could be. Thanks in advance for continuing the great work on this and ensuring Investoscope limps on :-)

Hi @Rod-Persky I just updated to the latest version you released and it doesn't work on my machine :-(
I updated the .py file with the same path as the previous version. When I run the new app, it opens with a list of shares, saying "up to date:" and then after a few seconds closes. Python doesn't do any connections to the internet.
If I immediately run the older version it cycles through and updates. None of my symbols were up to date.

I can use the old one (but would love the error checking in the new one, as I still get issues with the old one) but am happy to help troubleshoot the new release just let us know what you need. I've got a video of what happens if that helps?

The title is as ominous as it sounds

macOS Catalina comes with Python 3. It is at /usr/bin/python3.

Firstly I would like to say a big thank you for taking the time to create and maintain this. I love Investoscope and was gutted when it stopped working. I haven't been able to find anything else that meets my needs.

Does the script update the exchange rates in the application? I'm in the UK and use GBP but have some foreign shares, mainly US.

Hi Rod Persky,

Can you give the setup instruction in plain english? I don't even know where to start.
I read the instruction on the page. But I don't have any idea.

👍 please open investoscope haha... I'll fix this